[175900] in North American Network Operators' Group

home help back first fref pref prev next nref lref last post

Re: outages list down? Lightower, Worcester, MA fiber cut

daemon@ATHENA.MIT.EDU (Jared Mauch)
Fri Nov 7 08:44:07 2014

X-Original-To: nanog@nanog.org
From: Jared Mauch <jared@puck.nether.net>
In-Reply-To: <20141107123413.GM8489@angus.ind.WPI.EDU>
Date: Fri, 7 Nov 2014 08:43:57 -0500
To: Chuck Anderson <cra@WPI.EDU>
Cc: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org

RFO: underlying filesystem issues on the hypervisor caused the system to =
=E2=80=9Cfreak out=E2=80=9D (yes, that=E2=80=99s the technical term).  =
After a reboot, some fighting with java iKVM the system is back up.  =
Sorry for the trouble. =20

OT: (you can delete if you don=E2=80=99t care about my random stuff)

I have some bigger plans to start to split the various parts of puck =
into more distributed systems (e.g.: DNS services, mail lists, etc) of =
time, including the various open.*project sites and databases).

Last hardware refresh was in fall of 2011, so with the falling prices of =
SSD, etc it=E2=80=99s likely time to transition.

- Jared

> On Nov 7, 2014, at 7:34 AM, Chuck Anderson <cra@WPI.EDU> wrote:
>=20
> I can't get to the mailing list page, and neither can
> downforeveryoneorjustme.com:
>=20
> http://downforeveryoneorjustme.com/puck.nether.net
>=20
> I've sent a couple new email messages there and they haven't been
> delivered either.
>=20
> Here is the latest message I sent regarding the Lightower, Worcester,
> MA fiber cut:
>=20
> Date: Fri, 7 Nov 2014 07:26:55 -0500
> From: Chuck Anderson <cra@WPI.EDU>
> To: outages@outages.org
> Subject: Re: [outages] Lightower, Worcester, MA fiber cut
>=20
> Update as of 06:05:16 -0500:
>=20
> "Status has remained the same Lightower has two (2) 288 cables that
> were burnt through in a National Grid  enclosure. National Grid has
> informed us that there is an exposed high tension power line in this
> enclosure and there will be no access granted in the near
> future. National Grid has deemed this manhole unsafe at this
> time. Lightower is working on an alternate solution that will re-route
> fibers around the affected man hole still NO ETTR at this time. We
> apologize for this inconvenience."
>=20
> On Fri, Nov 07, 2014 at 04:04:27AM -0500, Chuck Anderson wrote:
>> Updates as of 01:41:40 -0500 and 02:58:11 -0500:
>>=20
>> "We have two (2) 288 cables that were burnt through in this National
>> Grid enclosure. National Grid has informed us that there is an =
exposed
>> high tension power line in this enclosure and there will no access
>> granted in the near future. National Grid has deemed this manhole
>> unsafe at this time. Lightower is working on an alternate solution
>> that will re-route fibers around the affected man hole. NO ETTR at
>> this time."
>>=20
>> My initial outage time was exactly 18:58:54 EST on two circuits.
>>=20
>> On Thu, Nov 06, 2014 at 08:49:16PM -0500, Chuck Anderson via Outages =
wrote:
>>> We lost several dark fiber circuits out of Lightower in Worcester, =
MA.
>>> There is a manhole fire down the street at Main St. & Highland
>>> St. that is most likely the cause.  Smoke is billowing out of =
several
>>> manholes.  NGrid, NStar, police & fire are on the scene.  Lightower
>>> has dispatched techs.  No ETR yet.
>>>=20
>>> One other of my circuits that leaves the building via a diverse path
>>> is still up luckily, or we'd be off the net entirely.


home help back first fref pref prev next nref lref last post