[146989] in North American Network Operators' Group
Re: ATT GigE issue on 11/19 in Kansas City
daemon@ATHENA.MIT.EDU (Stefan)
Wed Nov 30 09:54:24 2011
In-Reply-To: <56C0B9C7-937A-465B-B8C4-65D9E690EDB6@gmail.com>
Date: Wed, 30 Nov 2011 08:53:34 -0600
From: Stefan <netfortius@gmail.com>
To: Brad Fleming <bdflemin@gmail.com>
Cc: comptech@kc.rr.com, nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
On Wed, Nov 30, 2011 at 8:21 AM, Brad Fleming <bdflemin@gmail.com> wrote:
> On Nov 29, 2011, at 8:17 PM, <comptech@kc.rr.com> wrote:
>
>> We lost several of our GigE links to AT&T for 6 hours on 11/19, anyone e=
lse see this and get a root cause from AT&T? All I can get is that they bel=
ieve a change caused the issue.
>>
> We lost several (but not all) of our Optiman circuits on 11/19 at about 1=
0:20am. We were told the root issue was that all VLANs in one of their swit=
ches had been accidentally deleted / removed. We were never able to get any=
additional detail (like "how") but services were restored about 16:45.
+1 to the above - we received the following RFO, from the their NOC:
"All impacted VLANS were rebuilt to restore service. It is believed
there were some configuration changes that caused the VLAN troubles. A
case has been opened with Cisco to further investigate the root
cause."
***Stefan Mititelu
http://twitter.com/netfortius
http://www.linkedin.com/in/netfortius
