[33930] in North American Network Operators' Group
Re: From Microsoft's site
daemon@ATHENA.MIT.EDU (Marshall Eubanks)
Thu Jan 25 08:10:54 2001
Message-ID: <3A702616.962B22C8@21rst-century.com>
Date: Thu, 25 Jan 2001 08:11:50 -0500
From: Marshall Eubanks <tme@21rst-century.com>
Reply-To: tme@21rst-century.com
MIME-Version: 1.0
To: Valdis.Kletnieks@vt.edu
Cc: jlewis@lewis.org, nanog@merit.edu
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Errors-To: owner-nanog-outgoing@merit.edu
Vladis, et al;
Of course, we know (well :) that there have been multicast problems
from the MSDP storms from the RAMEN since Saturday before last. If there
have been wider network
problems caused by these MSDP storms, I would like to hear of them,
either on
or off list. I would like to give a report on this in Atlanta.
For those MSDPer's out there, we have good luck with rate limits to limit
the damage. I will be glad to share (off list?) the configs used.
Also, FWIW, there does not seem to have been a MSDP storm at 5:00 PM
or so
on Tuesday.
Regards
Marshall Eubanks
Valdis.Kletnieks@vt.edu wrote:
>
> On Thu, 25 Jan 2001 00:37:58 EST, jlewis@lewis.org said:
> > > Their management should be real embarrassed to take so long to back out the
> > > last-change.
> >
> > Somebody bitched a router config, and it took 22.5 hours to figure it out?
>
> Umm.. let's think more carefully here.
>
> A major *MAJOR* player is changing a config *during prime time*?
>
> Hell, we're not that big, and we get 3AM-7AM local. ANything else is
> emergency-only.
>
> So we'll assume that the *real* timeline was:
>
> 5PM something *else* melts
> 6:30PM change a config to stop THAT emergency
> 6:45PM notice you've scrogged it up
>
> <next 19 hours> try to decide which is worse, the DNS being screwed
> but your *local* operations are back online using local private
> secondaries, or DNS being OK but whatever was loose trashing the
> corporate backbone? Meanwhile, your OTHER set of network monkeys is
> busy fighting whatever fire melted stuff to start with...
>
> <META MODE="so totally hypothetical we won't even GO there...">
> They'd not be the first organization this week that had to make an
> emergency router config change because Ramen multicasting was melting their
> routers, or the first to not get it right on the first try.
>
> They'd merely be the ones thinking hardest how to put the right spin on it...
> </META>`
>
> I have *NO* evidence that Ramen was the actual cause other than it's this
> week's problem. However, I'm pretty sure that *whatever* happened,
> the poor router tech was *already* having a Very Bad Day before he ever
> GOT to the part where he changed the config.....
>
> Valdis Kletnieks
> Operating Systems Analyst
> Virginia Tech
--
Multicast Technologies, Inc.
10301 Democracy Lane, Suite 201
Fairfax, Virginia 22030
Phone : 703-293-9624 Fax : 703-293-9609
e-mail : tme@on-the-i.com http://www.on-the-i.com
