[27025] in North American Network Operators' Group
Re: RR.COM has backbone problems
daemon@ATHENA.MIT.EDU (Steve Sobol)
Mon Feb 7 14:26:21 2000
Message-ID: <389F1C30.C306E956@NorthShoreTechnologies.net>
Date: Mon, 07 Feb 2000 14:25:36 -0500
From: Steve Sobol <sjsobol@NorthShoreTechnologies.net>
MIME-Version: 1.0
To: Vadim Antonov <avg@kotovnik.com>
Cc: nanog@merit.edu, markh@va.rr.com
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Errors-To: owner-nanog-outgoing@merit.edu
I am cc'ing this to their head of security, who is NOT the right person to talk to, but who should know who
to forward this news to.
Vadim Antonov wrote:
> I tried to contact their NOC, but the bozo on call didn't
> even want to hear what the problem is, advising me to contact
> my cable provider (how helpful).
>
> Please if there's anyone with a clue from RR.COM / MEDIAONE.NET
> or any other Time Warner holding - wake up, you've got a big
> problem. It doesn't show on your network monitors, but it is sure
> to screw your customers.
>
> And, please track down who was on the NOC telephone ((703) 345-3416)
> at this time (see message headers) and kick him hard for being
> an idiot and not taking a real trouble report.
>
> It appears that somehow long packets between boxes
> 24.218.189.197 and 24.218.188.118 get corrupted:
>
> kitty 67: ping -s1500 24.218.188.118
> PING 24.218.188.118 (24.218.188.118): 1500 data bytes
> 1508 bytes from 24.218.188.118: icmp_seq=0 ttl=246 time=164.0 ms
> wrong data byte #1404 should be 0x7c but was 0xaa
> 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> <skipped>
> 1508 bytes from 24.218.188.118: icmp_seq=1 ttl=246 time=164.6 ms
> 1508 bytes from 24.218.188.118: icmp_seq=2 ttl=246 time=166.1 ms
> 1508 bytes from 24.218.188.118: icmp_seq=3 ttl=246 time=164.7 ms
> 1508 bytes from 24.218.188.118: icmp_seq=5 ttl=246 time=164.3 ms
> 1508 bytes from 24.218.188.118: icmp_seq=6 ttl=246 time=168.6 ms
> wrong data byte #1404 should be 0x7c but was 0xaa
> 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> <skipped>
> 1508 bytes from 24.218.188.118: icmp_seq=7 ttl=246 time=163.9 ms
> wrong data byte #924 should be 0x9c but was 0xaa
> 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> <skipped>
>
> kitty 70: ping -s1500 24.218.189.197
> PING 24.218.189.197 (24.218.189.197): 1500 data bytes
> 1508 bytes from 24.218.189.197: icmp_seq=0 ttl=248 time=144.3 ms
> 1508 bytes from 24.218.189.197: icmp_seq=1 ttl=248 time=83.2 ms
> 1508 bytes from 24.218.189.197: icmp_seq=2 ttl=248 time=82.4 ms
> 1508 bytes from 24.218.189.197: icmp_seq=3 ttl=248 time=80.8 ms
> 1508 bytes from 24.218.189.197: icmp_seq=4 ttl=248 time=80.9 ms
>
> kitty 72: /usr/sbin/traceroute 24.218.189.170
> traceroute: Warning: Multiple interfaces found; using 205.158.141.97 @ eth0
> traceroute to 24.218.188.118 (24.218.188.118), 30 hops max, 40 byte packets
> 1 192.168.203.1 (192.168.203.1) 19.680 ms 19.540 ms 19.521 ms
> 2 mpk.core.rtr.wavepath.net (205.158.140.129) 19.825 ms 19.699 ms 21.328 ms
> 3 pop.access.rtr.wavepath.net (205.158.140.249) 21.097 ms 21.252 ms 21.030 ms
> 4 pop.border.rtr.wavepath.net (205.158.140.1) 21.204 ms 21.146 ms 23.449 ms
> 5 us-ca-scl-web1-e8.rtr.concentric.net (205.158.7.97) 22.883 ms 21.793 ms 22.879 ms
> 6 us-ca-scl-border2-fe4-0d0.rtr.concentric.net (207.88.95.9) 24.269 ms 101.210 ms 23.812 ms
> 7 us-ca-scl-core2-g5-0-0.rtr.concentric.net (207.88.1.3) 23.506 ms 24.663 ms 22.793 ms
> 8 PacBellrr.com (198.32.128.50) 23.761 ms 25.667 ms 26.280 ms
> 9 pabrt02-pabrt03.rr.com (24.218.189.197) 26.108 ms 57.919 ms 24.778 ms
> 10 24.218.188.118 (24.218.188.118) 110.427 ms * 116.969 ms
> 11 bsgsr01-nygsr01.rr.com (24.218.188.21) 116.396 ms 115.738 ms 116.053 ms
> 12 cambrt01-bsgsr01.rr.com (24.128.191.58) 113.668 ms 114.166 ms 115.836 ms
> 13 cmrtr01-srp3.ne.mediaone.net (24.218.189.170) 113.589 ms 114.720 ms 113.175 ms
>
> (Note that the problem affects all _transit_ packets):
>
> kitty 78: ping -s1500 24.218.188.21
> PING 24.218.188.21 (24.218.188.21): 1500 data bytes
> 1508 bytes from 24.218.188.21: icmp_seq=1 ttl=245 time=525.0 ms
> 1508 bytes from 24.218.188.21: icmp_seq=2 ttl=245 time=171.2 ms
> 1508 bytes from 24.218.188.21: icmp_seq=3 ttl=245 time=173.8 ms
> 1508 bytes from 24.218.188.21: icmp_seq=4 ttl=245 time=174.3 ms
> wrong data byte #1404 should be 0x7c but was 0xaa
> 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> <skipped>
>
> Regards,
>
> --vadim
--
North Shore Technologies, Cleveland, OH http://NorthShoreTechnologies.net
Steve Sobol, President, Chief Website Architect and Janitor
sjsobol@NorthShoreTechnologies.net - 888.480.4NET - 216.619.2NET
One good thing about the Y2K hoopla: I haven't forgotten to write 2000 as
the year on a single one of my checks! :)
