[130660] in North American Network Operators' Group
RE: reachability problems Europe->US?
daemon@ATHENA.MIT.EDU (John van Oppen)
Thu Oct 7 12:44:34 2010
From: John van Oppen <jvanoppen@spectrumnet.us>
To: 'Heath Jones' <hj1980@gmail.com>
Date: Thu, 7 Oct 2010 16:44:11 +0000
In-Reply-To: <AANLkTikbgeFu1oXGW-g8z3ak=FcNGL2-ewzSwFAwh-EK@mail.gmail.com>
Cc: "nanog@nanog.org" <nanog@nanog.org>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
I know for certain it was gblx, noc confirmed, we saw this to multiple dest=
inations all with the outbound towards gblx (not just DFN). We are on the=
same GBLX pop the sites they are talking about are connected to (westin) a=
nd almost every path I see back to dfn (from seven upstreams in seattle) wa=
s via gblx not qwest, the only exceptions were level3's and Savvis' routes =
which are via AS1299.
I think the asymmetric routing was obfuscating the problem a bit for the gu=
ys attached to DFN.
John
-----Original Message-----
From: Heath Jones [mailto:hj1980@gmail.com]=20
Sent: Thursday, October 07, 2010 9:24 AM
To: John van Oppen
Cc: Thomas Schmid; nanog@nanog.org
Subject: Re: reachability problems Europe->US?
It seemed from the symptoms OP was seeing, that Qwest was the issue.
Has GLBX reported to you that they are having a fault? If not, perhaps
try tagging your exported routes to GLBX with 8010 as per this:
http://onesc.net/communities/as3549/
On 7 October 2010 16:59, John van Oppen <jvanoppen@spectrumnet.us> wrote:
> Global crossing is having major issues (since yesterday actually) in Seat=
tle. =A0 =A0Every path I see to dfn.de is via gblx and Microsoft hosts most=
of those sites out of the seattle area so they may be seeing the same issu=
e.
>
> Based on what we can see gblx has a broken port-channel or something simi=
lar here as random traffic (into) their network via our transit link gets b=
lack-holed. =A0 We could not even reach global crossing's own name servers =
for a while. =A0 =A0We gave up and turned down BGP yesterday until we hear =
from them. =A0 Based on graphs at the time things broke they appeared to be=
black-holing roughly 1/4 of what we were sending them.
>
>
> Thanks,
> John van Oppen
> Spectrum Networks / AS 11404
>
>
> -----Original Message-----
> From: Thomas Schmid [mailto:schmid@dfn.de]
> Sent: Thursday, October 07, 2010 6:10 AM
> To: Heath Jones
> Cc: nanog@nanog.org
> Subject: Re: reachability problems Europe->US?
>
> Hi,
>
> On 07.10.2010 14:35, Heath Jones wrote:
>>> Seems to be only source-prefix-based, but several ISPs in europe are af=
fected.
>> Can you post source and destination IP's ?
>
> source: 131.220.0.0/16, 212.201.68.0/22, 212.201.72.0/21,
> destination: 65.122.178.73, 63.228.223.104
>
> traceroute to 65.122.178.73 (65.122.178.73), 30 hops max, 40 byte packets
> =A01 =A0er-rz-gig-3-3.stw-bonn.de (131.220.99.62) =A01.792 ms =A01.275 ms=
=A01.125 ms
> =A02 =A0xr-bon1-te2-3.x-win.dfn.de (188.1.233.193) =A00.705 ms =A02.132 m=
s =A00.755 ms
> =A03 =A0xr-bir1-te2-3.x-win.dfn.de (188.1.144.9) =A01.477 ms =A01.936 ms =
=A01.051 ms
> =A04 =A0zr-fra1-te0-7-0-5.x-win.dfn.de (188.1.145.46) =A04.034 ms =A03.73=
4 ms =A04.957 ms
> =A05 =A064.213.78.237 (64.213.78.237) =A03.866 ms =A03.295 ms =A026.854 m=
s
> =A06 =A0jfk-brdr-04.inet.qwest.net (63.146.26.225) =A0119.511 ms =A092.73=
5 ms =A099.019 ms
> =A07 =A0* * *
>
> or quote from DE-CIX tech-list:
>
> [www.microsoft.com]
> -----------
> We also have some connectivity problems to ms, changing the bgp routing t=
o
> another tier 1 carrier don t resolve the problem
> -----------
>
> Cheers,
>
> =A0Thomas
>
>
