[182102] in North American Network Operators' Group
Re: Level3 routing issue US west coast
daemon@ATHENA.MIT.EDU (Joseph Jenkins)
Fri Jul 10 08:55:14 2015
X-Original-To: nanog@nanog.org
From: Joseph Jenkins <joe@breathe-underwater.com>
In-Reply-To: <5aea0a23de084d508d02aa46e4a08513@anx-i-dag02.anx.local>
Date: Fri, 10 Jul 2015 05:55:07 -0700
To: "nanog@nanog.org" <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
Level3 had an issue with one of their core routers in Los Angeles last =
night(7pm Pacific) and early this morning(1am Pacific). Last update to =
my trouble ticket had the issue still being reviewed by engineering, but =
that a core router was dropping packets.
> On Jul 10, 2015, at 3:59 AM, J=FCrgen Jaritsch <jj@anexia.at> wrote:
>=20
> Hi,
>=20
> does anyone else experience issues with the Level3 network at the US =
west coast? We see lots of broken paths like this:
>=20
> Packets =
Pings
> Host Loss% Snt Last =
Avg Best Wrst StDev
> 1. er-01.0v-00-03.anx01.klu.at.anexia-it.com 0.0% 231 0.6 =
0.5 0.2 18.1 1.2
> 2. cr-01.0v-08-06.anx01.klu.at.anexia-it.com 0.0% 231 0.5 =
9.9 0.3 361.1 40.1
> 3. cr-04.01-01-04.anx03.vie.at.anexia-it.com 0.0% 230 6.5 =
7.7 6.3 49.7 5.3
> 4. win-b4-link.telia.net 0.0% 230 6.6 =
6.8 6.4 20.2 1.5
> 5. level-ic-1573273-wien-b4.c.telia.net 0.0% 230 6.6 =
9.3 6.3 69.1 9.6
> 6. ae-2-70.edge1.SanJose3.Level3.net 38.4% 230 164.8 =
165.0 164.5 194.9 2.6
> 7. ae-2-70.edge1.SanJose3.Level3.net 45.9% 230 164.7 =
164.8 164.5 174.1 0.9
> 8. 4.53.208.102 34.3% 230 634.9 =
310.7 168.5 680.1 199.7
> 9. TenGE5-4.br01.seo01.pccwbtn.net 34.1% 230 412.0 =
455.2 304.9 954.6 203.4
> 10. sejong-telecom.ge5-3.br01.seo01.pccwbtn.net 40.6% 230 323.4 =
441.4 323.1 822.0 182.1
> 11. 211.115.201.92 38.4% 230 289.8 =
412.9 289.6 846.7 185.4
> 12. 61.250.89.2 35.8% 230 290.6 =
439.4 290.2 804.3 205.1
> 13. ???
>=20
> Trace from NYC is also broken:
>=20
> Packets =
Pings
> Host Loss% Snt =
Last Avg Best Wrst StDev
> 1. cr-01.0v-00-05.anx32.nyc.us.anexia-it.com 0.0% 30 =
0.4 4.3 0.4 57.8 13.3
> 2. nyk-b5-link.telia.net 0.0% 30 =
0.3 0.4 0.3 0.9 0.1
> 3. ???
> 4. ae-3-80.edge1.SanJose3.Level3.net 17.2% 30 =
71.7 73.2 71.7 98.7 5.5
> 5. ae-3-80.edge1.SanJose3.Level3.net 0.0% 30 =
71.8 71.8 71.7 72.0 0.1
> 6. 4.53.208.102 31.0% 30 =
569.6 250.7 70.5 579.3 231.2
> 7. 63.218.250.73 31.0% 30 =
672.6 355.5 178.0 672.6 232.1
>=20
>=20
> At 10:24 UTC+2 it was even more broken:
>=20
> Packets =
Pings
> Host Loss% Snt Last =
Avg Best Wrst StDev
> 1. er-01.0v-00-03.anx01.klu.at.anexia-it.com 0.0% 326 0.4 =
0.5 0.3 39.7 2.2
> 2. cr-01.0v-08-06.anx01.klu.at.anexia-it.com 0.0% 326 0.5 =
6.7 0.3 198.1 26.3
> 3. cr-04.01-01-04.anx03.vie.at.anexia-it.com 0.0% 326 6.6 =
7.6 6.4 43.6 4.5
> 4. win-b4-link.telia.net 0.0% 326 6.7 =
7.4 6.3 43.1 3.2
> 5. level-ic-1573273-wien-b4.c.telia.net 0.0% 326 6.9 =
9.2 6.3 73.2 10.1
> 6. ae-1-60.edge5.LosAngeles1.Level3.net 62.6% 326 164.7 =
165.5 164.5 176.7 1.5
> ae-2-70.edge1.SanJose3.Level3.net
> 7. ae-1-60.edge5.LosAngeles1.Level3.net 63.1% 326 164.8 =
165.8 164.6 204.2 3.9
> ae-2-70.edge1.SanJose3.Level3.net
> 8. 205.129.5.70 74.2% 326 799.9 =
487.2 169.0 799.9 305.7
> 4.53.208.102
> 9. TenGE5-4.br01.seo01.pccwbtn.net 77.2% 326 1359. =
701.0 308.7 3716. 510.6
> 10. sejong-telecom.ge5-3.br01.seo01.pccwbtn.net 75.1% 326 960.4 =
643.0 323.4 960.4 307.6
> 11. 211.115.201.92 68.9% 326 925.3 =
674.2 289.8 932.3 296.6
> 12. 61.250.89.2 72.9% 326 928.5 =
637.2 291.9 928.5 304.3
> 13. ???
>=20
>=20
> best regards
>=20
> J=FCrgen Jaritsch
> Head of Network & Infrastructure
>=20
> ANEXIA Internetdienstleistungs GmbH
>=20
> Telefon: +43-5-0556-300
> Telefax: +43-5-0556-500
>=20
> E-Mail: jj@anexia.at<mailto:jj@anexia.at>
> Web: http://www.anexia.at<http://www.anexia.at/>
>=20
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstra=DFe 140, 9020 =
Klagenfurt
> Gesch=E4ftsf=FChrer: Alexander Windbichler
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT =
U63216601
>=20
