[195551] in North American Network Operators' Group
Re: 100G - Whitebox
daemon@ATHENA.MIT.EDU (=?utf-8?Q?Fredrik_Korsb=C3=A4ck?=)
Sun Aug 20 13:25:00 2017
X-Original-To: nanog@nanog.org
From: =?utf-8?Q?Fredrik_Korsb=C3=A4ck?= <hugge@nordu.net>
In-Reply-To: <4A3FA52E-22E1-49C0-B533-F49FC7E27F64@pch.net>
Date: Sun, 20 Aug 2017 19:24:41 +0200
To: Bill Woodcock <woody@pch.net>
Cc: NANOG list <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
The only viable merchant silicon chip that would be useful for a IXP is from=
the StrataDNX-family which house the jericho/qumran/petra/arad chips from b=
roadcom. No packetbuffer in the exhangepoint will shred performance signific=
antly, especially when one of your bursty 100G customers starts sending data=
into 1/10G customers.=20
To the best of my knowledge the only one that offers DNX in whitebox-fashion=
is Agema and Edgecore. But why whitebox? Except on a very few occasions whi=
tebox is just "i like paying hardware and software on different invoices =3D=
whitebox" the TCO is just the same but. As an exchangepoint i also see that=
it can hard to reap the benefits of all the hipstershit going on in these N=
OS-startups, you want spanning-tree, port-security, something to loadbalance=
over links and perhaps a overlaying-technology if the IXP becomes to big an=
d distributed, like vxlan. This is to easy almost.=20
Whenever i see unbuffered mix-speed IXPs i ask if i can pay 25% of the portc=
ost since that is actually how much oumpfff i would get through the port.
// hugge @ 2603
> 20 aug. 2017 kl. 18:40 skrev Bill Woodcock <woody@pch.net>:
>=20
> Why don't we just swap out your 40g switch for a 100g switch? You've had t=
he 40g one for a while, and we anticipate upgrades every 18-24 months.=20
>=20
> -Bill
>=20
>=20
>> On Aug 20, 2017, at 08:46, Mike Hammett <nanog@ics-il.net> wrote:
>>=20
>> I first sent to an IX-specific mailing list, but as I have yet to see the=
message hit the list, I figured I would post it here as well.=20
>>=20
>> We've had multiple requests for 100G interfaces (instead of Nx10G) and no=
one seems to care about the 40G interfaces we have available.=20
>>=20
>> Looking at cost effective options brings us to whitebox switches. Obvious=
ly there's a wide range of hardware vendors and there are a few OSes availab=
le as well. Cumulus seems to be the market leader, while IPinFusion seems to=
be the most feature-rich.=20
>>=20
>> We're not doing any automation on the switches at this time, so it would s=
till need decent manual configuration. It wouldn't need a Cisco-centric CLI a=
s we're quite comfortable managing standard Linux-type config files. We're n=
ot going all-in on some overlay either given that we wouldn't be replacing o=
ur entire infrastructure, only supplementing it where we need 100G. I know t=
hat LINX has gone IPinfusion. What OS would be appropriate for our usage? I'=
m not finding many good comparisons of the OSes out there. I'm assuming any o=
f them would work, but there may be gotchas that a "cheapest that meets requ=
irements" doesn't quite unveil.=20
>>=20
>> Any particular hardware platforms to go towards or avoid? Broadcom Tomaha=
wk seems to be quite popular with varying control planes. LINX went Edgecore=
, which was on my list given my experience with other Accton brands. Fiberst=
ore has a switch where they actually publish the pricing vs. a bunch of myst=
ery.=20
>>=20
>> Thoughts?=20
>>=20
>>=20
>>=20
>>=20
>> -----=20
>> Mike Hammett=20
>> Intelligent Computing Solutions=20
>> http://www.ics-il.com=20
>>=20
>> Midwest-IX=20
>> http://www.midwest-ix.com=20
>=20
