[177519] in North American Network Operators' Group
RE: scaling linux-based router hardware recommendations
daemon@ATHENA.MIT.EDU (Tony Wicks)
Mon Jan 26 20:58:14 2015
X-Original-To: nanog@nanog.org
From: "Tony Wicks" <tony@wicks.co.nz>
To: "'Mike Hammett'" <nanog@ics-il.net>, <nanog@nanog.org>
In-Reply-To: <21180758.11197.1422322999684.JavaMail.mhammett@ThunderFuck>
Date: Tue, 27 Jan 2015 14:57:44 +1300
WTL-MailScanner-From: tony@wicks.co.nz
Errors-To: nanog-bounces@nanog.org
And the solution to this issue is - http://routerboard.com/ or =
http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 =
switch. Don't scoff until you have tried it, the price/performance is =
pretty staggering if you are in the sub 20gig space.
=20
-----Original Message-----
From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett
Sent: Tuesday, 27 January 2015 2:44 p.m.
To: nanog@nanog.org
Subject: Re: scaling linux-based router hardware recommendations
Aren't most of the new whitebox\open source platforms based on switching =
and not routing? I'd assume that the "cloud-scale" data centers =
deploying this stuff still have more traditional big iron at their =
cores.=20
The small\medium sized ISP usually is left behind. They're not big =
enough to afford the big new hardware, but all of their user's NetFlix =
and porn and whatever else they do is chewing up bandwidth. For example, =
the small\medium ISPs are at the Nx10GigE stage now. The new hardware is =
expensive, the old hardware (besides being old) is likely in a huge =
chassis if you can get any sort of port density at all.=20
48 port GigE switches with a couple 10GigE can be had for $100. A =
minimum of 24 port 10GigE switches (except for the occasional IBM switch =
) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more =
than just a couple 10GigEs are even more money, I'd assume.=20
I thought vMX was going to save the day, but it's pricing for 10 gigs of =
traffic (licensed by throughput and standard\advanced licenses) is =
really about 5x - 10x what I'd be willing to pay for it.=20
Haven't gotten a quote from AlcaLu yet.=20
Vyatta (last I checked, which was admittedly some time ago) doesn't have =
MPLS.=20
The FreeBSD world can bring zero software cost and a stable platform, =
but no MPLS.=20
Mikrotik brings most (though not all) of the features one would want... =
a good enough feature set, let's say... but is a non-stop flow of bugs. =
I don't think a week or two goes by where one of my friends doesn't =
submit some sort of reproducible bug to Mikrotik. They've also been =
"looking into" DPDK for 2.5 years now. hasn't shown up yet. I've used MT =
for 10 years and I'm always left wanting just a little more, but it may =
be the best balance between the features and performance I want and the =
ability to pay for it.=20
-----
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com=20
----- Original Message -----
From: "Mehmet Akcin" <mehmet@akcin.net>
To: "micah anderson" <micah@riseup.net>
Cc: nanog@nanog.org
Sent: Monday, January 26, 2015 6:06:53 PM
Subject: Re: scaling linux-based router hardware recommendations=20
Cumulus Networks has some stuff,=20
http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal=
-2014-final.pdf=20
Pretty decent presentation with more details you like.=20
Mehmet=20
> On Jan 26, 2015, at 8:53 PM, micah anderson <micah@riseup.net> wrote:=20
>=20
>=20
> Hi,
>=20
> I know that specially programmed ASICs on dedicated hardware like=20
> Cisco, Juniper, etc. are going to always outperform a general purpose=20
> server running gnu/linux, *bsd... but I find the idea of trying to use =
> proprietary, NSA-backdoored devices difficult to accept, especially=20
> when I don't have the budget for it.
>=20
> I've noticed that even with a relatively modern system (supermicro=20
> with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server=20
> adapters, and 16gig of ram, you still tend to get high percentage of=20
> time working on softirqs on all the CPUs when pps reaches somewhere=20
> around 60-70k, and the traffic approaching 600-900mbit/sec (during a=20
> DDoS, such hardware cannot typically cope).
>=20
> It seems like finding hardware more optimized for very high packet per =
> second counts would be a good thing to do. I just have no idea what is =
> out there that could meet these goals. I'm unsure if faster CPUs, or=20
> more CPUs is really the problem, or networking cards, or just plain=20
> old fashioned tuning.
>=20
> Any ideas or suggestions would be welcome!=20
> micah
>=20
