[177522] in North American Network Operators' Group
Re: scaling linux-based router hardware recommendations
daemon@ATHENA.MIT.EDU (Mike Hammett)
Mon Jan 26 21:20:22 2015
X-Original-To: nanog@nanog.org
Date: Mon, 26 Jan 2015 20:20:07 -0600 (CST)
From: Mike Hammett <nanog@ics-il.net>
To: nanog@nanog.org
In-Reply-To: <54C6F3AE.5040703@winterei.se>
Errors-To: nanog-bounces@nanog.org
Different (configuration) strokes for different folks. I look at a Cisco in=
terface now and say, "Who the hell would use this?" despite my decade old C=
isco training.=20
I was corrected offlist that Vyatta does do MPLS now... but I can't find an=
ything on it doing VPLS, so I guess that's still out.=20
The 5600's license (according to their SDNCentral performance report) appea=
rs to be near $7k whereas MT you can get a license for $80.=20
-----=20
Mike Hammett=20
Intelligent Computing Solutions=20
http://www.ics-il.com=20
----- Original Message -----
From: "Paul S." <contact@winterei.se>=20
To: nanog@nanog.org=20
Sent: Monday, January 26, 2015 8:10:54 PM=20
Subject: Re: scaling linux-based router hardware recommendations=20
Like Mike mentioned, the feature list in RouterOS is nothing short of=20
impressive -- problem is that pretty much everything in there is=20
inherently buggy.=20
That and one hell of a painful syntax-schema to work with too.=20
On 1/27/2015 =E5=8D=88=E5=89=8D 10:57, Tony Wicks wrote:=20
> And the solution to this issue is - http://routerboard.com/ or http://www=
.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don'=
t scoff until you have tried it, the price/performance is pretty staggering=
if you are in the sub 20gig space.=20
>=20
> -----Original Message-----=20
> From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett=20
> Sent: Tuesday, 27 January 2015 2:44 p.m.=20
> To: nanog@nanog.org=20
> Subject: Re: scaling linux-based router hardware recommendations=20
>=20
> Aren't most of the new whitebox\open source platforms based on switching =
and not routing? I'd assume that the "cloud-scale" data centers deploying t=
his stuff still have more traditional big iron at their cores.=20
>=20
> The small\medium sized ISP usually is left behind. They're not big enough=
to afford the big new hardware, but all of their user's NetFlix and porn a=
nd whatever else they do is chewing up bandwidth. For example, the small\me=
dium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the=
old hardware (besides being old) is likely in a huge chassis if you can ge=
t any sort of port density at all.=20
>=20
> 48 port GigE switches with a couple 10GigE can be had for $100. A minimum=
of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x =
to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a cou=
ple 10GigEs are even more money, I'd assume.=20
>=20
> I thought vMX was going to save the day, but it's pricing for 10 gigs of =
traffic (licensed by throughput and standard\advanced licenses) is really a=
bout 5x - 10x what I'd be willing to pay for it.=20
>=20
> Haven't gotten a quote from AlcaLu yet.=20
>=20
> Vyatta (last I checked, which was admittedly some time ago) doesn't have =
MPLS.=20
>=20
> The FreeBSD world can bring zero software cost and a stable platform, but=
no MPLS.=20
>=20
> Mikrotik brings most (though not all) of the features one would want... a=
good enough feature set, let's say... but is a non-stop flow of bugs. I do=
n't think a week or two goes by where one of my friends doesn't submit some=
sort of reproducible bug to Mikrotik. They've also been "looking into" DPD=
K for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm=
always left wanting just a little more, but it may be the best balance bet=
ween the features and performance I want and the ability to pay for it.=20
>=20
>=20
>=20
>=20
> -----=20
> Mike Hammett=20
> Intelligent Computing Solutions=20
> http://www.ics-il.com=20
>=20
> ----- Original Message -----=20
>=20
> From: "Mehmet Akcin" <mehmet@akcin.net>=20
> To: "micah anderson" <micah@riseup.net>=20
> Cc: nanog@nanog.org=20
> Sent: Monday, January 26, 2015 6:06:53 PM=20
> Subject: Re: scaling linux-based router hardware recommendations=20
>=20
> Cumulus Networks has some stuff,=20
>=20
> http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal=
-2014-final.pdf=20
>=20
> Pretty decent presentation with more details you like.=20
>=20
> Mehmet=20
>=20
>> On Jan 26, 2015, at 8:53 PM, micah anderson <micah@riseup.net> wrote:=20
>>=20
>>=20
>> Hi,=20
>>=20
>> I know that specially programmed ASICs on dedicated hardware like=20
>> Cisco, Juniper, etc. are going to always outperform a general purpose=20
>> server running gnu/linux, *bsd... but I find the idea of trying to use=
=20
>> proprietary, NSA-backdoored devices difficult to accept, especially=20
>> when I don't have the budget for it.=20
>>=20
>> I've noticed that even with a relatively modern system (supermicro=20
>> with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server=20
>> adapters, and 16gig of ram, you still tend to get high percentage of=20
>> time working on softirqs on all the CPUs when pps reaches somewhere=20
>> around 60-70k, and the traffic approaching 600-900mbit/sec (during a=20
>> DDoS, such hardware cannot typically cope).=20
>>=20
>> It seems like finding hardware more optimized for very high packet per=
=20
>> second counts would be a good thing to do. I just have no idea what is=
=20
>> out there that could meet these goals. I'm unsure if faster CPUs, or=20
>> more CPUs is really the problem, or networking cards, or just plain=20
>> old fashioned tuning.=20
>>=20
>> Any ideas or suggestions would be welcome!=20
>> micah=20
>>=20
