[177525] in North American Network Operators' Group
Re: scaling linux-based router hardware recommendations
daemon@ATHENA.MIT.EDU (David bass)
Mon Jan 26 22:21:45 2015
X-Original-To: nanog@nanog.org
In-Reply-To: <10EE5175-77DF-4A97-BA1C-BCD89BA6483E@gmail.com>
From: David bass <davidbass570@gmail.com>
Date: Mon, 26 Jan 2015 19:07:11 -0600
To: Phil Bedard <bedard.phil@gmail.com>
Cc: micah anderson <micah@riseup.net>, "<nanog@nanog.org>" <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
I'm also in the research stage of building our own router. I'm interested i=
n reading more if you can post links to some of this research and/or testing=
.=20
David
Sent from my iPad
> On Jan 26, 2015, at 6:45 PM, Phil Bedard <bedard.phil@gmail.com> wrote:
>=20
> Kind of unsurprisingly, the traditional network vendors are somewhat at=20=
> the forefront of pushing what an x86 server can do as well. Brocade=20
> (Vyatta), Juniper, and Alcatel-Lucent all have virtualized routers using=20=
> Intel's DPDK pushing 5M+ PPS at this point. They are all also tweaking=20=
> what Intel is providing, and they are the ones with lots of software=20
> developers with a lot of hardware and network programming experience. =20
>=20
> ALU claims to be able to get 160Gbps full duplex through a 2RU server with=
=20
> 16x10G interfaces and two 10-core latest-gen Xeon processors. Of course=20=
> that's probably at 9000 byte packet sizes, but at Imix type traffic it's=20=
> probably still pushing 60-70Gbps. They have a demo of lots of them in a=20=
> single rack managed as a single router pushing Tbps. =20
>=20
> A commerical offering you are going to pay for that kind of performance=20=
> and the control plane software. Over time though you'll see the DPDK type=
=20
> enhancements make it into standard OS stacks. Other options include=20
> servers with integrated network processors or NPs on a PCI card, there is=20=
> a whole rash of those type of devices out there now and coming out. =20
>=20
> Phil=20
>=20
>=20
>=20
>> On 1/26/15, 22:53, "micah anderson" <micah@riseup.net> wrote:
>>=20
>>=20
>> Hi,
>>=20
>> I know that specially programmed ASICs on dedicated hardware like Cisco,
>> Juniper, etc. are going to always outperform a general purpose server
>> running gnu/linux, *bsd... but I find the idea of trying to use
>> proprietary, NSA-backdoored devices difficult to accept, especially when
>> I don't have the budget for it.
>>=20
>> I've noticed that even with a relatively modern system (supermicro with
>> a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server
>> adapters, and 16gig of ram, you still tend to get high percentage of
>> time working on softirqs on all the CPUs when pps reaches somewhere
>> around 60-70k, and the traffic approaching 600-900mbit/sec (during a
>> DDoS, such hardware cannot typically cope).
>>=20
>> It seems like finding hardware more optimized for very high packet per
>> second counts would be a good thing to do. I just have no idea what is
>> out there that could meet these goals. I'm unsure if faster CPUs, or
>> more CPUs is really the problem, or networking cards, or just plain old
>> fashioned tuning.
>>=20
>> Any ideas or suggestions would be welcome!
>> micah
>=20
