[177517] in North American Network Operators' Group
Re: scaling linux-based router hardware recommendations
daemon@ATHENA.MIT.EDU (Sudeep Khuraijam)
Mon Jan 26 20:05:47 2015
X-Original-To: nanog@nanog.org
From: Sudeep Khuraijam <skhuraijam@liveops.com>
To: micah anderson <micah@riseup.net>, "nanog@nanog.org" <nanog@nanog.org>
Date: Mon, 26 Jan 2015 17:05:29 -0800
In-Reply-To: <87vbjt6tml.fsf@muck.riseup.net>
Errors-To: nanog-bounces@nanog.org
It really depends on the application that you are interested in beyond
forwarding, but not knowing that and to scale forwarding =B3at a
reasonable price", things have to come off cpu and become more customized
for forwarding, especially for low latency forwarding. The optimization
comes in minimizing packet tuple copies, off load to co-processors and
network coprocessors (some of which can be in NICs) and parallel
processing with some semblance of shared memory across, all of which
takes customization beyond CPU and Kernel which in itself needs to be
stripped down bare and embedded. Ultimately that=B9s what appliance vendor=
s
do with different levels of hardware/firmware customization depending on
ROI of features, speeds and price. A generic OpenSource compatible OEM
product with multi-gig ports will generally be at least half to 5th the
price of a high end latest server architecture server product with ability
to support 10 gig interfaces in the same forwarding performance range
(which are in the market for a different scale problem in compute and net
I/O but exist at a price point that make them exorbitant to solve
forwarding speed).
Cheers,
Sudeep Khuraijam
On 1/26/15, 2:53 PM, "micah anderson" <micah@riseup.net> wrote:
>
>Hi,
>
>I know that specially programmed ASICs on dedicated hardware like Cisco,
>Juniper, etc. are going to always outperform a general purpose server
>running gnu/linux, *bsd... but I find the idea of trying to use
>proprietary, NSA-backdoored devices difficult to accept, especially when
>I don't have the budget for it.
>
>I've noticed that even with a relatively modern system (supermicro with
>a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server
>adapters, and 16gig of ram, you still tend to get high percentage of
>time working on softirqs on all the CPUs when pps reaches somewhere
>around 60-70k, and the traffic approaching 600-900mbit/sec (during a
>DDoS, such hardware cannot typically cope).
>
>It seems like finding hardware more optimized for very high packet per
>second counts would be a good thing to do. I just have no idea what is
>out there that could meet these goals. I'm unsure if faster CPUs, or
>more CPUs is really the problem, or networking cards, or just plain old
>fashioned tuning.
>
>Any ideas or suggestions would be welcome!
>micah
>
