[158636] in North American Network Operators' Group
Re: TCP time_wait and port exhaustion for servers
daemon@ATHENA.MIT.EDU (Fred Baker (fred))
Wed Dec 5 17:06:57 2012
From: "Fred Baker (fred)" <fred@cisco.com>
To: Jon Lewis <jlewis@lewis.org>
Date: Wed, 5 Dec 2012 22:06:29 +0000
In-Reply-To: <Pine.LNX.4.61.1212051557220.26706@soloth.lewis.org>
Cc: NANOG <nanog@nanog.org>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
If you want to get into software rewriting, the simplest thing I might come=
up with would be to put TCBs in some form of LRU list and, at a point wher=
e you need a port back, close the TCB that least recently did anything. My =
understanding is that this was implemented 15 years ago to manage SYN attac=
ks, and could be built on to manage this form of "attack".
Or, change the period of time a TCB is willing to stay in time-wait. Instea=
d of 60 seconds, make it 10.
On Dec 5, 2012, at 1:11 PM, Jon Lewis wrote:
> On Wed, 5 Dec 2012, Ray Soucy wrote:
>=20
>> So if I rebuild the kernel to use a 20 second timeout, then that 30000
>> port pool can sustain 1500, and a 60000 port pool can sustain 3000
>> connections per second.
>>=20
>> The software could be re-written to round-robin though IP addresses
>> for outgoing requests, but trying to avoid that.
>=20
> It's kind of a hack, but you don't have to rewrite the software to get di=
fferent source IPs for different connections. On linux, you could do the f=
ollowing:
>=20
> *) Keep your normal default route
> *) Configure extra IPs as aliases (eth0:0, eth0:1,...) on the proxy
> *) Split up the internet into however many subnets you have proxy host IP=
s *) route each part of the internet to your default gateway tacking on "de=
v eth0:n".
>=20
> This will make the default IP for reaching each subnet of the internet th=
e IP from eth0:n.
>=20
> Of course you probably won't get very good load balancing of connections =
over your IPs that way, but it's better than nothing and a really quick fix=
that would give you immediate additional capacity.
>=20
> I was going to also suggest, that to get better balancing, you could peri=
odically (for some relatively short period) rotate the internet subnet rout=
es such that you'd change which parts of the internet were pointed at which=
dev eth0:n every so many seconds or minutes, but that's kind of annoying t=
o people like me (similar to the problem I recently posted about with AT&T =
3G data web proxy). Having your software round robin the source IPs would =
probably introduce the same problem/effect.
>=20
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> Senior Network Engineer | therefore you are
> Atlantic Net |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
>=20
