[29164] in North American Network Operators' Group
Re: PMTU-D: remember, your load balancer is broken
daemon@ATHENA.MIT.EDU (Valdis.Kletnieks@vt.edu)
Tue Jun 13 23:35:52 2000
Message-Id: <200006140333.e5E3XmL28888@black-ice.cc.vt.edu>
To: Marc Slemko <marcs@znep.com>
Cc: nanog@merit.edu
In-reply-to: Your message of "Tue, 13 Jun 2000 17:04:19 MDT."
<Pine.BSF.4.20.0006131643401.572-100000@alive.znep.com>
From: Valdis.Kletnieks@vt.edu
Date: Tue, 13 Jun 2000 23:33:48 -0400
Errors-To: owner-nanog-outgoing@merit.edu
On Tue, 13 Jun 2000 17:04:19 MDT, Marc Slemko <marcs@znep.com> said:
> Chances are that if you are using a load balancer for TCP connections,
> then it does not properly handle Path MTU Discovery. Examples of devices
Does anybody have any field experience on how much PMTU-D actually
helps? I just checked 'netstat -s' on an AIX box that runs a
stratum-2 NTP server, which accidentally had it enabled for
several weeks. Abridged output follows:
ip:
16357209 total packets received
18411 fragments received
5314999 path MTU discovery packets sent
0 path MTU discovery decreases detected
icmp:
Input histogram:
echo reply: 3635421
destination unreachable: 271455
AIX sends a test ICMP Echo to detect PMTU for UDP (which is where the
high icmp numbers came from). The main interface on the box is a
10BaseT, so the MTU gets nailed to 1500. As a result, I do *not* have
figures on how often we would have used a bigger MTU than 1500 - only
on whether there's still sub-1500 links out there. On the other
hand, at least in today's Internet, the Other End is still quite
likely to be 10BaseT or PPP.
Approximately 80% of the traffic this machine sees is from off-campus,
all over the US. We only got about 60% replies on the test ICMP Echo,
which constituted a good 40% of the entire traffic. In spite of this,
not once did the PMTU get fragmented below 1500.
Admittedly, PMTU-D for TCP is a lot less resource intensive (just
set the DF bit and see who salutes). However, it should be tripped
roughly the same percent of the time (if a packet needs fragmenting,
it needs fragmenting - it's probably rare that a TCP packet of a given
size would fit but the same size UDP would fragment).
It looks to me like a better Rule Of Thumb is just:
a) If you know that the path to a specific net has an MTU over
1500 all the way, set a route specifying the MTU.
b) If you're a webserver or something else providing service Out
There to random users, just nail the MTU at 1500, which will
work for any Ethernet/PPP/SLIP out there. And if you're load
balancing to geographically disparate servers, then your users
are probably Out There, with an MTU almost guaranteed to be 1500.
I assert that the chances of PMTU-D helping are in direct ratio to the
number of end users who have connections with MTU>1500 - it's almost
a sure thing that you probably won't have users with an MTU on their
last-hop that's bigger than their campus backbone and/or Internet
connection's MTU.
Is anybody seeing any documentable wins by using PMTU-D?
Valdis Kletnieks
Operating Systems Analyst
Virginia Tech
