[45822] in North American Network Operators' Group
RE: red [was Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?]
daemon@ATHENA.MIT.EDU (Brennan_Murphy@NAI.com)
Thu Feb 21 17:03:23 2002
Message-ID: <B481990C9658D411BD3C009027D6F5440247797D@SNC-5-87.nai.com>
From: Brennan_Murphy@NAI.com
To: smd@clock.org, nanog@merit.edu
Date: Thu, 21 Feb 2002 16:03:28 -0600
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----_=_NextPart_001_01C1BB23.97AE5A40"
Errors-To: owner-nanog-outgoing@merit.edu
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C1BB23.97AE5A40
Content-Type: text/plain;
charset="iso-8859-1"
Remember too that if you have multiple types of
flows on your link (FTP, TELNET, HTTP) the key isn't
so much to utilize your DS3 99% as it is to ensure
that the end user's expectations aren't upset. So,
you might use WRED to drop bulky FTP packets instead
of Voice, Video, Telnet, some HTTP, etc...in those cases
where you reach the threshold that you specify. Does an
FTP user really notice if their download starts at
160kbs and gyrates around between 30-150kbs..especially
if they are very large downloads? But a web page taking
12 seconds to load...not good.
-----Original Message-----
From: smd@clock.org [mailto:smd@clock.org]
Sent: Wednesday, February 20, 2002 1:02 PM
To: nanog@merit.edu
Subject: red [was Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?]
randy bush writes:
| you can turn on [x]red. it will make a better choice of which packets
| to drop. but packets will still be dropped [0].
fsvo "better". red drops packets randomly, a little earlier than
packets will be tail-dropped by being unable to find buffer space
in a queue. in most cases, tail-drop is worst for TCP connections
in slow-start, and leads to such fun things as web-page-slowdown-syndrome,
or ssh-stall-syndrome, with the attendant hitting of the "reload" button
or frustrated banging on the keyboard trying to get one's keys to echo
back, or trying to unstick the page of text being sent to /dev/tty on
the remote end.
"r" for random: this spreads out loss randomly, and in most cases,
this means that several different TCPs each observe a single lost
segment, which is quickly detected and causes an attendant slow-down,
but not a stall.
"e" for early: we have to keep the queue from filling, because
that leads to tail-drop, so we have to drop packets when we
could really send them. there is no way around this -- in virtually any
implementation, the mechanism to turn on red can be translated
into more fifo-queue space. the trade-off (early random dropping)
is usually worth it.
there are some corner cases where there is insufficient ability
to spread drops across enough flows, where resulting behaviour can
be alot like tail-drop (a huge difference between fast input speed
and slow output speed, with only a few bulk-transfer flows), but
these cases are rare, particularly where
average_segment_size / bottleneck_kbps <= 0.4, with the 0.4 being
a historical minimum target for queueing in the presence of flows
which experience long RTTs (like ones across oceans). in general,
you don't have to worry about that, though.
| you can increase buffers blah blah blah. but twenty tomatoes will still
| not fit in a fifteen tomato can.
yup, the trick is to deploy a red control law that drops just before
tail-dropping would start, so we don't lose packets unnecessarily,
yet early enough before tail-dropping would start that the senders
have time to back off in response to the detected lost segment,
which is driven by the RTT. drop too early, and you get an underused
line. drop too late and you get tail-drop, which also gives you
an underused line. drop just right, and you can keep your line
very very very close to 100% for sustained periods of time (hours!),
assuming a large enough set of TCP flows that random dropping spreads
the early dropping across different flows.
| [0] - corollary: qos mechanisims decide which packets to drop. but isps
| are paid not to drop any packets.
it is impossible to get 0 drop and enjoy statistical multiplexing gain.
statmux gain keeps the network cheap enough to use. red minimizes the
liklihood that anyone will complain about performance during those
inevitable periods when one's statistical bet goes wrong.
Sean.
------_=_NextPart_001_01C1BB23.97AE5A40
Content-Type: text/html;
charset="iso-8859-1"
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.19">
<TITLE>RE: red [was Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?]</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>Remember too that if you have multiple types of</FONT>
<BR><FONT SIZE=2>flows on your link (FTP, TELNET, HTTP) the key isn't</FONT>
<BR><FONT SIZE=2>so much to utilize your DS3 99% as it is to ensure</FONT>
<BR><FONT SIZE=2>that the end user's expectations aren't upset. So,</FONT>
<BR><FONT SIZE=2>you might use WRED to drop bulky FTP packets instead</FONT>
<BR><FONT SIZE=2>of Voice, Video, Telnet, some HTTP, etc...in those cases</FONT>
<BR><FONT SIZE=2>where you reach the threshold that you specify. Does an</FONT>
<BR><FONT SIZE=2>FTP user really notice if their download starts at</FONT>
<BR><FONT SIZE=2>160kbs and gyrates around between 30-150kbs..especially</FONT>
<BR><FONT SIZE=2>if they are very large downloads? But a web page taking</FONT>
<BR><FONT SIZE=2>12 seconds to load...not good. </FONT>
</P>
<P><FONT SIZE=2>-----Original Message-----</FONT>
<BR><FONT SIZE=2>From: smd@clock.org [<A HREF="mailto:smd@clock.org">mailto:smd@clock.org</A>]</FONT>
<BR><FONT SIZE=2>Sent: Wednesday, February 20, 2002 1:02 PM</FONT>
<BR><FONT SIZE=2>To: nanog@merit.edu</FONT>
<BR><FONT SIZE=2>Subject: red [was Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?]</FONT>
</P>
<BR>
<BR>
<P><FONT SIZE=2>randy bush writes:</FONT>
</P>
<P><FONT SIZE=2>| you can turn on [x]red. it will make a better choice of which packets</FONT>
<BR><FONT SIZE=2>| to drop. but packets will still be dropped [0].</FONT>
</P>
<P><FONT SIZE=2>fsvo "better". red drops packets randomly, a little earlier than</FONT>
<BR><FONT SIZE=2>packets will be tail-dropped by being unable to find buffer space</FONT>
<BR><FONT SIZE=2>in a queue. in most cases, tail-drop is worst for TCP connections</FONT>
<BR><FONT SIZE=2>in slow-start, and leads to such fun things as web-page-slowdown-syndrome,</FONT>
<BR><FONT SIZE=2>or ssh-stall-syndrome, with the attendant hitting of the "reload" button</FONT>
<BR><FONT SIZE=2>or frustrated banging on the keyboard trying to get one's keys to echo</FONT>
<BR><FONT SIZE=2>back, or trying to unstick the page of text being sent to /dev/tty on</FONT>
<BR><FONT SIZE=2>the remote end.</FONT>
</P>
<P><FONT SIZE=2>"r" for random: this spreads out loss randomly, and in most cases,</FONT>
<BR><FONT SIZE=2>this means that several different TCPs each observe a single lost</FONT>
<BR><FONT SIZE=2>segment, which is quickly detected and causes an attendant slow-down,</FONT>
<BR><FONT SIZE=2>but not a stall. </FONT>
</P>
<P><FONT SIZE=2>"e" for early: we have to keep the queue from filling, because</FONT>
<BR><FONT SIZE=2>that leads to tail-drop, so we have to drop packets when we</FONT>
<BR><FONT SIZE=2>could really send them. there is no way around this -- in virtually any</FONT>
<BR><FONT SIZE=2>implementation, the mechanism to turn on red can be translated</FONT>
<BR><FONT SIZE=2>into more fifo-queue space. the trade-off (early random dropping)</FONT>
<BR><FONT SIZE=2>is usually worth it.</FONT>
</P>
<P><FONT SIZE=2>there are some corner cases where there is insufficient ability</FONT>
<BR><FONT SIZE=2>to spread drops across enough flows, where resulting behaviour can</FONT>
<BR><FONT SIZE=2>be alot like tail-drop (a huge difference between fast input speed</FONT>
<BR><FONT SIZE=2>and slow output speed, with only a few bulk-transfer flows), but</FONT>
<BR><FONT SIZE=2>these cases are rare, particularly where </FONT>
<BR><FONT SIZE=2>average_segment_size / bottleneck_kbps <= 0.4, with the 0.4 being</FONT>
<BR><FONT SIZE=2>a historical minimum target for queueing in the presence of flows</FONT>
<BR><FONT SIZE=2>which experience long RTTs (like ones across oceans). in general,</FONT>
<BR><FONT SIZE=2>you don't have to worry about that, though.</FONT>
</P>
<P><FONT SIZE=2>| you can increase buffers blah blah blah. but twenty tomatoes will still</FONT>
<BR><FONT SIZE=2>| not fit in a fifteen tomato can.</FONT>
</P>
<P><FONT SIZE=2>yup, the trick is to deploy a red control law that drops just before</FONT>
<BR><FONT SIZE=2>tail-dropping would start, so we don't lose packets unnecessarily,</FONT>
<BR><FONT SIZE=2>yet early enough before tail-dropping would start that the senders</FONT>
<BR><FONT SIZE=2>have time to back off in response to the detected lost segment,</FONT>
<BR><FONT SIZE=2>which is driven by the RTT. drop too early, and you get an underused</FONT>
<BR><FONT SIZE=2>line. drop too late and you get tail-drop, which also gives you</FONT>
<BR><FONT SIZE=2>an underused line. drop just right, and you can keep your line</FONT>
<BR><FONT SIZE=2>very very very close to 100% for sustained periods of time (hours!),</FONT>
<BR><FONT SIZE=2>assuming a large enough set of TCP flows that random dropping spreads</FONT>
<BR><FONT SIZE=2>the early dropping across different flows.</FONT>
</P>
<P><FONT SIZE=2>| [0] - corollary: qos mechanisims decide which packets to drop. but isps</FONT>
<BR><FONT SIZE=2>| are paid not to drop any packets.</FONT>
</P>
<P><FONT SIZE=2>it is impossible to get 0 drop and enjoy statistical multiplexing gain.</FONT>
<BR><FONT SIZE=2>statmux gain keeps the network cheap enough to use. red minimizes the</FONT>
<BR><FONT SIZE=2>liklihood that anyone will complain about performance during those</FONT>
<BR><FONT SIZE=2>inevitable periods when one's statistical bet goes wrong.</FONT>
</P>
<P> <FONT SIZE=2>Sean.</FONT>
</P>
</BODY>
</HTML>
------_=_NextPart_001_01C1BB23.97AE5A40--
