[191577] in North American Network Operators' Group
Re: CDN Overload?
daemon@ATHENA.MIT.EDU (Martin Hannigan)
Thu Sep 22 03:55:28 2016
X-Original-To: nanog@nanog.org
From: Martin Hannigan <hannigan@gmail.com>
In-Reply-To: <1842564151.15662.1474504249587.JavaMail.mhammett@ThunderFuck>
Date: Wed, 21 Sep 2016 15:19:35 -1000
To: Mike Hammett <nanog@ics-il.net>
Cc: NANOG <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
Mike,
I will forward to the requisite group for a look. Have you brought this to o=
ur attention previously? I don't see anything. If you did, please forward me=
the ticket numbers or message(s) (peering@ is best) so wee can track down a=
nd see if someone already has it in queue.
Jared alluded to fasttcp a few emails ago. Astute man.
Best,
Martin Hannigan=20
AS 20940 // AS 32787
> On Sep 21, 2016, at 14:30, Mike Hammett <nanog@ics-il.net> wrote:
>=20
> https://docs.google.com/spreadsheets/d/1Jdm0dOBf81kSnXEvVfI6ZJbWFNt5AbYUV8=
CDxGwLSm8/edit?usp=3Dsharing=20
>=20
> I have made the anonymized answers public. This will obviously have some b=
ias to it given that I mostly know fixed wireless operators, but I'm hoping t=
his gets some good distribution to catch more platforms.=20
>=20
>=20
>=20
>=20
> -----=20
> Mike Hammett=20
> Intelligent Computing Solutions=20
>=20
> Midwest Internet Exchange=20
>=20
> The Brothers WISP=20
>=20
> ----- Original Message -----
>=20
> From: "Mike Hammett" <nanog@ics-il.net>=20
> To: "NANOG" <nanog@nanog.org>=20
> Sent: Wednesday, September 21, 2016 9:08:55 AM=20
> Subject: Re: CDN Overload?=20
>=20
> https://goo.gl/forms/LvgFRsMdNdI8E9HF3=20
>=20
> I have made this into a Google Form to make it easier to track compared to=
randomly formatted responses on multiple mailing lists, Facebook Groups, et=
c.=20
>=20
>=20
>=20
>=20
> -----=20
> Mike Hammett=20
> Intelligent Computing Solutions=20
>=20
> Midwest Internet Exchange=20
>=20
> The Brothers WISP=20
>=20
> ----- Original Message -----=20
>=20
> From: "Mike Hammett" <nanog@ics-il.net>=20
> To: "NANOG" <nanog@nanog.org>=20
> Sent: Monday, September 19, 2016 12:34:48 PM=20
> Subject: CDN Overload?=20
>=20
>=20
> I participate on a few other mailing lists focused on eyeball networks. Fo=
r a couple years I've been hearing complaints from this CDN or that CDN was b=
ehaving badly. It's been severely ramping up the past few months. There have=
been some wild allegations, but I would like to develop a bit more standard=
ized evidence collection. Initially LimeLight was the only culprit, but rece=
ntly it has been Microsoft as well. I'm not sure if there have been any othe=
rs.=20
>=20
> The principal complaint is that upstream of whatever is doing the rate lim=
iting for a given customer there is significantly more capacity being utiliz=
ed than the customer has purchased. This could happen briefly as TCP adjusts=
to the capacity limitation, but in some situations this has persisted for d=
ays at a time. I'll list out a few situations as best as I can recall them. S=
ome of these may even be merges of a couple situations. The point is to show=
the general issue and develop a better process for collecting what exactly i=
s happening at the time and how to address it.=20
>=20
> One situation had approximately 45 megabit/s of capacity being used up by a=
customer that had a 1.5 megabit/s plan. All other traffic normally held its=
elf within the 1.5 megabit/s, but this particular CDN sent excessively more f=
or extended periods of time.=20
>=20
> An often occurrence has someone with a single digit megabit/s limitation c=
onsuming 2x - 3x more than their plan on the other side of the rate limiter.=
=20
>=20
> Last month on my own network I saw someone with 2x - 3x being consumed ups=
tream and they had *190* connections downloading said data from Microsoft.=20=
>=20
> The past week or two I've been hearing of people only having a single conn=
ection downloading at more than their plan rate.=20
>=20
>=20
> These situations effectively shut out all other Internet traffic to that c=
ustomer or even portion of the network for low capacity NLOS areas. It's a D=
oS caused by downloads. What happened to the days of MS BITS and you didn't e=
ven notice the download happening? A lot of these guys think that the CDNs a=
re just a pile of dicks looking to ruin everyone's day and I'm certain that t=
here are at least a couple people at each CDN that aren't that way. ;-)=20
>=20
>=20
>=20
>=20
> Lots of rambling, sure. What do I need to have these guys collect as evide=
nce of a problem and who should they send it to?=20
>=20
>=20
>=20
>=20
> -----=20
> Mike Hammett=20
> Intelligent Computing Solutions=20
>=20
> Midwest Internet Exchange=20
>=20
> The Brothers WISP=20
>=20
>=20
>=20
