[184687] in North American Network Operators' Group
Re: Google's peering, GGC, and congestion management
daemon@ATHENA.MIT.EDU (Patrick W. Gilmore)
Thu Oct 15 10:35:45 2015
X-Original-To: nanog@nanog.org
From: "Patrick W. Gilmore" <patrick@ianai.net>
In-Reply-To: <20151014170714.GA16087@lud.polynome.dn42>
Date: Thu, 15 Oct 2015 10:35:41 -0400
To: NANOG list <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
--Apple-Mail=_9BB7E213-2282-4252-95D8-6BA91A02049B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
On Oct 14, 2015, at 1:07 PM, Baptiste Jonglez =
<baptiste@bitsofnetworks.org> wrote:
> In its peering documentation =
[https://peering.google.com/about/traffic_management.html],
> Google claims that it can drive peering links at 100% utilisation:
>=20
>> Congestion management
>>=20
>> Peering ports with Google can be run at 100% capacity in the short =
term,
>> with low (<1-2%) packet loss. Please note that an ICMP ping may =
display
>> packet loss due to ICMP rate limiting on our platforms. Please =
contact
>> us to arrange a peering upgrade.
>=20
> How do they achieve this?
The 100% number is silly. My guess? They=E2=80=99re at 98%.
That is easily do-able because all the traffic is coming from them. =
Coordinate the HTTPd on each of the servers to serve traffic at X bytes =
per second, ensure you have enough buffer in the switches for =
micro-bursts, check the NICs for silliness such as jitter, and so on. It =
is non-trivial, but definitely solvable.
Google is not the only company who can do this. Akamai has done it far =
longer. And Akamai has a much more difficult traffic mix, with -paying =
customers- to deal with.
> More generally, is there any published work on how Google serves =
content
> from its CDN, the Google Global Cache? I'm especially interested in =
two
> aspects:
>=20
> - for a given eyeball network, on which basis are the CDN nodes =
selected?
As for picking which GGC for each eyeball, that is called =E2=80=9Cmapping=
=E2=80=9D. It varies among the different CDNs. Netflix drives it mostly =
from the client. That has some -major- advantages over other CDNs. =
Google has in the past (haven=E2=80=99t checked in over a year) done it =
by giving each user a different URL, although I think they use DNS now. =
Akamai uses mostly DNS, although they have at least experimented with =
other ways. Etc., etc.
> - is Google able to spread traffic over distinct peering links for the
> same eyeball network, in case some of the peering links become
> congested? If so, how do they measure congestion?
Yes. Easily.
User 1 asks for Stream 1, Google sends them them to Node 1. Google =
notices Link 1 is near full. User 2 asks for Stream 2, Google sends them =
to Node 2, which uses Link 2.
This is possible for any set of Users, Streams, Nodes, and Links.
It is even possible to send User 2 to Node 2 when User 2 wants Stream 1. =
Or sending User 1 to Node 2 for their second request despite the fact =
they just got a stream from Node 1. There are few, if any, restrictions =
on the combinations.
Remember, they control the servers. All CDNs (that matter) can do this. =
They can re-direct users with different URLs, different DNS responses, =
302s, etc., etc. It is not BGP.
Everything is much easier when you are one of the end points. (Or both, =
like with Netflix.) When you are just an ISP shuffling packets you =
neither send nor receive, things are both simpler and harder.
--
TTFN,
patrick
--Apple-Mail=_9BB7E213-2282-4252-95D8-6BA91A02049B
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.28
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJWH7m9AAoJEG0dU8fZKHCvMMoP/1LLaez6jqga7rZQjqSn5Hav
ncqMVbyYE+cCXMkLgZtMd3QYiFZUuL32rJS9ocXNenuY+Yd71/dNsEKEr1vxFFGz
ByXuA4zGmYLNlK/TTHENm5wJLc9o3G0nki0VT5IpPTysqcCZ8QOXaPeG/rdoNUjo
HVby5XE+RzXAJnZrr9zBpId7mPHQvUJNUShUw20I+p1sxf4fDhDfRnTooU5kg+Q9
d0iPOtcT/aA4xvQ+YLwJ+Wkn+z9hvp1oGpy0vi3gZ6qMK5OYA3YfPJ2pxlk32KeI
r1AjUT/MbhKu36kTyrrxkP7FcXyyo8uswffRQt9qg5XLkDsuB0oWRnGsnqC2PoBH
r116oyM4oNOGUjzLg/kRe6yxi0WWr1uPaesmp69HhkSQMJGsA4BW8md+S7p6/bfj
kYR9FomwhMdtV3XkBDIJ8Lv7N8fyNIqMG+b6clQapnxl+X11qLYVqUZr2D2lgIO9
Q+slwjZVb1ze9CRZ+2jh8KF8FKrbp9c4bxwWKD2a4puOXguva6Ld82M8mE1zaws0
poeaW/FrzRh7AgzNPBL0nB9/kKTLUxd4HDDR0bD27sWj69nzE5cilhKQs0sQQG16
0/XcZDgJ1TucDbWdb1gpZlIB4XsCx1T/8ROpGj+CznxFEluK7n/X3mL3TOUTPSG5
T/m0XQOEeDIMfrnuA2N6
=KYTu
-----END PGP SIGNATURE-----
--Apple-Mail=_9BB7E213-2282-4252-95D8-6BA91A02049B--
