[189733] in North American Network Operators' Group
Re: Traffic engineering and peering for CDNs
daemon@ATHENA.MIT.EDU (Phil Rosenthal)
Mon Jun 6 10:38:26 2016
X-Original-To: nanog@nanog.org
From: Phil Rosenthal <pr@isprime.com>
In-Reply-To: <82C0CE81789FE64D8F4D152631918297B18660@MSG6.westman.int>
Date: Mon, 6 Jun 2016 08:36:19 -0600
To: Graham Johnston <johnstong@westmancom.com>
Cc: "nanog@nanog.org" <nanog@nanog.org>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
Hello,
> On Jun 6, 2016, at 7:36 AM, Graham Johnston <johnstong@westmancom.com> =
wrote:
>=20
> Lately I have been putting in some effort to maximize our IX =
connections by trying to work with the top 5-ish list of ASNs that still =
send us traffic via a paid transit connection despite the fact that we =
are both present on the same IX(s). In one case I missed the fact that =
one ASN wasn't using the IXs route-servers, that's on me for not =
spotting that one.
>=20
> Even with proper IX peering in place though it seems like some CDNs =
are better at using the IX connections than others. ASN 15169 for =
instance does an excellent job sending more than 99.99% of traffic via =
the IX connection; thank you. While others only seem to manage to send =
60 - 80% of traffic via the IX. What I am not understanding about the =
respective CDN's network wherein they don't send traffic to me through a =
consistent path? Is the content coming from widely different places and =
rather than transport it across their own network from a remote site =
they would rather hot-potato it out a local transit connection? Are =
their transit costs so low that they don't care about using an IX =
connection over transit unlike a small operator like me? Is this just a =
non-obvious issue wherein they maybe just can't originate enough of the =
traffic near the IX and therefore don't make use of the IX connection, =
again a hot-potato phenomenon?
Most CDN=E2=80=99s do not have a backbone. Transit costs are not free, =
but as most traffic is served by local nodes from cache, the costs of =
transport between locations in many cases is higher than just sending =
via transit. In some cases, the CDN may not have good mapping and may =
not be certain which node is best to serve your customers. In other =
cases, not all content exists on all nodes, and they may redirect to =
serve from the nodes which have the content. Finally, there may be an =
outage or capacity limits from the closest location, and another =
location may be serving to make up the shortfall.
>=20
> Secondly can someone explain to me why some CDNs want a gigabit or two =
of traffic to be exchanged between our respective networks before they =
would peer with me via a public IX? I totally get those kinds of =
thresholds before engaging in a private interconnect but I don't =
understand the reluctance with regard to a public IX, that they are =
already established at. Is it again just a simple case of bandwidth =
economics that operate at a different scale than I can comprehend?
>=20
This sounds like a surprisingly high threshold, but to some extent it =
boils down like this =E2=80=94 setting up sessions requires some time. =
In the ideal world, the peer is intelligent and has everything set up =
properly, but even in this case, it still requires some time for making =
sure things go up properly. Some (but not all) CDN=E2=80=99s have it =
automated to reduce this time. Some potential peering networks are =
poorly run, and will leak routes, not announce all of their routes, will =
not configure the sessions properly, etc =E2=80=94 this adds up to =
significantly more time. Before the CDN starts setting up peering with =
another network, it is not necessarily obvious if the potential peer is =
run by competent people or not. Many CDN=E2=80=99s are members of the =
route servers. If you are exchanging a small amount of traffic, and =
both you and the CDN are on the Route Server for the IX, there maybe no =
reason to set up direct sessions which will require both more =
coordination time for configuration, and more router cpu time/ram on an =
ongoing basis. =46rom the perspective of the CDN, most likely, 1Gbps or =
less is a perfectly reasonable amount of traffic to exchange to peers =
who are learned only via the route server, and not directly.
=20
> I'm hoping the community can shed some light on this for me as I'm =
trying to avoid grilling the operators that are working with me as I =
don't expect those front line individuals to necessarily have a full =
view of the factors at play.
>=20
> Thanks,
> Graham Johnston
> Network Planner
> Westman Communications Group
> 204.717.2829
> johnstong@westmancom.com<mailto:johnstong@westmancom.com>
> P think green; don't print this email.
>=20
Best Regards,
-Phil Rosenthal
ISPrime=
