[8707] in Commercialization & Privatization of the Internet
Re: NAP philosophy & IP technology
daemon@ATHENA.MIT.EDU (Erik E. Fair" (Your Friendly Postm)
Wed Dec 1 05:21:31 1993
From: "Erik E. Fair" (Your Friendly Postmaster) <fair@apple.com>
In-Reply-To: <9312010034.AA17991@psi.com>
To: "Roger Fajman" <RAF@cu.nih.gov>
Cc: com-priv@psi.com
Date: Wed, 01 Dec 93 02:20:40 -0800
Roger, redundant routing with load balancing into the Internet works
today. I've been doing it for four years at Apple Computer.
When I got here in 1988, Apple was on CSNET's X25NET at 9600 baud
(ouch!), and the A/UX group had just bought a T1 into BARRNET. I tied
that together by Feb '89 and rationalized our external network, and
made it work for both the "fail-over" cases, and for load balancing (to
the extent that such load could be balanced). I've been continuously
upgrading the links, and lowering costs as the market has improved,
ever since.
Today, we're still T1 to BARRNET, but we speak T1 SMDS with two
attachments to the Pac*Bell San Francisco LATA SMDS cloud to CERFNET's
two POPs on the SMDS. We also have a 56Kb/s connection to NEARNET from
our Cambridge office, but both that link, and our internal link to them
from Cupertino are so slow as to make tertiary backup routing for the
bulk of Apple through that connection silly - we seem to run our link
into BARRNET (where our default points normally) at around 200Kb/s
continuously, so we'd swamp the NEARNET link the instant we tried to
use it for everything.
This actually works too - twice in the last two months, Pac*Bell has
screwed up our T1 to BARRNET, and I didn't even notice until I checked
the monitors, because everything just flipped over to CERFNET,
automatically. When everything is up, we send traffic out both links.
There are some gotchas to doing what I do: the NSFNET backbone is
configured to accept routing announcements for about 18,000 network
when last I asked (IETF, last month), and we have about 1/3 of that
many routes in our key border routers (you can bet I don't flood my
entire AS with that many routes!). In August, I had to do some quickie
CPU and RAM upgrades to my key routers, because we ran out of RAM in
'em just holding the routing tables.
Secondly, the combinatorics of who we talk to most, and how the rest of
the Internet does its routing (i.e. what path they take to talk back to
us) affects which connections we use - if I were trying to get perfectly
balanced load, I'd have to get a list of who we talk to most, and then
figure out which NSP's to connect to after that. I usually prefer
symmetric routing (because TCP performs better) to balanced load.
To top it all off, the interconnections out there keep on getting
richer (i.e. more complex and harder to track), so no matter what I do
today, someone will change something tomorrow that will bollix my
careful balancing act if I try too hard to fine-tune it. Over-engineer
a little and be content to accept a 80% solution; you'll be happier.
I'm still waiting for the other shoe to drop after the announcement
that ANS CO+RE is joining the CIX - this will have a direct and
immediate impact on my routing tables when they actually do it, but I'm
not quite sure how it will shake out. I may have to filter out a bunch
of routes to get symmetric routing back again (whee!). Things like this
make it very important for me to talk periodically to my counterparts
at the service providers that we do business with (Vince Fuller at
BARRNET, Pushpendra Mohta at CERFNET, John Curran & Dan Long at
NEARNET), plus the other big players (AlterNet, EUNET, etc), to figure
out the "shape" of the Internet, and set my routing policies to match
it relatively closely.
So, it can be done. Apple's Internet connectivity is proof of it.
It just takes work, and careful thinking.
Erik E. Fair apple!fair fair@apple.com
P.S. Did I mention that private links to other companies (like IBM)
figure into this picture too?
