[8726] in Commercialization & Privatization of the Internet
Re: NAP philosophy & IP technology
daemon@ATHENA.MIT.EDU (Roger Fajman)
Wed Dec 1 21:48:11 1993
To: fair@apple.com
Cc: com-priv@psi.com
From: "Roger Fajman" <RAF@cu.nih.gov>
Date: Wed, 01 Dec 1993 21:43:10 EST
> Roger, redundant routing with load balancing into the Internet works
> today. I've been doing it for four years at Apple Computer.
I really didn't mean to suggest that it is impossible, just that it is
quite difficult -- so difficult that most people would be discouraged
from doing it. I think the rest of your message documents that rather
well:
> Today, we're still T1 to BARRNET, but we speak T1 SMDS with two
> attachments to the Pac*Bell San Francisco LATA SMDS cloud to CERFNET's
> two POPs on the SMDS. We also have a 56Kb/s connection to NEARNET from
> our Cambridge office, but both that link, and our internal link to them
> from Cupertino are so slow as to make tertiary backup routing for the
> bulk of Apple through that connection silly - we seem to run our link
> into BARRNET (where our default points normally) at around 200Kb/s
> continuously, so we'd swamp the NEARNET link the instant we tried to
> use it for everything.
>
> This actually works too - twice in the last two months, Pac*Bell has
> screwed up our T1 to BARRNET, and I didn't even notice until I checked
> the monitors, because everything just flipped over to CERFNET,
> automatically. When everything is up, we send traffic out both links.
>
> There are some gotchas to doing what I do: the NSFNET backbone is
> configured to accept routing announcements for about 18,000 network
> when last I asked (IETF, last month), and we have about 1/3 of that
> many routes in our key border routers (you can bet I don't flood my
> entire AS with that many routes!). In August, I had to do some quickie
> CPU and RAM upgrades to my key routers, because we ran out of RAM in
> 'em just holding the routing tables.
>
> Secondly, the combinatorics of who we talk to most, and how the rest of
> the Internet does its routing (i.e. what path they take to talk back to
> us) affects which connections we use - if I were trying to get perfectly
> balanced load, I'd have to get a list of who we talk to most, and then
> figure out which NSP's to connect to after that. I usually prefer
> symmetric routing (because TCP performs better) to balanced load.
>
> To top it all off, the interconnections out there keep on getting
> richer (i.e. more complex and harder to track), so no matter what I do
> today, someone will change something tomorrow that will bollix my
> careful balancing act if I try too hard to fine-tune it. Over-engineer
> a little and be content to accept a 80% solution; you'll be happier.
>
> I'm still waiting for the other shoe to drop after the announcement
> that ANS CO+RE is joining the CIX - this will have a direct and
> immediate impact on my routing tables when they actually do it, but I'm
> not quite sure how it will shake out. I may have to filter out a bunch
> of routes to get symmetric routing back again (whee!). Things like this
> make it very important for me to talk periodically to my counterparts
> at the service providers that we do business with (Vince Fuller at
> BARRNET, Pushpendra Mohta at CERFNET, John Curran & Dan Long at
> NEARNET), plus the other big players (AlterNet, EUNET, etc), to figure
> out the "shape" of the Internet, and set my routing policies to match
> it relatively closely.
>
> So, it can be done. Apple's Internet connectivity is proof of it.
> It just takes work, and careful thinking.
>
> Erik E. Fair apple!fair fair@apple.com
>
> P.S. Did I mention that private links to other companies (like IBM)
> figure into this picture too?
I don't think it's necessary to go into that now. :-)
The ordinary network manager (is that an oxymoron? :-) for a large
organization is likely to need a lot more redundancy and backup than
most organizations now have, but will not be able or willing to do what
you have.
Roger Fajman Telephone: +1 301 402 4265
National Institutes of Health BITNET: RAF@NIHCU
Bethesda, Maryland, USA Internet: RAF@CU.NIH.GOV
