[9319] in Commercialization & Privatization of the Internet
Re: The annointed
daemon@ATHENA.MIT.EDU (John Curran)
Wed Dec 29 01:33:13 1993
To: Marc Horowitz <marc@MIT.EDU>
Cc: Stephen Williams <sdw@meaddata.com>, steve@mon.cise.nsf.gov,
In-Reply-To: Your message of Tue, 28 Dec 1993 18:58:47 -0500.
Date: Wed, 29 Dec 1993 01:30:38 -0500
From: John Curran <jcurran@nic.near.net>
--------
] From: Marc Horowitz <marc@mit.edu>
] Subject: Re: The annointed
] Date: Tue, 28 Dec 93 18:58:47 EST
] ...
] Really? Explain this traceroute:
]
] > traceroute uu6.psi.com
] traceroute to uu6.psi.com (38.145.155.3), 30 hops max, 40 byte packets
] 1 nearnet-gw (192.231.148.253) 2 ms 2 ms 2 ms
] 2 bbn2-gw.near.net (131.192.170.1) 6 ms 6 ms 6 ms
] 3 harvard-gw.near.net (131.192.5.1) 7 ms 8 ms 7 ms
] 4 prospect-gw.near.net (131.192.32.3) 7 ms 6 ms 7 ms
] 5 mit2-gw.near.net (131.192.7.1) 7 ms 7 ms 7 ms
] 6 enss.near.net (192.233.33.6) 7 ms 6 ms 6 ms
] 7 t3-1.Hartford-cnss49.t3.ans.net (140.222.49.2) 12 ms 8 ms 10 ms
] 8 mf-0.Hartford-cnss48.t3.ans.net (140.222.48.222) 9 ms 9 ms 9 ms
] 9 t3-2.New-York-cnss32.t3.ans.net (140.222.32.3) 11 ms 11 ms 11 ms
] 10 t3-1.Washington-DC-cnss56.t3.ans.net (140.222.56.2) 17 ms 17 ms
] 11 mf-0.Washington-DC-cnss58.t3.ans.net (140.222.56.194) 17 ms 17 ms
] 12 t3-0.enss136.t3.ans.net (140.222.136.1) 18 ms 17 ms 18 ms
] 13 psi-nsf.psi.net (192.41.177.246) 23 ms 22 ms 23 ms
] 14 core.net155.psi.net (38.1.2.3) 178 ms 180 ms 184 ms
] 15 uu6.psi.com (38.145.155.3) 192 ms 170 ms 163 ms
]
] I just sent mail to my service provider (NEARnet. John are you there?
] :-) to explain this as well, because it does seem wrong. But it is
] most certainly happening.
I'd rather not be "summoned" to a public mailing list discussion, but
the customers are always right... (It would have been easier if you
waited for a response and then posted to com-priv, but no matter :-).
I'll try to add some clarity to both the particular traceroute and
the discussion in general, but like all Internet discussions it has to
prefaced with a warning that this is only the way things are _today_,
and things change very quickly in the Internet. Folks who would rather
not have partial tour of US Internet routing might want to skip ahead
several paragraphs.
The current routing for CIX members receiving transit via ANS CO+RE
is rather complicated, and it is the result of technology limitations
and operational limitations that were in place 2 years ago. The technology
limitation is that routing in the Internet is done based on the destination
_but not the source_ of the packet. This means that routing to a given
destination is the same regardless of the sender. The operational
limitation in affect at the time of the CIX/ANS CO+RE interconnect
was that the CIX interchange locations were limited to T1 capacity.
As a result of the above limitations, NEARNET has had to be very careful
about announcing networks to the CIX. For example, if MIT were announced
to the CIX, any and all traffic TO MIT from the clients of CIX networks
would take the T1 path via the CIX, rather than the higher speed T3 NSFNET
path. This would have a noticable impact on many of our 10Mb/s clients.
Likewise, we have to be careful about what routes from the CIX are used
to reach destinations, so that research traffic does not pass through the
CIX to reach it's destination.
The routing plan for the CIX/ANS CO+RE interconnection provides for
symmetric routing between research traffic, and symmetric routing for
what is clearly commercial traffic, and produces less than ideal routing
for intermediate cases. (The reason for "less than ideal" routing is due
to the first limitation above: routing is based only upon the destination,
and hence the "type" of the source of the traffic does not enter into the
routing decision.)
Your particular example above is one of the intermediate cases... Here's
why:
Your particular network (192.231.148) is "announced" as a _commercial_
network (i.e. non-NSFNET AUP compliant network). This is the result
of you purchasing NEARNET commercial routing, and NEARNET in turn paying
ANS CO+RE to carry this traffic. We send this traffic to ANS CO+RE via
the ENSS134 in Cambridge (a location where MIT, NEARNET, and Alternet
all peer with the NSFNET).
The other network (PSI's network 38) is "announced" as a NSFNET AUP
compliant network. This is perfectly understandable, as it allows
PSI to communicate with NSFNET sites presuming that appropriate care
is taken by the sender to conform to the NSFNET AUP. PSI has little
choice in this matter, if they would like network 38 to be reachable
from the significant number of NSFNET sites, the act of "announcing"
network 38 to NSFNET requires recognition of the NSFNET AUP. This
announcement to the NSFNET results in traffic from NSFNET (or CIX
members using ANS CO+RE for transit) going via PSI's connection to
the NSFNET.
Note that the path TO your site is as expected:
traceroute -g info.psi.com 192.231.148.253
traceroute to 192.231.148.253 (192.231.148.253), 30 hops max, 40 byte packets
1 bbn1-gw (192.52.71.1) 11 ms 3 ms 9 ms
2 mit2-gw (131.192.2.2) 3 ms 4 ms 4 ms
3 enss (192.233.33.6) 4 ms 7 ms 6 ms
4 t3-1.Hartford-cnss49.t3.ans.net (140.222.49.2) 6 ms 14 ms 6 ms
5 mf-0.Hartford-cnss48.t3.ans.net (140.222.48.222) 8 ms 20 ms 7 ms
6 t3-2.New-York-cnss32.t3.ans.net (140.222.32.3) 10 ms 13 ms 10 ms
7 t3-1.Washington-DC-cnss56.t3.ans.net (140.222.56.2) 14 ms 29 ms 15 ms
8 mf-0.Washington-DC-cnss58.t3.ans.net (140.222.56.194) 15 ms 27 ms 16 ms
9 t3-0.enss136.t3.ans.net (140.222.136.1) 16 ms 28 ms 17 ms
10 192.41.177.246 (192.41.177.246) 23 ms 23 ms 23 ms
11 core.net211.psi.net (38.1.2.1) 137 ms 120 ms 114 ms
12 info.psi.com (38.145.211.6) 137 ms 136 ms 149 ms
13 lan.syracuse.ny.psi.net (38.145.211.1) 110 ms 273 ms 425 ms
14 core.net222.psi.net (38.1.2.4) 313 ms 141 ms 152 ms
15 cix-west2.cix.net (149.20.3.3) 134 ms 117 ms 131 ms
16 149.20.5.2 (149.20.5.2) 148 ms 128 ms 135 ms
17 en-0.San-Francisco-cnss11.t3.ans.net (192.103.60.5) 135 ms 128 ms 120 ms
18 mf-0.San-Francisco-cnss8.t3.ans.net (140.222.8.222) 111 ms 636 ms 130 ms
19 t3-0.Chicago-cnss24.t3.ans.net (140.222.24.1) 116 ms 134 ms 126 ms
20 t3-0.Cleveland-cnss40.t3.ans.net (140.222.40.1) 129 ms 103 ms 123 ms
21 t3-1.Hartford-cnss48.t3.ans.net (140.222.48.2) 132 ms 127 ms 132 ms
22 mf-0.Hartford-cnss49.t3.ans.net (140.222.48.193) 134 ms 146 ms 135 ms
23 t3-0.enss134.t3.ans.net (140.222.134.1) 126 ms 125 ms 129 ms
24 mit2-gw (192.233.33.5) 125 ms 108 ms 121 ms
25 prospect-gw (131.192.7.3) 113 ms 135 ms 128 ms
26 bbn2-gw (131.192.137.2) 167 ms 116 ms 186 ms
27 openvision-gw (131.192.170.2) 131 ms 124 ms 117 ms
[This is a source-routed trace of the path that packets will take from a PSI
host to your network. Due to the asymmetric path, you need to explicitly
trace in both directions.] Hop #15 is the CIX-west router.
No one exactly "likes" this current ANS CO+RE/CIX routing arrangement and
recent developments (such as ANS CO+RE _joining_ the CIX, and higher speed
locations for interconnection) may now make it possible to eliminate this
policy-laden routing. Both the CIX and ANS should be commended for their
progress in this area. Long-term, there may be a sufficiently large
AUP-free Internet so that the default routing can be "commercial" (i.e.
no R&E AUP restriction applies) and special routing will be done to handle
the (non-commerial) exceptions.
] >> Everyone with a .com domain SHOULD be a commercial customer.
[Side note: Domain name does not _in any way_ reflect routing. If you have
a question about routing to a destination, ask your service provider... ]
] I pay for NEARnet's commercial routing service, which means ANS should
] be carrying my packets from NEARnet to the CIX. But the CIX isn't in
] the above traceroute. I would be interested to know if I did not
] subscribe to this service, if packets to nbc would be dropped on the
] floor, get ICMP Host or Net Unreachable, or go through.
If you did not have commercial routing, then the host accepting mail for
nbc.com would still be reachable on network 38. You would be accepting
responsibility for NSFNET AUP-compliant usage to reach it, however.
] One more question, if someone out there can answer it: Can anyone
] estimate the number of Internet sites/hosts/users, globally, who could
] get packets to, say, MIT, but not to nbc.com, due to AUP restrictions.
The only folks this should happen to (inability to reach NSFNET sites) are
those who are CIX routed, but explicitely ask not to be NSFNET routed because
they do not want to worry about AUP compliance. Last time I checked, this was
under 100 networks (compared with ~15000 networks total). Also note that the
connectivity that you would have (without NSFNET routing) varies greatly from
provider to provider; some well-known transit networks (such as Alternet) have
sufficient peers that the connectivity is rich even without NSFNET routing.
"The World" ran without NSFNET routing for some time... you could ask Barry
(privately ;-) about the connectivity.
/John
p.s. Also, Mark, no trouble ticket has been opened regarding your routing;
if you'd like it otherwise then send a note to "nearnet-ops". Thanks!
