[12996] in Athena Bugs
rsaix 7.7F: source routing
daemon@ATHENA.MIT.EDU (John Hawkinson)
Wed Dec 7 01:17:48 1994
To: bugs@MIT.EDU
Date: Wed, 07 Dec 1994 01:17:36 -0500
From: John Hawkinson <jhawk@MIT.EDU>
System name: oliver
Type and version: POWER 7.7F
Display type: colorgda
Source routing seems to fail under AIX.
CASE 1 -- How things should work
When connecting to yaz-pistachio (an Ultrix box) from lola-granola
(a NetBSD 1.0/current box) source routed through www.mit.edu:
[lola-granola!jhawk] /tmp> telnet @www:yaz 13
Trying 18.70.0.225...
Connected to yaz.
Escape character is '^]'.
Tue Dec 6 21:44:53 1994
Connection closed by foreign host.
All works fine. While doing this, we run a tcpdump of all packets
with options (thus, the IHL, or Internet Header Length, is
greater than 5 (the minimum value)) between the two hosts:
---cut
lola-granola# tcpdump 'ip[0] & 0xf != 5' and host yaz-pistachio and host lola-granola
tcpdump: listening on ep0
21:44:48.287323 LOLA-GRANOLA.MIT.EDU.1161 > YAZ-PISTACHIO.MIT.EDU.daytime: S 2045696001:2045696001(0) win 16384 <mss 1460,nop,wscale 84[len 73],opt-160:e52e0d6c0400420000004200000012000020af30559908002b2bedb6080047> [tos 0x10]
21:44:48.289805 YAZ-PISTACHIO.MIT.EDU.daytime > LOLA-GRANOLA.MIT.EDU.1161: S 927360000:927360000(0) ack 2045696002 win 16384 <mss 1460>
21:44:48.291691 LOLA-GRANOLA.MIT.EDU.1161 > YAZ-PISTACHIO.MIT.EDU.daytime: . ack 1 win 17520 [tos 0x10]
21:44:48.295022 YAZ-PISTACHIO.MIT.EDU.daytime > LOLA-GRANOLA.MIT.EDU.1161: P 1:27(26) ack 1 win 16384
21:44:48.296091 YAZ-PISTACHIO.MIT.EDU.daytime > LOLA-GRANOLA.MIT.EDU.1161: F 27:27(0) ack 1 win 16384
21:44:48.297935 LOLA-GRANOLA.MIT.EDU.1161 > YAZ-PISTACHIO.MIT.EDU.daytime: . ack 28 win 17494 [tos 0x10]
21:44:48.317396 LOLA-GRANOLA.MIT.EDU.1161 > YAZ-PISTACHIO.MIT.EDU.daytime: F 1:1(0) ack 28 win 17520 [tos 0x10]
21:44:48.318521 YAZ-PISTACHIO.MIT.EDU.daytime > LOLA-GRANOLA.MIT.EDU.1161: . ack 2 win 16384
^C
7749 packets received by filter
0 packets dropped by kernel
lola-granola#
---cut
We observe packets in both directions, indicating that granola sends
source routed packets to yaz and yaz sends them back source routed.
For the purposes of this test, we assume that the IP option selected
is indeed source routing, and that it is through www (otisberg).
CASE 2 -- AIX fails to return SR packets SR
This is the same as CASE 1, except that we replace yaz-pistachio (Ultrix)
with oliver (AIX).
The telnet:
[lola-granola!jhawk] /tmp> telnet @www:oliver 13
Trying 18.70.0.212...
Connected to oliver.
Escape character is '^]'.
Tue Dec 6 22:17:41 1994
Connection closed by foreign host.
The tcpdump:
---cut
lola-granola# tcpdump 'ip[0] & 0xf != 5 and host oliver and host granola'
tcpdump: listening on ep0
22:17:35.837493 LOLA-GRANOLA.MIT.EDU.1167 > OLIVER.MIT.EDU.daytime: S 150464000:150464000(0) win 16384 <mss 1460,nop,wscale 48[len 48],opt-79:e52e16d50c003e0000003e000000120002608c2f79b800000c05a233080047100030be0200003d> [tos 0x10]
22:17:35.840982 LOLA-GRANOLA.MIT.EDU.1167 > OLIVER.MIT.EDU.daytime: . ack 335673858 win 17520 [tos 0x10]
22:17:36.411225 LOLA-GRANOLA.MIT.EDU.1167 > OLIVER.MIT.EDU.daytime: . ack 27 win 17520 [tos 0x10]
22:17:36.412200 LOLA-GRANOLA.MIT.EDU.1167 > OLIVER.MIT.EDU.daytime: . ack 28 win 17520 [tos 0x10]
22:17:36.417236 LOLA-GRANOLA.MIT.EDU.1167 > OLIVER.MIT.EDU.daytime: F 0:0(0) ack 28 win 17520 [tos 0x10]
^C
47267 packets received by filter
0 packets dropped by kernel
lola-granola#
---cut
This is a bug, as a oliver fails to return packets with source routing
(that's why we do not see any of oliver's returning packets). In general,
when a host receives an IP packet with source routing enabled, it ought to
reverse that source route when sending back packets; see the documentation
section.
CASE 3: Some AIX programs fail to send packets source routed.
Again, using telnet as our example; from oliver.mit.edu (AIX), I attempt
to telnet to yaz-pistachio via way of www.mit.edu. It clearly fails
% /usr/athena/bin/telnet @www:yaz 13
Trying 18.70.0.225...
---> setsockopt (IP_OPTIONS): Not owner
telnet: setsockopt (IP_TOS) (ignored): Not owner
Connected to yaz.
Escape character is '^]'.
Wed Dec 7 00:07:33 1994
Connection closed by foreign host.
presumably as AIX setockopt() requires one to be root(?).
As root:
# telnet @www:yaz 13
Trying 18.70.0.225...
Connected to yaz.
Escape character is '^]'.
Wed Dec 7 00:17:22 1994
Connection closed by foreign host.
Unfortunately, the tcpdump yields nothing:
---cut
lola-granola# tcpdump 'ip[0] & 0xf != 5 and host oliver and host yaz'
tcpdump: listening on ep0
^C
35456 packets received by filter
0 packets dropped by kernel
---cut
This implies that AIX is utterly failing to send packets
source-routed. Having tcpdump print out the the entire IP header when
repeating the above telnet, we see:
---cut
lola-granola# tcpdump -s 68 -x 'host oliver and host yaz'
tcpdump: listening on ep0
00:23:15.032416 OLIVER.MIT.EDU.1482 > YAZ-PISTACHIO.MIT.EDU.daytime: S 1306745857:1306745857(0) win 16384 <mss 1460> [tos 0x10]
4510 2c00 8e0b 0000 3c06 4cee 1246 00d4
1246 00e1 05ca 000d 4de3 5c01 0000 0000
6002 4000 822a 0000 0204 05b4 c924
00:23:15.034544 YAZ-PISTACHIO.MIT.EDU.daytime > OLIVER.MIT.EDU.1482: S 17728000:17728000(0) ack 1306745858 win 16384 <mss 1460>
4500 2c00 6766 0000 3c06 f224 1246 00e1
1246 00d4 000d 05ca 010e 8200 4de3 5c02
6012 4000 ff0a 0000 0204 05b4 0000
00:23:15.035810 OLIVER.MIT.EDU.1482 > YAZ-PISTACHIO.MIT.EDU.daytime: . ack 1 win 16060 [tos 0x10]
4510 2800 8f0b 0000 3c06 4cf1 1246 00d4
1246 00e1 05ca 000d 4de3 5c02 010e 8201
5010 3ebc 180c 0000 0204 05b4 c924
00:23:15.038893 YAZ-PISTACHIO.MIT.EDU.daytime > OLIVER.MIT.EDU.1482: P 1:27(26) ack 1 win 16384
4500 4200 6866 0000 3c06 f20d 1246 00e1
1246 00d4 000d 05ca 010e 8201 4de3 5c02
5018 4000 31f9 0000 5765 6420 4465 6320
2037 2030 303a
00:23:15.039786 YAZ-PISTACHIO.MIT.EDU.daytime > OLIVER.MIT.EDU.1482: F 27:27(0) ack 1 win 16384
4500 2800 6966 0000 3c06 f226 1246 00e1
1246 00d4 000d 05ca 010e 821b 4de3 5c02
5011 4000 16ad 0000 0000 0000 0000
00:23:15.041776 OLIVER.MIT.EDU.1482 > YAZ-PISTACHIO.MIT.EDU.daytime: . ack 28 win 16034 [tos 0x10]
4510 2800 900b 0000 3c06 4cf0 1246 00d4
1246 00e1 05ca 000d 4de3 5c02 010e 821c
5010 3ea2 180b 0000 0000 0000 0000
00:23:15.066585 OLIVER.MIT.EDU.1482 > YAZ-PISTACHIO.MIT.EDU.daytime: F 1:1(0) ack 28 win 16060 [tos 0x10]
4510 2800 970b 0000 3c06 4ce9 1246 00d4
1246 00e1 05ca 000d 4de3 5c02 010e 821c
5011 3ebc 17f0 0000 be80 3be5 4ace
00:23:15.067711 YAZ-PISTACHIO.MIT.EDU.daytime > OLIVER.MIT.EDU.1482: . ack 2 win 16384
4500 2800 6a66 0000 3c06 f225 1246 00e1
1246 00d4 000d 05ca 010e 821c 4de3 5c03
5010 4000 16ac 0000 0000 0000 0000
^C
6854 packets received by filter
0 packets dropped by kernel
---cut
(tcpdump will fail to decode the IP header telling us which host sends
what unless we have it look at the first 68 bytes, even though the IP header
is only 48). For those of you who don't recall the IP header, page 10 of [IP]
gives us:
3.1. Internet Header Format
A summary of the contents of the internet header follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Options | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Looking at the first packet:
00:23:15.032416 OLIVER.MIT.EDU.1482 > YAZ-PISTACHIO.MIT.EDU.daytime: S 1306745857:1306745857(0) win 16384 <mss 1460> [tos 0x10]
4510 2c00 8e0b 0000 3c06 4cee 1246 00d4
1246 00e1 05ca 000d 4de3 5c01 0000 0000
6002
It looks like a perfectly normal (0x45 & 0xF _does_ equal 5), so
our tcpdump syntax wasn't off...
The other standard app which can use source routing, also fails
(traceroute -g):
# /usr/athena/etc/traceroute -g www yaz
traceroute to YAZ-PISTACHIO.MIT.EDU (18.70.0.225), 30 hops max, 40 byte packets
1 YAZ-PISTACHIO.MIT.EDU (18.70.0.225) 8 ms 4 ms 4 ms
We can see the normal behavior of this without resorting to examining
IP packets. Under Ultrix, f'rinstance:
[hodge-podge!jhawk] ~> /usr/athena/etc/traceroute -g www yaz
traceroute to YAZ-PISTACHIO.MIT.EDU (18.70.0.225), 30 hops max, 40 byte packets
1 NW12A-RTR-W20-ETHER.MIT.EDU (18.70.0.1) 4 ms 12 ms 4 ms
2 ANXIETY-CLOSET.MIT.EDU (18.181.0.21) 4 ms 4 ms 4 ms
3 W20-RTR-SIPB-ETHER.MIT.EDU (18.181.0.1) 4 ms 4 ms 4 ms
4 YAZ-PISTACHIO.MIT.EDU (18.70.0.225) 8 ms 16 ms 4 ms
whereas a pure traceroute yields:
[hodge-podge!jhawk] ~> /usr/athena/etc/traceroute yaz
traceroute to YAZ-PISTACHIO.MIT.EDU (18.70.0.225), 30 hops max, 40 byte packets
1 YAZ-PISTACHIO.MIT.EDU (18.70.0.225) 0 ms 0 ms 4 ms
[It should be noted that traceroute -g fails under Solaris; since telnet @:
works, this is probably a traceroute bug and thus the subject of another
bug report, but nevertheless this deserves mention...]
Documentation:
All of the referenced machines, except for www, lie on the same
subnet (viz. 18.70 or W20-ETHER).
Page 13 of [IP] states:
The options may appear or not in datagrams. They must be
implemented by all IP modules (host and gateways). What is optional
is their transmission in any particular datagram, not their
implementation.
Page 50 of [TCP] states:
If the lower level is IP (or other protocol that provides this
feature) and source routing is used, the interface must allow the
route information to be communicated. This is especially important
so that the source and destination addresses used in the TCP
checksum be the originating source and ultimate destination. It is
also important to preserve the return route to answer connection
requests.
[CLARK] Page 8 (not a standards track RFC, merely informational):
Source routes are already in use in the internet, and many
implementations will wish to be able to take advantage of them. The
following sorts of usages should be permitted. First, a user, when
initiating a TCP connection, should be able to hand a source route into
TCP, which in turn must hand the source route to IP with every outgoing
datagram. The user might initially obtain the source route by querying
a different sort of name server, which would return a source route
instead of an address, or the user may have fabricated the source route
manually. A TCP which is listening for a connection, rather than
attempting to open one, must be prepared to receive a datagram which
contains a IP return route, in which case it must remember this return
route, and use it as a source route on all returning datagrams.
[STD3], the definitive reference, page 35:
(c) Source Route Options
A host MUST support originating a source route and MUST
be able to act as the final destination of a source
route.
If host receives a datagram containing a completed
source route (i.e., the pointer points beyond the last
field), the datagram has reached its final destination;
the option as received (the recorded route) MUST be
passed up to the transport layer (or to ICMP message
processing). This recorded route will be reversed and
used to form a return source route for reply datagrams
(see discussion of IP Options in Section 4). When a
return source route is built, it MUST be correctly
formed even if the recorded route included the source
host (see case (B) in the discussion below).
For those of you wondering where the @: syntax of telnet comes from,
/mit/telnet/src/telnet/telnet/commands.c, line 2771:
---cut
* Source route is handed in as
* [!]@hop1@hop2...[@|:]dst
* If the leading ! is present, it is a
* strict source route, otherwise it is
* assmed to be a loose source route.
*
* We fill in the source route option as
* hop1,hop2,hop3...dest
* and return a pointer to hop1, which will
* be the address to connect() to.
---cut
References:
[IP] 0791 Internet Protocol. J. Postel. Sep-01-1981. (Format: TXT=97779
bytes) (Obsoletes RFC0760)
[TCP] 0793 Transmission Control Protocol. J. Postel. Sep-01-1981. (Format:
TXT=177957 bytes)
[CLARK] 0814 Name, addresses, ports, and routes. D.D. Clark. Jul-01-1982.
(Format: TXT=24663 bytes)
[BRADEN] 0003 Host Requirements. R. Braden. October 1989. (Format: TXT=528939
bytes) (Also RFC1122, RFC1123)
--jhawk
