[8938] in athena10
Comcast's DNS servers + Kerberos + dialup.mit.edu = fail
daemon@ATHENA.MIT.EDU (Geoffrey Thomas)
Tue Apr 17 02:37:40 2012
Date: Tue, 17 Apr 2012 02:37:30 -0400 (EDT)
From: Geoffrey Thomas <geofft@MIT.EDU>
To: debathena@MIT.EDU
Message-ID: <alpine.DEB.2.00.1204170214400.8722@dr-wily.mit.edu>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
For posterity (from -c linerva -i barnowl earlier today):
Comcast's residential DNS IPs (75.75.75.75 and 75.75.76.76) are anycast
addresses for muliple machines, which means that when looking up
*.dialup.mit.edu, which is load-balanced at the DNS level, there's no
guarantee that two successive client lookups (from a stub resolver) will
hit the same server, and thus are more likely than not to give you two
different results, because the first DNS server you hit is happily caching
your result while you talk to the second DNS server.
This means that if you open an SSH connection to athena.dialup.mit.edu and
ssh then asks the Kerberos libraries to get tickets for
athena.dialup.mit.edu, the Kerberos libraries are likely to canonicalize
the name into another server than the existing SSH connection and give you
a service ticket for a the wrong dialup, leading to various random but
probable failures setting up a Kerberos connection.
Combined with the underlying issue behind Trac #315 where a failed keyex
aborts the connection, as opposed to falling back to another keyex method
(like checking the server's RSA key), this manifests as athena.dialup
giving "Connection closed" messages more often than not from a Debathena
box on a Comcast residential connection if the user has active tickets.
I recommend we give some thought to one or more of
1) giving up on GSSAPIKeyExchange (#315 is almost excuse enough, but
combining it with #787 and this issue makes it something of a real
problem), at least until we can teach SSH to do keyex fallback
2) changing athena.dialup from DNS-level load balancing to IP-level, by
assigning a distinct IP to athena.dialup and having every athena.dialup
host include the host/athena.dialup.mit.edu key in its keytab in
addition to its own (and turning off the GSSAPIStrictAcceptorCheck);
this is basically the configuration of the scripts.mit.edu pool, except
that ssh connections aren't actually load-balanced.
This would also have some UI benefits for non-GSSAPI users, since they
would only get a host key prompt/warning for the single IP once.
--
Geoffrey Thomas
geofft@mit.edu
