[88684] in North American Network Operators' Group
NANOG36-NOTES 2006.02.15 talk 1 ipv6fix (and boy, does it need it)
daemon@ATHENA.MIT.EDU (Matthew Petach)
Wed Feb 15 11:59:21 2006
Date: Wed, 15 Feb 2006 08:58:00 -0800
From: Matthew Petach <mpetach@netflight.com>
To: "nanog@nanog.org" <nanog@nanog.org>
Errors-To: owner-nanog@merit.edu
------=_Part_3833_16057896.1140022680932
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Morning intro notes--don't forget to fill out
your SURVEYS!!!!
six lightening talks signed up, should be very
cool. If you have slides, get them to Steve
Feldman start with!
Wireless movie after break should be cool to watch.
Ren? Steve mistakenly introduces her, she corrects
them. Don't forget to give feedback via the Survey
forms!!
2006.02.15 v6fix: Wiping the Slate Clean for IPv6
Kenjiro Cho, WIDE/IIJ, Ruri Hiromi, WIDE/Intec NetCore
Will be talking about their efforts to deploy
IPv6, called v6fix.
v6fix is an effort to solve problems in the current
v6 deployment.
focuses on v4/v6 dual stack environments.
it's a technical analysis of real world problem
Kenjiro will talk about tools and measurements.
deployment status
majority of equipment out there is v6 available
from major vendors
still many applications and appliances just work
with v4
v6 is starting to get into various business fields
Many people lack knowledge/experience with v6.
when non-experts hit problems, they're clueless.
Example: illiteracy.
Hotel internet systems have instructions for guest.
troubleshooting: if you have IPv6 enabled, please
disable IPv6--brochure in guest room.
Cause of problem: combination.
DNS redirection returns specific A record for AAAA
clients stub-resolver accepts the A for AAAA, can't
get out.
Wiping the slate clean for the v6
faulty behaviours only 1% and combinatorial often, but
could be fatal to deployment.
slow fallback to v4 after v6 errors
misbehaving DNS resolvers
filtering of ICMPv6
DNS misconfigurations
poorly configured tunnels
lack of peering or v6 paths
v6fix activities (research group)
identify/analysze/solve real-world tech problems
in v6 deployment.
Enemy: "after disabling v6, my problems went away"
Cooperation needed between researches, implementers, ops.
v6fix topics
harmful effects of the on-link assumption.
misbehaving DNS servers and resolvers
slow fallback to v4 after v6 failures
Examples:
case 1: DNS loop at hotel
real story of hotel internet system--went to same room,
investigated.
DNS is intercepted, redirected to signup page
ipv6 users can't get beyond first page
hotel instructions say to disable v6
erroneus DNS redirection system and stub-resolver
redirection system always returns specific A record
when getting non-A queries
client's stub resolver queries AAAA for any address,
blindly accepts A return response.
case 2: DNS server slowdown
Japanse ISP
ISP upgraded a DNS cache to BIND9, recieved complaints
about slowdown.
recompiling BIND9 with --disable-ipv6, fixed problem,
reported to JANOG
Caused by older BIND9 w/o IPv6 connectivity
server w/o v6 connectivity always tries to talk over v6,
ends up failing back to v4 after timeouts
fixed in BIND9.2.5 and 9.3.1
Common factors
1 problems appear only with specific combinatorial conditions
2 implementors and operators didn't notice until reported
3 even for professionals, not easy to track down problems.
Kenjiro Cho, Tools:
v6 tools and measurement results
Goal: to understand the macro-level v6 healthiness
current methodologies
wide area meaasuremetn of behaviours of 2nd/3rd level
DNS servers
dual stack issues
DNS server measurements of .jp domain
AAAA responses: 0.13% DNS servers can't deal with
AAAA requests
Most are lame delegation type errors.
ignore AAAA queries
respond with RCODE 3 ("name error") NXDOMAIN
dual-stack path analysis
measurement techniques specifically designed for
dual-stack
take measurements for v4 and v6 at same time
compare v6 results with v4 results
extract problems that exist in v6 only
methodology
dual-stack node discovery
create dual-stack node list by monitoring DNS AAAA replies.
dual stack ping
run ping/ping6 to target dual-stack nodes
select a few representative nodes per site (/48) by RTT
dual-stack traceroute
trace/trac6 to selected nodes
visula v6 MTU to look at issues
visualize path issues
distribution of v6/v4 RTTs
4000 ping targets v4 on x-axis, v6 on y axis
individual nodes far above unity line--leaf issues
paths and PMTU visualization
from NYSERNET to ARIN sites
Many of ARIN paths via jp! (lack of peering)
From ISC to ARIN sites--paths look much better, but
lots of blue =3D=3D lots of tunnels
Abilene case: well known problem.
Abilene trying to encourage v6 adoption
no AUP, tunnel services for v6
but ended up with horrible v6 paths, mostly with tunnels
ISPs are reluctant to move to paid v6 connectivity
Abilene thinking about suspending its relaxed AUP for v6
tool tries to illustrate such issues, convince users to
move to native v6
dual stack traceroute to ABILENE from WIDE (v4 upper,
v6 lower)
similar RTTs/hops for v4/v6; native dual-stack paths
dual-stack trace to ABILENE from IIJ
similar RTTs, but different paths: currently more common
dualstack traceroue to ABILINE fro ES
v6 RTTs much larger than v4: roundabout tunnels
Conclusion: faulty behaviours are only 1% and often
combinatorial, but can be fatal to acceptance of v6
slow fallback to v4 on v6 errors
knowledge sharing
need to realize the dangers of harmful of adoption of v6
cooperation among researchers, implementers, and ops
need to act now, or will bring negative impact to v6
deployment
Acknowlegements
v6fix members, etc.
http://v6fix.net/
overview, documents, and fact database.
contact at v6fix.net
reports of issues are welcome
------=_Part_3833_16057896.1140022680932
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
<br>
Morning intro notes--don't forget to fill out<br>
your SURVEYS!!!!<br>
<br>
six lightening talks signed up, should be very<br>
cool. If you have slides, get them to Steve<br>
Feldman start with!<br>
<br>
Wireless movie after break should be cool to watch.<br>
Ren? Steve mistakenly introduces her, she corrects<br>
them. Don't forget to give feedback via the Survey<br>
forms!!<br>
<br>
2006.02.15 v6fix: Wiping the Slate Clean for IPv6<br>
Kenjiro Cho, WIDE/IIJ, Ruri Hiromi, WIDE/Intec NetCore<br>
<br>
Will be talking about their efforts to deploy<br>
IPv6, called v6fix.<br>
<br>
v6fix is an effort to solve problems in the current<br>
v6 deployment.<br>
focuses on v4/v6 dual stack environments.<br>
it's a technical analysis of real world problem<br>
Kenjiro will talk about tools and measurements.<br>
<br>
deployment status<br>
majority of equipment out there is v6 available<br>
from major vendors<br>
still many applications and appliances just work<br>
with v4<br>
v6 is starting to get into various business fields<br>
Many people lack knowledge/experience with v6.<br>
when non-experts hit problems, they're clueless.<br>
<br>
Example: illiteracy.<br>
Hotel internet systems have instructions for guest.<br>
troubleshooting: if you have IPv6 enabled, please<br>
disable IPv6--brochure in guest room.<br>
Cause of problem: combination.<br>
DNS redirection returns specific A record for AAAA<br>
clients stub-resolver accepts the A for AAAA, can't<br>
get out.<br>
<br>
Wiping the slate clean for the v6<br>
faulty behaviours only 1% and combinatorial often, but<br>
could be fatal to deployment.<br>
slow fallback to v4 after v6 errors<br>
misbehaving DNS resolvers<br>
filtering of ICMPv6<br>
DNS misconfigurations<br>
poorly configured tunnels<br>
lack of peering or v6 paths<br>
<br>
v6fix activities (research group)<br>
identify/analysze/solve real-world tech problems <br>
in v6 deployment.<br>
Enemy: "after disabling v6, my problems went away"<br>
Cooperation needed between researches, implementers, ops.<br>
<br>
v6fix topics<br>
harmful effects of the on-link assumption.<br>
misbehaving DNS servers and resolvers<br>
slow fallback to v4 after v6 failures<br>
<br>
Examples:<br>
case 1: DNS loop at hotel<br>
real story of hotel internet system--went to same room,<br>
investigated.<br>
DNS is intercepted, redirected to signup page<br>
ipv6 users can't get beyond first page<br>
hotel instructions say to disable v6<br>
erroneus DNS redirection system and stub-resolver<br>
redirection system always returns specific A record<br>
when getting non-A queries<br>
client's stub resolver queries AAAA for any address,<br>
blindly accepts A return response.<br>
<br>
case 2: DNS server slowdown<br>
Japanse ISP<br>
ISP upgraded a DNS cache to BIND9, recieved complaints<br>
about slowdown.<br>
recompiling BIND9 with --disable-ipv6, fixed problem,<br>
reported to JANOG<br>
Caused by older BIND9 w/o IPv6 connectivity<br>
server w/o v6 connectivity always tries to talk over v6,<br>
ends up failing back to v4 after timeouts<br>
fixed in BIND9.2.5 and 9.3.1<br>
<br>
Common factors<br>
1 problems appear only with specific combinatorial conditions<br>
2 implementors and operators didn't notice until reported<br>
3 even for professionals, not easy to track down problems.<br>
<br>
Kenjiro Cho, Tools:<br>
v6 tools and measurement results<br>
Goal: to understand the macro-level v6 healthiness<br>
current methodologies<br>
wide area meaasuremetn of behaviours of 2nd/3rd level<br>
DNS servers<br>
dual stack issues<br>
<br>
DNS server measurements of .jp domain<br>
AAAA responses: 0.13% DNS servers can't deal with <br>
AAAA requests<br>
Most are lame delegation type errors.<br>
ignore AAAA queries<br>
respond with RCODE 3 ("name error") NXDOMAIN<br>
<br>
dual-stack path analysis<br>
measurement techniques specifically designed for<br>
dual-stack<br>
take measurements for v4 and v6 at same time<br>
compare v6 results with v4 results<br>
extract problems that exist in v6 only<br>
methodology<br>
dual-stack node discovery<br>
create dual-stack node list by monitoring DNS AAAA replies.<br>
dual stack ping<br>
run ping/ping6 to target dual-stack nodes<br>
select a few representative nodes per site (/48) by RTT<br>
dual-stack traceroute<br>
trace/trac6 to selected nodes<br>
visula v6 MTU to look at issues<br>
visualize path issues<br>
<br>
distribution of v6/v4 RTTs<br>
4000 ping targets v4 on x-axis, v6 on y axis<br>
individual nodes far above unity line--leaf issues<br>
<br>
paths and PMTU visualization<br>
from NYSERNET to ARIN sites<br>
<br>
Many of ARIN paths via jp! (lack of peering)<br>
<br>
From ISC to ARIN sites--paths look much better, but<br>
lots of blue =3D=3D lots of tunnels<br>
<br>
Abilene case: well known problem.<br>
Abilene trying to encourage v6 adoption<br>
no AUP, tunnel services for v6<br>
but ended up with horrible v6 paths, mostly with tunnels<br>
ISPs are reluctant to move to paid v6 connectivity<br>
Abilene thinking about suspending its relaxed AUP for v6<br>
tool tries to illustrate such issues, convince users to<br>
move to native v6<br>
<br>
dual stack traceroute to ABILENE from WIDE (v4 upper, <br>
v6 lower)<br>
similar RTTs/hops for v4/v6; native dual-stack paths<br>
<br>
dual-stack trace to ABILENE from IIJ<br>
similar RTTs, but different paths: currently more common<br>
<br>
dualstack traceroue to ABILINE fro ES<br>
v6 RTTs much larger than v4: roundabout tunnels<br>
<br>
Conclusion: faulty behaviours are only 1% and often<br>
combinatorial, but can be fatal to acceptance of v6<br>
slow fallback to v4 on v6 errors<br>
<br>
knowledge sharing<br>
need to realize the dangers of harmful of adoption of v6<br>
cooperation among researchers, implementers, and ops<br>
need to act now, or will bring negative impact to v6<br>
deployment<br>
<br>
Acknowlegements<br>
v6fix members, etc.<br>
<br>
<a href=3D"http://v6fix.net/">http://v6fix.net/</a><br>
overview, documents, and fact database.<br>
<br>
contact at <a href=3D"http://v6fix.net">v6fix.net</a><br>
reports of issues are welcome<br>
<br>
<br>
<br>
<br>
------=_Part_3833_16057896.1140022680932--
