[180412] in North American Network Operators' Group
Re: AWS Elastic IP architecture
daemon@ATHENA.MIT.EDU (Owen DeLong)
Tue Jun 2 05:05:04 2015
X-Original-To: nanog@nanog.org
From: Owen DeLong <owen@delong.com>
In-Reply-To: <CAL9jLabzwHRON-d=jhT+qKc=s-CcDzYSyzarW9auRfKZMm+Xzw@mail.gmail.com>
Date: Tue, 2 Jun 2015 10:01:38 +0100
To: Christopher Morrow <morrowc.lists@gmail.com>
Cc: "nanog@nanog.org" <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
> On Jun 1, 2015, at 4:30 PM, Christopher Morrow =
<morrowc.lists@gmail.com> wrote:
>=20
> On Mon, Jun 1, 2015 at 3:06 AM, Owen DeLong <owen@delong.com =
<mailto:owen@delong.com>> wrote:
>>=20
>>> On May 31, 2015, at 7:46 PM, Christopher Morrow =
<morrowc.lists@gmail.com> wrote:
>>>=20
>>> On Sun, May 31, 2015 at 9:07 PM, Owen DeLong <owen@delong.com> =
wrote:
>>>> As I said before:
>>>>=20
>>>> Host Virtual (vr.org <http://vr.org/>)
>>>> Softlayer (softlayer.com <http://softlayer.com/>)
>>>> Linode (Linode.com <http://linode.com/>)
>>>>=20
>>>> All have full dual-stack support.
>>>=20
>>> <snip>
>>>=20
>>>>> At the risk of feeding the troll...
>>>>>=20
>>>>> This isn't just an AWS problem.
>>>=20
>>> So... ok. What does it mean, for a customer of a cloud service, to =
be
>>> ipv6 enabled?
>>=20
>> It means that I should be able to develop my cloud application(s) =
with full native IPv6 support and expect to be able to serve IPv4-only, =
IPv6-only, and dual-stack customers using native IP stacks on my virtual =
machines without requiring external proxies, translators, etc. from the =
cloud service provider.
>>=20
>=20
> Ok, I suppose as a long term goal that seems fine. I don't get why
> 'ipv6 address on my vm' matters a whole bunch (*in a world where v4 is
> still available to you I mean), but as a long term goal, sure fine.
Short term, I don=E2=80=99t want to have to pay to develop my =
application twice.
I want to develop once for IPv6 and be done. Using IPV6_V6ONLY=3DFALSE
socket option, I should be able to support both protocols from an =
application
developed for IPv6 on machines which have both protocol capabilities.
If you deliver to my native IPv6 or native IPv4 address, then i get real =
source
addresses of the originator of the connection and I can develop my =
application
normally and not have to re-engineer it around a protocol change later.
If you don=E2=80=99t, then, if I care about the source of the connection =
request in my
application, I either have to write the application to look elsewhere =
for it if the
connection came in on IPv4, or, I have to get even more creative about =
it.
Worse, it means that I=E2=80=99m having to maintain IPv4-specific cruft =
in my application
some of which may be necessary to support IPv6 connections that arrive =
over
IPv4 because I can=E2=80=99t get native IPv4.
No, this is not a long-term goal, it=E2=80=99s a short-term need. =
Otherwise, my development
costs are increased and my development time is extended. Additionally, =
the
extra code creates additional opportunities for errors, security holes, =
etc.
In short, there=E2=80=99s nothing good about not being able to develop a =
native application
and there are many draw-backs.
> In the short term though, that was my question: "What should be
> prioritized, and then get that information to aws/gce/rackspace/etc
> product managers" because I suspect their engineers are not silly,
> they know that v6 should be part of the plan... but the PM folk are
> saying: "No one is asking for this v6 thing, but lots of people want
> that other shiney bauble...so build the shiney!!=E2=80=9D
I have little doubt that this is true. However, but the time all of =
their customers
start screaming for IPv6, do you think they=E2=80=99re going to be =
ready/willing/able
to wait the 6-18 months it will take to deploy it after that point? I =
think they=E2=80=99re
going to be coming in crisis mode wondering how they can get IPv6 turned
on yesterday.
>=20
>>> What really matters for a cloud service user? What information could
>>> be surfaced to the cloud providers in order to get the most =
important
>>> ipv6 'stuff' done 'now=E2=80=99?
>>=20
>> Ideally, simple native routing of IPv6 to provisioned hosts should =
suffice in most cases.
>>=20
>>> o Is it most important to be able to address ever VM you create with
>>> an ipv6 address?
>>=20
>> Yes.
>>=20
>=20
> long term sure, short term though.. really?? isn't it more important
> to be able to terminate v6 services from 'public' users? (and v4 as
> well because not everyone has v6 at home/work/mobile)
Yes, but I need to be able to terminate the v6 services on the v6 socket
on my VM, not on some proxy that masks the source address.
If you have a 6->4 proxy/nat/whatever that somehow hands me a packet
with an IPv6 from address in the IPv4 packet header, that=E2=80=99s a =
pretty neat trick.
If you have operating system stacks that will support this and hand said =
source
address to the application through the standard accept() API, that=E2=80=99=
s an even
better trick.
If you don=E2=80=99t have that for every platform supported on AWS =
(Linux/BSD/Win/etc.),
then I think we need the above mentioned native connectivity, no?
>>> o Is it most important to be able to talk to backend services =
(perhaps
>>> at your prem) over ipv6?
>>=20
>> It=E2=80=99s hard to imagine how you could provide the first one =
above without having this one come along for the ride.
>>>=20
>>> o Is it most important that administrative interfaces to the VM
>>> systems (either REST/etc interfaces for managing vms or 'ssh'/etc) =
be
>>> ipv6 reachable?
>>=20
>> This would be the one where I would most be able to tolerate a delay.
>>=20
>>> o Is it most important to be able to terminate ipv6 connections (or
>>> datagrams) on a VM service for the public to use?
>>=20
>> If you can=E2=80=99t get the first one, this might be adequate as a =
short-term fallback for some applications. However, it=E2=80=99s far =
from ideal and not all that useful.
>>=20
>=20
> I agree it's not ideal, and I was making a list of 'short term goals'
> that could be prioritized and get us all to the v6 utopia later.
The thing is that in most cases, the sum of implementing this and =
managing it over a 3-6 month period is more effort than simply routing =
native IPv6 and adding IPv6 to your provisioning systems. As such, =
it=E2=80=99s hard for me to justify putting this short term goal in =
place vs. just deploying native IPv6 connectivity.
>>> I don't see, especially if the vm networking is unique to each
>>> customer, that 'ipv6 address on vm' is hugely important as a
>>> first/important goal. I DO see that landing publicly available
>>> services on an ipv6 endpoint is super helpful.
>>=20
>> Here=E2=80=99s the thing=E2=80=A6 In order to land IPv6 services =
without IPv6 support on the VM, you=E2=80=99re creating an environment =
where:
>>=20
>> 1. The services basically have to be supported by some =
form of proxy/nat/etc.
>> So long as you are using a supported L4 protocol, that =
might be fine, but not everything fits in TCP/UDP/ICMP.
>> Generally supporting GRE, IKE, and =
application-specific protocols becomes an issue.
>>=20
>=20
> I figured the simplest path from v6 to v4 was: "Rip the header and
> extension headers off, make a v4 packet, deliver to vm=E2=80=9D
How do you do that without masking the source address of the IPv6 host =
at the other end of the connection?
If you don=E2=80=99t have a good answer to that question=E2=80=A6 This =
fails.
> aside from "yes, your request came from <ipv6 literal>" that should
> 'just work', you do have to maintain some state to deliver back,
> depending on the implementation (but even that is probably able to be
> hidden from the 'user' and provisioned/capacity planned independent of
> the 'user=E2=80=99).
Sure, this is all fine and dandy from the user perspective, but it sucks =
in very big ways from the application developer perspective.
Imagine running a service (let=E2=80=99s say not a web-based service for =
the moment) with 100s of thousands or even millions of users from all =
over the world. Now, imagine if every connection in your application =
logs came from 192.9.200.5. How, exactly, do you track where your users =
are coming
from? How do you tell who connected from what network? How do you track =
down the abuser after you find him in log entries which share the
same source address as everything else?
>=20
>> 2. The developer has to develop and maintain an =
IPv4-compatible codebase rather than be able to use
>> the dual stack capabilities of a host with =
IPV6_V6ONLY=3DFALSE in the socket options. This delays the
>> ability to produce native IPv6 applications.
>>=20
>> 3. Proxies and translators add complexity, increase =
fragility, and reduce performance.
>>=20
>=20
> I think this point is cogent, but ... also part of the pie that 'aws'
> pays for you the user... Or rather they run a 'service' which takes
> care of this, has slas and slos...
> "All packets delivered with 99.99% having Xms extra latency!"
> "Service has 99.999% uptime!"
> "Throughput comparable (99.99% at peak, 99% of the time) to
> straight/native v4=E2=80=9D
I have yet to see a proxy/nat implementation that delivers on those =
numbers.
Forgive my skepticism, but until this is demonstrated, I think I=E2=80=99d=
rather focus efforts on native deployments than miserable bandaids.
>>> Would AWS (or any other cloud provider that's not currently up on =
the
>>> v6 bandwagon) enabling a loadbalanced ipv6 vip for your public =
service
>>> (perhaps not just http/s services even?) be enough to relieve some =
of
>>> the pressure on other parties and move the ball forward meaningfully
>>> enough for the cloud providers and their customers?
>>=20
>> For the reasons outlined above, I don=E2=80=99t really think so.
>=20
> ok, I figured that for short-term while the providers figure out: "Oh
> yea, this v6 thing is pretty simple in our encap'd world... why didn't
> we just turn it on originally?"
>=20
> getting v6 endpoints with little hassle for the 'user' (vm user) would
> help us all a bunch.
It might help some, but given the extent to which it complicates =
application development and breaks things unless you=E2=80=99re running =
the simplest case of web server, I don=E2=80=99t think it helps much.
Owen
