IPv6 in Docker containers

Docker is great. It's a client-server based tool that makes it easy to run something in isolated Linux containers (LXC). Linux containers usually need some non-trivial configuration to be set up and run. Docker helps with this by providing easy methods to build container images and ready to run LXC configurations for running them. Unfortunately, it ignores IPv6... 

About LXC and Docker

For those of you who don't know Linux containers: LXC is a lightweight virtual system mechanism in Linux to isolate processes from the host system. The usual virtualization (kvm, qemu, xen) boots a new kernel as a guest system and isolates the guest's access to the hardware. Containers however run on the same kernel as the host system does, but they get a completely isolated view to this kernel. I.e. a container has its own file system, network devices, process space, etc. Therefore, containers are a very lightweight virtualization mechanism witch much less overhead in memory and processing time compared to VMs. Linux containers are comparable to Zones in Solaris or Jails in FreeBSD.

Update: Since version 0.9, Docker speaks directly to the kernel by default rather than using LXC commands. For the following IPv6 workaround, Docker needs to be running with the LXC execution driver, i.e. start the Docker daemon with --exec-driver=lxc and make sure to have the LXC package installed.

Besides preparing a root filesystem, Docker also sets up networking for a container when you run it. The host running the Docker server gets a new network device (docker0) using the private address range 172.17.42.1/16. This device is bridged with every running container (veth* devices) and provides this host-only network to the running containers. If you want to expose an IP port from a container to the outside (i.e. make a service running inside a container available to access from the internet), Docker can automatically set up iptables forwarding rules to do so.

This works fine with IPv4, but what about IPv6?

IPv6

Unfortunately, Docker does not have any support for IPv6. Very unfortunate, because IPv6 is the (long overdue) future internet protocol and much easier to handle (e.g. more than enough addresses and no annoying NAT).

However, there is a way to route IPv6 into containers. You just need to modify the container configuration manually.

Let's assume that the host has the IPv6 address range 1111:2222:3333:4444::2/64. Pick a common subnet address range for all containers, e.g. 1111:2222:3333:4444::8888:1/112. Assign the first subnet address to the docker0 device (ensure that IPv6 forwarding is on and that your firewall is configured properly).

$ ip -6 addr add 1111:2222:3333:4444::8888:1/112 dev docker0

All traffic for this subnet should now be routed to the docker0 device. Now you need to make sure that the apropriate containers can handle it. The container's network device needs to get a subnet address too, e.g. 1111:2222:3333:4444::8888:20/112. For this to happen, we need to add the following statements to the LXC configuration of this container:

lxc.network.flags = up
lxc.network.ipv6 = 1111:2222:3333:4444::8888:20/112
lxc.network.ipv6.gateway = 1111:2222:3333:4444::8888:1

This is equivalent to running ip -6 addr add 1111:2222:3333:4444::8888:20/112 dev eth0 and ip -6 route add default via 1111:2222:3333:4444::8888:1 dev eth0 inside the container every time it is run.

Docker allows us to specify arbitrary LXC configuration statements when running a container by using the -lxc-conf= option with docker run.

$ docker run \
    -lxc-conf="lxc.network.flags = up" \
    -lxc-conf="lxc.network.ipv6 = 1111:2222:3333:4444::8888:20/112" \
    -lxc-conf="lxc.network.ipv6.gateway = 1111:2222:3333:4444::8888:1" \
    -p 80:80 \
    webserver

This runs the container named "webserver", exposes IPv4 port 80 inside the container as port 80 on the host and routes the given IPv6 address into the container.

I admit that the above command is a bit lengthy to type in every time. However, containers with an IPv6 address are most probably started automatically, so these options can easily be added to the upstart configuration, init.d script or whatever.

Ideas

It would be nice to have an easier and more flexible way to specify an IPv6 address for a Docker container. Maybe if Docker would recognize one or more ADDRESS statements in a Dockerfile that specify the static address a container should get when being run. This could also cover static IPv4 addresses. However, by assigning static addresses at build time, a container would lose a lot of its flexibility (its host independence).

This reminds me of the same problem when assigning volumes that are shared with the host. You need to make sure that the host has the right volume the container needs and add a command line option to mount it into the container. Same as with IP addresses.

My idea of dealing with this would be a generic resource request/provide mechanism in Docker. Container build files could state named resources they need (need volume "webpages" mounted to /var/www, need IP address "webserver"). Hosts on the other hand could provide those named resources. (e.g. because they are listed in a host-specific configuration file). If a host does not have a resource, an arbitrary script could be run to somehow make the resource available (e.g. by mounting NFS). If a resource can not be provided, the container fails to run.

My server

I recently switched my server from running multiple VMs to running Docker containers. I published most of the scripts and configuration files in my infrastructure repository on Github.

Updated April 22, 2014: Added lxc.network.flags=up and hint on using --exec-driver=lcx

• Tweet