ipv6 KVM multiple vms ip config

Christopher Simpson

Feb 26, 2021 • 7 min read

I want to spin up multiple vms using kvm and ipv6. IPv4 is exhausted and expensive. How do I do that?

Does my machine support KVM?

You need to find out if your CPU support virtualisation.

To do this, cat your cpuinfo file, cat /proc/cpuinfo and if you have an Intel CPU, look for the 'vmx' (which is for Intel VT-x) or 'smv' (for AMD SVM). These features can be enabled in the BIOS if they're not already enabled- and your CPU supports virtualisation. See also nixCraft

Forget KVM for a moment. We want to give each VM a public ip address (just like when you create a VM on Digital Ocean, AWS EC2, Hetzener or Vultr etc) you get given a public ip address. We want one our VMs too!

How to add an IPv6 address to your host

We are going to

Find out our servers IPv6 address using the ip command
Create a new IPv6 address (each vm we create will have its own IPv6 address)
Try and ping the new IPv6 address from another host outside the network

Find out the servers assigned IPv6 address

If your provider supports IPV6, they will have assigned your server an IPv6 address. Hopefully you've been given a large block to use.

IPv6 addresses have 128 bits. IPv4 only had 32. In IPv6, a /64 address means the first 64 bits are the network address, and the other 64 bits available for hosts. This means you'll have 2^64 addressed (a lot).

Get your current ipv6 address:

ip -6 address
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2001:ba8:1f1:f043::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:5eff:fe00:44c/64 scope link 
       valid_lft forever preferred_lft forever

The output above means (check man ip to see the options)

You use ping6 to ping global IPv6 address, but you can only locally ping6 link local addresses (because they're only locally rotatable).

eth0 is an interface, which has two IPv6 addresses
- inet6 2001:ba8:1f1:f043::2/64 scope global which is a globally routable ip address (globally unique)
- inet6 fe80::216:5eff:fe00:44c/64 scope link which is a link-local ipv6 address. Which is only routable locally within the local network. Similar to an IPv4 private address but not the same. IPv6 is cool. IPv6 supports auto configuration of hosts on an IPv6 enabled network, which is cool because it means you don't have to manually assign IPv4 addresses to each host, or rely on a DHCP server for that- but that's another topic)

We want to add another globally routable IPv6 address.

Our IPv6 network address given to us by our provider is the first half: 2001:ba8:1f1:f043::2/64

Let's add another IPv6 address!

We already have 2001:ba8:1f1:f043::2/64 so to keep it simple we'll create another IPv6 address on the eth0 interface: 2001:ba8:1f1:f043::3/64

Before: We just have one IPv6 address assigned to the interface:

ip -6 addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2001:ba8:1f1:f043::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:5eff:fe00:44c/64 scope link 
       valid_lft forever preferred_lft forever

Now add another IPv6 address to the interface eth0:

sudo ip -6 address add 2001:ba8:1f1:f043::3 dev eth0

If you accidentally add the wrong address, you can delete it using

sudo ip -6 address del 2001:ba8:1f1:f043::3 dev eth0

After: Now the additional IPv6 address appears attached to the interface (this server only has one interface eth0 but you might have multiple for redundancy.

ip -6 addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2001:ba8:1f1:f043::3/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 2001:ba8:1f1:f043::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:5eff:fe00:44c/64 scope link 
       valid_lft forever preferred_lft forever

Let's try and pin6 the address!

First try locally from the server:

ping6 2001:ba8:1f1:f043::3
PING 2001:ba8:1f1:f043::3(2001:ba8:1f1:f043::3) 56 data bytes
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=1 ttl=64 time=0.043 ms
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=2 ttl=64 time=0.043 ms
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=3 ttl=64 time=0.073 ms
^C
--- 2001:ba8:1f1:f043::3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.043/0.053/0.073/0.014 ms

Then try remotely from your local machine over the internet. Because this is a global IPv6 address, it should reply to pings (unless you have firewall rules preventing this):

ping6 2001:ba8:1f1:f043::3
PING 2001:ba8:1f1:f043::3(2001:ba8:1f1:f043::3) 56 data bytes
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=1 ttl=55 time=22.4 ms
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=2 ttl=55 time=16.8 ms
64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=3 ttl=55 time=19.0 ms
^C
--- 2001:ba8:1f1:f043::3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 16.862/19.452/22.406/2.280 ms

Fantastic! Above you can see the server is responding also from over the internet. Notice the response time is obviously longer (2.2ms) compares to locally (0.014 ms).

Fantastic! We know know how to add, route and ping IPv6 addresses. But what about VMs? We want to spin up a VM, and have this IP address assigned to the VM, and then automate the whole process (this is 'exactly' what happens when you buy a VPS from Amazon EC2, Digital Ocean etc).

Notice that this does not add the ip address permanently. If you reboot the server, the assignment is gone. For that you need to use netplan or interfaces config (see Bitfolk's excellent guide IPv6 guide for Ubuntu and Centos).

Creating a KVM instance / VPS

Goal: I want to start an Ubuntu VPS server, and assign it the IPv6 address we created earlier. Let's give the VPS 25GB of storage and 1GB of ram.

Install KVM on Ubuntu 20.04

This is based on the excellent guide by Cyberciti.

libvirt-daemon-system
- toolkit to interact with the virtualization capabilities of recent versions of Linux: Supports QEMU, KVM, XEN, OpenVZ, LXC, and VirtualBox
libvirt-clients
virtinst
- command line tool which provides an easy way to provision operating systems into virtual machines.
libosinfo-bin
- Operatiing system information database
libguestfs-tools
- is a set of tools for accessing and modifying virtual machine (VM) disk images. You can use this for viewing and editing files inside guests, scripting changes to VMs

Install the required tools

sudo apt install bridge-utils libvirt-daemon-system libvirt-clients virtinst libosinfo-bin libguestfs-tools

IPv6 is confusing to read because there's different ways to write an address. Why? Because IPv6 addresses are long, and we're lazy to type you can shorten them (if you follow the rules).

This an an example. These are all the same IPv6 address:

2001:db8:0:0:1:0:0:1
2001:0db8:0:0:1:0:0:1
2001:db8::1:0:0:1
2001:db8::0:1:0:0:1
2001:0db8::1:0:0:1
2001:db8:0:0:1::1
2001:db8:0000:0:1::1
2001:DB8:0:0:1::1
See IETF or Ciscopress for details

Providers which support IPv6

For this to work your server and network provider needs to support IPv6.

Bitfolk for example gives you a /64 ip6 network address.

Please note I don't believe Bitfolk support nested virtualisation so you can't use them for nested KVM, but are a fantastic VPS provider nonetheless.

Notes

Static bridge with netplan

https://www.cs.wcupa.edu/rkline/linux/virtual.html

install uvtool-libvirt

uvt-kvm: error: libvirt: storage volume 'guest1.qcow' exists already
DO:

virsh vol-list --details uvtool
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1.qcow

Host netplan:

/etc/netplan/01-netcfg.yaml

### Hetzner Online GmbH installimage
network:
  version: 2
  renderer: networkd
  ethernets:
    enp4s0:
      match:
        macaddress: 54:04:a6:b2:00:ee
      dhcp4: no
      dhcp6: no

  bridges:
    vmbr0:
      macaddress: 54:04:a6:b2:00:ee
      interfaces: [enp4s0]
      dhcp4: no
      dhcp6: no
      addresses:
        - 88.198.64.28/32
        - 2a01:4f8:140:4486::2/64
        - 2a01:4f8:140:4486::3/64
      routes:
        - on-link: true
          to: 0.0.0.0/0
          via: 88.198.64.1
        - on-link: true
          to: 0::/0
          via: fe80::1
      gateway6: fe80::1
      nameservers:
        addresses:
          - 213.133.98.98
          - 213.133.100.100
          - 213.133.99.99
          - 2a01:4f8:0:1::add:1010
          - 2a01:4f8:0:1::add:9898
          - 2a01:4f8:0:1::add:9999

guest ip netplan /tmp/guest1-netplan

version: 2
ethernets:
  enp1s0:
    addresses:
      - 2a01:4f8:140:4486::3/64
    gateway6: fe80::1
    nameservers:
      addresses:
        - 2a01:4f8:0:1::add:1010
        - 2a01:4f8:0:1::add:9898
        - 2a01:4f8:0:1::add:9999
    routes:
      - to: 2a01:4f8:140:4486::2/64
        via: ::/0
        scope: link

Create a vps with password login (can also pass cloud-init file to overide)
See http://manpages.ubuntu.com/manpages/bionic/man1/uvt-kvm.1.html

uvt-kvm create \
    --ssh-public-key-file ~/guest-ssh-keys/guest1.pub \
    --password password \
    --memory 4096 \
    --disk 40 \
    --cpu 2 \
    --bridge vmbr0 \
    --network-config /tmp/guest1-netplan \
    --packages language-pack-en,openssh-server,mosh,git,vim,puppet,screen,ufw \
    guest1 arch="amd64" release=disco label="minimal release"

virsh destroy guest1
virsh undefine guest1
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1.qcow
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1-ds.qcow

I tried to ping6 the network address from outside my server (`ping6 2001:ba8:1f1:f043:0:0:0:0`) , but Source routing is deprecated by RFC5095. Although it works on some hosts still. Providers which support IPv6