I want to spin up multiple vms using kvm and ipv6. IPv4 is exhausted and expensive. How do I do that?
Does my machine support KVM?
You need to find out if your CPU support virtualisation.
To do this, cat your cpuinfo file,
cat /proc/cpuinfo and if you have an Intel CPU, look for the 'vmx' (which is for Intel VT-x) or 'smv' (for AMD SVM). These features can be enabled in the BIOS if they're not already enabled- and your CPU supports virtualisation. See also nixCraft
Forget KVM for a moment. We want to give each VM a public ip address (just like when you create a VM on Digital Ocean, AWS EC2, Hetzener or Vultr etc) you get given a public ip address. We want one our VMs too!
How to add an IPv6 address to your host
We are going to
- Find out our servers IPv6 address using the
- Create a new IPv6 address (each vm we create will have its own IPv6 address)
- Try and ping the new IPv6 address from another host outside the network
Find out the servers assigned IPv6 address
If your provider supports IPV6, they will have assigned your server an IPv6 address. Hopefully you've been given a large block to use.
IPv6 addresses have 128 bits. IPv4 only had 32. In IPv6, a
/64 address means the first 64 bits are the network address, and the other 64 bits available for hosts. This means you'll have 2^64 addressed (a lot).
Get your current ipv6 address:
ip -6 address 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000 inet6 2001:ba8:1f1:f043::2/64 scope global valid_lft forever preferred_lft forever inet6 fe80::216:5eff:fe00:44c/64 scope link valid_lft forever preferred_lft forever
The output above means (check
man ip to see the options)
You use ping6 to ping global IPv6 address, but you can only locally ping6 link local addresses (because they're only locally rotatable).
- eth0 is an interface, which has two IPv6 addresses
- inet6 2001:ba8:1f1:f043::2/64 scope global which is a globally routable ip address (globally unique)
- inet6 fe80::216:5eff:fe00:44c/64 scope link which is a link-local ipv6 address. Which is only routable locally within the local network. Similar to an IPv4 private address but not the same. IPv6 is cool. IPv6 supports auto configuration of hosts on an IPv6 enabled network, which is cool because it means you don't have to manually assign IPv4 addresses to each host, or rely on a DHCP server for that- but that's another topic)
We want to add another globally routable IPv6 address.
Our IPv6 network address given to us by our provider is the first half: 2001:ba8:1f1:f043::2/64
Let's add another IPv6 address!
We already have
2001:ba8:1f1:f043::2/64 so to keep it simple we'll create another IPv6 address on the
Before: We just have one IPv6 address assigned to the interface:
ip -6 addr show eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000 inet6 2001:ba8:1f1:f043::2/64 scope global valid_lft forever preferred_lft forever inet6 fe80::216:5eff:fe00:44c/64 scope link valid_lft forever preferred_lft forever
Now add another IPv6 address to the interface eth0:
sudo ip -6 address add 2001:ba8:1f1:f043::3 dev eth0
If you accidentally add the wrong address, you can delete it using
sudo ip -6 address del 2001:ba8:1f1:f043::3 dev eth0
After: Now the additional IPv6 address appears attached to the interface (this server only has one interface
eth0 but you might have multiple for redundancy.
ip -6 addr show eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000 inet6 2001:ba8:1f1:f043::3/128 scope global valid_lft forever preferred_lft forever inet6 2001:ba8:1f1:f043::2/64 scope global valid_lft forever preferred_lft forever inet6 fe80::216:5eff:fe00:44c/64 scope link valid_lft forever preferred_lft forever
Let's try and pin6 the address!
First try locally from the server:
ping6 2001:ba8:1f1:f043::3 PING 2001:ba8:1f1:f043::3(2001:ba8:1f1:f043::3) 56 data bytes 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=1 ttl=64 time=0.043 ms 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=2 ttl=64 time=0.043 ms 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=3 ttl=64 time=0.073 ms ^C --- 2001:ba8:1f1:f043::3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.043/0.053/0.073/0.014 ms
Then try remotely from your local machine over the internet. Because this is a global IPv6 address, it should reply to pings (unless you have firewall rules preventing this):
ping6 2001:ba8:1f1:f043::3 PING 2001:ba8:1f1:f043::3(2001:ba8:1f1:f043::3) 56 data bytes 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=1 ttl=55 time=22.4 ms 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=2 ttl=55 time=16.8 ms 64 bytes from 2001:ba8:1f1:f043::3: icmp_seq=3 ttl=55 time=19.0 ms ^C --- 2001:ba8:1f1:f043::3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 16.862/19.452/22.406/2.280 ms
Fantastic! Above you can see the server is responding also from over the internet. Notice the response time is obviously longer (2.2ms) compares to locally (0.014 ms).
Fantastic! We know know how to add, route and ping IPv6 addresses. But what about VMs? We want to spin up a VM, and have this IP address assigned to the VM, and then automate the whole process (this is 'exactly' what happens when you buy a VPS from Amazon EC2, Digital Ocean etc).
Notice that this does not add the ip address permanently. If you reboot the server, the assignment is gone. For that you need to use netplan or interfaces config (see Bitfolk's excellent guide IPv6 guide for Ubuntu and Centos).
Creating a KVM instance / VPS
Goal: I want to start an Ubuntu VPS server, and assign it the IPv6 address we created earlier. Let's give the VPS 25GB of storage and 1GB of ram.
Install KVM on Ubuntu 20.04
This is based on the excellent guide by Cyberciti.
- toolkit to interact with the virtualization capabilities of recent versions of Linux: Supports QEMU, KVM, XEN, OpenVZ, LXC, and VirtualBox
- command line tool which provides an easy way to provision operating systems into virtual machines.
- is a set of tools for accessing and modifying virtual machine (VM) disk images. You can use this for viewing and editing files inside guests, scripting changes to VMs
Install the required tools
sudo apt install bridge-utils libvirt-daemon-system libvirt-clients virtinst libosinfo-bin libguestfs-tools
IPv6 is confusing to read because there's different ways to write an address. Why? Because IPv6 addresses are long, and we're lazy to type you can shorten them (if you follow the rules).
This an an example. These are all the same IPv6 address:
- See IETF or Ciscopress for details
Providers which support IPv6
For this to work your server and network provider needs to support IPv6.
- Bitfolk for example gives you a /64 ip6 network address.
Please note I don't believe Bitfolk support nested virtualisation so you can't use them for nested KVM, but are a fantastic VPS provider nonetheless.
Static bridge with netplan
uvt-kvm: error: libvirt: storage volume 'guest1.qcow' exists already
virsh vol-list --details uvtool
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1.qcow
### Hetzner Online GmbH installimage network: version: 2 renderer: networkd ethernets: enp4s0: match: macaddress: 54:04:a6:b2:00:ee dhcp4: no dhcp6: no bridges: vmbr0: macaddress: 54:04:a6:b2:00:ee interfaces: [enp4s0] dhcp4: no dhcp6: no addresses: - 18.104.22.168/32 - 2a01:4f8:140:4486::2/64 - 2a01:4f8:140:4486::3/64 routes: - on-link: true to: 0.0.0.0/0 via: 22.214.171.124 - on-link: true to: 0::/0 via: fe80::1 gateway6: fe80::1 nameservers: addresses: - 126.96.36.199 - 188.8.131.52 - 184.108.40.206 - 2a01:4f8:0:1::add:1010 - 2a01:4f8:0:1::add:9898 - 2a01:4f8:0:1::add:9999
guest ip netplan /tmp/guest1-netplan
version: 2 ethernets: enp1s0: addresses: - 2a01:4f8:140:4486::3/64 gateway6: fe80::1 nameservers: addresses: - 2a01:4f8:0:1::add:1010 - 2a01:4f8:0:1::add:9898 - 2a01:4f8:0:1::add:9999 routes: - to: 2a01:4f8:140:4486::2/64 via: ::/0 scope: link
Create a vps with password login (can also pass cloud-init file to overide)
uvt-kvm create \ --ssh-public-key-file ~/guest-ssh-keys/guest1.pub \ --password password \ --memory 4096 \ --disk 40 \ --cpu 2 \ --bridge vmbr0 \ --network-config /tmp/guest1-netplan \ --packages language-pack-en,openssh-server,mosh,git,vim,puppet,screen,ufw \ guest1 arch="amd64" release=disco label="minimal release"
Login with : virsh console guest1
virsh destroy guest1
virsh undefine guest1
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1.qcow
virsh vol-delete /var/lib/uvtool/libvirt/images/guest1-ds.qcow
I tried to ping6 the network address from outside my server (`ping6 2001:ba8:1f1:f043:0:0:0:0`) , but
Source routing is deprecated by RFC5095. Although it works on some hosts still. Providers which support IPv6