SmartOS – Migrate, backup and restore KVM VM’s

Recently I had to migrate my VM’s to a new SmartOS host. At first I thought I would be able to use vmadm send and vmadm receive commands but these commands did not seem to work. I found some documentation on Github which helped me along however I had to make some changes to the proposed steps to actually succeed in my goal. Below is the way how I was able to backup and restore my VM’s.

Backing up a KVM virtual machine

First backup the VM definitition file (replace ${UUID} with the VM’s UUID from vmadm list):

vmadm get ${UUID} > ${UUID}.json

Find which zfs filesystems/datasets you need to backup:

vmadm get ${UUID} | json zfs_filesystem disks

Example:

# vmadm get 13ac95c5-16e3-4743-b2a7-b0e4f63cb6fd|json zfs_filesystem
 zones/13ac95c5-16e3-4743-b2a7-b0e4f63cb6fd

 # vmadm get 13ac95c5-16e3-4743-b2a7-b0e4f63cb6fd|json disks
 [
 {
 "path": "/dev/zvol/rdsk/zones/13ac95c5-16e3-4743-b2a7-b0e4f63cb6fd-disk0",
 "boot": true,
 "model": "virtio",
 "media": "disk",
 "zfs_filesystem": "zones/13ac95c5-16e3-4743-b2a7-b0e4f63cb6fd-disk0",
 "zpool": "zones",
 "size": 8096,
 "compression": "off",
 "refreservation": 8096,
 "block_size": 8192
 }
 ]

In the example above I used two seperate vmadm get commands with the datasets to be backed up shown in bold.

Stop the VM from within the guest operating system or by issueing a:

vmadm stop ${UUID}

Create a snapshot, backup the snapshotted dataset to a file and remove the snapshot.

zfs snapshot zones/${UUID}@sending
zfs send -p zones/${UUID}@sending > ${UUID}.zfs
zfs destroy zones/${UUID}@sending

Repeat this process for the other datasets (disk0,etc)

zfs snapshot zones/${UUID}-disk0@sending
zfs send -p zones/${UUID}-disk0@sending > ${UUID}-disk0.zfs
zfs destroy zones/${UUID}-disk0@sending

Restoring/migrating the VM

Copy the *.zfs, *.json files to the target machine

Import the filesystems from the backup files:

zfs receive zones/${UUID} < ${UUID}.zfs
zfs destroy zones/${UUID}@sending
zfs receive zones/${UUID}-disk0 < ${UUID}-disk0.zfs
zfs destroy zones/${UUID}-disk0@sending

Now the datasets have been recreated we are ready to create the VM.
First edit the *.json file to include “nocreate”: true, in the “disks” subset:

Example:

  "disks": [
    {
      "path": "/dev/zvol/rdsk/zones/47d84c19-2eae-48c6-a976-109a4c9b9db2-disk0",
      "nocreate": true,      
      "boot": true,
      "model": "virtio",
      "media": "disk",
      "size": 4960,
      "zfs_filesystem": "zones/47d84c19-2eae-48c6-a976-109a4c9b9db2-disk0",
      "zpool": "zones",
      "compression": "off",
      "refreservation": 0,
      "block_size": 8192
    }

Create the VM by importing the (editted) definition file:

vmadm create < ${UUID}.json

Finally, If all went well the temporary *.zfs, *.json files and the original VM can be removed with vmadm delete.

Sources:
README.migration on Github
man vmadm
man zfs