VMware backup script

I have been searching for simple and easy script to backup selected virtual machines running on esxi server, which will:

• run on linux – do not require windows machine; can’t use PowerCLI
• do not require direct access to esxi host using ssh (as other backups scripts like ghettoVCB.sh do) and will use API
• support multiple esxi hosts managed by vcenter
• have clear code – that means no perl hell (including big vsphere perl sdk)
• have short code
• can be put into cron

I was unable to find any to comply my requirements. I’m learning python, so I searched for some python vsphere libs and found psphere. Then backup.py has born.

How it works? Script will go trought all vms running on esxi host or vcenter and search for vm names listed in backup.list file. If vm is on list, then snapshot is created and copied to backup datastore (as thin provision disk) with virtual machine configuration file (.vmx). After copying snapshot is removed. At the end, backup.py will search for old backups (variable KeepDays) and delete them.

In my setup I’m using separate 1.5 TB disk as datastore [backup] to store backups, but you can use for example NFS datastore exported from some ZFS storage with deduplication or from some NAS device.

Script is hardcoded to use first Datacenter in vcenter, if you have more, you have to modify it. Plain esxi without vcenter don’t care about datacenters, so it is all ok.

Download including psphere here backup.zip

#!/usr/bin/python


from psphere.client import Client
from psphere.managedobjects import VirtualMachine, VirtualDiskManager, FileManager, Datacenter
from datetime import datetime, date, timedelta
import time, logging

# datastore name where to backup
BackupToDatastore="[backup]"
# how many days keep backups
KeepDays=1
client = Client("host","root","password")
logging.basicConfig(level=logging.INFO,format='%(asctime)s - %(message)s',datefmt='%Y-%m-%d %H:%M:%S')
logger = logging.getLogger("esxibackup")
logging.getLogger("psphere").setLevel("WARN")

dclist=Datacenter.all(client)
dc=dclist[0]

vmlist = VirtualMachine.all(client)

vdm=client.sc.virtualDiskManager
fileManager=client.sc.fileManager


for vm in vmlist:
    logger.debug("Found vm: %s" %vm.name)
    logger.debug(vm.config.files.vmPathName)
    logger.debug(vm.config.datastoreUrl[0].url)
    logger.debug(vm.config.files.vmPathName)
    if vm.name in open('backup.list').read():
                logger.info("Found vm %s on backup list" %vm.name)
                destination=BackupToDatastore+" "+datetime.now().strftime("%Y-%m-%d")+"/"+vm.name
                fileManager.MakeDirectory(name=destination,
                        datacenter=dc,
                        createParentDirectories=True)
                task=fileManager.CopyDatastoreFile_Task(sourceName=vm.config.files.vmPathName,
                        sourceDatacenter=dc,
                        destinationName=destination+"/%s.vmx" %(vm.name),
                        destinationDatacenter=None,
                        force=True)
                logger.info("Creating virtual machine %s snapshot" %vm.name)
                task=vm.CreateSnapshot_Task(name="backup",
                                        description="Automatic backup "+datetime.now().strftime("%Y-%m-%d %H:%M:%s"),
                                        memory=False,
                                        quiesce=True)

                while task.info.state in ["queued", "running"]:
                        time.sleep(5)
                        task.update()
                logger.info("Snapshot created")
                snapshot=task.info.result

                for device in vm.config.hardware.device:
                        if device.__class__.__name__ == 'VirtualDisk':
                                logger.info("Backuping %s to %s" %(device.backing.fileName,destination))
                                dstSpec = client.create("VirtualDiskSpec")
                                dstSpec.adapterType="pvscsi"
                                dstSpec.diskType="thin"
                                task=vdm.CopyVirtualDisk_Task(sourceName=device.backing.fileName,
                                                                                  sourceDatacenter=dc,
                                                                                  destName=destination+"/"+device.backing.fileName.rsplit('/')[1],
                                                                                  destDatacenter=None,
                                                                                  destSpec=None,
                                                                                  force=False)
                                while task.info.state in ["queued", "running"]:
                                        time.sleep(5)
                                        task.update()
                                        logger.debug("task update")
                                if task.info.state == "success":
                                        elapsed_time = task.info.completeTime - task.info.startTime
                                        logger.info("Elapsed time: %s" %elapsed_time)
                                elif task.info.state == "error":
                                        logger.error("ERROR: The task finished with an error. If an error was reported it will follow.")
                                        try:
                                                logger.error("ERROR: %s" % task.info.error.localizedMessage)
                                        except AttributeError:
                                                logger.error("ERROR: There is no error message available.")
                                else:
                                        logger.error("UNKNOWN: The task reports an unknown state %s" % task.info.state)

                logger.info("Removing virtual machine %s snapshot" %vm.name)
                task=snapshot.RemoveSnapshot_Task(removeChildren=True)
                while task.info.state in ["queued", "running"]:
                        time.sleep(5)
                        task.update()
                        logger.debug("task update")


for ds in dc.datastore:
        if ds.name==BackupToDatastore[1:-1]:
                task=ds.browser.SearchDatastore_Task(datastorePath=BackupToDatastore)
                while task.info.state in ["queued", "running"]:
                        time.sleep(1)
                        task.update()
                if task.info.state == "success":
                        for file in task.info.result.file:
                                try:
                                        backup_date=datetime.strptime(file.path,"%Y-%m-%d")
                                        from_date = date.today()-timedelta(days=KeepDays)
                                        if backup_date.date() < from_date:
                                                logger.info("Deleting old backup %s" %file.path)
                                                delete_task=fileManager.DeleteDatastoreFile_Task(name=BackupToDatastore+"/"+file.path,datacenter=dc)
                                                while delete_task.info.state in ["queued", "running"]:
                                                        time.sleep(1)
                                                        delete_task.update()
                                except ValueError:
                                        logger.info("Ignoring folder '%s'" %file.path)

Sample output from backup

2014-06-27 03:56:56 - Found vm zabbix.iwik.org on backup list
2014-06-27 03:56:56 - Creating virtual machine zabbix.iwik.org snapshot
2014-06-27 03:57:07 - Snapshot created
2014-06-27 03:57:07 - Backuping [raid1] zabbix.iwik.org/zabbix.iwik.org.vmdk to [backup] 2014-06-27/zabbix.iwik.org
2014-06-27 04:00:20 - Elapsed time: 0:03:11.276205
2014-06-27 04:00:20 - Backuping [raid1] zabbix.iwik.org/zabbix.iwik.org_1.vmdk to [backup] 2014-06-27/zabbix.iwik.org
2014-06-27 04:04:44 - Elapsed time: 0:04:22.287747
2014-06-27 04:04:44 - Removing virtual machine zabbix.iwik.org snapshot
2014-06-27 04:05:15 - Found vm irc.post.sk on backup list
2014-06-27 04:05:15 - Creating virtual machine irc.post.sk snapshot
2014-06-27 04:05:25 - Snapshot created
2014-06-27 04:05:25 - Backuping [raid1] irc.post.sk/irc.post.sk.vmdk to [backup] 2014-06-27/irc.post.sk
2014-06-27 04:06:57 - Elapsed time: 0:01:28.828372
2014-06-27 04:06:57 - Removing virtual machine irc.post.sk snapshot
2014-06-27 04:07:03 - Found vm pbx.iwik.org on backup list
2014-06-27 04:07:04 - Creating virtual machine pbx.iwik.org snapshot
2014-06-27 04:07:14 - Snapshot created
2014-06-27 04:07:14 - Backuping [raid1] pbx.iwik.org/pbx.iwik.org.vmdk to [backup] 2014-06-27/pbx.iwik.org
2014-06-27 04:08:35 - Elapsed time: 0:01:17.608250
2014-06-27 04:08:35 - Removing virtual machine pbx.iwik.org snapshot
2014-06-27 04:08:42 - Deleting old backup 2014-06-25

For consistent backup of virtual machines running mysql databases see my next post