VMware backup script

I have been searching for simple and easy script to backup selected virtual machines running on esxi server, which will:

  • run on linux – do not require windows machine; can’t use PowerCLI
  • do not require direct access to esxi host using ssh (as other backups scripts like ghettoVCB.sh do) and will use API
  • support multiple esxi hosts managed by vcenter
  • have clear code – that means no perl hell (including big vsphere perl sdk)
  • have short code
  • can be put into cron

I was unable to find any to comply my requirements. I’m learning python, so I searched for some python vsphere libs and found psphere. Then backup.py has born.

How it works? Script will go trought all vms running on esxi host or vcenter and search for vm names listed in backup.list file. If vm is on list, then snapshot is created and copied to backup datastore (as thin provision disk) with virtual machine configuration file (.vmx). After copying snapshot is removed. At the end, backup.py will search for old backups (variable KeepDays) and delete them.

In my setup I’m using separate 1.5 TB disk as datastore [backup] to store backups, but you can use for example NFS datastore exported from some ZFS storage with deduplication or from some NAS device.

Script is hardcoded to use first Datacenter in vcenter, if you have more, you have to modify it. Plain esxi without vcenter don’t care about datacenters, so it is all ok.

Download including psphere here backup.zip

#!/usr/bin/python


from psphere.client import Client
from psphere.managedobjects import VirtualMachine, VirtualDiskManager, FileManager, Datacenter
from datetime import datetime, date, timedelta
import time, logging

# datastore name where to backup
BackupToDatastore="[backup]"
# how many days keep backups
KeepDays=1
client = Client("host","root","password")
logging.basicConfig(level=logging.INFO,format='%(asctime)s - %(message)s',datefmt='%Y-%m-%d %H:%M:%S')
logger = logging.getLogger("esxibackup")
logging.getLogger("psphere").setLevel("WARN")

dclist=Datacenter.all(client)
dc=dclist[0]

vmlist = VirtualMachine.all(client)

vdm=client.sc.virtualDiskManager
fileManager=client.sc.fileManager


for vm in vmlist:
    logger.debug("Found vm: %s" %vm.name)
    logger.debug(vm.config.files.vmPathName)
    logger.debug(vm.config.datastoreUrl[0].url)
    logger.debug(vm.config.files.vmPathName)
    if vm.name in open('backup.list').read():
                logger.info("Found vm %s on backup list" %vm.name)
                destination=BackupToDatastore+" "+datetime.now().strftime("%Y-%m-%d")+"/"+vm.name
                fileManager.MakeDirectory(name=destination,
                        datacenter=dc,
                        createParentDirectories=True)
                task=fileManager.CopyDatastoreFile_Task(sourceName=vm.config.files.vmPathName,
                        sourceDatacenter=dc,
                        destinationName=destination+"/%s.vmx" %(vm.name),
                        destinationDatacenter=None,
                        force=True)
                logger.info("Creating virtual machine %s snapshot" %vm.name)
                task=vm.CreateSnapshot_Task(name="backup",
                                        description="Automatic backup "+datetime.now().strftime("%Y-%m-%d %H:%M:%s"),
                                        memory=False,
                                        quiesce=True)

                while task.info.state in ["queued", "running"]:
                        time.sleep(5)
                        task.update()
                logger.info("Snapshot created")
                snapshot=task.info.result

                for device in vm.config.hardware.device:
                        if device.__class__.__name__ == 'VirtualDisk':
                                logger.info("Backuping %s to %s" %(device.backing.fileName,destination))
                                dstSpec = client.create("VirtualDiskSpec")
                                dstSpec.adapterType="pvscsi"
                                dstSpec.diskType="thin"
                                task=vdm.CopyVirtualDisk_Task(sourceName=device.backing.fileName,
                                                                                  sourceDatacenter=dc,
                                                                                  destName=destination+"/"+device.backing.fileName.rsplit('/')[1],
                                                                                  destDatacenter=None,
                                                                                  destSpec=None,
                                                                                  force=False)
                                while task.info.state in ["queued", "running"]:
                                        time.sleep(5)
                                        task.update()
                                        logger.debug("task update")
                                if task.info.state == "success":
                                        elapsed_time = task.info.completeTime - task.info.startTime
                                        logger.info("Elapsed time: %s" %elapsed_time)
                                elif task.info.state == "error":
                                        logger.error("ERROR: The task finished with an error. If an error was reported it will follow.")
                                        try:
                                                logger.error("ERROR: %s" % task.info.error.localizedMessage)
                                        except AttributeError:
                                                logger.error("ERROR: There is no error message available.")
                                else:
                                        logger.error("UNKNOWN: The task reports an unknown state %s" % task.info.state)

                logger.info("Removing virtual machine %s snapshot" %vm.name)
                task=snapshot.RemoveSnapshot_Task(removeChildren=True)
                while task.info.state in ["queued", "running"]:
                        time.sleep(5)
                        task.update()
                        logger.debug("task update")


for ds in dc.datastore:
        if ds.name==BackupToDatastore[1:-1]:
                task=ds.browser.SearchDatastore_Task(datastorePath=BackupToDatastore)
                while task.info.state in ["queued", "running"]:
                        time.sleep(1)
                        task.update()
                if task.info.state == "success":
                        for file in task.info.result.file:
                                try:
                                        backup_date=datetime.strptime(file.path,"%Y-%m-%d")
                                        from_date = date.today()-timedelta(days=KeepDays)
                                        if backup_date.date() < from_date:
                                                logger.info("Deleting old backup %s" %file.path)
                                                delete_task=fileManager.DeleteDatastoreFile_Task(name=BackupToDatastore+"/"+file.path,datacenter=dc)
                                                while delete_task.info.state in ["queued", "running"]:
                                                        time.sleep(1)
                                                        delete_task.update()
                                except ValueError:
                                        logger.info("Ignoring folder '%s'" %file.path)


Sample output from backup

2014-06-27 03:56:56 - Found vm zabbix.iwik.org on backup list
2014-06-27 03:56:56 - Creating virtual machine zabbix.iwik.org snapshot
2014-06-27 03:57:07 - Snapshot created
2014-06-27 03:57:07 - Backuping [raid1] zabbix.iwik.org/zabbix.iwik.org.vmdk to [backup] 2014-06-27/zabbix.iwik.org
2014-06-27 04:00:20 - Elapsed time: 0:03:11.276205
2014-06-27 04:00:20 - Backuping [raid1] zabbix.iwik.org/zabbix.iwik.org_1.vmdk to [backup] 2014-06-27/zabbix.iwik.org
2014-06-27 04:04:44 - Elapsed time: 0:04:22.287747
2014-06-27 04:04:44 - Removing virtual machine zabbix.iwik.org snapshot
2014-06-27 04:05:15 - Found vm irc.post.sk on backup list
2014-06-27 04:05:15 - Creating virtual machine irc.post.sk snapshot
2014-06-27 04:05:25 - Snapshot created
2014-06-27 04:05:25 - Backuping [raid1] irc.post.sk/irc.post.sk.vmdk to [backup] 2014-06-27/irc.post.sk
2014-06-27 04:06:57 - Elapsed time: 0:01:28.828372
2014-06-27 04:06:57 - Removing virtual machine irc.post.sk snapshot
2014-06-27 04:07:03 - Found vm pbx.iwik.org on backup list
2014-06-27 04:07:04 - Creating virtual machine pbx.iwik.org snapshot
2014-06-27 04:07:14 - Snapshot created
2014-06-27 04:07:14 - Backuping [raid1] pbx.iwik.org/pbx.iwik.org.vmdk to [backup] 2014-06-27/pbx.iwik.org
2014-06-27 04:08:35 - Elapsed time: 0:01:17.608250
2014-06-27 04:08:35 - Removing virtual machine pbx.iwik.org snapshot
2014-06-27 04:08:42 - Deleting old backup 2014-06-25

For consistent backup of virtual machines running mysql databases see my next post

12 thoughts on “VMware backup script

  1. I am getting while running the above script at line 36.
    C:\Python34>python.exe D:\scripts\backup\backup.py
    File “D:\scripts\backup\backup.py”, line 36
    logger.info(“Found vm %s on backup list” %vm.name)
    ^
    TabError: inconsistent use of tabs and spaces in indentation

    Thanks,
    Ankoji

  2. Hi, maybe copy&paste or editor issue? Did you used script copied from webpage itself or from zip file? Please try from zip in article.

  3. I checked your script. However when I think backup a server it gets for example from 200GB to 20GB.
    The point is if I wanted to recover this server, those files at 20GB will weight 200GB when rsyncing them.
    If it’s vm does not matter, but if there are 4, a long time could take for transferring all files.
    Any solution?
    Thanks!

  4. Hi,
    when copying VM to backup datastore, it is done as thin-provison disks. If your 200GB VM has only 20GB allocated, backup will have just 20GB. But do not copy thin-provision disks by rsync. Setup NFS datastore and copy it using API or backup your VMs directly to NFS datastore.
    To restore just connect backup datastore to new esxi server and import VMs. You can do vMotion after that.

  5. Does this work with the free version of ESXI? I’m getting the following error:

    ha-nfc-file-manager
    [backup] 2014-09-24/PowerMonitor
    ha-datacenter
    true

    Traceback (most recent call last):
    File “./backup.py”, line 40, in
    createParentDirectories=True)
    File “/home/mephi/backupESXI/old/psphere/__init__.py”, line 355, in func
    **kwargs)
    File “/home/mephi/backupESXI/old/psphere/client.py”, line 173, in invoke
    result = getattr(self.service, method)(_this=_this, **kwargs)
    File “/usr/lib/python2.7/dist-packages/suds/client.py”, line 542, in __call__
    return client.invoke(args, kwargs)
    File “/usr/lib/python2.7/dist-packages/suds/client.py”, line 602, in invoke
    result = self.send(soapenv)
    File “/usr/lib/python2.7/dist-packages/suds/client.py”, line 653, in send
    result = self.failed(binding, e)
    File “/usr/lib/python2.7/dist-packages/suds/client.py”, line 708, in failed
    r, p = binding.get_fault(reply)
    File “/usr/lib/python2.7/dist-packages/suds/bindings/binding.py”, line 265, in get_fault
    raise WebFault(p, faultroot)
    suds.WebFault: Server raised fault: ‘Current license or ESXi version prohibits execution of the requested operation.’

  6. Hello
    I don’t kwon python.

    When I run the script return this error

    W:\Source\Tgarijo\VmWareBackup>python backup.py
    Traceback (most recent call last):
    File “backup.py”, line 4, in
    from psphere.client import Client
    File “W:\Source\Tgarijo\VmWareBackup\psphere\client.py”, line 117
    except URLError, e:
    ^
    SyntaxError: invalid syntax

    thank You

  7. I’m not sure. It may be python version? What version are you running? Try it with 2.7.

  8. Hi,

    Is there an error in the script? It specifies, but does not use the dstSpec, making it copy the spec from the original file, so thick becomes thick again instead of thin.

    I have tried to fiddle with it but, can’t seem to get it working properly.. :-S

    Rgds

  9. Do you happen to have an end to end documented guide to support the implementation? Specifically I am unsure exact where the psphere client should be installed and the script executed from – i.e. the host or target server or just from a desktop client which secure (shell) access to the VMs you want to back up? Additionally, whilst I assume the backup script is a simple text file can you provide an example of the format of the file for me to follow? Last, where exactly do you read the backup list from? again is the the host or target server or desktop client? Some simple, documented, these are the steps you take would take some of the guess work out of it for me.

    Much appreciated.

    Bernard

  10. Hi, there is no more documentation. You can just run it somewhere, where you can run python script(s) – some linux VM for example.
    List of vms to backup is just plain text file with name of vm – one vm per line. Put it in same directory as backup script.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>