Handling virtual machine files on Linux

Posted on Tue 19 April 2011 in Virtualization

While i was working with Linux/KVM/Xen i made some notice for daily commands and why do it this way. Maybe someone can use this information for reducing time to wait for finished copy jobs or free unused disk space.

rsync is our friend:

Many of my virtual machines are using sparse file container to reduce the disk space they are using active. If a machine needs to be replaced and or those sparse files need to change there location to a different machine or filesystem you should use the right command to be sure your sparse file will not expand to full size on the new location.

I use rsync to do this job because it's kicking asses. I am a friend of tools they are powerful instead of many small tools. Read the man pages of rsync and you know what i mean.

The option we are looking for in this case is -S or --sparse. This option will handle the sparse files as we want it. Our sparse file will be exactly using the active taken space like on source filesystem.

Because of the size of VM's it's nice to see a progress while the copy job is running, so use --progress to be sure you get the transfer rate you are expecting.

Here an example for pulling a machine file to a new machine:

rsync -avS --progress <user>@<sourcemachine>:/<directory>/<file> <destinationdirectory>/

Some times you maybe have an "normal" full size vm file and you need to convert it into a sparse file. In this case i prefer the way to copy the file via:

cp --sparse=always <sourcefile> <destinationfile>

or you can use rsync local as well if you prefer the one tool policy.

Be sure before you start this actions to have enough disk space and backup of you files. Sparse file can help you to reduce the footprint of your VM's to the real needed space but you should monitor the growing of you files if you made an overcommit of your physical storage or lun. The virtual machine will run into real trouble if disk space on such a overcommited storage is full.