October 12, 2013
by andrijars
Folks from CloudStack thought it was fun to release CloudStack 4.2 with incomplete/incorrect documentation, so after you update CloudStack to 4.2, all your system VMs are not going to work, once you restart them…
BUG described here: https://issues.apache.org/jira/browse/CLOUDSTACK-4826 with some proposed solutions
A full solutions is very well explained here: http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/
In case the URL is down, here is a very much copy/paste from the original author, all credits goes to him.:
Step 1): Mount your secondary storage to your management server with the following:
mount -t nfs {ip_of_storage_server}:[path_to_secondary_storage] /mnt
Step 2): Download the latest version of the templates:
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt -u http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 -h kvm -F
Step 3): Find the name of the old template in the database:
USE cloud
SELECT install_path FROM template_host_ref WHERE template_id=3;
Sample output:
template/tmpl/1/3//44d12c68-82f6-3d46-8073-9db0e835b94c.qcow2
Step 4): write down the name of the .qcow2 file that your given in the previous step.
Step 5): from the management server locate the new template on the mounted secondary storage:
cd /mnt/template/tmpl/1/3/
Step 6): rename the .qcow2 file in that folder to the name we copied from the database.
Step 7): edit the template.properties file in the same folder and change both instances of the old name to the new one.
Step 8): we need to reset the cached template in the database:
UPDATE template_spool_ref SET download_pct='0',download_state='NOT_DOWNLOADED',state='NULL',local_path='NULL',install_path='NULL',template_size='0' WHERE template_id='3';
Step 9): Unmount your secondary storage from the management server:
umount /mnt
Step 10): disable the zone from the management UI.
Step 11): update the database records for your system VMs to be ‘Stopped’. You will need to do this for both the Secondary Storage VM and the Console Proxy. The ID of the system VM is the number in it’s name, for example; s-34-VM,, would have an ID of ’34′.
UPDATE vm_instance SET state='Stopped' where id='{id_of_system_vm}';
Step 12): From the management UI, destroy both the system VMs.
Step 13): Once both system VMs have been destroyed, re-enable the zone.
Step 14): Tail the management log and watch for the VMs to start.
tail -f /var/log/cloudstack/management/management-server.log
Admin Tweets 