CentOS KVM and CEPH – client side setup

Once you have deployed the almighty CEPH storage, you will want to be able to actualy use it (RBD).

Before we begin, some notes:

Current CEPH version:  0.67 (“dumpling”).
OS: Centos 6.4 x86_64 (running some VMs on KVM, basic CentOS qemu packages, nothing custom)

Since CEPH RBD module was first introduced with kernel 2.6.34 (and current RHEL/CentOS kernel is 2.6.32) – that means we need a newer kernel.

So, one of the options for the new kernel is, 3.x from elrepo.org:

rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install kernel-ml # will install 3.11.latest, stable, mainline
     # or
yum --enablerepo=elrepo-kernel install kernel-lt # will install 3.0.latest, long term supported

If you want that new kernel to boot by default, edit /etc/grub.conf, and change the Default=1 to Default=0, and reboot.

That’s for the Kernel side.

If you want to use qemu with RBD, you will also need newer qemu packages – CEPH does provide some “untested”, “use it at your own risk” packages, that are rhel/centos based verions, just with RBD support added – well, running them on productional cloud, and all seems fine so far for me…

wget http://ceph.com/packages/ceph-extras/rpm/rhel6/x86_64/qemu-kvm-
wget http://ceph.com/packages/ceph-extras/rpm/rhel6/x86_64/qemu-img-
wget http://ceph.com/packages/ceph-extras/rpm/rhel6/x86_64/qemu-kvm-tools-

After done,  in order to actually install those CEPH-provided versions of binaries, we need to download and install CEPH, that is the required libraries librados and librbd:

rpm --import 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc'
rpm -Uvh http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm
yum install ceph

If all fine (will not talk here about creating the /etc/ceph.conf file and keyring for the client – maybe another post), then we are ready to install CEPH version of qemu packages:

First, we need to remove the original packages from Centos 6, then install the new ones:

rpm -e --nodeps qemu-img
rpm -e --nodeps qemu-kvm
rpm -e --nodeps qemu-kvm-tools
rpm -Uvh qemu-*

And… that’s it. If you run the   qemu-img | grep “Supported formats”  you will see RBD as being one od those.
You may shutdown and restart all your existing VMs, in order to load the new qemu binaries.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: