Install some prerequisites:
yum groupinstall "Development Libraries" "Development Tools" yum install mercurial python-devel dev86 iasl ncurses-devel ncurses \ glib2-devel glib2 openssl-devel yajl-devel libuuid-devel libuuid \ pciutils-devel pciutils texinfo kernel-xen bridge-utils gnutls gnutls-devel \ libxml2 libxml2-devel libnl libnl-devel libxslt libxslt-devel pygtk2 xorg-x11-xauth \ xorg-x11-fonts* device-mapper* gnome-python2-gconf pygtk2-libglade dbus-x11 \ gtk-vnc-python netcf netcf-devel netcf-libs vte vte-devel
Pull the source code and build Xen.
cd /usr/src hg clone -r RELEASE-4.1.2 http://xenbits.xensource.com/xen-4.1-testing.hg cd xen-4.1-testing.hg/ make dist -j4 make install
Build and install Libvirt management tools.
cd /usr/src wget http://libvirt.org/sources/libvirt-0.9.12.tar.gz tar -zxf libvirt-0.9.12.tar.gz cd libvirt-0.9.12/ ./configure --prefix=/usr make -j4 make install ldconfig cd /usr/src wget http://virt-manager.org/download/sources/virtinst/virtinst-0.600.1.tar.gz tar -zxf virtinst-0.600.1.tar.gz cd virtinst-0.600.1/ python setup.py install cd /usr/src wget http://virt-manager.org/download/sources/virt-manager/virt-manager-0.9.1.tar.gz tar -zxf virt-manager-0.9.1.tar.gz cd virt-manager-0.9.1/
Install the xen-enabled Dom0 kernel:
yum install http://au1.mirror.crc.id.au/repo/kernel-xen-release-6-3.noarch.rpm yum install kernel-xen
Edit /etc/grub.conf, make changes to the first ‘xen.gz’ line and change the next two lines to start with ‘module’.
kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin module /vmlinuz-126.96.36.199-2.el6xen.x86_64 ro root=UUID=efff8fe3-523b-4620-a01f-d948cd43c49a rd_MD_UUID=836f9712:2e50a8a6:b1eabaa6:19f7ff34 rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto rd_MD_UUID=0698abc6:b72a8b69:f1e3b4e8:2a9fc55f rd_NO_LVM rd_NO_DM rhgb quiet module /initramfs-188.8.131.52-2.el6xen.x86_64.img
Setup the init scripts.
chkconfig --add xencommons chkconfig --add xend chkconfig --add xen-watchdog chkconfig --add libvirtd chkconfig --add libvirt-domains chkconfig xencommons on chkconfig xend on chkconfig xen-watchdog on chkconfig libvirtd on chkconfig libvirt-domains on
Now it’s time to reboot, and manage your domains with virt-install and/or virt-manager.
Some issues that I haven’t been able to solve.
1. CentOS paravirtual domains hang during bootup with Kudzu due to the VNC framebuffer. They also fail to poweroff after shutdown and hang with 100% CPU usage. This doesn’t appear to be a libvirt specific issue; I could replicate it with pure Xen as well. To work around this, disable VNC framebuffer using virt-install with –nographics as follows.
virt-install -n centos -r 2048 --file /dev/vg0/centos --os-variant=rhel6 -\ -nographics -p -l http://mirror.fastserv.com/centos/6/os/x86_64/ -b virbr0 -d
Unfortunately, now virt-manager doesn’t know how to access text console. You have to use virsh [domain] console from the command line from now on.
2. Libvirt Xen driver does not support managedsave and ends up terminating the DomU ungracefully when Dom0 reboots. If you happen to have DomU on a MD RAID-backed LVM, this will crash Dom0 with a kernel oops as MD attempts to go read-only with domains still attached. If anyone knows a workaround I am keen to this. Until then, I really can’t use this setup in any production environment.
3. Trying to work around this, if you change /etc/sysconfig/libvirt-domains to shutdown the DomU’s instead of trying to (unsuccessfully) save them, libvirt attempts to shutdown Domain-0 and hangs the shutdown process until the timeout (default 300 seconds) is reached.
4. CentOS 6.2 seems to have a buggy e1000e driver (at least when used on an X9SCL+-F motherboard) and at one point went completely offline requiring a hard power cycle. Research reveals I’m not the only one with issues with this combination.
My final thoughts are that Xen+Libvirt are certainly not a production ready combination. Every time I thought I solved a problem I uncovered several more and finally gave up after (3). Unfortunately, I don’t have enough time to work these bugs out and had to use Ubuntu+KVM in a crunch to get things done.