RHEL 5 includes the now ancient Xen 3.0 hypervisior. A lot has been improved since then, especially in the current 3.3 release. Additionally, RedHat now owns the company behind KVM, so it is unlikely they will spend much time backporting Xen stuff for RHEL 5.3 or the likes.
Xen is a proven hypervisor. It works well on lots of hardware, including servers without hardware virtualization and older 64-bit Opterons that wont run 64-bit guests in the likes of VMWare. Since the OS is usually paravirtualized, performance is top notch. By making an OS aware of the environment it is running in, you can optimize it for virtualization. KVM is playing catchup here, realizing that paravirtualization is still ideal for many things.
Okay, so we are using or want to use Xen. Others have already built the packages we need, thankfully!
Head over to http://www.gitco.de/repo/ and grab the repo for your arch. (Most likely wget http://www.gitco.de/repo/CentOS5-GITCO_x86_64.repo in /etc/yum.repos.d/ for the uninitiated).
If you already have Xen installed, you may need to remove and readd it.
yum groupremove Virtualization yum groupinstall Virtualization
You'll also get some updated tools like Virtual Machine Monitor 0.6.0 that make it easier to install newer guests such as Fedora 10 or Ubuntu. Sweet!
Double check /etc/sysconfig/kernel. It should be set to kernel-xen. Likewise, check /boot/grub.conf and make sure that the Xen kernel is the default if the aforementioned was not done beforehand.
Xen 3.3 and Link Bonding
See my previous post for general information, but it gets harder.
This one is a nightmare. In my previous post, I detailed how to get Xen to work with link aggregation with Xen 3.0. Well, it doesn't work in 3.3. Xen decides that it still owns eth0 and completely destroys your bond0 setup.
Like these people, I've come to the conclusion that the integrated network scripts suck. This is alarming since you'd think link bonded setups would be the norm for Xen setups.
The quick fix is to let the OS handle networking. We do that like so: add a br0 interface and tell the bond to bridge with it.
DEVICE=br0 ONBOOT=yes BOOTPROTO=none IPADDR=10.0.6.201 NETMASK=255.255.255.0 GATEWAY=10.0.6.1 NO_ALIASROUTING=yes TYPE=Bridge
Then, edit your /etc/sysconfig/network-scripts/ifcfg-bond0 and add "BRIDGE=br0" and comment out any IP related information (since you are now defining that in the bridge. Head over to /etc/sysctl.conf and add:
net.ipv4.ip_forward = 1
Now, edit your Xen VMs in /etc/xen/ or /etc/xen/auto and change xenbr0 to br0:
vif = [ 'mac=ee:cc:aa:88:66:44, bridge=br0', ]
Okay, now disable the Xen networking garbage. Open /etc/xen/xend-config.sxp and comment out anything that looks like (network-script ....).
Almost done, but wait! RHEL 5.2 has a bug that prevents the bridge coming up on a bonded interface. Hopefully this will make the 5.3 cut or be pushed to 5.2, but until then go here. Download the new patch into /etc/sysconfig/network-scripts/ and run patch -p0 \< ifup-eth.patch for instance.
Reboot. You now have Xen 3.3 goodness on a big Ethernet channel! Post a comment if you have any trouble or questions.