Installing Xen Virtualization

[Leo.Prince]

Hi,

Xen is a very popular and advanced than openvz virtualization as it supports two major approaches

1, Paravirtualization

2, Hardware-assisted virtualization

Here we starts the steps to install Xen.

1, Make sure that SELinux is disabled or permissive:

Code: [Select]

vi /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

#       enforcing - SELinux security policy is enforced.

#       permissive - SELinux prints warnings instead of enforcing.

#       disabled - SELinux is fully disabled.

SELINUX=disabled

# SELINUXTYPE= type of policy in use. Possible values are:

#       targeted - Only targeted network daemons are protected.

#       strict - Full SELinux protection.

SELINUXTYPE=targeted

x!

If you had to modify /etc/sysconfig/selinux, please reboot the system:

Code: [Select]

reboot
2 Creating A Network Bridge

We needed to set up network bridges on our xen host node to avail the network to it's VMs as if it is having network as it's own. For that we needed to install a package bridge-utils...

Code: [Select]

yum install bridge-utils
It is not completed with that. We needed to configure bridging in the node.

Create the file /etc/sysconfig/network-scripts/ifcfg-br0 (please use the IPADDR, PREFIX, GATEWAY, DNS1 and DNS2 values from the /etc/sysconfig/network-scripts/ifcfg-eth0 file); make sure you use TYPE=Bridge, not TYPE=Ethernet

Code: [Select]

vi /etc/sysconfig/network-scripts/ifcfg-br0

DEVICE="br0"

NM_CONTROLLED="yes"

ONBOOT=yes

TYPE=Bridge

BOOTPROTO=none

IPADDR=192.168.1.15

PREFIX=24

GATEWAY=192.168.1.1

DNS1=8.8.8.8

DNS2=8.8.4.4

DEFROUTE=yes

IPV4_FAILURE_FATAL=yes

IPV6INIT=no

NAME="System br0"
Modify /etc/sysconfig/network-scripts/ifcfg-eth0 as follows (comment out BOOTPROTO, IPADDR, PREFIX, GATEWAY, DNS1, and DNS2 and add BRIDGE=br0)

Code: [Select]

vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE="eth0"

NM_CONTROLLED="yes"

ONBOOT=yes

HWADDR=00:1E:90:F3:F0:02

TYPE=Ethernet

#BOOTPROTO=none

#IPADDR=192.168.0.15

#PREFIX=24

#GATEWAY=192.168.1.1

#DNS1=8.8.8.8

#DNS2=8.8.4.4

DEFROUTE=yes

IPV4_FAILURE_FATAL=yes

IPV6INIT=no

NAME="System eth0"

UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03

BRIDGE=br0
Now restart the network.

Code: [Select]

service network restart
If you have configured so far correctly, Your network will be up with out issues. Otherwise you needed to revert and try via KVM 

After restarting network, use command

Code: [Select]

ifconfig It will show like this

Code: [Select]

[root@xen ~]# ifconfig

br0       Link encap:Ethernet  HWaddr 00:1E:90:F3:F0:02

         inet addr:192.168.1.15  Bcast:192.168.1.255  Mask:255.255.255.0

         inet6 addr: fe80::21e:90ff:fef3:f002/64 Scope:Link

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:17 errors:0 dropped:0 overruns:0 frame:0

         TX packets:29 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:1196 (1.1 KiB)  TX bytes:2794 (2.7 KiB)

eth0      Link encap:Ethernet  HWaddr 00:1E:90:F3:F0:02

         inet6 addr: fe80::21e:90ff:fef3:f002/64 Scope:Link

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:4554 errors:0 dropped:0 overruns:0 frame:0

         TX packets:3020 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:6249612 (5.9 MiB)  TX bytes:254928 (248.9 KiB)

         Interrupt:25 Base address:0x6000

lo        Link encap:Local Loopback

         inet addr:127.0.0.1  Mask:255.0.0.0

         inet6 addr: ::1/128 Scope:Host

         UP LOOPBACK RUNNING  MTU:16436  Metric:1

         RX packets:3 errors:0 dropped:0 overruns:0 frame:0

         TX packets:3 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:1304 (1.2 KiB)  TX bytes:1304 (1.2 KiB)

[root@xen ~]#
3, Now Installing Xen

First of all check if your CPU supports hardware virtualization. Try the command

Code: [Select]

egrep '(vmx|svm)' --color=always /proc/cpuinfo
The out put will be something similar to the below pasted

Code: [Select]

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat dts tpr_shadow vnmi flexpriority ept vpid
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat dts tpr_shadow vnmi flexpriority ept vpid

If the above command doesn't shows any output, That means your machine supports only paravirtualization bot hardware Virtualization.

As CentOS 6 is based on RedHat 6, and RedHat has dropped support for Xen in version 6, we need to get Xen from a third-party repository. We can enable the repo as follows

Code: [Select]

yum install wget

yum install http://au1.mirror.crc.id.au/repo/kernel-xen-release-6-3.noarch.rpm


To install xen, Then we have to simply run the yum command 

Code: [Select]

yum install kernel-xen xen
This will install the Xen and Xen kernel in our CentOS machine. Before we can boot the system with the Xen kernel, please check your GRUB bootloader configuration. We open /boot/grub/menu.lst or /etc/grub/grub.conf

Code: [Select]

vi /boot/grub/menu.lst
The first listed kernel should be the Xen kernel that we've recently installed

Code: [Select]

[...]

title CentOS (2.6.32.54-1.el6xen.x86_64)

       root (hd0,0)

       kernel /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM

       initrd /initramfs-2.6.32.54-1.el6xen.x86_64.img

[...]
We needed to modify it as pasted below, In order to boot the machine with Xen hypervisor kernel up on reboot.

Code: [Select]

[...]

title CentOS (2.6.32.54-1.el6xen.x86_64)

       root (hd0,0)

       kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin

       module /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM

       module /initramfs-2.6.32.54-1.el6xen.x86_64.img

[...]
Change the value of default to 0 (so that the first kernel (the Xen kernel) will be booted by default)

Code: [Select]

[...]

default=0

[...]
The complete grub.conf should look something like this

Code: [Select]

# grub.conf generated by anaconda

#

# Note that you do not have to rerun grub after making changes to this file

# NOTICE:  You have a /boot partition.  This means that

#          all kernel and initrd paths are relative to /boot/, eg.

#          root (hd0,0)

#          kernel /vmlinuz-version ro root=/dev/mapper/VolGroup00-LogVol00

#          initrd /initrd-[generic-]version.img

#boot=/dev/sde

default=0

timeout=5

splashimage=(hd0,0)/grub/splash.xpm.gz

hiddenmenu

title CentOS (2.6.32.54-1.el6xen.x86_64)

       root (hd0,0)

       kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin

       module /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM

       module /initramfs-2.6.32.54-1.el6xen.x86_64.img

title CentOS (2.6.32-220.el6.x86_64)
root (hd0,0)

       kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM

       initrd /initramfs-2.6.32-220.el6.x86_64.img
Before we reboot, we install the libvirt and python-virtinst (which contains the virt-install tool which we will use later on to install Xen VMs) packages

Code: [Select]

yum install libvirt python-virtinst
Because the libvirt package from CentOS 6/RedHat 6 has no support for Xen, we must rebuild it with Xen support. To do this, we install a few prerequisites now

Code: [Select]

yum groupinstall 'Development Tools'

Code: [Select]

yum install python-devel xen-devel libxml2-devel xhtml1-dtds readline-devel ncurses-devel libtasn1-devel gnutls-devel augeas libudev-devel libpciaccess-devel yajl-devel sanlock-devel libpcap-devel libnl-devel avahi-devel libselinux-devel cyrus-sasl-devel parted-devel device-mapper-devel numactl-devel libcap-ng-devel netcf-devel libcurl-devel audit-libs-devel systemtap-sdt-devel
We needed to see our libvirt version to proceed further. Use

Code: [Select]

rpm -qa | grep libvirt
It will give out put like this

Code: [Select]

[root@xen ~]# rpm -qa | grep libvirt

libvirt-client-0.9.4-23.el6_2.4.x86_64

libvirt-0.9.4-23.el6_2.4.x86_64

libvirt-python-0.9.4-23.el6_2.4.x86_64

[root@xen ~]#
It's 0.9.4, so we download the appropriate src.rpm package into /root/src and install it

Code: [Select]

mkdir /root/src

cd /root/src

wget http://vault.centos.org/6.2/os/Source/SPackages/libvirt-0.9.4-23.el6.src.rpm

rpm -i libvirt-0.9.4-23.el6.src.rpm
The last command may show some warnings which you can ignore.   

Next we patch Xen support into the libvirt sources

Code: [Select]

wget http://pasik.reaktio.net/xen/patches/libvirt-spec-rhel6-enable-xen.patch

cd /root/rpmbuild/SPECS

cp -a libvirt.spec libvirt.spec.orig

patch -p0 < ~/src/libvirt-spec-rhel6-enable-xen.patch
Now we can build a new libvirt package.

Code: [Select]

rpmbuild -bb libvirt.spec
At the end of the build process, You can see some verbose like this

Code: [Select]

Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-0.9.4-23.el6.x86_64.rpm

Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-client-0.9.4-23.el6.x86_64.rpm

Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-devel-0.9.4-23.el6.x86_64.rpm

Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-lock-sanlock-0.9.4-23.el6.x86_64.rpm

Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-python-0.9.4-23.el6.x86_64.rpm
Wrote: /root/rpmbuild/RPMS/x86_64/libvirt-debuginfo-0.9.4-23.el6.x86_64.rpm
Go to the directory where the new packages have been created (/root/rpmbuild/RPMS/x86_64/ in this case)..

Code: [Select]

cd /root/rpmbuild/RPMS/x86_64/
Then install the new libvirt packages (with Xen support) as follows

Code: [Select]

rpm -Uvh --force libvirt-0.9.4-23.el6.x86_64.rpm libvirt-client-0.9.4-23.el6.x86_64.rpm libvirt-python-0.9.4-23.el6.x86_64.rpm
Afterwards, we reboot the system

Code: [Select]

reboot
The system should now automatically boot up with the new Xen kernel. After the system has booted, we can check that by running

Code: [Select]

[root@xen ~]# uname -r

2.6.32.54-1.el6xen.x86_64

[root@xen ~]#
With that we can be happy that it is booted up with the new Xen kernel.

To list the xen VMs use

Code: [Select]

xm list
It will show you some outputs as follows with the default container Domain-0

Code: [Select]

[root@xen ~]# xm list

Name                                        ID   Mem VCPUs      State   Time(s)

Domain-0                                     0  1024     1     r-----     18.9

[root@xen ~]#
Instead of using the xm command, We can now use from now on the "virsh" command to manage Xen VMs. This is the preferred way as we are using libvirt.

Code: [Select]

virsh list
It will show output like this

Code: [Select]

[root@xen ~]# virsh list

Id Name                 State

----------------------------------

 0 Domain-0             running
That is it. Your Xen hardware node is ready to use.