ubuntu 10.04 beta server DomU on XenServer 5.6 beta

I’m one that always likes to test out the latest and greatest.  As frustrating as it can sometimes be, I do enjoy it.  So once I had ubuntu 9.10 server up on XenServer, I wanted to try ubuntu 10.04 server beta.  It is almost exactly the same as 9.10, but changes in grub have made it a little more involved.

What kernel?

The xen kernel is gone as all support was built into the server kernel as of ubuntu 8.10.  Now with 10.04 server beta, ubuntu actually uses the generic-pae kernel.  If you check the config file under /boot you will see that all the Xen and PV stuff is added to generic-pae.  You are good to go with the out of box kernel.

Pygrub, ext4, and Grub 2

Pygrub and xe-edit-bootloader cannot handle an ext4 partition.  Make sure the partition containing /boot is ext3.  xe-boot-loader cannot handle grub 2, but can be edited to allow the reading of grub.cfg.  Pygrub has some support for grub 2.  Out of the box it can handle ubuntu 9.10, if /boot is not a separate partition, but it cannot handle the format of the 10.04 grub.conf.  This is due to options after the menu entry title and the use of quotes in more areas.  Fortunately there is a XenSource patch floating around to address that.  More further down.

  1. First create a VM in XenCenter.  I used the other install media template.
  2. Install ubuntu to your liking. Just in case anything goes wrong with getting console output, it is recommended that you install the SSH server.  If you want to use something other than ext3, make a small first partition for /boot formatted as ext3.  Now you can make the remaining partitions whichever file system you like.  Once done, update the OS if you desire.
  3. The console for the DomU in PV mode will be hvc0.  You will need to tell getty to use this.  Copy the existing tty1.conf and edit it to replace “tty1” with “hvc0”.
    sudo cp /etc/init/tty1.conf /etc/init/hvc0.conf
    nano -w /etc/init/hvc0
  4. If you so not want to alter scripts on the XenServer take note of the default menu entry in grub.cfg as you will need to place the options in XenServer.  If you do not, you will at least have to alter xe-edit-bootloader.
  5. Shutdown the VM.
  6. Retrieve the UUID for the VM.  Assuming your VM is named “ubuntu10”:
    xe vm-list name-label=ubuntu10 params=uuid
  7. Retrieve the VBD UUID for the storage attached to the VM.
    xe vm-disk-list uuid=<UUID>
  8. Clear out the HVM boot policy.
    xe vm-param-set uuid=<UUID> HVM-boot-policy=
  9. Set the PV bootloader to pygrub.
    xe vm-param-set uuid=<UUID> PV-bootloader=pygrub
  10. Lastly, set the VBD for the VM to be bootable.
    xe vbd-param-set uuid=<VBD UUID> bootable=true
  11. Since pygrub cannot handle the grub.cfg format of ubuntu 10.04, we must make some alterations on XenServer.  If you do not want to do this, skip to step 14 and specify the kernel in XenServer.
  12. If you made /boot a separate partition, add support to pygrub by editing /usr/bin/pygrub.  Go to line 395 and you should see the file list for grub 2.  It is only searching for /boot/grub/grub.cfg.  Like in the grub section, it needs to search for the file when /boot is the root of the partition also.  Add /grub/grub.cfg to that list.  Otherwise you can skip this step.
  13. Now we need to patch /usr/lib/python2.4/site-packages/grub/GrubConf.py.  Create a patch file there and paste the patch contents from XenSource.  This one is from 3/24/2010.  There is one on the XenSource list from 3/15/2010, but it could not patch a few sections of my file.  Then patch the file as you normally would.  “man patch” if you are unfamiliar.  I highly recommend manually making a back up copy of GrubConf.py!  Once patched successfully pygrub will be able to parse grub.cfg from ubuntu 10.04.  Skip to step <next step>.
  14. If you are hesitant to edit the pygrub python script or GrubConf.py, simplify specify the arguments you need for the VM.  Specify the kernel, ramdisk, and other options that are present in the VMs grub.cfg.  Make sure you use the entries from your grub file, not these specific ones.
    xe vm-param-set uuid=<UUID> PV-bootloader-args="--kernel=vmlinuz-2.6.31-14-server \
    --ramdisk=initrd.img-2.6.31-14-server"
    xe vm-param-set uuid=<UUID> PV-args="root=UUID=<UUID from grub root entry> ro quiet splash"
  15. Start the VM and login.  If your console fails to appear or seems to hang, try connecting to the VM using SSH.  If you missed they step about getting getty to output to hvc0, the console will appear to be hung.  In actuality it is just not displaying the login prompt on hvc0.
  16. Attach the xs-tools.iso to the VM and mount the image.  Mount will claim /dev/cdrom1 does not exist if it has not yet been attached to the VM in XenCenter.
    sudo mount /dev/cdrom1 /mnt
  17. Install the XenServer tools.  Make sure you use the proper file for your architecture (amd64 or i386).  A quick “ls” will confirm the version of the utilities in you xs-tools.iso.  My file name my differ.
    sudo dpkg -i /mnt/Linux/xe-guest-utilities_5.5.901-562_i386.deb
  18. Unmount the iso image and then detach it in XenCenter.
    sudo umount /mnt
  19. You are now up and running.  Update the system with aptitude if you like.  Keep in mind that if you update your kernel and you specified PV-bootloader-args, you will have to go back and update those accordingly.  I still have some issue like XenCenter not reporting the NIC IP and no performance data.  Hopefully that can be resolved soon so I can update this article.

Troubleshooting

I need to use xe-edit-bootloader.

To add grub2 support, edit /opt/xensource/bin/xe-edit-bootloader. On line 18 add the grub2 configuration files to the list.

default_file_list="/boot/grub/grub.cfg /grub/grub.cfg /boot/grub/menu.lst /grub/menu.lst"

After updating the kernel it continues to boot the old kernel.

If the new kernel is listed as the first and default entry in your grub file, make sure your PV-bootloader-args are not specified.  Retrieve the UUID of the VM using step 6 from the article.  Check the PV-bootloader-args parameter.

xe vm-param-list uuid=<UUID>

If you have specified PV-bootloader-args you need to update them to specify the new kernel.  You can use xe-edit-bootloader to find these values.  If you have decided to use pygrub to read the grub config at boot, then clear out the PV-bootloader-args and PV-args fields.

xe vm-param-set uuid=<UUID> PV-bootloader-args=
xe vm-param-set uuid=<UUID> PV-args=

Can’t I just use PV-kernel and PV-ramdisk?

These parameters on a VM are actually for when the kernel and ramdisk on on the XenServer (Dom0) filesystem.  The article here is for using the kernel inside the actual VM (DomU) itself.  For that, PV-bootloader-argsd is used.

Can’t type at login prompt.

I ran into one instance where I could see the login prompt but nothing I typed appeared.  First, make sure you click inside the console window.  If that does not resolve the issue, close and reopen XenCenter.  This fixed the issue when I encountered it.

PV is not working and I need to get the VM back up.

It is possible to get the VM back up if either PV mode will just not work or you need to go back and redo a step.  To do this simply shutdown the VM and respecify the HVM-boot-policy parameter. This parameter is case-sensitive!

xe vm-param-set uuid=<UUID> HVM-boot-policy="BIOS order"

To return to PV mode, clear the HVM-boot-policy parameter.

xe vm-param-set uuid=<UUID> HVM-boot-policy=

About Adam Oliver

Adam Oliver has been working in the IT field for over 10 years and is a Sales Engineer for Citrix Systems, Inc. Follow Adam on twitter at http://twitter.com/theadamoliver. Find out more about Citrix at www.citrix.com.
This entry was posted in Virtualization, XenServer and tagged , , . Bookmark the permalink.

6 Responses to ubuntu 10.04 beta server DomU on XenServer 5.6 beta

  1. Geek Rohit says:

    Hey, I just downloaded the Stable Ubuntu 10.04 server, tried out your guide, and it works!. Now it shows performance data like RAM usage, and disk speed , network performance, though it still doesnt show the NIC IP.

    Thanks for the great guide!

  2. pepoluan says:

    Thanks for the wonderful tip! 😀

    I simplified your step-by-step guide here:

    http://rapidshare.com/files/389459273/Inst_UbSrv1004_DomU_XenSrv55.rar (9 KiB)

    I also included a step to ‘fix’ XenTools in rc.d

  3. Zijad says:

    It worked perfectly!
    Many thanks!

  4. Bronko says:

    Hey, one more solution with disregarding of point 11 up to 14 (without Dom0 manipulation, because it’s not acceptable even more for a template creation):

    At point 4 downgrade from grub2 to grub:

    # aptitude purge grub2 grub-pc
    # aptitude install grub
    # update-grub
    # grub-install /dev/sda (yes, at these time still sda)

    Now you have to edit the well known /boot/grub/menu.lst. Comment the second entry after title line and add in this point the root line:

    # uuid xxxxxx
    root (hd0,0)

    Continue at point 5 and don’t forget to ignore point 11 up to 14.

    If you upgrade the kernel in the future leave your menu.lst during the installation process and manual edit the title, kernel and initrd line to the new kernel version.

    It works fine…

  5. Adam Oliver says:

    Bronko,

    You’re right, that should work perfectly. I prefer to modify the DomU as the changes are just scripts that read the grub files. No binary changes. I would also not want to downgrade a package in the VM which is why I provide the instructions on manually specifying the PV-args. Then no modification is required on either the host or guest.

  6. stiV says:

    if you use XenServer 5.6 (stable), you can forget about (4) and (11-14), pygrub can handle grub.cfg out of the box 😉

Leave a Reply

Your email address will not be published. Required fields are marked *