Foaa

This article is about setting up Endian 2.3 on a XEN machine without hardware virtualization (hvm).

We work in a shared office space. It’s none of those big anonymous spaces where you can rent in on a monthly basis, but it does not belong to us alone. We share it with p4930 (architects) and control-b (we work together). However, some of the architects are using windows machines (the rest of us either linux or mac) which got ill with a virus. This ugly bugger happend to be a spam-bot and sent out a lot of UCE via our VDSL uplink. The Telekom (T-Online, our internet provider) decided to block outgoing SMTP on port 25 (which was just the beginning of a lot of more problems), which was quite reasonable. Until then, our network setup was shared between the architects and our side (control-b and fortrabbit), using the architects uplink as fallback, if our VDSL goes offline. This was mainly the legacy of of the previous tenant, who wired the whole office with 100Mbit network hidden in nice panels around the office space. The main router was an old Linksys with DD-WRT installed, not fully capable to route the full downstream bandwidth of 50Mbit, but working and quite easy to configure.

Now it was time to go separate ways, and we decided to go with Endian on our site.

We wanted it to be a VM in our XEN based office server. The server does not support hardware virtualiszation so the “normal” approaches (HVM) did not work. So here is what we did..

Getting a XEN image of Endian

First off, you need to get VirtualBox (or VMware or such would work also). Then you download the current Endian 2.3 Community edition and install it in VirtualBox. This should be quite easy and no problems should occure. I gave it 4 Gigs of dynamic space, more would be better. We also did setup our network cards properly from the beginning. At least give it two NICs, one for the green (LAN) and red (WAN) interface. Now this is done, boot and install Endian on your VM. You could also go on with the configruation via browser, but i didnt.

Now begins the hard part. Shut down Endian and convert it’s harddisk (VDI-file) to an raw image (img-file).

#> VBoxManage internalcommands converttoraw ~/.VirtualBox/HardDisks/your-endian-image.vdi ~/Desktop/raw-image.img
VirtualBox Command Line Management Interface Version 3.0.10
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

Converting image "/home/uk/.VirtualBox/HardDisks/your-endian-image.vdi" with size 4294967296 bytes (4096MB) to raw...

This could take some time, depending on the size of your virtual harddisk (it writes of course the full length, even if you made it dynamic). After this is done, you need to dump the partitions of the raw-image file via sfdisk:

#> sfdisk -d ~/Desktop/raw-image.img
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
# partition table of /home/uk/Desktop/raw-image.img
unit: sectors

/home/uk/Desktop/raw-image.img1 : start=        1, size=  1030175, Id=83, bootable
/home/uk/Desktop/raw-image.img2 : start=  1030176, size=  1016064, Id=82
/home/uk/Desktop/raw-image.img3 : start=  2046240, size=   205632, Id=83
/home/uk/Desktop/raw-image.img4 : start=  2251872, size=  6136704, Id= 5
/home/uk/Desktop/raw-image.img5 : start=  2251873, size=  2060351, Id=83
/home/uk/Desktop/raw-image.img6 : start=  4312225, size=  4076351, Id=83

This shows all partitions Endian has created during installation. Now you have to extract the relevant partitions. Check with Endian and you’ll see:

root@endianbox:~ # mount
/dev/hda1 on / type ext3 (rw,noatime)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
/dev/hda6 on /var type ext3 (rw,mand,noatime)
/dev/hda3 on /var/efw type ext3 (rw,noatime)
/dev/hda5 on /var/log type ext3 (rw,noatime)
none on /dev/pts type devpts (rw,gid=5,mode=620)

root@endianbox:~ # df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda1               506964    423492     57720  89% /
/dev/hda6              2006124     82612   1821604   5% /var
/dev/hda3               101192      5000     91052   6% /var/efw
/dev/hda5              1013940     19416    943016   3% /var/log

Ok, this means:

• raw-image.img1 is /dev/hda1 is root (“/”)
• raw-image.img2 is swap
• raw-image.img3 is /dev/hda3 is /var/efw
• raw-image.img4 doesnt matter (extended)
• raw-image.img5 is /dev/hda5 is /var/log
• raw-image.img6 is /dev/hda6 is /var

Now extract all those images into single raw-files via dd:

#> dd if=~/Desktop/raw-image.img of=~/Desktop/raw-root.img skip=1 count=1030175
#> dd if=~/Desktop/raw-image.img of=~/Desktop/raw-var-efw.img skip=2046240 count=205632
#> dd if=~/Desktop/raw-image.img of=~/Desktop/raw-var-log.img skip=2251873 count=2060351
#> dd if=~/Desktop/raw-image.img of=~/Desktop/raw-var.img skip=4312225 count=4076351

Mount image-files

Now you have all img-files extracted you can mount them. Like this:

#> mount -o loop ~/Desktop/raw-root.img /mnt/tmp
#> mount -o loop ~/Desktop/raw-var.img /mnt/tmp/var
#> mount -o loop ~/Desktop/raw-var-efw.img /mnt/tmp/var/efw
#> mount -o loop ~/Desktop/raw-var-log.img /mnt/tmp/var/log

Now you could either go to Make some fixes and copy each image to your XEN machine or (like I did) put everything in a single new image file..

Create new disk (image file)

#> dd if=/dev/zero of=~/Desktop/endian-disk.img count=4096 bs=1M
#> losetup -f ~/Desktop/endian-disk.img

Check which loop-device your new disk got:

#> losetup -a
/dev/loop0: [fc03]:11552516 (/home/uk/Desktop/endian-disk.img)

Format your new disk as ext3:

#> mkfs.ext3 /dev/loop0

And then mount it and copy anything on it:

#> mount /dev/loop0 /mnt/tmp2
#> rsync -avp /mnt/tmp/ /mnt/tmp2/

(You could propably use cp -ra /mnt/tmp /mnt/tmp2 instead)

Make some fixes

Before unmounting the new disk, make some fixes. First off, edit /etc/businfotab, to fix the mappings to your network cards (obsolete PCI addresses, replace with XEN-network addresses):

#> cat /mnt/tmp2/etc/businfotab
# Generated by ethconfig
eth0    XEN-eth0
eth1    XEN-eth1
eth2    XEN-eth2

Edit the /etc/fstab file:

#> cat /mnt/tmp2/etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
none            /dev/pts        devpts  gid=5,mode=620  0       0
/dev/xvda1      none            swap    sw              0       0
/dev/xvda2      /               ext3    defaults,errors=remount-ro 0 1

Remove the obsolete /boot directory, we dont need this for XEN anymore!

#> rm -rf /mnt/tmp2/boot

(be sure to do this on this not on your PC working on )

And another one, you have to add the console to /mnt/tmp2/etc/inittab

#> cat /mnt/tmp2/etc/inittab
#
# $Id: inittab,v 1.6 2003/12/24 19:51:23 riddles Exp $
#
id:3:initdefault:

l0:0:wait:/etc/rc.d/rc.halt halt
l6:6:wait:/etc/rc.d/rc.halt reboot

si::sysinit:/etc/rc.d/rc.sysinit

# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -r now

# Run gettys in standard runlevels
1:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console xvc0
2:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console tty2
3:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console tty3
4:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console tty4
5:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console tty5
6:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console tty6

# s0:2345:respawn: /sbin/agetty -n -l /usr/sbin/efw-console -L 38400 ttyS0 vt100

The important part is the line

1:2345:respawn: /sbin/mingetty --autologin root --loginprog /usr/sbin/efw-console xvc0

with the xvc0 console. You could probably delete the lines with tty[2-6], but you could also just let them be and go on.

Copy kernel modules

Ok, it’s nearly done. All you have do to now is to copy the kernel modules you use in your XEN machine into the new device. They are in /lib/modules/your-kernel-version/ (eg /lib/modules/2.6.28-16-generic). Just copy the whole kernel-version-directory into the mounted volume under /mnt/tmp2/lib/modules.

Now just umount and detach the loop device

#> umount /mnt/tmp2
#> losetup -d /dev/loop0

Finish

Now you are nearly done. Copy the whole image to your XEN machine and setup the cfg-file for it. I prefered to have it in LVM, not as a loop-image, so i did before:

#xen-machine> lvcreate -n "endian-swap" -L 1G domu-storage
#xen-machine> lvcreate -n "endian-disk" -L 4G domu-storage
#xen-machine> cat endian-disk.img > /dev/domu-storage/endian-disk

If you prefer working with loop-images, just create a swap-file:

#xen-machine> dd if=/dev/zero of=/path/to/your/domus/endian-swap.img count=1024 bs=1M

Now create your new XEN config file, it should look like this (depending on your XEN-machine):

#xen-machine> cat /etc/xen/endian.cfg
#
# !! REPLACE WITH YOUR KERNEL !!
#
kernel      = '/boot/vmlinuz-2.6.26-bpo.1-xen-686'
ramdisk     = '/boot/initrd.img-2.6.26-bpo.1-xen-686'
memory      = '512'

#
#  Your disks
#
root        = '/dev/xvda2 ro'
disk        = [
       # WITH LVM:
       'phy:/dev/domu-storage/endian-swap,xvda1,w',
       'phy:/dev/domu-storage/endian-disk,xvda2,w',
       # WITH IMAGES:
       #'file:/path/to/domus/endian-swap.img,xvda1,w',
       #'file:/path/to/domus/endian-disk.img,xvda2,w',
 ]

#
#  Hostname
#
name        = 'endian'

#
#  Networking
#
vif         = [
       'ip=192.168.100.1,bridge=xen-br1,vifname=endian_green',
       'ip=192.168.1.10,bridge=xen-br0,vifname=endian_red'
]

#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

vcpus = 1
extra = '2 console=xvc0 xencons=xvc0'

And boot it with

#xen-machine> xm create /etc/xen/endian.cfg

All done..

Of course: no warranties or garranties of any kind. If something terrible happens to you or your hardware, dont blame me. Redistribute, whatever, please.

Links / More:

• http://www.neobiker.de/wiki/index.php?title=XEN_Kernel_EFW (german)
• http://www.heise.de/ct/projekte/c-t-Debian-Server-284111.html (german)

Changelog:

• Friday, November 13 2009: Update rsync command, added changes to /etc/inittab, changed fdisk -a to losetup -a