How to setup diskless Ubuntu 12.04 with read-write root partition

Well, this how-to for the diskless Ubuntu setup is not usual. The main point is that all our infrastructure is based on CentOS, but sometimes (like in this case) we must support different Linux distributions for our clients. That is why all server-side configuration will be related to CentOS, and client-side to Ubuntu.

In difference to the official DisklessHowto, this one allows you to prepare a shared installation which can be used by any number of clients simultaneously. All changes to the mounted root will resist in ramfs, so no changes will be written to the NFS, and everything will be gone after restart. This feature in some (many) cases is treated as an advantage.

However, it is recommended to read  DisklessHowto from the Ubuntu site, since all basics related to network boot for Ubuntu described there much better. Also, I took some parts from their documentations to my howto. I believe they wouldn't mind.

Requirements

• DHCP server
• TFTP server
• NFS server

 All my servers uses CentOS, so I will describe server-side the configuration only for this system.
Current server setup is based on CentOS 6.2 x86_64 with IP 192.168.1.10.

Step by step

I. DHCP

I will not describe basics of DHCP configuration, you can easily google it. In order to make PXE work, you need to put just these two option to your DHCP configuration:

filename pxelinux.0;
next-server 192.168.1.10;

Where next-server provides with an address of your TFTP server. It can be put in almost any section of dhcpd.conf: global, class, subnet, pool, group or host.

II. TFTP

1. Install tftp daemon and syslinux package. Syslinux is available from rpmforge.

# yum install tftp-server syslinux

# chkconfig tftp on

# service tftp start 

2. Copy pxelinux files to tftp directory:

# cp /usr/share/syslinux/pxelinux.0 /tftpboot/
# cp /usr/share/syslinux/vesamenu.c32 /tftpboot/

3. Configure pxelinux

# mkdir /tftpboot/pxelinux.cfg

# cat << EOF > /tftpboot/pxelinux.cfg/default
DEFAULT vesamenu.c32
TIMEOUT 600
ONTIMEOUT BootLocal
PROMPT 0
MENU TITLE My PXE Server (by TORNADO)
ALLOWOPTIONS 1
menu width 80
menu rows 15
MENU TABMSGROW 24
MENU MARGIN 10
NOESCAPE 1
LABEL BootLocal
    localboot 0
    TEXT HELP
    Boot to local hard disk
    ENDTEXT
LABEL UBUNTU_1204_DISKLESS
    MENU LABEL Ubuntu 12.04 (64-bit) DISKLESS
    KERNEL Ubuntu/12.04/x86_64/vmlinuz-3.2.0-20-generic
    APPEND root=/dev/nfs nfsroot=192.168.1.10:/srv/nfsroot/Ubuntu/12.04/x86_64,ro initrd=Ubuntu/12.04/x86_64/initrd.img-3.2.0-20-generic ip=dhcp aufs=tmpfs
    TEXT HELP
    Boot the Ubuntu 12.04 64-bit Diskless
    ENDTEXT
EOF 

III. NFS

1. Install nfs

# yum install nfs-utils

# chkconfig nfs on

# service nfs start

2. Add /srv/nfsroot to exports

# cat << EOF > /etc/exports
/srv/nfsroot *(ro,async,no_root_squash,no_subtree_check,no_all_squash)
EOF

3. Apply exports

# exportfs -r

IV. Prepare installation

1. Install Ubuntu
Generally you have two ways.

1. Use debootstrap.
   I used this option to prepare some tiny installations (like network boot for number of POS terminals).
2. Install the Ubuntu on the real or virtual system and copy it to NFS server.
   In this article I will follow this way, because I want to prepare a usual Ubuntu installation.

After system was installed and configured as you want, you need to prepare it for network boot.

2. Modify /etc/network/interfaces to set eth0 configuration type to manual:

iface eth0 inet manual

3. Configure /etc/fstab to be looking like this:

# /etc/fstab: static file system information.
#
#                
proc            /proc           proc    defaults        0       0
/dev/nfs       /               nfs    defaults          1       1

4.  Change the following options in /etc/initramfs-tools/initramfs.conf:

MODULES=netboot
BOOT=nfs
DEVICE=eth0

NOTE: If the client source installation you copied the files from should remain bootable and usable from local hard disk, restore the former BOOT=local and MODULES=most options you changed in /etc/initramfs-tools/initramfs.conf. Otherwise, the first time you update the kernel image on the originating installation, the initram will be built for network boot, giving you "can't open /tmp/net-eth0.conf" and "kernel panic". Skip this step if you no longer need the source client installation.

5. Add to /etc/initramfs-tools/modules line:

aufs

6. Copy aufs module to /etc/initramfs-tools/scripts/modules

$ sudo cp /lib/modules/$(uname -r)/kernel/ubuntu/aufs/aufs.ko /etc/initramfs-tools/scripts/modules

7. Copy the following script to /etc/initramfs-tools/scripts/init-bottom as 00_aufs_init (0755):

#!/bin/sh -e

case $1 in
  prereqs)
    exit 0
    ;;
esac

for x in $(cat /proc/cmdline); do
  case $x in
    root=*)
      ROOTNAME=${x#root=}
      ;;
    aufs=*)
      UNION=${x#aufs=}
        case $UNION in
          LABEL=*)
            UNION="/dev/disk/by-label/${UNION#LABEL=}"
            ;;
          UUID=*)
            UNION="/dev/disk/by-uuid/${UNION#UUID=}"
            ;;
        esac    
      ;;
  esac
done

echo "Union=$UNION"

if [ -z "$UNION" ]; then
    exit 0
fi

modprobe -b aufs && echo "OK: modprobe -b aufs" || echo "ERR: modprobe -b aufs"

# make the mount points on the init root file system
mkdir /aufs /ro /rw && echo "OK: mkdir /aufs /ro /rw" || echo "ERR: mkdir /aufs /ro /rw"

# mount read-write file system
if [ "$UNION" = "tmpfs" ]; then
  mount -t tmpfs rw /rw -o noatime,mode=0755 && echo "OK:  mount -t tmpfs rw /rw -o noatime,mode=0755 " || echo "ERR:  mount -t tmpfs rw /rw -o noatime,mode=0755"
else
  mount $UNION /rw -o noatime
fi

# move real root out of the way
mount --move ${rootmnt} /ro && echo "OK: mount --move ${rootmnt} /ro" || echo "ERR: mount --move ${rootmnt} /ro"

mount -t aufs aufs /aufs -o noatime,dirs=/rw:/ro=ro && echo "OK: mount -t aufs aufs /aufs -o noatime,dirs=/rw:/ro=ro" || echo "ERR: mount -t aufs aufs /aufs -o noatime,dirs=/rw:/ro=ro"

# test for mount points on union file system
[ -d /aufs/ro ] || mkdir /aufs/ro
[ -d /aufs/rw ] || mkdir /aufs/rw

mount --move /ro /aufs/ro && echo "OK: mount --move /ro /aufs/ro" || echo "ERR: mount --move /ro /aufs/ro"
mount --move /rw /aufs/rw && echo "OK: mount --move /rw /aufs/rw" || echo "ERR: mount --move /rw /aufs/rw"

# strip fstab off of root partition
grep -v $ROOTNAME /aufs/ro/etc/fstab > /aufs/etc/fstab

mount --move /aufs /root && echo "OK: mount --move /aufs /root" || echo "ERR: mount --move /aufs /root"

exit 0 

To be honest this script isn't mine. I found it some time ago and I don't remember where. If you are the author or you know who he is, please tell me in comments so I put your name here.

8. Build a new initrd image:

$ sudo update-initramfs -k $(uname -r) -c -b /root/

9. Now you can copy this all on server by executing the following command on client:

$ sudo rsync -a --exclude=tmp/* --exclude=proc/* --exclude=sys/* --exclude=dev/* / username@192.168.1.10:/srv/nfsroot/Ubuntu/12.04/x86_64/

Be careful with all slashes! rsync treats source or destination in different way if slash is omitted.
"username" must have write permissions to /srv/nfsroot/Ubuntu/12.04/x86_64.

10. Copy kernel and new initrd image to a proper folder on tftp. Run it ON THE SERVER

# cp /srv/nfsroot/Ubuntu/12.04/x86_64/boot/vmlinuz-$(uname -r) username@192.168.1.10:/tftpboot/Ubuntu/12.04/x86_64/
# cp /srv/nfsroot/Ubuntu/12.04/x86_64/root/initrd.img-$(uname -r) username@192.168.1.10:/tftpboot/Ubuntu/12.04/x86_64/

Notes

Remember, that changes to the root filesystem is limited by you RAM. This means that you will not be able to copy a 4GB video to /tmp if you have only 2GB of RAM.

I have prepared this post without access to my test environment, so some small mistakes are possible. If you find any, please comment.