Category: Computers

Building an Ubuntu Live CD from Scratch

( This document was written in connection with the one day Live Cd workshop held at college of Engineering Attingal. )

1. Introduction

This document describes how to build a minimal live CD based on ubuntu 8.10.. The most recent version of this document can be found at http://brainstorms.in.

2. System Requirements

  1. A Host System with ubuntu 8.10 installed. ( Other versions will also work)
  2. Plenty of Disk space.
  3. A steady Internet connection or access to ubuntu repositories.
  4. Root access on the host system.

3 Exploring an existing ubuntu CD.

In order to understand the working of a live CD, let us look at an existing ubuntu CD

If you have the CD with you , put it into your CD drive. If you have an image , mount it on the loop device.

# mount -o loop ubuntu-8.10-desktop-i386.iso /cdrom

The directory structure of a typical ubuntu CD look as below.

autorun.inf dists isolinux pics preseed ubuntu wubi.exe

casper install md5sum.txt pool README.diskdefines umenu.exe

The exe files and autorun.inf are for installing under MS Windows. Some of the other files are self explanatory. Our main focus will be on isolinux and casper directories..

The isolinux directory contains files needed to boot from the live CD, Infact, isolinux belongs to a set of bootloaders written by Peter. H Anwin. The home page is here. As I mentioned isolinux , is the bootloader.. If you look at isolinux subdirectory, you will find several files. Some of them will be as shown below.

isolinux.bin isolinuc binary

isolinux.cfg Configuration File

boot.cat boot catlogue

bootlogo Logo

There are several other files. They are mainly for providing language support. ( Please feel free to look at some of them). The isolinux.cfg includes several other configuration files corresponding to differnt boot options. We will write a simple isolinux.cfg later.

The casper directory contains the following.

filesystem.manifest filesystem.squashfs vmlinuz

filesystem.manifest-desktop initrd.gz

vmlinuz and initrd.gz are the kernel and ram image from which isolinux will boot the system.

The filesystem.squashfs contains a compressed image of the filesystem. When we boot the live CD, this image will be uncompressed and moounted as the root filesystem.. All remastering programs and custom distros try to rebuild this image.. Filesystem,manifest and filesystem.manifest-desktop are text files which contain a list of packages inside the squashfs image.

The squashfs image can be loop mounted like an iso image.

# mount -o loop filesystem.squashfs /mnt /tmp

4 Preparing the host system.

Ensure that the Ubuntu 8.10 is installed on the host system. Log to the host system and update the system using apt. (I am assuming that you have direct access to Internet..)

a) Install necessary packages

apt-get install syslinux squashfs-tools mkisofs sbm

b) Prepare a working directory

Open a terminal on the host system, Become root.

$sudo su

#

Create a temporary work directory.

# mkdir work

Inside work create the following.

# mkdir work/filesys

#mkdir work/image

#mkdir work/image/casper

#mkdir work/image/isolinux

#mkdir work/image/install

We will be building the iso image from the image directory. We will populate the image directory one by one and finally build a CD image. The filesys directory is a temporary area where we will build the root filesystem of the Live CD.

5 Building a Squashfs image

For building the compressed root image,

we will build a bare minimum filesystem under filesys directory and then convert it into squashfs.

Step 1.

Install debootstrap on the host system

# apt-get install debootstrap.

Debootstrap is a program to bootstrap a basic debian system.

# cd work

# debootstrap –arch i386 intrepid filesys

This will create a bare minimum filesystem inside the filesys directory. If you look inside the filesys directory, you can see that the structure resembles root directory as shown below.

bin dev home lib mnt proc sbin sys usr vmlinuz
boot etc initrd.img media opt root srv tmp var

Step 2

Copy the /etc/resolv.conf and /etc/apt/sources.list to the corresponding locations inside the filesys directrory.

# cp /etc/resolv.conf filesys/etc/resolv.conf

# cp /etc/apt/sources.list filesys/etc/apt/sources.list
Step 3

Now, we will use chroot command to change the root directory to filesys.

# chroot filesys

You will get back the # prompt. However inside this terminal you will be referring to filesys directory as / .
Now run the following ( You are inside chroot )
# mount /proc

# mount /sys

# mount -t devpts none /dev/pts
The above commands mount the respective directories inside chroot.
Step 4

Update the system. (Note that we are inside the chroot jail not on your host system.)

# apt-get update

To avoid locale issues and in order to import GPG keys set the following shell variables.

#export HOME=/root

#export LC_ALL=C
Step 5

Install packages needed for Live System:

#apt-get install –yes ubuntu-standard casper

#apt-get install –yes discover  laptop-detect os-prober

#apt-get install –yes linux-generic

The package ubuntu-standard will install most of the packages needed for a working installation.

The other packages are mainly for hardware detection. Linux generic will install the kernel..

Step 6

Install additional  packages.

You can get the list of currently installed packages with

# dpkg -l

You can remove some of the packages and try to save space.

Step 7

Cleanup the unwanted files.

#apt-get clean
#rm -rf /tmp/*
#rm /etc/resolv.conf
#umount -l -f /proc

#umount -l -f /sys

#umount /dev/pts

#exit

Now you are out of chroot and your filesys directoy is ready to be compressed into a squashfs image.

Step 8

# mksquashfs filesys image/casper/filesystem.squashfs -e filesys/boot

Note: If the system you are building will only be used as a live system and you will never use it to install, you can exclude the /boot folder. The live system boots from outside the chroot and so the /boot folder is not used. If you will be using ubiquity to install the custom-built system, you will need to keep the /boot folder. Use this instead of the previous command to preserve the /boot folder.

# mksquashfs filesys image/casper/filesystem.squashfs.

6 Populate rest of the casper directory

Step 1

Copy kernel and initrd image.

#cp filesys/boot/vmlinuz-2.6.**-**-generic image/casper/vmlinuz

# cp filesys/boot/initrd.img-2.6.**-**-generic image/casper/initrd.gz

Step 2

Create manifest:

Run the following commands.

# chroot filesys  dpkg-query -W --showformat='${Package} ${Version}\n' |  tee image/casper/filesystem.manifest

#cp -v image/casper/filesystem.manifest{,-desktop}
# REMOVE=’ubiquity casper live-initramfs user-setup discover  xresprobe os-prober libdebian-installer4′
# for i in $REMOVE

do

sed -i “/${i}/d” image/casper/filesystem.manifest-desktop

done

Now all the files in the Casper directory are ready.

7 Building the isolinux directory

The isolinux directory contains files needed for booting the live CD.

Step 1

Copy isolinux and memtest to isolinux directory
#cp /usr/lib/syslinux/isolinux.bin image/isolinux/

#cp /boot/memtest86+.bin image/install/memtest
#cp /boot/sbm.img image/install/
Step 2

Create an isolinux.txt file in image/isolinux to display at boot time.

Eg.

This is a Live CD created at College of Engineering Attingal.For the default live system, enter “live”.
To verify the CD for errors, enter “check”.
To run memtest86+, enter “memtest”

Step 2a (optional)

If you put the name of a speacilly encoded image file file in the beginning of isolinux.txt ,it will be displayed when we boot up.

^Xsplash.rle

The filename should be preceded by ^X .

( To enter ^X in vi editor, do ctrl-V and follow with ctrl-X).

To create the splash.rle file, create an image 480 pixels wide, and convert it to 15 colors, indexed, using the gimp. Save it as .bmp.

Install the netpbm package.

#apt-get install netpbm

#bmptoppm splash.bmp > splash.ppm #ppmtolss16 '#ffffff=7' < splash.ppm > splash.rle

Step 3

Create an isolinux.cfg file in image/isolinux/  /usr/share/doc/syslinux/syslinux.doc is an idea place to start  looking for information on  various  configuration options. Here is an example  that you can use as a starting point.

DEFAULT live
LABEL live
menu label ^Start or install Ubuntu
kernel /casper/vmlinuz
append file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.gz quiet splash —
LABEL check
menu label ^Check CD for defects
kernel /casper/vmlinuz
append boot=casper integrity-check initrd=/casper/initrd.gz quiet splash —
LABEL memtest
menu label ^Memory test
kernel /install/memtest
append –
LABEL hd
menu label ^Boot from first hard disk
localboot 0x80
append –
DISPLAY isolinux.txt
TIMEOUT 300
PROMPT 1

8 Create miscellaneous files

Step 1

Create a diskdefines:

vi image/README.diskdefines

example:

#define DISKNAME Ubuntu 8.10 “Intrepid” – Release i386 **Remix**

#define TYPE binary

#define TYPEbinary 1

#define ARCH i386

#define ARCHi386 1

#define DISKNUM 1

#define DISKNUM1 1

#define TOTALNUM 0

#define TOTALNUM0 1

Step 2

Calculate MD5

# cd image && find . -type f -print0 | xargs -0 md5sum > md5sum.txt

9 Create and Test the CD image

Create iso image using the following command


#mkisofs -r -V "MyUbuntu" -cache-inodes -J -l -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ../ubuntu-remix.iso .

# cd ..

The boot.cat file will be automatically created.

Now, the image is ready for testing. You can install virtual box and mount the image as a cdrom and try booting from it.

10 To Do

Installing the live CD into a hard disk

Ubiquity.

One day workshop on Re-mastering Live Cds

I am planning to conduct a  one day workshop on re-mastering Ubuntu CDs on 25th of April 2009. The  agenda for the workshop will be

1)  Exploring the ubuntu Live CD
i) isolinux – How it works  Config files.
Customizing  boot splash and  boot messages.
ii) Casper Directory –
What is squashfs How to  mount it
iii) Other files

2) Building an Ubuntu CD from scratch
i) Requirements
ii)  Build  CD
iii) Test it with virtual box
iv) Unpacking an existing Live CD
v) Adding additional applications .

3)   Automating Re mastering
i) Remaster sys
ii) Reconstructor
iii) Miscellaneous topics. Usplash
4) Building a USB boot disk and carrying your distro

If you are interested   in attending,  email me.

Learning device drivers

I wrote this charater device driver for learning how they work.This is actually an assignment that we got today. .  Prof. TS Reddy  from JNTU helped me to debug the code.  I am planning to make a tutorial on it once I reach home. The course is becoming more and more exciting.  The internet access is very limited now. I will update when I have net access and free time.

Generating number sequences in shell scripts

I learned this technique from a fellow participant .

$ yes |nl |head -100 |cut -f1

You can remove the  new lines as below.

$  yes |nl |head -100 | cut -f1| tr ‘\n’ ‘\t’

Cool.

Otherwise, my course is progressing well. I wrote a simple file transfer daemon and implemented  the client..  Now, I have to do the same with pthreads and non blocking calls.  We had the first glimpse of kernel programming today when some kernel data structures for process and memory managment was iintroduced. I have to read  a lot to get the exact picture.

At JNTU Hyderabad for Prepare Future

I am presently attending a 2 weeks training program  conducted by  CDAC Hyderabad  at JNTU. This  is 2 weeks faculty updation program named Prepare future. I am attending Linux internals and driver development under system software track.

The course syllabus looks pretty impressive as given below.

  • Introduction to Linux System Programming
  • Linux Architecture
  • Linux Shell
  • GNU Tool Chain (GCC, GDB, MAKE, GPROF & GCONV)
  • System Calls and Working with Files
  • Linux Environment
  • Process Management & IPC
  • POSIX Compliant Thread Programming
  • Socket Programming
  • Linux Kernel Programming and Module Programming
  • Character Drivers
  • Concurrency, Race Conditions & IOCTL
  • Timers & Character Drivers
  • Block Drivers & Network Drivers
  • Introduction to USB Drivers

The course started today with an introduction to Linux architecture, system calls and programming interface. There  was nothing new to me, but I enjoyed the lecture by Ms Lakshmi  as  it refreshed my ideas about several concepts.   The second session was an introduction to shell,gcc and make.  It was not  a perfect lecture and the inexperience of the faculty was quite visible.

The afternoon was spent in the lab., with exercises on gcc make and gdb.  The lab sessions are held on  OpenSuse. In fact I am using it for the first time. As a long time debian user I have some discomfort.May be I will catch up by tomorrow.

The interesting thing about the program is that I have an option to sit in the lab as long as I like.  I feel like I was in CUSAT computer science department.

There are nearly 15 other participants. Most of them are from research organisations like CDAC , DRDO and DOE.  Even though the course is for faculty updation  I don’t find many faculty members attending the training. ( I am the only one from Kerala.)