This HOWTO shows how oneSIS can be used to manage many diskless clients from a single NFSroot filesystem. For more information on network booting and NFSroot, refer to the NFSroot HOWTO.

This HOWTO is broken up into several sections. The sections range in scope from how to set up a very simple system consisting of a single class of diskless nodes to setting up more complex systems and detailing some tricks you can do with oneSIS. For additional information, be sure to refer to the oneSIS Administrator's Manual.

Create a root image

Before any nodes can boot into an NFSroot image, we'll need to create the image that the nodes are going to use. To do this we'll simply copy the root filesystem from any installed linux machine. We could copy the local machine's root, but lets assume that there is another machine on the network that has a freshly installed version of Redhat Enterprise Linux 3 Update 4 that we want to use. The actual distribution doesn't matter to oneSIS, or to the copy-rootfs command. For this example, lets assume the machine we want to duplicate the image of is called 'rook', and root has ssh keys set up to access that machine without entering a password.

Example 1: Copying the root image from a remote machine

# copy-rootfs -r rook /var/lib/oneSIS/image

This may take a while depending on the speed of the network between the machines. A typical root image can be anywhere from 1 to 6 gigabytes. The entire root filesystem of the remote machine will be copied to /var/lib/oneSIS/image on the local machine. Note: if the remote machine uses a dynamic /dev filesystem such as udev or devfs, the contents of that /dev filesystem will not be copied over. If that is the case, you will need to manually populate /dev in your image with some static device files, or some stuff may not work at boot time. This will be fixed in future releases of oneSIS.

Define the distribution

Configuring the target distribution is one things that must be done in every oneSIS configuration. oneSIS needs to know which linux distribution the image is built on so it can alter the boot scripts to use a read-only root filesystem. Each distribution does it differently. The 'distro-patches' directory included with oneSIS contains patches for many distributions. Each patch has a distro name, and some have a distro version. In our example, we are using Redhat Enterprise 3 Update 4, so we need to specify that as follows with a DISTRO directive in the oneSIS configuration file of the image, /etc/sysimage.conf. In this case, we would edit /var/lib/oneSIS/image/etc/sysimage.conf.

Example 2: Specify your linux distribution

# /etc/sysimage.conf

The oneSIS patch file for RedHat Enterprise 3 Update 4 is named redhat-el-ws3-u4.patch. Note how it corresponds to our DISTRO directive. Everything up to the first dash is the distro name. Everything after the first dash (except .patch) is the distro version. Capitalization is ignored in the DISTRO directive -- it assumes the filename of the distro patch is all lowercase. Now let's run mk-sysimage on our image to make sure everything is OK. You should see something similar to this:

# mk-sysimage /var/lib/oneSIS/image

oneSIS: Applying patch: /usr/share/oneSIS/distro-patches/redhat-el-ws3-u4.patch
oneSIS: Altering file: /var/lib/oneSIS/image/etc/inittab

This means that the distro patch was successful. Your NFSroot nodes will now come up with a read-only root filesytem.

If there is no patch for your target linux distribution included with oneSIS you can either read this for info on porting oneSIS to your distribution or send a request to someone on the onesis-users mailing list to create a patch for your distribution. Great. The worst should now be over with.

Decide on a runlevel

Many linux distributions are commonly configured to run X-windows when they first come up. Most distributions (except for Gentoo) control this by simply by changing the default runlevel. If you don't want your client nodes to try to run X11, edit your /etc/inittab and change the default runlevel to 3.

Example 3: Changing the default runlevel

# /etc/inittab

Disable unnecessary services

Many linux distributions these days come with a whole lot of daemons running that are both unnecessary and unwanted on NFSroot clients. As a very first step, lets disable most of those undesired daemons. Each distribution does this differently. Redhat has the 'chkconfig' utility to help with this. Suse uses 'insserv'. Take the time to go through the entire list of start scripts and turn off as many as you can get away with. Note: some of these are actually important: it is up to you to determine which ones are needed and which are not. Be aware that the fewer services you have running, the easier things are down the road.

Configure some writable directories

One task that is generally global for every node is to create some writable directories for the distribution's boot scripts to store temporary files. If you were to boot your nodes NFSroot now, they may come up just fine, but you'll probably notice many errors on the console such as these:

Example 4: 'Read-only filesystem' errors

/etc/rc.d/rc.sysinit: line 661: /var/run/utmp: Read-only file system
touch: creating `/var/log/wtmp': Read-only file system
touch: creating `/var/run/': Read-only file system
touch: creating `/var/lock/subsys/sshd': Read-only file system
chmod: changing permissions of `/var/run/utmp': Read-only file system
/etc/rc.d/rc.sysinit: line 819: /var/log/dmesg: Read-only file system

These 'Read-only file system' errors come up because common applications and start scripts need to write to places such as /var/run to store temporary files. Lets make sure that these directories are writable by linking them into /ram.

Example 5: Getting rid of 'Read-only filesystem' errors

# /etc/sysimage.conf
LINKDIR /var/run
LINKDIR /var/log
LINKDIR /var/lock/subsys

Run mk-sysimage on your image again:

# mk-sysimage /var/lib/oneSIS/image

oneSIS: Creating LINKDIR: /var/lib/oneSIS/image/var/log
oneSIS: Creating LINKDIR: /var/lib/oneSIS/image/var/run
oneSIS: Creating LINKDIR: /var/lib/oneSIS/image/var/lock/subsys

Now, when a node boots, it will be able to write into these directories. Since any files created in these directories are actually being written into RAM, they won't be persistent across a reboot, but that's OK for these files.

Proceed with configuring oneSIS

There are many ways to use oneSIS to manage your environment. There are also many tricks you can do with oneSIS that may not be obvious at first. The rest of these pages are dedicated to showing some examples of what oneSIS can do, and how to use it to set up your environment. It may prove useful to read through the simpler setups even if you are planning on building a more complex environment. If you run into any trouble, don't hesitate to send questions to

  • The simplest setup: one class of diskless nodes
  • A typical setup for a cluster image
  • More to come soon ...



    Copyright © 2004-2007 Sandia Corp. All rights reserved.
    Last Modified: 05/7/05