Commit 4f5489f0 authored by Leigh B. Stoller's avatar Leigh B. Stoller
Browse files

Text of email message describing custom images.

parent d9a0ceae
From: Leigh Stoller <>
To: "Dorsch, Matthew D." <>
Cc: Testbed Project <>
Subject: RE: Scheduling a test for all 40 PC nodes
In-Reply-To: <>
References: <>
X-Mailer: VM 6.92 under Emacs 20.7.1
FCC: ~/mail/outgoing
--text follows this line--
> From: "Dorsch, Matthew D." <>
> Subject: RE: Scheduling a test for all 40 PC nodes
> Date: Fri, 22 Jun 2001 11:25:55 -0400
> Is there a particular command availble to the NS file that would
> instruct the system to copy our image to the extra partition then boot it?
> The available docs indicate it is possible to boot my own image, but do not
> indicate the process for this.
Many parts of the custom OS support are in place, but there are still some
pieces missing. In addition, 40 nodes presents a special challenge because
of the amount of bandwidth needed to load 40 multigigbyte images.
First off, you use an NS command (an extension we added) to tell the system
what OS to boot. For example:
tb-set-node-os $nodeA RHL62-STD
Where RHL62-STD is what we call an OSID. There are several builtin OSIDs
(for our linux and freebsd images, and a few others). You will create an
OSID of your own to describe your custom OS. There is a web interface to do
It has not been used by anyone outside our group, so please let us know if
its confusing. The OSID you pick needs to be globally unique, and it will
let you know if you have picked something that is already used.
The OSID describes the OS. Next you need to describe the actual disk
image. We call that an IMAGEID :-) There is supposed to be a web interface
to do that, but I have not had time to gen one up. Basically, the IMAGEID
says where the disk file lives (usually in /proj), where to load it on the
disk (what slices, how many slices), and most importantly, what OSID is on
each slice. When the node boots, our first level boot program talks to the
testbed database to find out what slice to boot from. As you have probably
guessed, the tb-set-node-os command tells what OSID, and the IMAGEID maps
that to a particular slice on each node. If the web interface is not in
place by the time you get this far, I'll just stick the info in by hand for
you. Lets say you end up calling your OSID "RHL62-SANDS" and your IMAGEID
Okay, now you have your image loaded on one of our testbed machines and its
all ready to go. The thing to do now is snapshot the disk (or slice) using
our FS compression utility. Basically, this is a domain specific
compression program that knows how to read the linux filesystem, and
compress all of the allocated blocks, and skip all the unallocated blocks.
When you get this far, I will describe how to create this snapshot.
Once you have the snapshot, we have a special kernel that knows how to lay
the image back onto the disk, and its how we can load our 6.5 gig image in
just 88 seconds! The backend script to do that is called os_load, and you
invoke it like this:
os_load SANDS-RHL62-IMAGE pc34
You would use this command only to test that the image is created properly
and loading okay, by starting an experiment with a single node, running the
os_load, and then checking to see that it worked okay by rebooting (You can
change the OSID for a node after an experiment starts via the web
Now you are ready to put it all together. You would do this in your NS
tb-set-node-os $nodeM RHL62-SANDS
One of two things will happen at this point. Either one of us will preload
your image on all 40 nodes, or we will add the little bit of glue that
takes your tb-set-node-os command and autoloads the image on the nodes that
get allocated to the experiment. This isn't that hard to do, but to do it
so that the switch and network are not overwhelmed is slightly more
challenging. But, we will figure something out by then. Promise!
Note to testbed. I'm archiving this message in doc/custimage.txt
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment