Emulab.Net is an NSF/DARPA sponsored project, with additional support from these sponsors. As such, eligibility to use Emulab is primarily granted to other NSF/DARPA sponsored projects, as well as current university research projects. There are exceptions of course. If you are unsure about your eligibility to use Emulab, please feel free to send us an email inquiry.
If you are new to the Testbed, simply click on the "Start Project" link on the Emulab Home Page. You will need to fill in the forms with your personal information and information about the project. Then click on the "Submit" button. Within a few days you will be contacted via email with an approval message. More information about starting projects can be found in Authorization Page.
If you already have an Emulab account, and wish to start a second project, first log into the Web Interface. Then select the "Start Project" link; all of the personal information will already be filled in. You will need to complete just the project information section.
If you are new to the Testbed, simply click on the "Join Project" link on the Emulab Home Page. You will need to fill in the form with your personal information, and provide the name of the project you are trying to join (typically, the Project Leader will have told you the name of the project). Then click on the "Submit" button, and wait for the project leader to approve you. When approved you will receive an email message saying so, and you can then log into the Testbed.
Once you have been approved to start (or join) your first project, you will be able to log into Emulab's user machine, users.emulab.net. We require that all Emulab users use ssh. For example, if your Emulab account name is "joe", then you would do:
ssh users.emulab.net -l joe
Your password starts out the same as the password you initially supplied to the Start (or Join) web page.
There are several useful (although not required) programs installed on users.emulab.net in /usr/testbed/bin. You should edit your dot files to include that directory in your search path.
Yes. You may join (and/or start) as many projects as you like, subject to Emulab administrative policies.
Yes. You can change your Emulab Web password and your Emulab login password (the password you use to log into users.emulab.net, as well as nodes in your experiments). To change your Web password, simply click on the "Update User Information" in the menu to your left, and then enter your new password in the location provided. To change your login password, use the unix passwd utility when logged into users.emulab.net.
Yes. To designate a TA, you must first create a project group. A project group is a lot like a unix group, and in fact unix groups is the mechanism used to protect members of one group from members of another group. When you create a group, you designate a group leader who is responsible for approving users who apply to join the group. Group leaders may also terminate experiments that have been created by members of the group. As Project Leader, you may also shift members of your project in and out of your project's groups as you like, and you are automatically a member of all groups within your project. As a convenience, all new projects are created with one new group, termed the default group. As its name implies, whenever the group is left unspecified in a form, it defaults to the project group (this allows you to create a project without any sub groups at all; new members join the default group, new experiments are created in the default group, etc.).
Project groups are created via the Project Information link at your left. Simply go to the project page in which you want to create a group, and look for the "Create New Group" link. More information on project groups is available via the Emulab Documentation page in the Groups Tutorial.
If you cannot find an answer to your question in the Emulab Documentation, then you can send us an email message. We will try to answer your question as quickly as we can.
Yes, we have an extensive tutorial on using the Testbed.
Yes, we provide a GUI that gives you an easy to use drawing palette on which you can place nodes, lans, and links. Testbed specific attributes such as operating system, hardware type, and link/lan characteristics, may be attached to each object. With a single click, you can instantiate your new topology on the Testbed as an experiment in one of your projects. Alternatively, you can save the auto-generated NS file on your machine, edit as required, and then submit it later when creating an experiment.
To access the GUI, please log in and go to the Begin Experiment page. Note: you need a Java compliant browser.
You can ask for as many nodes as are currently available! You can click on the "Node Reservation Status" link at your left to see how many nodes are currently free. If you ask for more than are currently available, your experiment will be rejected (you will receive email notification shortly after you submit your NS file to the web interface).
We urge all new Emulab users to begin with a small 3-4 node experiment so that you will become familiar with NS syntax and the practical aspects of Emulab operation.
You can keep them as long as you need them, subject to our Node Usage Policies. In general, you should do your work, and then terminate your experiment as soon as you're done with it. If you're not done with it, but are through for a while, you should probably "swap out" your experiment (See the question What is Swapping in this FAQ). It is especially important to swap out your experiment if you're through with it for the weekend. Emulab usually gets heavy use on the weekends by users who need to make very large experiments, so it is important to leave as many nodes available as possible.
For example, say you need 50 nodes but there are only 40 free. In general, getting this many nodes is going to require intervention from Testbed Operations, if only so we can ask other experimenters to free up nodes, if possible. Please send us email if you are not able to able to get the number of nodes you need for your experiment.
Another alternative is to use the Batch System. If your experiment is amenable to being batched (does not require human intervention to start and stop), then you can submit a batch request, which will be serviced when enough nodes become available. Typically, you would start out with a few nodes, getting used to the batch system and creating whatever scripts are needed to make the experiment batchable. Then scale up to larger numbers of nodes. Thats the easiest way of getting a lot of nodes!
Yes. Project leaders get root access to all of the nodes in all of
the experiments that are running in their project. Project members
get root if their project leader grants them root access, when the
leader approves the group membership request.
Root privileges are granted via the sudo
command. The
tutorial describes
this in more detail.
Yes. Each of the PCs has its own serial console line that you can interact with using the unix tip utility. To "tip" to "pc1" in your experiment, ssh into users.emulab.net, and then type tip pc1 at the unix prompt. You may then interact with the serial console. The console output is also saved for each node so that you may look at it it later. For each node, the console log is stored as /var/log/tiplogs/pcXXX.run. This run file is created when nodes are first allocated to an experiment, and the unix permissions of the run files permit just members of the project to view them. When the nodes are deallocated, the run files are cleared, so if you want to save them, you must do so before terminating the experiment.
The Sharks also have serial console lines, but because of the limited number of serial ports available on users.emulab.net, only one Shark, the last or "eighth", on each shelf has a console line attached. To tip to that shark, you would type tip shXX at the unix prompt, where "XX" is the shark shelf number. The shark shelf number is the first digit in the name. Using shark sh16-8 as an example, the shelf number is sixteen, and the number of the node on the shelf is eight.
Yes. Each of the PCs is independently power controlled. If your node becomes wedged, or otherwise unresponsive, you can use the node_reboot command, as discussed in the Emulab Tutorial.
The Sharks are also power controlled, but because of the limited number of power ports available, the entire shelf of 8 sharks is on a single controller. The node_reboot does its best to cleanly reboot individual sharks, but if a single shark is unresponsive, the entire shelf will be power cycled.
If you manage to corrupt a disk (or slice), no worries. You can easily repair the damage yourself by reloading a fresh copy of the default disk image. You will of course lose anything you have stored on that disk; it is a good idea to store only data that can be easily recreated, or else store it in your project directory in /proj. Disk reloading is covered in more detail in the Emulab Tutorial.
Each project has its own directory, rooted at /proj, which is available via NFS to all of the nodes in experiments running in that project. For example, when the "RON" project was created, a directory called /proj/RON was also created. This directory is owned by the project creator, and is in the unix group "RON." Its permission (mode) is 770; read/write/execute permitted by the project creator and by all of the members of the project RON, but protected against all access by people outside the RON project.
Project members are encouraged to store any files needed by their experiments in the corresponding /proj project directory.
Yes. All of the files in your home directory on /users, and all of the files in your project directory in /proj are filesaved. While we can restore lost files in an emergency, we encourage you to back up critical data on your own to avoid (possibly long) delays in conducting your experiments.
No! The nodes in your experiment are not filesaved. Any changes
you make to the local filesystems will be lost if the event of a
disk failure. We plan to provide a mechanism for experimenters to
create snapshots of their node state, but that is not done yet. In
the meantime, any files that must not be lost should be stored in
the project directory (/proj/
Swapping is when you (or we) temporarily swap your experiment out, releasing all of the nodes in the experiment. Your experiment is still resident in the Emulab database, and you can see its status in the web interface, but no nodes are allocated. Once an experiment is swapped out, you can swap it back in via the web interface by going to the Experiment Information page for your experiment, and clicking on the swapin option.
The swappable checkbox in the Begin Experiment web page is used to determine what experiments can be automatically swapped by the testbed scheduling system. Note that all experiments are capable of being swapped; even if you do not check the swappable box, you are free to swap your own experiments as you like. The only difference is that the testbed scheduling system will not consider your experiment when looking for experiments to swap out. You will sometimes notice that the Experiment Information page does not contain the swap link. That is because experiments cannot be swapped when they are in transition. For example, when the experiment is being swapped in (say, after first being created) the link will disappear until the experiment is fully swapped in, and it is capable of being swapped out. You will need to occasionally reload the page so that the updated state is recognized and the swap link appears.
Be aware that we do not currently save any files that you may have placed on your nodes. When your experiment is swapped back in, you will likely get different nodes, and with fresh copies of the disk images. For that reason, you should not swap your experiment out unless you make arrangements to save and restore any state you need.
Make sure to take a look at our Node Usage Policies as well.
We have a command called portstats
that allows you access
to some of the port counters on our switches. To use it, you'll need
to ssh to users.emulab.net. 'portstats <pid>
<eid>
' will get you stats for all experimental interfaces in
your experiment. Run 'portstats -h
' to get a list of other
options, such as different sets of stats.
Note that the numbers returned by portstats
do not get
reset between experiments.
Please see the Hardware Overview page for a description and count of the computers that comprise the Testbed.
If you click on the "Node Reservation Status" link in the menu to your left, you will see a summary of the number of nodes (by type) that are currently available, followed by a listing of the reservation status of each individual node.
Yes! You can specify the delay, bandwidth, and packet loss rate between any two nodes in your topology. Bandwidth and delay are specified in the NS duplex-link statement, while packet loss rate is specified with the Emulab tb-set-link-loss extension to NS. You may also specify delay, bandwidth, and packet loss rate between nodes in a regular LAN.
Please see the Extensions page for a summary of all Emulab NS extensions, and the Emulab Tutorial for an example.
Yes! If your NS file specified traffic shaping on a link, then you can subsequently modify those parameters after the experiment has been swapped in. Note that you cannot convert a non shaped link into a shaped link; you can only modify the traffic shaping parameters of a link that is already being shaped. To modify the parameters, log into users.emulab.net and use the delay_config program. This program requires that you know the symbolic names of the individual links. This information is available via the web interface on the Experiment Information page. The command line syntax for delay_config will be displayed when the -h option is given.
Yes! However, access to those other parameters is slightly more difficult since you cannot specify them in your NS file. First off, you should log into users.emulab.net and read the man page for ipfw. Refer to the section on dummynet; ipfw is the user interface for the Dummynet traffic shaper. As noted in previous section above, you can alter the traffic shapping parameters of any delayed link (one in which you have specified a bandwidth, delay, or PLR that causes a delay node to be inserted). However, you will need to log into the delay node for the link you wish to modify and interact with ipfw directly. The easiest approach would be to make a copy of /etc/testbed/rc.delay and edit the pipe commands as desired (or replace the pipe commands with "queue" commands). The pipe commands are indexed by number; the mapping from pipe number to virtual link is available via the web interface on the Experiment Information page for your experiment. Be sure to leave the rest of the contents of the file as is. Once you have your changes made, simply execute the file using the sudo command.
Please see the Software Overview page for a description of the Operating Systems that can be run on each of the Testbed nodes.
When a choice of OS is available, you may specify which one you prefer for each node in the NS file using the Emulab tb-set-node-os extension to NS. When your experiment is configured, the appropriate disk image will be loaded on your nodes, and the selected operating system will boot up on each.
Please see the Extensions page for a summary of all Emulab NS extensions, and the Emulab Tutorial for an example.
Yes! If have an RPM (or more than one) that is appropriate for loading on the OS you have selected, you can arrange to have them loaded automatically when your experiment is configured. The Emulab NS extension tb-set-node-rpms is used in the NS file to specify a list of RPMS to install. You may specify a different list for each node in the experiment. When the node first boots after the experiment is configured, each of the RPMs will be installed (but only RPMs that have not already been installed).
Please see the Extensions page for a summary of all Emulab NS extensions, and the Emulab Tutorial for an example.
Yes! You can arrange to run a single program or script when your node boots. The script is run as the UID of the experiment creator, and is run after all other node configuration (including RPM installation) has completed. The exit status of the script (or program) is reported back and is made available for you to view in Experiment Information link in the menu at your left. The Emulab NS extension tb-set-node-startup is used in the NS file to specify the path of the script (or program) to run. You may specify a different program for each node in the experiment.
Please see the Extensions page for a summary of all Emulab NS extensions, and the Emulab Tutorial for an example.
By default, we do not setup any static routes or run any routing daemon on nodes in an experiment. However, we do provide several options for experimenters, which are described in the "Setting up IP routing between nodes" section of the Emulab Tutorial.
If your application requires synchronization to determine when all of the nodes in your experiment have started up and are ready to proceed, then you can use the Testbed's ready bits mechanism. The ready bits are really just a way of determining how many nodes have issued the ready command, and is returned to the application as a simple N of M string, where N is the number that have reported in, and M is the total number of nodes in the experiment. Applications can use this as a very simplistic form of barrier synchronization, albeit one that can be used just once and one that does not actually block!
Use of the ready bits is described in more detail in the Emulab Tutorial and in the Testbed Master Control Daemon documentation.
Yes! You can run your own OS (or a customized version of an Emulab supported OS) on any of the PCs. You can also run OSKit kernels on the PCs. Each of the PCs is partitioned with two DOS partitions large enough to hold the typical OS installation. The 1st and 2nd partitions are each 3GB. The 3rd partition is 500MB, and is labeled as Linux Swap. The 4th partition is the remainder of the disk, and varies in size depending on the pc type. We recommend that you use the 1st or 2nd partition; using the 4th partition will restrict the number of machines that you can run your OS on since it varies in size. Note that you must leave the MBR (Master Boot Record) in sector 0 alone, and that your custom partition must contain a proper DOS boot record in the first sector.
Please note that while users are free to customize their disks and install their own operating systems, Emulab staff will not be able to offer more than encouragement and advice! We cannot install the OS for you, and we cannot load CDROMS, floppy disks, or tape drives! We do provide an easy way for you to boot FreeBSD from a memory based filesystem (MFS) so that you can more easily work with the disk (in case it is not possible to install your OS on a live disk). Beyond that, you are pretty much on your own!
Many users had great success with customizing an Emulab supported OS (FreeBSD or Linux), and then creating a disk image that is autoloaded when the experiment is swapped in. We strongly encourage people to use this approach whenever possible! There is more information available in the Custom OS section of the Emulab Tutorial.
Each node has a partition at the end of the disk that you can use if
you wish. In Linux, the partition is /dev/hda4
; in FreeBSD,
it's /dev/ad0s4 . There is no filesystem on this partition,
so you'll need to create it yourself. For example, in Linux:
mkfs /dev/hda4;
mount /dev/hda4 /mnt;
This partition is only 6 Gigs, the size of the leftover space on our
smallest drives. If you need more space than this, it would be possible
to enlarge this partition on some machines (for example, our pc850s
have 40 GB disks,) but that is outside the scope of this FAQ.
By default, the testbed startup scripts currently start two daemons in addition to the OS's standard set. Other daemons may be started depending on the network services you ask for in your ns file (see below).
Unconditionally started daemons:
healthd
- A low overhead hardware health monitor.
This deamon periodically polls the machine's health monitoring
hardware and sends this information back to our boss
node for analysis. The hardware is polled once per second, and a
status datagram is sent out once every five minutes.
Healthd
's overhead is quite low, but it can be safely
killed and disabled from startup if you're worried about possible
side effects. It is started by
/etc/testbed/rc.healthd
.
slothd
- A low overhead usage analysis tool.
Slothd
is important to efficient testbed utilization
and should run on every node whenever possible. Its overhead is
almost negligible (essentially less than running 'ls -l
/dev'
once per hour), and should not interfere with your
work. However if your experiment is exceptionally sensitive, then
you may arrange with us to disable slothd
. Please
note that we will restart this daemon if it is not running unless
prior arrangements have been made.
Conditionally started daemons:
gated
- A network routing daemon.
If you have requested automatic routing on your nodes with the
tb-set-ip-routing
command in your NS file, this will
start gated
on all of your nodes.
We have left all daemons started by the operating systems' default
configurations (such as cron
) enabled, so you should also
look at them if you are concered about running processes affecting
your experiment.
Yes. Emulab blocks all of the low numbered ports (ports below 1024), with the exception of ports 20 and 21 (FTP), 22 (Secure Shell), and 80 (HTTP). This is for the protection of experimenters, as well as to ensure that an errant application cannot become the source of a Denial of Service attack to sites outside of Emulab. If your application requires external access to other low numbered ports, please contact us to make special arrangements.
The most common reason is that your topology includes nodes which are not directly connected, and you have not setup any routing. Refer to "How can I turn on routing or set up routes automatically in my nodes?" for details. If you cannot send packets between two machines which are directly connected (via a link or a lan), then there are two possibilities: either the nodes did not properly negotiate their speed and duplex with the Cisco switch, or the physical wire is loose or bad. In these cases, you should contact us for help.