This is a snapshot of the current source code for the software that runs
Netbed (Emulab++) at the University of Utah, as well as testbeds at other
sites:
http://www.emulab.net/
This is a snapshot of a rapidly-evolving system, rather than an engineered
release. It contains all Netbed source required to build a running testbed, but
interaction with Utah will almost certainly be required to get things up and
running.
Netbed is being actively developed, so there will be features on deployed
Netbeds that are not included in this snapshot. Contact us (contact info at the
bottom of this file) if you are interested in receiving an updated version of
this source.
IMPORTANT NOTE
See the file LICENSE for restrictions on this snapshot.
In summary, you can't redistribute it, use it for commercial
purposes, must give appropriate credit to Utah and Emulab,
and return improvements to us. These terms may become more
liberal in a future formal release.
HOW TO SETUP
Copy and localize the 'defs-example' file for your environment
and run "./configure". See doc/setup.txt for details.
This source currently only builds on FreeBSD, and requires several packages;
the dependencies can be found in the Makefile for our 'meta-port' in
install/ports/emulab-boss .
HOW TO UPDATE
Start by looking at doc/update-testbed.txt.
DOCUMENTATION
This is a snapshot, not a formal distribution, so don't expect
great documentation.
doc/* contains documentation for developers.
www/*.html and www/doc/*.html contain end-user documentation.
Good places to start would include:
- The doc/papers directory, which contains papers published about Netbed
- The user tutorial (www/tutorial/tutorial.html)
- The user FAQ (www/faq.html)
- The user authorization document (www/auth.html)
- Some powerpoint slides on Emulab internals, found at:
http://www.cs.utah.edu/flux/testbed-doc/internals/
These include two files especially useful for setting up your own
testbed: "build-operate.ppt" and the first part of "security-all.ppt".
- Software architecture preliminary diagram (doc/arch-diagram.ps)
- Notes on the source tree and architecture (doc/arch.txt)
- Overview of some major systems (doc/overview.txt)
- Diagrams and brief explanations of the state machines used in the
system (www/doc/states.html and www/doc/*.gif)
- The instructions for building a running system from scratch (doc/setup*.txt)
- Mail we have exchanged with others about hardware recommendations
(doc/hardware-mail.mbox)
QUICK TOUR
- The Database maintains most testbed state.
- LibDB (db/libdb.pm.in) is the main DB interface for testbed software.
- sql/database-create.sql contains the DB schema.
- The Web Interface (www/*) is the primary means of managing the system.
- startexp (tbsetup/startexp.in) is the backend for experiment creation;
Important scripts it calls include tbprerun (tbsetup/tbprerun.in)
and tbswap (tbsetup/tbswap.in).
- ns2ir (tbsetup/ns2ir/*) converts NS files to DB state (via TCL).
- assign (assign/*) maps experiments to available hardware.
- snmpit (tbsetup/snmpit.in) configures VLANs on switch hardware.
- frisbee (os/frisbee.redux/*) uses multicast to rapidly deploy disk images.
- tmcd (tmcd/*) is contacted by nodes as they boot up, to get configuration.
- capture and console (capture/*, tip/*) provide serial line access to nodes.
SOURCE TREE ORGANIZATION
account - Account creation and management
apache - Apache HTTPD configuration
assign - Resource allocation software
capture - Node serial line server software
cdrom - Software related to CD-booting local and remote machines
db - Variety of interfaces to testbed state
dhcpd - DHCPD configuration
doc - Documentation (mainly internal)
event - Event System
hyperviewer - 3D experiment visualization tool
install - Site installation scripts
ipod - "Ping of Death" software
ir - NS-to-database interpreter (in TCL)
lib - C/C++ interface libraries
mote - Initial testbed support for Berkeley/Crossbow motes
named - BIND configuration files
os - Client-side software (disk loading, route calculation, etc.)
patches - Patches needed to compile standard software packages
pxe - Software related to PXE-booting machines
rc.d - Server-side startup scripts
security - Tools for secure boss node access
sensors - Node monitoring daemons
sql - Database schema and initialization
ssl - SSL Certificate configurations
sysadmin - Apache Log Roller
tbsetup - Experiment creation and manipulation
testsuite - Automated testing suite
tip - Node serial line client software
tmcd - Testbed master control daemon and client (for node configuration)
tools - Misc. infrastructure tools
utils - Testbed configuration and maintenence tools
vis - Experiment visualization
www - Web Interface / User Documentation
xmlrpc - Programatic XML-RPC interface to testbed
HELP/CONTACT INFORMATION
For help and to report bugs and problems, mail
testbed-ops@flux.utah.edu
Don't be shy!
We are happy to get on the phone with you, too,
for "big picture" issues and problems.
Jay Lepreau, lepreau@cs.utah.edu, 801-581-4285, and the Utah testbed crew.
February 1, 2004
]
Timothy Stack
authored
Checkin some changes related to experiment automation and vnode feedback: * configure, configure.in: Add sensors/canaryd/feedbacklogs template. * db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment" table that holds environment variable names and values. * event/lib/event.c: Allocate memory of the right size for event_notifications. * event/program-agent/GNUmakefile.in: Add version.c file and add install targets for the man page. * event/program-agent/program-agent.8: Man page describing the program-agent daemon. * event/program-agent/program-agent.c: Add a bunch of convenience features: let the user specify the working directory for commands; save output to separate files on every invocation of an agent; let the user specify a timeout for a command; make the set of environment variables sane and add vars given in the NS file in the opt array; a "status" file containing process information is written out when children are collected. Internal changes: child processes are collected immediately, instead of waiting for the next START event, so we can send back COMPLETE events; the daemon now runs with a real-time priority, to increase the chances of receiving events. * event/proxy/evproxy.c: Made it bidirectional so the program-agent's COMPLETE events make it back to the scheduler. * event/sched/error-record.c: Change the default log directory. * event/sched/event-sched.h, event/sched/event-sched.c: Setup an environment similar to a program-agent to run the user's log digester. * event/sched/node-agent.cc: Add a handler for the SNAPSHOT event that runs create_image for the node. * event/sched/simulator-agent.h, event/sched/simulator-agent.cc: Let the user specify a "DIGESTER" script that digests the log files into a summary of the results. Add event handler for remapping a vnode experiment. * event/sched/timeline-agent.c: Accept the RUN event as well as the START event. * os/GNUmakefile.in: Install the install-tarfile.1 man page. * os/install-tarfile: Automatically chown/chgrp any files that do not have valid user or group IDs, the new owner will be the user that swapped in the experiment. Include the install directory in the DB file. Add a "list" mode that just dumps what files have been installed and where. Add a "force" option so the user can forcefully install the file, even though the DB says its already there. * os/install-tarfile.1: Man page describing the install-tarfile tool. * os/syncd/GNUmakefile.in: Install man pages on ops. * sensors/canaryd/GNUmakefile.in: Link canaryd statically and install "feedbacklogs" tool. * sensors/canaryd/canaryd.c: Dump dummynet pipe data. * sensors/canaryd/canarydEvents.c: Log errors. * sensors/canaryd/feedbacklogs.in: Tool used to generate feedback data from canaryd log files. * sensors/slothd/GNUmakefile.in: Install digest-slothd on ops. * sensors/slothd/digest-slothd: Fix some bugs and write out an "alert" file with all the nodes/links that were overloaded. * tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode" argument that lets you specify that you want to wait for the disk to finish loading and/or wait for the node to come back up in the new OS. * tbsetup/power.in: Remove debugging printf. * tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl, tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some quoting problems with event-sequences. Add -expected-exit-code and -tag options to the "$program run" event. Add -digester to the "$ns report" event that lets the user specify a program to run to digest the log files. * tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling factor for feedback nodes to 1%, instead of 100%. * tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that returns the values in "virt_user_environment". Return new program agent fields: dir, timeout, and expected_exit_code. * tmcd/common/GNUmakefile.in: Install rc.canaryd. * tmcd/common/bootvnodes: Add hack to boost the program-agents to a real-time priority, they can't do it from inside the jail. * tmcd/common/rc.canaryd: Rc script for canaryd. * tmcd/common/watchdog: Don't fail outright if there is a bad line in the battery.log * tmcd/common/rc.progagent: Append "userenv" data to the program-agent config file. * utils/GNUmakefile.in: Install loghole and its man page on ops. * utils/loghole.1: Document "clean" command and the change in loghole directories. * utils/loghole.in: Add "clean" command and parallelization. * xmlrpc/emulabserver.py.in: Add "virt_user_environment" table. Order the eventlist by "idx" and time, needed for sequences. And removed unnecessary nologin checks.