Commit ea264fe6 authored by Leigh B. Stoller's avatar Leigh B. Stoller
Browse files

Many weeks of updates. My eyes and fingers hurt.

parent 347ad1b2
stoller 2004/02/17 15:06:34 MST
Modified files:
tmcd/common bootvnodes rc.linktest rc.slothd
rc.testbed runstartup update vnodesetup
tmcd/freebsd delaysetup prepare rc.conf
rc.delayagent rc.kname rc.mfs
tmcd/freebsd/jail rc.injail rc.local
tmcd/linux delaysetup
prepare testbed
tmcd/plab rc.inplab
Added files:
tmcd/common ifsetup rc.bootsetup
tmcd/common/config rc.accounts rc.config rc.delays
rc.hostnames rc.ifconfig rc.keys rc.misc
rc.mounts rc.progagent rc.route rc.rpms
rc.simulator rc.startcmd rc.syncserver
rc.tarfiles rc.template rc.trafgen
Removed files:
tmcd/common bootsetup rc.agents rc.progagent rc.setup
tmcd/freebsd ifsetup
tmcd/freebsd/jail jailsetup
tmcd/plab plabsetup
Medium size cleanup of the client side code. The main impetus behind
this was to add soft reconfig support so that nodes could be
reconfigured without having to reboot them. This appears to work, and
has been tested with jails getting moved around. I've also tested the
new code on the MFS, but still no testing has been done on PLAB nodes.
The main change is that most of the code moved out of, and
was split into constituent rc scripts, each of which does its own
thing, including cleaning up and preparing for making an image. Most
of that central knowledge has been moved out into the scripts. Still more
to do but this was a good start.
Revision Changes Path
1.19 +7 -8 testbed/tmcd/common/
1.12 +105 -41 testbed/tmcd/common/bootvnodes
1.97 +304 -2246 testbed/tmcd/common/
1.9 +35 -1 testbed/tmcd/common/
1.2 +3 -0 testbed/tmcd/common/rc.linktest
1.3 +2 -1 testbed/tmcd/common/rc.slothd
1.2 +2 -2 testbed/tmcd/common/rc.testbed
1.5 +11 -4 testbed/tmcd/common/runstartup
1.7 +44 -70 testbed/tmcd/common/update
1.20 +32 -49 testbed/tmcd/common/vnodesetup
1.12 +4 -3 testbed/tmcd/common/watchdog
1.43 +7 -8 testbed/tmcd/freebsd/
1.13 +26 -31 testbed/tmcd/freebsd/delaysetup
1.42 +40 -15 testbed/tmcd/freebsd/
1.3 +3 -3 testbed/tmcd/freebsd/
1.27 +4 -1 testbed/tmcd/freebsd/prepare
1.4 +4 -2 testbed/tmcd/freebsd/rc.conf
1.9 +7 -5 testbed/tmcd/freebsd/rc.delayagent
1.2 +7 -0 testbed/tmcd/freebsd/rc.kname
1.6 +152 -38 testbed/tmcd/freebsd/rc.mfs
1.4 +1 -1 testbed/tmcd/freebsd/
1.4 +3 -5 testbed/tmcd/freebsd/
1.30 +146 -72 testbed/tmcd/freebsd/jail/
1.4 +138 -45 testbed/tmcd/freebsd/jail/rc.injail
1.5 +2 -2 testbed/tmcd/freebsd/jail/rc.local
1.38 +4 -4 testbed/tmcd/linux/
1.4 +30 -18 testbed/tmcd/linux/delaysetup
1.24 +31 -12 testbed/tmcd/linux/
1.24 +4 -1 testbed/tmcd/linux/prepare
1.5 +1 -1 testbed/tmcd/linux/
1.6 +1 -0 testbed/tmcd/linux/testbed
1.10 +4 -2 testbed/tmcd/plab/
1.5 +214 -37 testbed/tmcd/plab/rc.inplab
stoller 2004/02/16 08:34:35 MST
Modified files:
Add new -f flag to assign_wrapper, and have tbswap use that flag when
tbswap is doing a swapretry (this happens during a normal swapin, when
a node fails). In this case, we want to fix all the nodes (overriding
user setting) when replacing failed nodes!
Revision Changes Path
1.161 +17 -5 testbed/tbsetup/
1.34 +8 -5 testbed/tbsetup/
ricci 2004/02/13 15:42:46 MST
Modified files:
doc setup-cisco.txt
Recommend turning off MAC aging for the control network.
Revision Changes Path
1.4 +6 -0 testbed/doc/setup-cisco.txt
stoller 2004/02/12 14:26:13 MST
Modified files:
Fix problem with trivial links not getting lindelays. In the case that
a 100Mb link is mapped to a trivial link, a linkdelay has to be
inserted since the actual loopback bandwidth is much higher. The
problem is that unlike other delays where requires_delay() will return
true, this decision has to wait until after assign does the mapping
and we find out that a link was mapped to a trivial link. When this
happens, I insert a linkdelay "on the fly" when processing the link
statements from assign.
Revision Changes Path
1.160 +76 -35 testbed/tbsetup/
ricci 2004/02/12 10:46:22 MST
Modified files:
Unbreak vtypes by putting entries for them into the %node_type_linkbw
This means we have to call LoadVirtTypes() from a slightly different
place, so that we have the virt types before calling LoadPhysInfo(),
where %node_type_linkbw is built.
Revision Changes Path
1.159 +25 -1 testbed/tbsetup/
stoller 2004/02/12 08:43:37 MST
Modified files:
Removed files:
* Removed startexp, and merged its contents into batchexp. There has been
no reason for the separation for a long time, and it made maintence more
difficult cause of duplication between batchexp and startexp (batch was
the sole user of startexp). Cleaner solution.
* Check argument processing for batchexp, swapexp, endexp to make sure the
taint checks are correct. All three of these scripts will now be
available from ops. I especially watch the filename processing, which was
pretty loose before and could allow some to grab a file on boss by trying
to use it as an NS file (scripts all runs as user of course). The web
interface generates filenames that are hard to guess, so rather then
wrapping these scripts when invoked from ops, just allow the usual paths
(/proj, /groups, /users) but also /tmp/$uid-XXXXXX.nsfile pattern, which
should be hard enough to guess that users will not be able to get
anything they are not supposed to.
* Add -w (waitmode) options to all three scripts. In waitmode, the backend
detaches, but the parent remains waiting for the child to finish so it
can exit with the appropriate status (for scripting). The user can
interrupt (^C), but it has no effect on the backend; it just kills the
parent side that is waiting (backend is in a new session ID). Log outout
still goes to the file (available from web page) and is emailed.
Revision Changes Path
1.103 +4 -3 testbed/tbsetup/
1.45 +399 -62 testbed/tbsetup/
1.30 +59 -13 testbed/tbsetup/
1.52 +92 -15 testbed/tbsetup/
stoller 2004/02/12 08:37:10 MST
Modified files:
www beginexp.php3 beginexp_form.php3
beginexp_xml.php3 menu.php3 plab_ez.php3
www/buildui bui.php3
Make the new version of beginexp_xml live now that its been through
the paper effort for everyone. I've redirected everything to
beginexp_html.php3 (the front end to the XML version), including the
plab and bui pages.
I've stuck a redirect in the old beginexp.php3, to make it easy to
bring back the old version in case something goes wrong. Just comment
out the lines at the top and reinstall, along with changing the other
links to it.
Revision Changes Path
1.45 +7 -1 testbed/www/beginexp.php3
1.30 +2 -2 testbed/www/beginexp_form.php3
1.4 +3 -2 testbed/www/beginexp_xml.php3
1.132 +2 -8 testbed/www/menu.php3
1.6 +2 -2 testbed/www/plab_ez.php3
1.11 +2 -2 testbed/www/buildui/bui.php3
ricci 2004/02/11 16:42:23 MST
Modified files:
tmcd tmcd.c
Change the alias behavior in dohosts() . We used to only give aliases
for nodes that are not directly connected if we had a route to them.
Now, we give them to everyone.
This should make hosts file generation go much faster for large
Revision Changes Path
1.208 +16 -74 testbed/tmcd/tmcd.c
stoller 2004/02/11 15:04:15 MST
Modified files:
Kill off all of the old osid handling that nfree did. When a node is
released, all of def_boot_osid,temp_boot_osid,next_boot_osid is cleared.
bootinfo currently uses the nodes free/alloc status to determine if the
node gets PXEWAIT, but will eventually switch to looking at the osids;
A node with no osids set (as will be when free) will go into PXEWAIT. This
will allow os_load of free nodes!
Change temporary locked pid/eid to a real pid/eid (emulab-ops/nfree-locked)
to avoid transient problems in tmcd, where isalive would cause tmcd to
core dump cause the reserved table had a nonexistent pid/eid setting.
Revision Changes Path
1.74 +12 -30 testbed/db/
kwebb 2004/02/10 14:13:05 MST
Modified files:
. configure
tbsetup/ipassign README
tbsetup/ipassign/src Assigner.h
Added files:
tbsetup/ipassign/etc autocheck.conf
tbsetup/ipassign/src FixedPartition.h Partition.h
SearchPartition.h SquareRootPartition.h
Merge of ipassign stuff from commvirtsig-branch into main trunk.
Revision Changes Path
1.199 +1 -1 testbed/configure
1.206 +1 -1 testbed/
1.102 +10 -3 testbed/tbsetup/
1.3 +8 -4 testbed/tbsetup/ipassign/
1.4 +12 -9 testbed/tbsetup/ipassign/README
1.5 +10 -10 testbed/tbsetup/ipassign/
1.2 +0 -4 testbed/tbsetup/ipassign/src/
1.2 +3 -3 testbed/tbsetup/ipassign/src/Assigner.h
1.2 +9 -70 testbed/tbsetup/ipassign/src/
1.2 +4 -2 testbed/tbsetup/ipassign/src/ConservativeAssigner.h
1.2 +52 -71 testbed/tbsetup/ipassign/src/
1.2 +16 -7 testbed/tbsetup/ipassign/src/Framework.h
1.4 +33 -16 testbed/tbsetup/ipassign/src/
1.2 +1 -1 testbed/tbsetup/ipassign/src/
1.2 +1 -1 testbed/tbsetup/ipassign/src/
1.2 +1 -1 testbed/tbsetup/ipassign/src/
1.2 +6 -0 testbed/tbsetup/ipassign/src/
1.4 +3 -1 testbed/tbsetup/ipassign/src/
1.3 +28 -10 testbed/tbsetup/ipassign/src/
stoller 2004/02/09 16:21:02 MST
Modified files:
www newimageid_ez.php3 newosid.php3
Added files:
www osiddefs.php3
Minor cleanup of how we do features, osnames, and op_modes. Moved the
lists to a common "header" file, and change newosid and newimageid_ez
to grab from there. Also add notion of admin-only features (veths,
mlinks) and op_modes (NORMALv2) while I transition to new stuff.
Revision Changes Path
1.29 +43 -43 testbed/www/newimageid_ez.php3
1.14 +45 -28 testbed/www/newosid.php3
1.9 +38 -25 testbed/www/newosid_form.php3
stoller 2004/02/09 10:31:16 MST
Modified files:
os/syncd emulab-sync.c
Add SIGALRM handling to cut the connection timeout from usual 60
seconds it takes the kernel to decide the other machine is not
responding (as will happens as machines are rebooting), down to 5
seconds, using a SIGARLM.
Revision Changes Path
1.4 +37 -3 testbed/os/syncd/emulab-sync.c
stoller 2004/02/09 09:38:16 MST
Modified files:
os/syncd emulab-syncd.c
Add proper daemonization, which is turned off when -d (debug) flag
used. Add .pid file creation in /var/run to kill daemon from the new
rc.syncserver script in tmcd client code. Add -v flag to increase
server verbosity. Add -l argument to specify where to write the log
Revision Changes Path
1.3 +34 -20 testbed/os/syncd/emulab-syncd.c
duerig 2004/02/06 11:52:48 MST
Modified files: (Branch: commvirtsig-branch)
tbsetup/ipassign/src Assigner.h
Added files: (Branch: commvirtsig-branch)
tbsetup/ipassign/src FixedPartition.h Partition.h
SearchPartition.h SquareRootPartition.h
* Fixed netmask calculation bug.
* Added the option to search for a good partitioning
* Set up a pluggable partition framework for future changes
Revision Changes Path +10 -10 testbed/tbsetup/ipassign/ +0 -4 testbed/tbsetup/ipassign/src/ +3 -3 testbed/tbsetup/ipassign/src/Assigner.h +9 -70 testbed/tbsetup/ipassign/src/ +4 -2 testbed/tbsetup/ipassign/src/ConservativeAssigner.h +52 -71 testbed/tbsetup/ipassign/src/ +16 -7 testbed/tbsetup/ipassign/src/Framework.h +6 -0 testbed/tbsetup/ipassign/src/ +3 -1 testbed/tbsetup/ipassign/src/ +28 -10 testbed/tbsetup/ipassign/src/
stoller 2004/02/05 18:46:15 MST
Modified files: (Branch: commvirtsig-branch)
Hacky changes for node reconfiguration, committed to branch only.
When reboot not selected for swapmod, set the nodes to RES_RECONFIG in
tbswap. In os_setup, nodes in RES_RECONFIG get a reconfig instead of
the reboot. Note that if reconfig fails (say, cause the image has no
reconfig script in it!), it node_reboot falls back to plain reboot
(note, node_reboot -r does the reconfig).
Problem: Not sure that pcvm nodes will always be moved out of ISUP and
so the wait for ISUP might terminate too soon.
Also some minor changes to make os_setup see the cancel flag and
hopefully end a lot sooner. These could go into the main branch if
Revision Changes Path +34 -30 testbed/tbsetup/ +11 -5 testbed/tbsetup/
ricci 2004/02/05 15:09:32 MST
Modified files: (Branch: commvirtsig-branch)
Add the ability to run ipassign instead of staticroutes for route
Revision Changes Path +28 -4 testbed/tbsetup/
ricci 2004/02/05 10:42:14 MST
Modified files: (Branch: commvirtsig-branch)
Several changes:
Take a '-m' option to take the multiplexing factor given by the user,
instead of my lame hardcoded default.
Handle node-hints - not totally slick, we just pick one hint for the
conglomerate and go with it.
Make binpack_sort() aware of CPU and memory - this should help out
with Tim's experiments, in which that's what he's tweaking.
Data structure changes for desires so that we can do the above.
Some bufixes for the parser so that we can do node-hints.
Revision Changes Path +103 -49 testbed/tbsetup/
stoller 2004/02/05 09:57:15 MST
Modified files: (Branch: commvirtsig-branch)
Change bogus check against FBSD-STD, to a check of the osfeature list.
The point is determine if the osid a user has selected support
mulitplexed links and veths. This needs more work; I am currently
requiring both veth/mlink support whenever a nonvirt node is in a link
or lan with a virt node. This is pretty much what the code did before;
I just changed how the support is determined. Needs more work,
Revision Changes Path +20 -9 testbed/tbsetup/
stoller 2004/02/05 09:14:02 MST
Modified files:
sql database-create.sql database-migrate.txt
Add 'veths' and 'mlinks' to the list of possible osfeatures in
is_info table.
Revision Changes Path
1.225 +1 -1 testbed/sql/database-create.sql
1.118 +6 -0 testbed/sql/database-migrate.txt
kwebb 2004/02/04 20:36:52 MST
Modified files: (Branch: commvirtsig-branch)
ipassign_wrapper is now ready for prime time.
Fixed the nasty underconstrained clause in the virt_nodes ips column update
SQL clause that bit me yesterday/today.
Added logic to find the src ip address for a route, and fixed up a few more
things to prepare the script for integration.
Revision Changes Path +49 -16 testbed/tbsetup/ipassign/Attic/
ricci 2004/02/04 17:33:56 MST
Modified files: (Branch: commvirtsig-branch)
Handle fixed nodes. The main rule regarding fixed nodes is that we can
fix each conglomerate to at most one physical node. So, we can
1) Nodes that are not fixed
2) Nodes that are fixed, as long as they are fixed to the same place
3) Nodes that are fixed with nodes that are not fixed
Revision Changes Path +20 -5 testbed/tbsetup/
stoller 2004/02/04 17:14:41 MST
Modified files:
Minor whackery. Add "-r" option to node_reboot to "reconfig" a pnode
instead of rebooting. If the reconfig fails, fail through to reboot.
A reconfig will "halt" the current vnodes (or remove ones that are no
longer assigned to the node) reconfig the pnode, and then restart the
vnodes that are still assigned to the node (or create new ones for
newly assigned vnodes). A halt stops the vnode, but leaves the
vnode filesystem intact.
Not bothering to reconfig individual vnodes yet since thats pretty
much like a reboot of a vnode. Difference in time is tiny.
Revision Changes Path
1.49 +49 -6 testbed/tbsetup/
shash 2004/02/04 15:42:49 MST
Modified files: (Branch: commvirtsig-branch)
www modifyexp.php3
Added -e option to web modify to restart event system. Checked on by
default if the experiment is active.
Revision Changes Path +7 -1 testbed/www/modifyexp.php3
shash 2004/02/04 02:39:58 MST
Modified files: (Branch: commvirtsig-branch)
Yet another fix which causes trouble when vnodes move around pnodes
arbitrarily. The code even though was written a long time back wasn't
tested coz of doing fix-node during modify. Without fix-node, vnodes
move around like crazy even if the ns file never changes in any way
during a modify. With this fix, as many vnodes that were previously
reserved, as possible, will be reused as long as assign ends up
using the same PCs as were reserved before.
Revision Changes Path +75 -22 testbed/tbsetup/
kwebb 2004/02/03 16:16:30 MST
Modified files: (Branch: commvirtsig-branch)
Fixes to build/install of ipassign and related tools.
Revision Changes Path +2 -4 testbed/tbsetup/ipassign/ +33 -16 testbed/tbsetup/ipassign/src/ +1 -1 testbed/tbsetup/ipassign/src/ +1 -1 testbed/tbsetup/ipassign/src/ +1 -1 testbed/tbsetup/ipassign/src/
ricci 2004/02/03 14:29:02 MST
Modified files:
Add conglomeration of links as well as LANs - this can help assign's
runtime quite a bit, because it has less scoring work to do.
Add METIS as a coarsening algorithm - we shoot for making
conglomerates of size 10.
Revision Changes Path
1.2 +397 -36 testbed/tbsetup/
ricci 2004/02/03 11:37:35 MST
Modified files: (Branch: commvirtsig-branch)
Two changes when doing a swap-update:
Give all virtnodes the desire 'already_reserved' so that assign will
try to place them onto nodes the experimenter already has allocated.
Give assign 'node-hint's for nodes that were mapped before, so that
we can start from something that is presumably a pretty good
solution. Hopefully this will make assign converge faster and/or
improve the solution.
Revision Changes Path +19 -0 testbed/tbsetup/
ricci 2004/02/03 11:04:30 MST
Modified files: (Branch: commvirtsig-branch)
Add an 'already_reserved' feature to nodes we already have reserved
if an $exempt_eid was given - this is so that assign can use them in
preference to allocating new nodes.
Revision Changes Path +15 -2 testbed/tbsetup/
stoller 2004/02/03 10:46:14 MST
Modified files: (Branch: commvirtsig-branch)
Two changes for the commvirtsig branch only.
* Set the node cmd_line to the appropriate kernel when selecting a
jail or delay node host. Currently, jail (and sim) phosts get
/kernel.jail, and delay nodes get /kernel.delay. This is passed to
pxeboot via bootinfo, which boots the proper kernel, thus avoiding
the double reboot we all know and loathe.
WARNING: You have to use new (FBSD47-UPDATE) images for this to
actually work right, since the current default inage does not rename
the kernel after reboot. Mostly this will not be a problem, but it
might be.
Be sure to use tb-set-jail-os and tb-set-delay-os in your NS files
too (to FBSD47-UPDATE at least) for the same reason.
* My requires_delay() change from many weeks ago that needs more
testing before committing to main branch. Have faith all ye little
guinea pigs.
Revision Changes Path +126 -36 testbed/tbsetup/
stoller 2004/02/03 09:46:31 MST
Modified files:
tbsetup/ns2ir nstb_compat.tcl
Add tb-set-jail-os and tb-set-delay-os to set the experiment wide
jail and delay node OSIDs. When set, will override anything in the
node_types table. Use with caution.
Revision Changes Path
1.16 +3 -0 testbed/tbsetup/ns2ir/nstb_compat.tcl
1.47 +4 -0 testbed/tbsetup/ns2ir/
1.63 +11 -0 testbed/tbsetup/ns2ir/
1.62 +28 -0 testbed/tbsetup/ns2ir/
shash 2004/02/02 17:51:08 MST
Modified files: (Branch: commvirtsig-branch)
A new tb-* command tb-fix-current-resources 0/1 tells assign_wrapper
whether to do a fix-node of current resources or not. This is useful
when virtual node weights are changed during a swap modify. For
example, when a virtual node had a CPU desire of 10 and it became
800 during a swap modify, doing a fix-node on previous allocation
won't let assign do the correct thing and will sometimes cause
In order to support this, we need to move current reserved
pnodes/virtnodes to a holding experiment emulab-ops/oldreserved
and then back to current experiment. This was the solution that
worked best in the face of UNIQUE KEY issues in the reserved
table. The reserved table needs the combination pid-eid-vname
to be unique.
Revision Changes Path