* remove check for gather arg, use TESTINFO from tmcc hwinfo
  instead. Only call tmcc once. Let non-MFS boot collect data.
* tmcc hwinfo not return a TESTINFO line. Handle the LOGDIR, COLLECT
  and CHECK indicators
* Do to variable scoping make sure everything runs in the shell.
  Do not 'exit' from called programs. Source called programs instead
  of starting new shells.
* Nothing wrong with the which() function I put in since the
  MFS/busybox installation does not have one, but if we have a real
  which(1) program use it.
* output format changes and programming space changes
* remove check for gather arg, use TESTINFO from tmcc hwinfo
  instead. Only call tmcc once. Let non-MFS boot collect data.
* tmcc hwinfo not returning a TESTINFO line. Handle the LOGDIR, COLLECT
  and CHECK indicators
* Do to variable scoping make sure everything runs in the shell.
* Do not 'exit' from called programs. Source called programs instead of
  starting new shells.
* Nothing wrong with the which() function I put in since the MFS/busybox
  installation does not have one, but if we have a real which(1) program
  use it.
* call sub-programs by sourcing not by starting new shell.
* oops mfsmode is not set until initialize is run. also set -e -u to
  catch this type of thing.

Checknode tests once again run in standard images and can collect
data.

* All tests can now be "sourced" from gatherinv to insure everything
  runs in a single shell.
* All tests can also be called standalone.
* MFS mode and non-MFS mode honor the CHECK and COLLECT flags.
* Create, if necessary, a new project save directory.
* This will be needed once every-time checknodes is run in a new project.
  path is /proj/{pid}/nodecheck.
* If script can't run quit with a 'return 0' so the rest of the checks
  can continue.

Change filename gatherinv -> checknode

* Can't run in MFS mode if not in the emulab-ops pid

Merge rc.config changes into the real script.
Reduce rc.nodecheck varients to a single shell script and get rid of rc subdir.
Add copyrights.
Add GNUmakefile install target (but not used yet).
Replace some hardwired paths with vars from paths.sh.

script, which I am using from the XEN code to localize a guest.

In order to hook in via the "generic vnode" path for setting up
blockstores under FreeNAS, I've done a bit of shuffling in order to
make things more OS-independent and reusable.

* mkvnode.pl

Moved to clientside/tmcc/common.  OS-dependent bits (really only some
IPtables stuff) abstracted, and moved to tmcc/linux/libvnode.pm.

* libvnode.pm

Moved generic vnode stuff to a new module.  Moved miscellaneous
utility functions to a new module.  Left OS-specific stuff.  Not
really sure if what is left should be merged into libsetup/liblocsetup
or left here - deferring this decision for now.

* libgenvnode.pm

New module currently containing generic vnode stuff.  Currently, the
VNODE_* predicates are here.

* libutil.pm

New module containing miscellaneous utility functions (fatal,
mysystem, mysystem2, setState, etc.)

Files referencing libvnode.pm have been updated, as have the relevant
Makefiles.

This officially drops the pretense that fs nodes can operate with minimal
Emulab software. If you have a seperate fs node, it had better be dedicated
to Emulab!

However, it still doesn't do everything. In particular, accounts are not
installed. This has never been needed for serving NFS, but is needed for
the samba stuff to work correctly.

Also, you cannot do an fs node software install from boss yet as we do not
mount fs filesystems on boss. You really cannot do a full ops install from
boss either since we don't mount ops' /usr/local/etc/emulab directory.

It works like this. Certain nodes that are on the node control net
(right now just subbosses, but ops coming soon) can set static ARP entries
for the nodes they serve. This raises the bar for (but does not eliminate
the possibility of) nodes spoofing servers. Currently this is only for
FreeBSD.

When such a server boots, it will early on run /etc/rc.d/arplock.sh
which will in turn run /usr/local/etc/emulab/fixarpinfo. fixarpinfo
asks boss via an SSL tmcc call for "arpinfo" (using SSL ensures that the
info coming back is really from boss). Tmcd on boss returns such arpinfo
as appropriate for the node (subboss, ops, fs, etc.) along with the type
of lockdown being done. The script uses this info to update the ARP
cache on the machine, adding, removing, or making permanent entries
as appropriate.

fixarpinfo is intended to be called not just at boot, but also whenever
we might need to update the ARP info on a server. The only other use right
now is in subboss_dhcpd_makeconf which is called whenever DHCP info may
need to be changed on a subboss (we hook this because a call to this script
might also indicate a change in the set of nodes served by the subboss).
In the future, fixarpinfo might be called from the newnode path (for ops/fs,
when a node is added to the testbed), the deletenode path, or maybe from
the watchdog (if we started locking down arp entries on experiment nodes)

The type of the lockdown is controlled by a sitevar on boss,
general/arplockdown, which can be set to 'none', 'static' or 'staticonly'.
'none' means do nothing, 'static' means just create static arp entries
for the given nodes but continue to dynamically arp for others, and
'staticonly' means use only this set of static arp entries and disable
dynamic arp on the control net interface. The last implies that the server
will only be able to talk to the set of nodes for which it got ARP info.

As mentioned, tmcd is responsible for returning the correct set of arp
info for a given request. The logic currently is:

 * Only return ARP info to nodes which are on the CONTROL_NETWORK.
   If the requester is elsewhere (e.g., Utah's boss and ops are currently
   segregated on different IP subnets) then this whole infrastructure
   does not apply and nothing is returned.

 * If the requester is a subboss, return info for all other servers that
   are on the node control network as well as for the set of nodes
   which the subboss serves.

 * If the requester is an ops or fs node, again return info for all
   other servers and info for all testnodes or virtnodes whose control
   net IP is on the node control net.

 * Otherwise, return nothing.

One final note is that the ARP info for servers such as boss/ops/fs or
the gateway router is not readily available in most Emulab instances
since those machines are not in the DB nodes or interfaces tables.
Eventually we will fix that, but for now the info must come from new
site variables. To help initially populate those variables, I added
the utils/update_sitevars script which attempts to determine which
servers are on the node control net and gathers the appropriate IP and
MAC info from them.

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* statements, the licensing status of
the source files was unclear.

This commit is intended to clear up that confusion.

Most Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the Affero GNU General Public License, version 3
(AGPLv3).

Most Utah-copyrighted files related to ProtoGENI are distributed under
the terms of the GENI Public License, which is a BSD-like open-source
license.

Some Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the GNU Lesser General Public License, version 2.1
(LGPL).

directory.

Also add support for talking to non elvin-compat nodes on other CMs by
having the event clients contact another pubsubd instance on the home
boss, which will gateway the events back and forth, munging them as
needed.

 * added new tmcd directory with Ubuntu (really, Debian) specifics
 * fixed up GNUmakefiles to not do "-g wheel" when creating directories
 * other, relatively minor tweaks

    from link-agent to provide dynamic link modification.  Calls
    liblocsetup::os_ifdynconfig_cmds to get a list of commands to change
    the iface config; if this function does not exist, it tries to do its
    best via ifconfig.

* tmcd/linux/liblocsetup.pm: Added support for dynamic configuration of
    ath and flex900 ifacetypes.  Also added more static support for both,
    including adhoc/monitor modes, rts threshold, etc, for ath; carrier
    threshold and rx gain for flex900 (sometimes these params really need
    to be set so our gnu radios can actually talk...).

* event/link-agent/link-agent.c: Do link up/down/modify by calling
    ifdynconfig with the event params.  Backwards compat with old
    functionality.

being taken.

The basic strategy is to have node_reboot (when -p option supplied)
invoke a special command on the node that will cause the shutdown
procedure to run prepare as it goes single user, but before the
network is turned off and the machine rebooted. The output of the
prepare run is capture and send back via the tmcd BOOTLOG command and
stored in the DB, so that create_image can dump that to the logfile
(so that the person taking the image can know for certain that the
prepare ran and finished okay).

On linux this is pretty easy to arrange since reboot is actually
shutdown and shutdown runs the K scripts in /etc/rc.d/rc6.d, and at
the end the node is basically single user mode. I just added a new
script to run prepare and send back the output.

On FreeBSD this is a lot harder since there are no decent hooks.
Instead, I had to hack up init (see tmcd/freebsd/init/{4,5,6}) with
some simple code that looks for a command to run instead of going to a
single user shell. The command (script) runs prepare, sends the output
back to tmcd, and then does a real reboot.

Okay, so how to get -p passed to node_reboot? I hacked up the
libadminmfs code slightly to do that, with new 'prepare' argument
option. This may not be the best approach; might have to do this as a
real state transition if problems develop. I will wait and see.

Also, I changed www/loadimage.php3 to spew the output of the
create_image to the browser.

Checkin some changes related to experiment automation and vnode feedback:

	* configure, configure.in: Add sensors/canaryd/feedbacklogs
	template.

	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
	table that holds environment variable names and values.

	* event/lib/event.c: Allocate memory of the right size for
	event_notifications.

	* event/program-agent/GNUmakefile.in: Add version.c file and
	add install targets for the man page.

	* event/program-agent/program-agent.8: Man page describing the
	program-agent daemon.

	* event/program-agent/program-agent.c: Add a bunch of convenience
	features: let the user specify the working directory for commands;
	save output to separate files on every invocation of an agent; let
	the user specify a timeout for a command; make the set of
	environment variables sane and add vars given in the NS file in
	the opt array; a "status" file containing process information is
	written out when children are collected.  Internal changes: child
	processes are collected immediately, instead of waiting for the
	next START event, so we can send back COMPLETE events; the daemon
	now runs with a real-time priority, to increase the chances of
	receiving events.

	* event/proxy/evproxy.c: Made it bidirectional so the
	program-agent's COMPLETE events make it back to the scheduler.

	* event/sched/error-record.c: Change the default log directory.

	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
	environment similar to a program-agent to run the user's log
	digester.

	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
	that runs create_image for the node.

	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
	Let the user specify a "DIGESTER" script that digests the log
	files into a summary of the results.  Add event handler for
	remapping a vnode experiment.

	* event/sched/timeline-agent.c: Accept the RUN event as well as
	the START event.

	* os/GNUmakefile.in: Install the install-tarfile.1 man page.

	* os/install-tarfile: Automatically chown/chgrp any files that do
	not have valid user or group IDs, the new owner will be the user
	that swapped in the experiment.  Include the install directory in
	the DB file.  Add a "list" mode that just dumps what files have
	been installed and where.  Add a "force" option so the user can
	forcefully install the file, even though the DB says its already
	there.

	* os/install-tarfile.1: Man page describing the install-tarfile
	tool.

	* os/syncd/GNUmakefile.in: Install man pages on ops.

	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
	install "feedbacklogs" tool.

	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.

	* sensors/canaryd/canarydEvents.c: Log errors.

	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
	data from canaryd log files.

	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.

	* sensors/slothd/digest-slothd: Fix some bugs and write out an
	"alert" file with all the nodes/links that were overloaded.

	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
	argument that lets you specify that you want to wait for the disk
	to finish loading and/or wait for the node to come back up in the
	new OS.

	* tbsetup/power.in: Remove debugging printf.

	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
	quoting problems with event-sequences.  Add -expected-exit-code
	and -tag options to the "$program run" event.  Add -digester to
	the "$ns report" event that lets the user specify a program to run
	to digest the log files.

	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
	factor for feedback nodes to 1%, instead of 100%.

	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
	returns the values in "virt_user_environment".  Return new program
	agent fields: dir, timeout, and expected_exit_code.

	* tmcd/common/GNUmakefile.in: Install rc.canaryd.

	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
	a real-time priority, they can't do it from inside the jail.

	* tmcd/common/rc.canaryd: Rc script for canaryd.

	* tmcd/common/watchdog: Don't fail outright if there is a bad line
	in the battery.log

	* tmcd/common/rc.progagent: Append "userenv" data to the
	program-agent config file.

	* utils/GNUmakefile.in: Install loghole and its man page on ops.

	* utils/loghole.1: Document "clean" command and the change in
	loghole directories.

	* utils/loghole.in: Add "clean" command and parallelization.

	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
	Order the eventlist by "idx" and time, needed for sequences.  And
	removed unnecessary nologin checks.

dying to see whats going on. These changes can go into new images; it will
be ignored in non ElabInElab experiments;

rc.inelab: A terrible hack that I now regret. The whole point of this
script to allow inner boss/ops to 1) play by the standard state machine goo
(ISUP) for the outer emulab. That way os_setup, node_reboot, etc all work
when dealing with setting up the inner elab. 2) Since the inner
control can fall on any of the interfaces (as picked by assign), we
need to still do ifc/route configuration using tmcd. At the time I did
this, I expected that swapmod/swapin/swapout would be possible and
that boss/ops would be coming in on different nodes using different
interfaces for the inner control net, and that I would want to ask
each reboot. Well, I think swapping an inner emulab is a really long
way off, maybe so far off I'll be retired by that time.

rc.mkelab: A lot of very ugly code that turns a node into an inner ops
or boss. I won't bother to describe this code other than to say the
operating model for this rev is to create a fully functional inner
emulab based on the DB state provided by the outer emulab. Currently,
the current testbed software is expected to be in the /proj tree, and
a full boss/ops install is done, followed by all kinds of
customizations (like building accounts for members of the project,
etc) to make it a fully function emulab with just a single project;
the project in which the inner emulab was created.

the source of the link.  Otherwise when installing an MFS with something
like DESTDIR=/mnt, you wind up with links like:

lrwxr-xr-x   1 root  wheel   25 Nov  9 11:45 lib@ -> /mnt/usr/local/etc/emulab

which do not resolve when the MFS is booted.

This assumes that DESTDIR is only used for temporary mounting of filesystems
that are normally mounted elsewhere (and I believe that has always been the
intent of DESTDIR, at least in BSD).  If DESTDIR reflected the true path,
then the links will now be wrong.

just a bare skeleton of the client side is installed; just enough to
send state events when ops boots. Since ops has to boot before boss
(NFS mounts) the main script goes into the background and retries
until success.

Note that the ssl version of tmcc is installed, and so the proper pem
files must be installed on ops. Thats ctrlnode.pem from the ssl obj
directory as well as emulab.pem (public part of CA) into /etc/emulab.

possible to:

	gmake client
	sudo gmake client-install

on a FBSD4, FBSD5, RHL7.3, and RHL9.0 client node.

There are still some dependencies that are not explicit and which would
prevent a build/install from working on a "clean" OS.  Two that I know of are:
you must install our version of the elvin libraries and you must install boost.

this was to add soft reconfig support so that nodes could be
reconfigured without having to reboot them. This appears to work, and
has been tested with jails getting moved around. I've also tested the
new code on the MFS, but still no testing has been done on PLAB nodes.

The main change is that most of the code moved out of libsetup.pm, and
was split into constituent rc scripts, each of which does its own
thing, including cleaning up and preparing for making an image. Most
of that central knowledge has been moved out into the scripts. Still more
to do but this was a good start.

arguments, passes keyfile name, pidfile name, logfile name, etc.

this by moving (well, copying) the rcs file in the CVS tree so that we do
not lose the history.

event.pm (swigged event code).

Add symlink from /usr/testbed/lib to /usr/local/etc/emulab.

  code that was in libsetup has moved into this library, and underwent a
  giant cleaning and pumping up. The interface from your typical perl
  script now looks like this:

  use libtmcc;

  if (tmcc(TMCCCMD_STATUS, "optional arguments", \@tmccresults) < 0) {
      warn("*** WARNING: Could not get status from server!\n");
      return -1;
  }
  foreach my $me (@tmccresults) {
	print "bite $me";
  }

  The arguments and results are optional values. There is a fourth optional
  value that is a hash of config options (basically converted to command
  line switches passed to tmcc). For example, to set the timeout on an
  individual call, pass a fourth argument like:

	("timeout" => 5)

  There is also a way to set global options so that all subsequent tmcc
  calls are affected:

	configtmcc("timeout", 5);

  I'll probably clean this up a bit to avoid the direct strings.

  The result list is a list of strings. Since we are trending away from
  using tmcc to transfer large amounts of data, I think this is okay.

* A new tmcc.pl which does little more than load libtmcc and use it.
  This will become the new tmcc, with the existing C version becoming a
  backend binary for it.

* All of the perl scripts in tmcd have been changed to use the new
  library. I left the few uses of tmcc in shell scripts alone since they
  were of the simple variety (mostly "state" command).

* And again, if you have read this far, you will learn why I bothered with
  all this. Well, the existing code was really bad and it was getting out
  of control. Sort of like a squid that was getting harder to control as
  its rotting tenticles slithered into more and more scripts. Anyway ...

  More important, my goal is to use the libtmcc library to add caching.  I
  have not worked out the details yet, but I am envisioning a configuration
  file, perhaps generated initially by tmcd, of all of the config
  values. If the library finds that file, it sucks the info out of the file
  instead of going to tmcd. Eventually, this config file would be generated
  as part of experiment swapping and stored in the DB, but thats a longer
  term project, and perhaps orthogonal (how we fill the cache is not as
  important as adding the ability to use a cache, right?).

  Note that certain operations (like "state" and "ready") are flagged by
  the library to always bypass the "cache".

inside, but rather I do just enough to get the card booted (using
Abhijeet's minicom/expect scripts, but with changes to support the
configuration coming from tmcd. I also create a file of interface and
routeadd directives, so that the network configures properly, but
thats about it. Getting a more complete client side environment that
includes perl and sshd for the arm will have to wait.

configing the file.

server additions, made by Eric, Tim and Jay.

their per-node batch command jobs. Termination of batch jobs does not
happen until all nodes have reported status.