knows that some tests failed. Otherwise we say all test completed
okay, and then print a bunch of errors!

expected, then use the current test. If the latency is higher then
expected, then allow for some fudge. Using 3ms fudge, since as Jay
points out, its either really close, or its totally broken. This will
reduce false positives, especially when using endnodeshaping.

a link cause the BW or Loss is out of range; post the messages back
through the event system so that it ends up in the log or on the web
page.

Bring linktest in line with the new event system world order and do some
generic cleanup.  I'm pretty sure I broke backwards compatibility with
old disk images though...

	* configure, configure.in: Remove linktest.h.in from the list of
	template files.

	* event/linktest/GNUmakefile.in: Don't make subdirectories for
	executables since it can mess up the dependencies.  Don't install
	tb_compat.tcl stuff anymore.  Check for '/usr/local/bin/pathrate'
	and friends, and print out a warning if they are not found.  Make
	a version file for the linktest daemon.

	* event/linktest/linktest.c: Only subscribe to the events we care
	about 'STOP/KILL'.  Subscribe to TIME START events and kill any
	linktests that are in progress.  Send events through the local
	elvind instead of running tevc.

	* event/linktest/linktest.pl.in: Don't run the modified NS anymore
	to generate the topology file, use gentopofile to do that.  Send
	COMPLETE events to indicate that the tests were finished instead
	of STOP so it works with event-sequences.  Use emulab-sync to
	report errors between linktest daemons.  In addition to sending
	REPORT events, send LOG events to the SIMULATOR agent so the
	messages end up in the report e-mail.  Fix wait_all so it collects
	child processes immediately and doesn't leave zombies lying
	around.  Make the static route test work with topologies that
	aren't fully connected.  Change ownership of the linktest error
	files from root to the swapper.  Bump latency tolerance up to
	2.5ms.

	* event/linktest/ltevent.c: Use event_schedule instead of
	event_notify so the events go through the scheduler.

	* event/linktest/run_linktest.pl.in: Wait for COMPLETE events
	insetad of STOPs.

	* event/linktest/linktest.h.in: Removed.

	* event/sched/error-record.c: Include linktest error output in the
	report e-mail.

	* event/sched/event-sched.c: Linktest now sends COMPLETE events
	for START events.

	* tbsetup/gentopofile.in, tmcd/common/config/rc.topomap: Piggy
	back the generation and management of the linktest config file
	onto the generation of the topomap.

	* tbsetup/ns2ir/sim.tcl.in: Add "linktest" event to the Simulator
	object that runs linktest at level 3 (latency, static routing, and
	loss).

	* tmcd/common/libsetup.pm: Add TMLTMAP constant that contains the
	path to the node local linktest config file.

	* tmcd/common/rc.linktest: Pass '-u' option to the linktest daemon
	so it knows what user should to change ownership of files to.

	* tmcd/freebsd/jail/mkjail.pl: Copy the linktest config file
	(ltmap) into the jail.

Differentiate RHL9 and use a different build of ns.

the "exec" function.  A list arg to exec used to be considered untained,
but no more.  So run all the args through a hokey ".*" pattern match to
untaint them.

This was preventing linktest from working on RHL9 (and presumably FBSD5).

* The per-experiment event scheduler now runs on ops instead of boss.
  Boss still runs elvind and uses events internally, but the user part
  of the event system has moved.

* Part of the guts of eventsys_control moved to new script, eventsys.proxy,
  which runs on ops and fires off the event scheduler. The only tricky part
  of this is that the scheduler runs as the user, but killing it has to be
  done as root since a different person might swap out the experiment. So,
  the proxy is a perl wrapper invoked from a root ssh from boss, which
  forks, writes the pid file into /var/run/emulab/evsched/$pid_$eid.pid,
  then flips to the user and execs the event scheduler (which is careful
  not to fork). Obviously, if the kill is done as root, the pid file has to
  be stored someplace the user is not allowed to write.

* The event scheduler has been rewritten to use Tim's C++ interface to the
  sshxmlrpc server on boss. Actually, I reorg'ed the scheduler so that it
  can be built either as a mysql client, or as RPC client. Note that it can
  also be built to use the SSL version of the XMLRPC server, but that will
  not go live until I finish the server stuff up. Also some goo for dealing
  with building the scheduler with C++.

* Changes to several makefiles to install the ops binaries over NFS to
  /usr/testbed/opsdir. Makes life easier, but only if boss and ops are
  running the same OS. For now, using static linking on the event scheduler
  until ops upgraded to same rev as boss.

* All of the event clients got little tweaks for dealing with the new CNAME
  for the event system server (event-sever). Will need to build new images
  at some point. Old images and clients will continue to work cause of an
  inetd hack on boss that uses netcat to transparently redirect elvind
  connections to ops.

* Note that eventdebug needs some explaining. In order to make the inetd
  redirect work, elvind cannot be listening on the standard port. So, the
  boss event system uses an alternate port since there are just a few
  subsystems on boss that use the server, and its easy to propogate changes
  on boss. Anyway, the default for eventdebug is to connect to the standard
  port on localhost, which means it will work as expected on ops, but will
  require -b argument on boss.

* Linktest changes were slightly more involved. No longer run linktest on
  boss when called from the experiment swapin path, but ssh over to ops to
  fire it off. This is done as the user of course, and there are some
  tricks to make it possible to kill a running linktest and its ssh when
  experiment swapin is canceled (or from the command line) by forcing
  allocation of a tty. I will probably revisit this at some point, but I
  did not want to spend a bunch of time on linktest.

* The upgrade path detailed in doc/UPDATING is necessarily complicated and
  bound to cause consternation at remote sites doing an upgrade.

writable by adding a umask(0002) call.

1. The Linktest daemon, linktest.c, now listens for a KILL event. If received,
   the daemon will kill the linktest.pl child process and all of its subchildren.
2. The daemon also listens for SIGSTP events from the linktest.pl child, and
   will kill the linktest.pl process and its children if linktest.pl dies
   unexpectedly.
3. Locking has been implemented in linktest.c to ignore requests to start linktest
   while a current run is executing.
4. The controller script run_linktest.pl now includes the following new options:
   -t   allows the user to specify a timeout in seconds for Linktest.
   -v   prints out better feedback from the Linktest run as it proceeds.

Major remaining items are:
1. Avoid NFS mount hups
2. More testing, especially on vnodes

removal of unnecessary TEST option. Pending change: still need to move this to standard emulab style.

Last-minute tweaks to error ranges (wider latency tolerance due to occasional errors when running at low bandwidth, plus wider Pathrate tolerance to avoid false positives while waiting for tuning data).
Added more info when aborting run_linktest.pl
Added more time to prevent a race condition between collector startup and test during the loss and bandwidth tests.

Removed hardcoded "boss" from linktest.pl and run_linktest.pl
Changed historical log location.

Modified ltevent.c to use keyfiles.
Added a ROADMAP.
Linktest is now ready for first deploy.

better support for experiment failures (99% on latency and loss),
better error messages.

and wait for linktest events, and integrated it into the perl scripts.

At this point I'm working on small potatoes such as the rc.setup script
to invoke linktest daemon and nailing down where to put platform-specific
stuff such as the ns-linktest build and the customized tb-compat.

Also am testing linktest in cases where I've deliberately inserted an error
to make sure it catches it.

This includes an updated client-install.

However, this is not yet ready for builds due to certain libraries not
present on the nodes and ops. For those I will be writing a static
linked c program to handle the linktest "done" event in both linktest.pl
and run_linktest.pl.

and addded parameter passing the daemon.

Also fixed up the makefile for compilation and installation (client).

two-way assign). Has been tested on 5-node linux and bsd expts. Still chatty,
still does not log. Also does not yet incorporate changes from discussion
today, namely running ns on ops.