Note that the shared-node-listener should be doing this, but this
is easier.

So mike doesn't get worried...

receiving Geni style events from event enabled clusters.

On clusters where CLUSTER_PORTAL is defined, start up an SSL enabled pubsub
notification forwarder, to send geni style events to the portal pubsubd.

daemon.

every now and then. Seems to happen a lot on the racks, and
there are lots of them.

Currently it is configured (hardwired) to run every 15 minutes, even that
may be too frequent as things don't happen too fast in lease-world.

Otherwise, pubsubd won't start til after the testbed startup. Since
checknodes_daemon wants to send an event, it will hang forever if pubsubd
is not running.

with testbed-control, and then I reboot boss, I do not want the
daemons to start up until I call testbed-control again.

No more frisbeelauncher or assorted subboss frisbee stuff.

More work on the hierarchical configuration for subboss. When doing host-based
authentication, allow client to pass an explicit host (IP) to the mserver.
If the mserver is configured to allow it, that IP is used for authenticating
the request instead of the caller's IP. Add a default ("null") configuration
so the mserver can operate out-of-the-box with no config file. The goal of
these two changes is for an mserver instance with the default config and a
proxy option to serve the needs of a subboss node (i.e., so no explicit
configuration will be needed).

currently does is probe the known and enabled CMs and every 24 hours,
to see what version they are running (which says if they are online)
and then sends email to geni-dev-utah.

that they all write proper pid files in /var/run. You can not actually
"stop" the testbed daemons from the command line.

First cut at a daemon that does regular checkups of the testbed
hardware/software.

	* configure, configure.in: Add tbsetup/checkup directory.

	* db/audit.in: Add a listing of stuck checkups.

	* install/boss-install.in: Add 'elabckup' user.

	* rc.d/3.testbed.sh.in: Startup the checkup_daemon.

	* sql/database-create.sql, sql/database-migrate.txt: Add the
	checkups tables.

	* tbsetup/GNUmakefile.in: Descend into the checkup directory.

	* tbsetup/checkup: The checkup daemon, man page, and
	  associated scripts.

	* tbsetup/ptopgen.in: Add a feature with a value of 0.9 to
	  prereserved nodes to keep them from being allocated unless
	  they're really wanted.

	* utils/firstuser.in: Add some other options so the script can be
	  used to create other pseudo users.

mysqld hangs that cause the entire system to grind to a halt. The
basic theory of operation is like this:

* Once a minute fork a child (protected by a 60 second timeout) to
  connect to the DB and issue a simple query. If the child can access
  the DB okay, it exits with a zero status.

* If the alarm fires, the child is killed. This indicates that mysqld
  is no longer responding in a reasonable amount of time (60 seconds).
  We shift into trying to restart mysqld:

     * Send mysqld a TERM. Wait for 30 seconds.

     * Try query again; typically, the situation will not have changed one
       bit, but I do it anyway.

     * If mysqld was running, send it a kill -9. Wait for 15 seconds.

     * Start mysqld. Wait for 5 seconds.

     * Try query again. If query succeeds, we are done, and no one
       will have to deal with it Sunday morning at 6am (thanks Tim).

     * If query still fails, send email and give up trying to do fix
     ...

grabwebcams startup since it is no longer needed.

The new version of the webcam page connects directly to the cameras
using a URL stored in the DB (webcams table). The URL includes the
user (elabman) and the password, which is fine; not likely to leak
out the password and even if we did, no biggie since its on the private
network and no one can get to it anyway.

I have reduced the frame rate to 5fps (compression 50%) as specified
in the URL, and even thats a lot of bandwidth.

I will work on providing mpeg feeds later!

sitevars to determine if the Robot Lab is open or closed. The sitevars:

* 'robotlab/override' - Override other settings and forcibly turn the lab
  "on" or "off" (open or close). When the lab is turned off, new
  experiments cannot swap in and the current experiment is immediately
  swapped out.

* 'robotlab/exclusive' - The robot lab is exclusive use. Best to not mess
  with this sitevar :-)

* 'robotlab/opentime' - The time that the robot lab opens in the
  morning. The default is 07:00, but feel free to change this as you like.

* 'robotlab/closetime' - The time that the robot lab closes in the
  evening. The default is 18:00, but feel free to change this as you like.

* 'robotlab/open' - The robot lab is open or closed. DO NOT MESS WITH THIS!
  It is updated by the robomonitord script and intended to be used by
  admission control (not done yet).

The robomonitord script runs and periodically (every 2 minutes) wakes up
and looks at the various sitevars above. The lab is open during the day,
Monday through Friday, and closed on weekends. It is also supposed to be
closed on holidays, but I have not added that yet.

15 minutes before the lab is to be closed, a warning message is sent to the
swapper of the experiment running on the robot testbed, that their
experiment is going to be swapped soon. When the Robot lab is closed
(either cause the close time was reached, or because the lab was forcibly
closed with the override), the current experiment is immediately swapped
out.

I know, this is hopelessly bogus, but it will do until we feel like adding
a "Lab" datatype to the system.

(like stated).

start before trying to use it. Silly, I know.