This is analogous to what we do in bootinfo to ensure we don't fire off
excessive PXEBOOTING/BOOTING events. The DB file is in the new
/usr/testbed/db directory.

Long ago Leigh added code in the bootlog command to account for the fact that
you might not get all the data associated with a TCP-based command in one
read from the socket. This case also showed up with the hostkeys command
with uploaded multiple pubkeys.

So I moved the code up to handle_requests so that for all TCP-based
commands, we read til we get EOF.

Leigh should do this right!

Significant hackary involved. Similar to exports_setup, there is a boss-side
script and an ops-side script to handle creation and destruction of the ZFS
clones that are used for the NFS filesystem. The rest was all about when to
invoke said scripts.

Creation is easy, we just do a clone whenever the TBAdminMfsSelect is called
to "turn on" node admin mode. Destruction is not so simple. If we destroyed
the clone on the corresponding TBAdminMfsSelect "off" call, then we could
yank the filesystem out from under the node if it was still running in the
MFS (e.g., "node_admin -n off node"). While that would probably be okay in
most uses, where at worst we would have to apod or power cycle the node, we
try to do better. TBAdminMfsSelect "off" instead just renames the clone
(to "<nodeid>-DEAD") so that it stays available if the node is running on
it at the time, but ensures that it will not get accidentally used by any
future boot. We check for, and destroy, any previous versions for a node
every time we invoke the nfsmfs_setup code for that node. We also destroy
live or dead clones whenever we call nfree. This ensures that all MFSes
get cleaned up at experiment swapout time.

1. Fixes to allow specific versions of images to be exported; the existing
   path was reverting back to the highest numbered version. So far this has
   not come up, but will with APT and Cloud.

2. Add version argument to image_metadata.php, mostly as a convenience. So
   rather then using the version specific uuid, you can use the image uuid,
   with a version argument. This is actually more sensible, except for one
   important fact; it is not possible to locate a deleted image this way,
   since the image descriptor is gone (only the version descriptors are in
   the DB).

   But I went ahead and did it cause there is still some question as to
   whether we care about being able to export a deleted image. We do not
   expose these URLs at this time, but you can use one.

    <emulab:failure_action action="nonfatal"/>

We tried this 14 or so years ago and it didn't work on FreeBSD 4.7.
But this does work on Linux at least.

A concession to performance. Previously, PXEBOOTING was reported on
the PXE DHCP request and BOOTING on the following OS-originated request.
This is conceptually ideal, as that is what those states were intended
to mean, but it causes two synchronous "reportboot" command executions
from dhcpd for every node boot. Worse, the time gap between the second,
OS-originated DHCP call and the first explicit state reported by the
node itself (e.g., TBSETUP or RELOADSETUP) is generally small enough
that the node reported state often arrived at stated before the BOOTING
state from dhcpd. This can cause excess node reboots or other undesirable
behaviors from stated.

So now we only invoke "reportboot" on the first PXE-originated call and
tell reportboot to send both PXEBOOTING and BOOTING events at that time.
This does not eliminate the race condition above, but makes it unlikely
as there is the whole kernel boot process (10s of seconds) between the
dhcpd state reports and the first node state report.

users yet.

all users. Cloudlab added to the list, but not exposed except to
admins and studly users.

2.68 is installed on FreeBSD 8.x.

is not killed, the image upload will fail, and it takes 30 minutes for
frisbeed to idle exit. We really need to know if the frisbeed process has
any clients for this.

add for VMs, but we were never deleting them and after a while
sshd refuses to start.

Do not remove iscsi startup files in the guest on XEN44.