GitLab

The pieces are:

 * A monitoring daemon (clientside/protogeni/shared-node-monitor), which
   runs in each Xen shared node dom0, and gathers node and interface
   statistics.  It communicates these over the event system with...

 * A listening daemon (event/monitoring/shared-node-listener), running
   on the local boss node, and inserts everything it hears into the
   MySQL database "monitoring".  This database is also used by...

 * A site-wide monitoring daemon (protogeni/scripts/mondbd), periodically
   polling the normal testbed database (nodes and interfaces tables) and
   copying snapshots of the important pieces into the monitoring database.

 * The monitoring database is also read by a WWW front end invoked as a
   WSGI process by Apache.  This front end is developed by the GPO and
   modified to work with ProtoGENI, and is maintained in an independent
   repository.

Including one piece of incorrect code that might have prevented us
from getting a meaningful error message on a failed RPC.

The biggest changes were related to 10.0's replacing gcc with clang
and the switch to a new package system. The former required various
fixes to Makefiles that hardwired "gcc" and also missing prototypes
in source files that clang is particularly obnoxious about.

There was also accomodating the normal evolution of the ports tree.
New variables were needed in many of the Makefiles for this. Hopefully,
all the changes I made are backward compatible, but I don't actually
care that much since I hope to never, ever have to remake those older
packages! A new emulab version of the m2crypto port was needed to
reflect that m2crypto is no longer installed as an .egg file.

Casualties:

  * Emulab assign. As of this commit, assign does not build with
    clang. It will build, but immediately core dumps if built with
    a gcc port (gcc46 below). THIS NEEDS TO BE FIXED!

  * Emulab nsverify. This requires building a pure ns-2.34, which
    is old and does not compile with clang. Yo...

Can't do the untainting for all cases in libosload*.  The untainting
is now hooked into stated, where we catch the nodes as they send
along their "RELOADDONE" events to update their taint state according
to the final state of their partitions.

Also, clean up logging a smidge.

Log would be mail messages in stated-mail.log rather than actually emailing them.
Fewer regular log messages.
Avoid scanning a list unnecessarily if not in debug mode.
Use mysql to pick out certain osfeatures.
Bug fix: typo would let stated block when sent a SIGUSR1.

Cannot figure out why this happened, but at least don't crash if it
happens again!

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* state...

This happens at PRObE with very large experiment swapins (500+ nodes).
Don't know why it happens, but ingoring it is safe.

NOTE: currently only for FreeBSD 7.3 installs because that is the only
set of boss/ops/fs packages I have built so far!

This mostly involved minor changes to event agents. Too often we were
passing a pointer to a "long" to *get_int32, which on a 64-bit x86 OS would
fill the wrong half of a 64-bit variable. There was also one instance of
TCL code that had to be tweaked to account for 32- vs 64-bit.

These changes also required regeneration of SWIG stubs and an ugly change
to the SWIG generated code to use va_copy rather than direct assignment in
a couple of places.

Also related to SWIG is ensuring that the components that go into the
perl/python stub .so files are built with PIC. The amd64 linker requires
this.

The meta-ports had to be changed to reflect that linuxthreads and
ulsshxmlrpcpp don't work on amd64. The former had little effect as we
had mostly eliminated uses of linuxthreads already. The one thing that
did change was that we do not build nfstrace on amd64 (and we don't
currently use this anyway). Removing ulsshxmlrpcpp required switching
to the new event scheduler (event/new_sched) that Ryan did awhile back.
Note that it is only "new" in the sense that it uses a standard XMLRPC
package, there should be no functional differences. However, to be safe
we only use new_sched as the standard scheduler on 64-bit server installs.

Finally, added support to elabinelab setup to do a 64-bit server install.
Just specify FBSD73-64-STD as the boss/ops/fs osid and rc.mkelab should
do the rest.

That is pretty much it other than some random nits here and there.

Previously a number of invalid transitions were allowed due to the way the
check was coded. Note that this change has no functional impact as detected
invalid transitions were allowed anyway. Essentially this just changes what
gets logged.

When a node with the secure boot dongle is freed, it goes into PXEWAIT in
the context of the secure MFS. Previously we remained in "secure mode"
(i.e., did not terminate with a TPMSIGNOFF) while a node was in this state.
If the next use of the node, just booted from the OS that was already on
the disk, then we never signed off properly.

Now we sign off before entering PXEWAIT. I thought that this would be the
easiest alternative to fixing the problem..HaHaHa..not! Because now we have
to restart the secure boot path (i.e., reboot) if the result of coming out
of PXEWAIT is a request to reload the disk (i.e., if we are continuing the
secure disk load path).

Ideally this would have required only modifications to the state machines
for SECUREBOOT/LOAD, but as you can see by the presence of stated.in in the
modified files, this was not the case. The change required some additional
"finesse" to get it working. See the comments in stated.in and bootinfo_mysql.c
if you really care.

_display_info prints the status and related information about the virtual disk created.

reload MFSs to generate an ISUP cause that will screw up the reload by
clearing it. How do we prevent checkGenISUP() from generating an ISUP?
Well, by adding an osfeature to the MFSs that say they generate
ISUP. Sheesh.

Otherwise this blows up the swapout of a firewalled elabinelab.

This will get removed once I have time to figure out why we sometimes
get stuck here...

that section code as well, to use Node.pm methods.

1. Support for "one-shot" PXE booting ala the one-shot osid. Switches to
   pxelinux to boot WinPE and then switch back after done. Painful now
   because we have to HUP dhcpd everytime we change the PXE path, but we
   may be able to fix this in the future by going all-pxelinux-all-the-time.

2. Added pxe_select, analogous to os_select, for changing the pxe_boot_path
   including the one time path.

3. Added the WIMRELOAD state machine to shepherd a node through the process.
   Still has some rough edges and may need refining.

which says that any source state is okay, for the specified target
state. Basically a convenience.

clientside requres it to be there. As noted in previous revision, the
full emulab build always includes the clientside subdir, but a
clientside build will not necessarily include the rest of Emulab.