We have been running it for a long time on mothership, and the Wisc
NFS server needs all the help it can get...

pubsubd. The main point is that instead of being able to run the SSL
pubsubd at the Mothership only, any site can be a Portal and needs to
run it. So for example my elabinelab is a real Portal, which is very
handy for testing.

In particular, make it work with d430s as inner nodes by supporting
an "sio2" pxeboot, the 64-bit MFS, and a more modern kernel in the MFS.

Geni style events to the portal pubsubd (on the SSL port).

a dependency on a shared /proj, shame shame shame.

clusters.

The template configurations in the new ntpd subdir also address the
recent NTP amplification attacks that have been going on recently.

NTP configuration is controlled by a few defs-* variables:

NTPSERVER: boss|ops|fs|<external-server-name-or-IP>
  Default: "ops"
  Normally, one of boss, ops, or fs is designated as a local NTP server
  but this can be set to a fully qualified name of some other machine.
  If NTPSERVER is set to an external server, then boss/ops/fs are made
  clients of that server just as any testbed node is.

EXTERNAL_NTPSERVER[1-4]: <external-server-name-or-IP>
  Default: "[0-3].pool.ntp.org"
  If NTPSERVER is one of boss/ops/fs, then these values are used as the
  upstream servers for the local server. These can be changed to four of
  your favorite NTP servers.

NTPDRIFTFILE: <path>
  Default: "/var/db/ntp.drift"
  If NTPSERVER is one of boss/ops/fs, then this is the name of the drift
  file for the local server.

This may also fix some problems that people have had with their name server
when starting with this config--some of the IP addrs/masks were not right.

Also, remove defs-example-privatecnet since defs-example has been pretty
much the same thing for many years.

Documentation to follow.

This is not as general or as useful as you might think.
Right now it simply controls what device gets used as console when
customizing the MFSes. "sio" is the default. Other choices correspond
to pxeboot variants: sio2, vga, null. Choosing vga also sets the magic
VGAONLY setting in the frisbee MFS ensuring that any customized (via slicefix)
FreeBSD image doesn't try to use the serial port as console.

A real solution for choosing node consoles would not be per-testbed.
It would be per-node-type and per-node. Right now the console type is
selected in that fashion via a combination of the pxe_boot_path and
frisbee/admin MFS OSIDs. At some point we should make the console setting
explicit.

quailified!

Primarily, so we can let frisbee upload to boss work through our firewall
without opening a gaping hole.

YOU MUST RECONFIGURE YOUR TREE before installing.

Repeating myself since not everything got committed last time...

In the lastest version of SelfLoader they fixed the taint problem.  It is
literally one line of code (plus updating version string) so I just made
a patch.  Also, fixup boss/ops-install to apply the patch and change the
 default to use the SelfLoader again.

(eventual) support for NFS servers without race conditions!

This means no NFS between nodes and ops/fs. There are still NFS mounts of
ops on boss however.

Added new defs-* variable NOSHAREDFS, which when set non-zero will disable
the export of NFS filesystems to nodes.  Involved lots of little changes:

 * /users, /proj, and /share filesystems are not exported to nodes.

 * Returned mount info now includes an FSTYPE key which will be set to "LOCAL"
   if NOSHAREDFS is in effect (by default it is set to "NFS-RACY"; more on
   this later).  In the case where it is set to LOCAL, the other mount lines
   no longer contain REMOTE=foo settings.  Because of this change,
   THE TMCD VERSION NUMBER HAS BEEN BUMPED TO 32.

 * The client rc.mounts script will now create local versions of /users/*,
   /proj/<pid>, and /share when FSTYPE=LOCAL.  It first runs mkextrafs to
   create a large partition for these, since someday we will likely want
   to pre-populate these with a non-trivial amount of data.  Right now,
   the only thing that is put in the user's homedir is the standard dotfiles
   for the OS and the Emulab authorized_keys file (so you can login).

 * Linktest had to be modified to fetch the various results files (via
   loghole) rather than just assuming they were in /proj.  And also changed
   to invoke tevc with the local copy of the event key so it won't try to
   read it over NFS.

 * create_image was modified to ssh to the node and run the imagezip
   command, capturing the output of ssh.  This is controlled via the "-s"
   option which defaults to on for a NOSHAREDFS system, but can also be
   used on a normal system.

 * elabinelab's can be configured with/without a shared FS via the
   CONFIG_SHAREDFS attribute (note polarity change) which defaults to 1.

Another new defs-* variable, NFSRACY, will some day allow you to specify
(by setting to 0) that your NFS server does NOT have the nefarious mountd
race condition when changing /etc/exports.  Currently, this defaults to 1
since all versions of FreeBSD supported as an "fs" node have this "feature."
Rumor has it that FreeBSD 8 does not have this problem nor, presumably,
would a Linux NFS server.

The only use of this variable right now is to set the FSTYPE returned by the
tmcd "mounts" call, which in turn is used by one client script, rc.topomap
(via a libsetup function) to determine whether it should try copying
the topo file multiple times.

Random: add python2.6 to list of python's checked for in configure.
Random: resync defs-example-privatecnet with defs-example.
Random: did a little code-pissin here and there.

port speed and duplex when the switch does not support the stack mib.

its really a hugely stripped down Emulab boss install, using a very
short version of install/boss-install to get a few things into place.

I refactored a few things in both the protogeni code and the Emulab
code, and whacked a bunch of makefiles and configure stuff. The result
is that we only need to install about 10-12 files from the Emulab
code, plus the protogeni code. Quite manageable, if you don't mind
that it requires FreeBSD 6.X ... Still, I think it satisfies the
requirement that we have a packaged clearinghouse that can be run
standalone from a running Emulab site.

But opsrecv and friends were never officially installed, so took care of
that too (conditional on PELABSUPPORT in the defs file).

Two-day boondoggle to support "/scratch", an optional large, shared filesystem
for users.  To do this, I needed to find all the instances where /proj is used
and behave accordingly.  The boondoggle part was the decision to gather up all
the hardwired instances of shared directory names ("/proj", "/users", etc.)
so that they are set in a common place (via unexposed configure variables).
This is a boondoggle because:

1. I didn't change the client-side scripts.  They need a different mechanism
   (e.g., tmcd) to get the info, configure is the wrong way.

2. Even if I had done #1 it is likely--no, certain--that something would
   fail if you tried to rename "/proj" to be "/mike".  These names are just
   too ingrained.

3. We may not even use "/scratch" as it turns out.

Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
To maintain my illusion in the future you should:

1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
   et.al. functions where possible.  If not possible, make sure they run
   through configure and use @PROJROOT_DIR@, etc.

2. Use the configure method for python, C, php and other languages.

3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
   you should call to determine if an NS, template parameter, tarball or
   other file are in "an acceptable location."  Use these functions where
   possible.  They know about the optional "scratch" filesystem.  Note that
   the perl function is over-engineered to handles cases that don't occur
   in nature.

Make a version of the example which shows an unroutable control network.

Entailed new instructions for manual setup as well as integration into
elabinelab framework.  First, the manual path:

setup.txt, setup-boss.txt, setup-ops.txt and new setup-fs.txt:
    Updated to reflect potential for separate fs node.  The org here
    is a little dicey and could be confusing with ops+fs vs. ops and fs.
    Has not been field tested yet.

*/GNUmakefile.in: new fs-install target.

configure, configure.in, defs-*:
    Somewhat unrelated, make min uid/gid to use be a defs setting.
    Also add config of fs-install.in script.

boss-install.in, ops-install.in and new fs-install.in:
    Handle distinct fs node.  If you have one, fs-install is run before
    ops-install.  All scripts rely on the defs file settings of FSNODE
    and USERNODE to determine if the fs node is seperate.

utils/checkquota.in:
    Just return "ok" if quotas are not used (i.e., if defs file FS_WITH_QUOTA
    string is null.

install/ports/emulab-fs:
    Meta port for fs node specific stuff.  Also a patch for the samba port
    Makefile so it doesn't drag in CUPs, etc.  Note that the current samba
    port Makefile has this change, I am just backporting to our version.

Elabinelab specific changes:

elabinelab-withfs.ns:
    NS fragment used in conjunction with
	tb-elab-in-elab-topology "withfs"
    to setup inner-elab with fs node.

elabinelab.ns:
    The hard work on the boss side.  Recognize seperate-fs config and handle
    running of rc.mkelab on that node.  fs setup happens before ops setup.

rc.mkelab:
    The hard work on the client side.  Recognize FsNode setup as well as
    differentiate ops+fs from ops setup.

Related stuff either not part of the repo or checked in previously:
    emulab-fs package

setup-network.txt file.