Made the following changes to the clientside code to support Linux
firewalls:

- Made os_fwconfig_line() actually do something.
- getfwconfig() adds an 'IPS' hash to the fwinfo hash.  This contains
  the IP address for each host, much like how the 'MACS' hash contains
  the MAC address for each host.  This is needed because ebtables (which
  is needed for ARP proxying) doesn't resolve hostnames.

Rules are stored in firewall/iptables-fw-rules.  Syntax is similar to
fw-rules, but without the rule number (since iptables doesn't use rule
numbers).  These should be equivalent to our ipfw-based rules, but I
haven't tested every case yet to confirm this.  I'm sure some changes
will be necessary.

TBBackGround().

Added TBDebugTimeStampsOff()

stale.

build_fake_macs generates fake mac addresses for the inside and outside
halves of a veth.  For openvz vnodes, we have to uniquely address
both halves.  tmcd gives us the vmac for the inside of the container;
it is basically 00:00:ipOct0:ipOct1:ipOct2:ip0ct3.  Normally, for openvz
veths, this works fine, because only the inside of the container ever sees
the vmac.  BUT, if we're not using openvz veths (i.e., using macvlan devices),
we might not have inside/outside halves of the veth.  Consequently, we
have to give the device a unique mac addr that is unique in both the
root and container contexts.  This is trivial for non-shared vhosts, but
if the vhost is shared, we can't just use the vmac as specified above.  So,
we do the following:

    # We have to set the locally administered bit (0x02) in the first
    # octet, and we can't set the unicast/multicast bit (0x01).  So
    # we have the first two octets to play with, minus those two bits,
    # leaving us with 14 total bits.  But then, for veths, we need a
    # a MAC for the root context, and for the container.  So there goes
    # another bit.
    #
    # So, what we're going to do is, if the vmid fits in 13 bits,
    # take the 5 MSB and shift them into bits 3-7 of the first octet,
    # and take the 8 LSB and make them the second octet.  Then, we
    # always set bit 2, and the container MAC gets bit 8 set.

Of course, this requires getifconfig to check for these "hacked" vmacs
when ifsetup configures interfaces inside the container -- so now
getifconfig checks for these special hacked vmacs if it can't find
a device with the vmac itself.  Good times...

Firewalls now work with nodes which require a subboss. Had to introduce new
firewall rules which skipped around the checks that no packets to/from
node control net IPs should pass through the firewall, if the IP in question
belongs to a subboss (since subboss is on the node control network). It
actually checks for all Emulab servers (boss, ops, fs or any subboss),
so the code should work for an Emulab install which has a non-segmented
control network in which all servers were in the same subnet as the nodes.

In addition to the new rules, we also had to pass in additional information
via "tmcc firewallinfo" giving the IP/MAC of those server nodes that are on
the node control network. We use this to establish ARP entries on the
inside network so that nodes can find the servers. Since the existing
client-side firewall code in libsetup.pm would blow up if it got a line
that it didn't recognize, I had to bump the tmcd version number and add
some conditional code to tmcd.c:dofwinfo() to not return the extra info for
old versions.

Added a couple of new firewall variables EMULAB_BOSSES and EMULAB_SERVERS
that are used in the new rules. Fixed the support scripts in firewall/
to properly initialize these variables.

IMPORTANT: tmcd looks up boss, ops, fs, and subbosses in the interfaces
table to find their IPs and MAC addresses. By default, we do not create
such interface table entries for boss/ops/fs. We have them at Utah for
other reasons. These entries are only needed if you have a non-segmented
control network (or a subboss) and you want to firewall such nodes.
The script to initialize the firewall variables (initfwvars.pl) will
print out a warning for configurations that are affected and don't have
the entries.

directory.

If nodes.pxe_boot_path is set to '/tftpboot/pxelinux/<something>', then
dhcpd_makeconf will set the (pxeboot) filename to /tftpboot/pxelinux.0
and symlink the node's config file (/tftpboot/pxelinux.cfg/<mac>) to
/tftpboot/pxelinux.cfg/<something>.

In other words, we can customize pxelinux to some small degree, using one
of some small number of pre-existing configurations. We were using pxelinux
before for plab-in-elab and we will also need it for loading WinPE for
configuring Windows7 images. For the latter we will set the pxe_boot_path
to /tftpboot/pxelinux/winpe.

Anyway, ideally we would allow the user to specify a pxelinux config file
through the NS file, but need to think about the implications of that some
more. Small steps...

Original version broke openvz vnode host setup of vnode experimental
interfaces.

The current state of affairs is that you can only set the "link emulation"
style at the experiment level, and that forces link emulation on physical
nodes as well as virtual nodes. Thus you are forced into using either 802.11
tagged vlan emulation or an OS that supports something called "veths" on
physical nodes.

From the comment:

   This is a very, very, very special case. If a non-encapsulating veth
   interface (veth-ne) maps 1-to-1 with an underlying physical interface,
   we want to just use the physical interface instead. This allows OSes
   (on physical nodes) which don't support a veth device (i.e., most of
   them) to talk to vnodes which are using veth-ne style.

   This can go away once we have separated the notion of multiplexing
   links from encapsulating links (a historical conflation) so that we
   don't have to force virtual devices onto physical nodes just because
   some virtual nodes in the same experiment require multiplexed links.

getmanifest grabs any service configuration, and per-service hook
information.

runbootscript runs one of our client-side boot scripts (or the
replacement the user specified), and any hooks (via runhooks).

The big backward compatibility issue is that we no longer store running
frisbeed info in the DB.  This means that loadinfo could not return
address:port info to clients and thus old frisbee MFSes could no longer
work.  While not a show stopper to require people to update their MFS first,
I made a token effort to implement backward compat as follows.

When an old frisbee MFS does "tmcc loadinfo" (as identified by a tmcd
version < 33), tmcd will invoke "frisbeehelper" to startup a daemon.
Sound like frisbeelauncher?  Well sorta, but vastly simplified and I only
want this to be temporary.  The helper just uses the frisbee client to make
a "proxy" request to the localhost master server.  The Emulab configuration
of the master server now allows requests from localhost to proxy for another
node.

frisbeehelper is also used by webfrisbeekiller to kill a running daemon
(yes, just like frisbeelauncher).  It makes a proxy status request on
localhost and uses the returned info to identify the particular instance
and kill it.

(eventual) support for NFS servers without race conditions!

This means no NFS between nodes and ops/fs. There are still NFS mounts of
ops on boss however.

Added new defs-* variable NOSHAREDFS, which when set non-zero will disable
the export of NFS filesystems to nodes.  Involved lots of little changes:

 * /users, /proj, and /share filesystems are not exported to nodes.

 * Returned mount info now includes an FSTYPE key which will be set to "LOCAL"
   if NOSHAREDFS is in effect (by default it is set to "NFS-RACY"; more on
   this later).  In the case where it is set to LOCAL, the other mount lines
   no longer contain REMOTE=foo settings.  Because of this change,
   THE TMCD VERSION NUMBER HAS BEEN BUMPED TO 32.

 * The client rc.mounts script will now create local versions of /users/*,
   /proj/<pid>, and /share when FSTYPE=LOCAL.  It first runs mkextrafs to
   create a large partition for these, since someday we will likely want
   to pre-populate these with a non-trivial amount of data.  Right now,
   the only thing that is put in the user's homedir is the standard dotfiles
   for the OS and the Emulab authorized_keys file (so you can login).

 * Linktest had to be modified to fetch the various results files (via
   loghole) rather than just assuming they were in /proj.  And also changed
   to invoke tevc with the local copy of the event key so it won't try to
   read it over NFS.

 * create_image was modified to ssh to the node and run the imagezip
   command, capturing the output of ssh.  This is controlled via the "-s"
   option which defaults to on for a NOSHAREDFS system, but can also be
   used on a normal system.

 * elabinelab's can be configured with/without a shared FS via the
   CONFIG_SHAREDFS attribute (note polarity change) which defaults to 1.

Another new defs-* variable, NFSRACY, will some day allow you to specify
(by setting to 0) that your NFS server does NOT have the nefarious mountd
race condition when changing /etc/exports.  Currently, this defaults to 1
since all versions of FreeBSD supported as an "fs" node have this "feature."
Rumor has it that FreeBSD 8 does not have this problem nor, presumably,
would a Linux NFS server.

The only use of this variable right now is to set the FSTYPE returned by the
tmcd "mounts" call, which in turn is used by one client script, rc.topomap
(via a libsetup function) to determine whether it should try copying
the topo file multiple times.

Random: add python2.6 to list of python's checked for in configure.
Random: resync defs-example-privatecnet with defs-example.
Random: did a little code-pissin here and there.

(cherry picked from commit 22900058)

- Add support for new tmcd dhcpdconf command to watchdog
- Fix dhcpd.conf template so that ddns-update-style is set to none
- Pull some utility functions from boss's libtestbed.pm into the client
  libtestbed.pm for use by subboss_dhcpd_makeconf and daemon_wrapper.
- Add stuff to simplify getting control interface IP address.

In dhclient-exit-hooks we stash the vnode name in $BOOTDIR/vmname where
libsetup would expect it.

In libsetup.pm we set the event server to the physical host (based on
node names!)

In libvnode_xen.pm we reflect that these hacks are now done elsewhere!

the tb-set-node-os command with a second optional argument; if that is
present, the first arg is the child OS and the second is the parent OS.
We add some new features in ptopgen (OS-parentOSname-childOSname) based
off a new table that maps which child OSes can run on which parents, and
the right desires get added to match.  We setup the reloads in os_setup
along with the parents.  Also needed a new opmode, RELOAD-PCVM, to handle
all this.

For now, users only have to specify that their images can run on pcvms, a
special hack for which type the images can run on.  This makes sense in
general since there is no point conditionalizing childOS loading on
hardware type at the moment, but rather on parentOS.  Hopefully this stuff
wiill mostly work on shared nodes too, although we'll have to be more
aggressive on the client side garbage collecting old frisbee'd images for
long-lived shared hosts.

I only made these changes in libvtop, so assign_wrapper folks are left in
the dark.

Currently, the client side supports frisbee.  Only in openvz for now, and
this probably breaks libvnode_xen.pm.  Also in here are some openvz
improvements, like ability to sniff out which network is the public
control net, and which is the fake virtual control net.

Code never worked, we were just lucky.

by allowing virtual nodes from multiple experiments at the same time.

Of course, attaching tc filters via IMQ doesn't seem to work inside
openvz...

bootvnodes/vnodesetup/mkX.pl train in favor of my own strawman design for
a more generic form of virt node support.  The strawman is incomplete and
probably wrong in places, but I had to abandon the quest for anything
better for now.  For now, uses same server side stuff as jails.

another bit (/etc/emulab/isremded) that controls whether remote nodes are
fully reserved to a single experiment (should behave just like local
nodes, except they talk to Emulab like remote nodes).  Corresponding
changes in the rc scripts...

raw geni node.

(and vnodes). Each time a node is allocated to an experiment it gets a
new root password (using the node_attributes table). The watchdog has
a new section that resets the root password (defaults to hourly).  We
still using a common password in the image to avoid totally bricking
ourselves, but once a node boots into an experiment it gets a new root
password.

This prevents hundreds of nodes with the same password, and all of the
problems associated with that.

vservers can be configured with experimental interfaces.  Think duct tape
and baling wire here...

 * commmon/bootvnodes: did some code refactoring in anticipation of
   something that was never needed.  Oh well, it looks purdy anyway!

 * common/libsetup.pm: LINUXJAILED() predicate to indentify local Linux
   vserver setups.  getlocalevserver() to return the IP/hostname of the
   "local" event server.

 * common/rc.{linktest,linkagent,progagent,trace,trafgen}: use the
   getlocalevserver() function for use with -s options (Linux vserver
   based vnodes cannot bind to localhost to talk to the pnode pubsubd)

 * common/config/rc.ifconfig: run this for Linux vservers, put out iface
   map for veths as well as physical interfaces

 * common/delaysetup: add -j vnodeid option, will need this at some point

 * linux/liblocsetup.pm: veth (actually etun) setup for Linux vserver vnodes

 * linux/mkvserver.pl: first cut at getting interfaces configured in
   vservers, do all the necessary etun/br plumbing (NOT a pretty sight...)

 * linux/vserver/rc.invserver (moved here from linux/rc.invserver):
   run linktest in local vservers (though it won't run yet due to NFS
   problems in vservers)

 * linux/vserver/vserver-cnet.sh: statically configure the control net
   in a vserver, no DHCP here!

 * linux/vserver/vserver-{init,rc}.sh: two parts of the Funky Interface
   Setup Dance that run inside the vserver