GitLab

  code that was in libsetup has moved into this library, and underwent a
  giant cleaning and pumping up. The interface from your typical perl
  script now looks like this:

  use libtmcc;

  if (tmcc(TMCCCMD_STATUS, "optional arguments", \@tmccresults) < 0) {
      warn("*** WARNING: Could not get status from server!\n");
      return -1;
  }
  foreach my $me (@tmccresults) {
	print "bite $me";
  }

  The arguments and results are optional values. There is a fourth optional
  value that is a hash of config options (basically converted to command
  line switches passed to tmcc). For example, to set the timeout on an
  individual call, pass a fourth argument like:

	("timeout" => 5)

  There is also a way to set global options so that all subsequent tmcc
  calls are affected:

	configtmcc("timeout", 5);

  I'll probably clean this up a bit to avoid the direct strings.

  The result list is a list of strings. Since we are trending away from
  using tmcc to transfer large amounts of data, I think this is okay.

* A new tmcc.pl which does little more than load libtmcc and use it.
  This will become the new tmcc, with the existing C version becoming a
  backend binary for it.

* All of the perl scripts in tmcd have been changed to use the new
  library. I left the few uses of tmcc in shell scripts alone since they
  were of the simple variety (mostly "state" command).

* And again, if you have read this far, you will learn why I bothered with
  all this. Well, the existing code was really bad and it was getting out
  of control. Sort of like a squid that was getting harder to control as
  its rotting tenticles slithered into more and more scripts. Anyway ...

  More important, my goal is to use the libtmcc library to add caching.  I
  have not worked out the details yet, but I am envisioning a configuration
  file, perhaps generated initially by tmcd, of all of the config
  values. If the library finds that file, it sucks the info out of the file
  instead of going to tmcd. Eventually, this config file would be generated
  as part of experiment swapping and stored in the DB, but thats a longer
  term project, and perhaps orthogonal (how we fill the cache is not as
  important as adding the ability to use a cache, right?).

  Note that certain operations (like "state" and "ready") are flagged by
  the library to always bypass the "cache".

batch jobs will end! There were two problems; 1) The socket was
744, so mere users were not able to run tmcc. 2) libsetup was not
even using the proxy socket cause it was never looking for it.

killvserver() and getanyppid() which caused vnodesetup to fail when it
shouldn't have.

tmcd (which is bad, since tying up the tmcd threads blocks all nodes
in the testbed). The old functionality is left in tmcd for now.

On the server side, a new web page (www/spewrpmtar.php3) receives a
request for a file, along with the nodeid (pcXXX) making the request,
and the secret key that is generated for each new experiment and
transfered to the node via tmcd. If the key matches, the operation is
handed off to tbsetup/spewrpmtar.in which verifies that the file is in
the list of rpm/tar files for that node, and then spits it out to
stdout. The web page uses fpassthru() to send the file out to the
client. The client is using wget, and is required to use https (the
web page checks).

At present, the external script is run as the creator of the
experiment, and gid of the experiment. Perhaps this is not a good
idea. In any event, the file must be in the list of rpm/tarfiles,
either owned by the experiment creator or with a group of the
experiment, and the file must reside in either /proj or /groups.
I use the realpath() function to make sure there are no symlink tricks
pointing to outside those filesystems. I use the standard NFS read goo to
prevent transient mount problems that we all know and love.

 - regexp that matched the slice user wasn't working now that we're only
   looking for emulab_<exp_index>.  Witholding this change from libplab.py
   for now until we push out a new tarball with this file

like we do for jailed nodes.

Eliminated the second call. The output from the first time is now used
to construct the file $BOOTDIR/tmcc.ifconfig

to avoid hup-me-and-i-will-screw-ya mountd/NFS bug.

Use new -t option (use tmcd) in jails/plab

address of the interface instead of the dest&mask, which was bogus
anyway, and now wrong in the presence of variable netmasks for the
destination.

code (asking tmcd twice for routes, ifconfig, etc). Shashi will fix
later.

routing (-V option to vnodesetup).  I was going to do this just for
jails that used gated, but there is no efficient way to do this
(I don't want to make a redundent "tmcc routing" call for every jail
just for the very rare case they are using gated).

Since it is the Right Thing To Do anyway, I'll just go for it now
while we only have one external user.

There is a possibility of unexpected consequences on mini, but I'll
cross that bridge when I get to it (or more likely, when Leigh gets
to it :-)

inside, but rather I do just enough to get the card booted (using
Abhijeet's minicom/expect scripts, but with changes to support the
configuration coming from tmcd. I also create a file of interface and
routeadd directives, so that the network configures properly, but
thats about it. Getting a more complete client side environment that
includes perl and sshd for the arm will have to wait.

swamping tmcd.

Add a -V option to mkjail.pl telling it to use the virtual control net
IP (172.16.0.1) for the default route instead of the real control net
IP (155.101.132.1).  In this case, it also assigns a virtual control net
IP alias (172.17.<pnode>.0) to the physical interface so that the
aforementioned routes can actually be installed.

Add the same -V option (only valid with -j) in vnodesetup so that we
can get it through from bootvnodes.  (Also cleaned up the usage message
some)

The virtual control net address/mask are currently constants in mkjail.pl.
At some point they can/should be changed to config time options so that we
can easily use this on minibed too.

In theory, this (-V) shouldn't be an option and we should just use
virtual control net routing all the time.  But I want to get some more
testing and we have to resolve the minibed conflicts first.  So for now,
this is an option and it is off by default.

Haven't tied this option in with gated yet, so creating a vnode experiment
with Session routing still won't work.  If you want to do that, create the
experiment, watch gated blow up right and left, login to all the physical
nodes and change bootvnodes to add -V to the invocation of vnodesetup,
and then reboot everything.  Now it should be working!

idle loop while waiting for a signal in a vserver.  Moved vnode
killing/halting into the signalling process for vservers, and made the
running vnodesetup just clean up after itself in these cases.  Rewrote
the process of killing a vserver to use proc to get the process list,
and to not kill any of its parent processes.  This was necessary to
make it possible to report back to Emulab when node teardown is
complete.

per-jail software.

(necessary in order to play nicely with vnode_setup).  rc.vinit now
figures out he nodeid based on the file in which libsetup stores it.

use CONFDIR will work properly. Not tested yet ...

configing the file.

grab that from the bossname using tmcc (be nice if bsd/linux had a
consistent way to get the domainname).

the project is different. This is not supposed to happen, but does
when reloading. Sigh.

Jail changes.

iron out.  TMCD/libsetup now has a plabconfig commands that parallels
the jailconfig command.  The Plab boot process has been added to
libsetup and a -p option has been added to vnodesetup to parallel the
-j option.  Parts of the code that were Jail-specific, but labeled
just as vnode stuff have been renamed.  $vnodedir in vnodesetup has
been removed, since it was redunant with libsetup's CONFDIR, and
CONFDIR is much more intelligent.

server additions, made by Eric, Tim and Jay.

their per-node batch command jobs. Termination of batch jobs does not
happen until all nodes have reported status.

* Parser: Added new tb command to set the name of the sync server:

	tb-set-sync-server <node>

  This initializes the sync_server slot of the experiment entry to the
  *vname* of the node that should run the sync server for that
  experiment. In other words, the sync server is per-experiment, runs
  on a node in the experiment, and the user gets to chose which node
  it runs on.

* tmcd and client side setup. Added new syncserver command which
  returns the name of the syncserver and whether the requesting node
  is the lucky one to run the daemon:

    SYNCSERVER SERVER='nodeG.syncserver.testbed.emulab.net' ISSERVER=1

  The name of the syncserver is written to /var/emulab/boot/syncserver
  on the nodes so that clients can easily figure out where the server
  is.

  Aside: The ready bits are now ignored (no DB accesses are made) for
  virtual nodes; they are forced to use the new sync server.

* New os/syncd directory containing the daemon and the client. The
  daemon is pretty simple. It waits for TCP (and UDP, although that
  path is not complete yet) connections, and reads in a little
  structure that gives the name of the "barrier" to wait for, and an
  optional count of clients in the group (this would be used by the
  "master" who initializes barriers for clients). The socket is saved
  (no reply is made, so the client is blocked) until the count reaches
  zero. Then all clients are released by writting back to the
  sockets, and the sockets are closed. Obviously, the number of
  clients is limited by the numbed of FDs (open sockets), hence the
  need for a UDP variant, but that will take more work.

  The client has a simple command line interface:

    usage: emulab-sync [options]
    -n <name>         Optional barrier name; must be less than 64 bytes long
    -d                Turn on debugging
    -s server         Specify a sync server to connect to
    -p portnum        Specify a port number to connect to
    -i count          Initialize named barrier to count waiters
    -u                Use UDP instead of TCP

    The client figures out the server by looking for the file created
    above by libsetup (/var/emulab/boot/syncserver). If you do not
    specify a barrier "name", it uses an internal default. Yes, the
    server can handle multiple barriers (differently named of course)
    at once (non-overlapping clients obviously).

    Clients can wait before a barrier in "initialized." The count on
    the barrier just goes negative until someone initializes the
    barrier using the -i option, which increments the count by the
    count. Therefore, the master does not have to arrange to get there
    "first." As an example, consider a master and one client:

	nodeA> /usr/local/etc/emulab/emulab-sync -n mybarrier
	nodeB> /usr/local/etc/emulab/emulab-sync -n mybarrier -i 1

    Node A waits until Node B initializes the barrier (gives it a
    count).  The count is the number of *waiters*, not including the
    master. The master is also blocked until all of the waiters have
    checked in.

    I have not made an provision for timeouts or crashed clients. Lets
    see how it goes.