GitLab

Also add partial support for 11.2 MFS (just kernel right now,
binaries are still 10.3).

Docker containers may be (and default to, and in the shared host case,
must be) deprivileged; thus, they cannot mount devices, much less tell
the kernel (via iscsi userspace tools, etc) to make devices.

Therefore, we must setup any storage backing devices (temp LVs, iscsi
attachments) outside the container.  This commit makes that possible for
rc.storage and linux liblocstorage.  Basically, rc.storage now supports
(for the Linux liblocstorage and Docker) the -j vnodeid calling
convention; and if it's being called on behalf of a vnodeid, it uses
per-vnodeid fstab for any mounts, storage.conf for its state; etc.

I modified libvnode_docker to *not* create virtual networks for
remote blockstore links, because those are pinned to /30s, and thus I
have no client blockstore link address to place on a device in the root
context.  However, I (ab)used the existing Docker network setup for the
blockstore links, and that all happens the same as it used to; we just
no longer create the Docker virtual network nor attach the container to
it.

Finally, I modified tmcd dostorageconfig slightly to return
HOSTIP/HOSTMASK for remote blockstores; and now
libsetup::getstorageconfig will use HOSTIP in preference to its own
HOSTID->HOSTIP translation.  I had to do this so that libvnode_docker in
the root context would not have to go through the mess of translating
HOSTID on behalf of a vnode.

as per issue #440.

If $ETCDIR/ltmap-gzip exists on a clientside node, rc.topomap will only
download the gzip'd versions of ltmap/ltpmap into $BOOTDIR, and
linktest.pl will use them.  Those files are TMLTMAPGZ() and TMLTPMAPGZ().

This is important for multi-thousand node exps, where the lt*map files
easily grow to 250MB or more (and are compressible to 25:1 or so!);
saves CoW virtual disk blocks and raw disk space.  And now that commit
67cd8518 means nodetype no longer uses ltpmap, linktest is the only
consumer of lt*map files.

Provide automated setup of an ssh keypair enabling root to login without
a password between nodes. The biggest challenge here is to get the private
key onto nodes in such a way that a non-root user on those nodes cannot
obtain it. Otherwise that user would be able to ssh as root to any node.
This precludes simple distribution of the private key using tmcd/tmcc as
any user can do a tmcc (tmcd authentication is based on the node, not the
user).

This version does a post-imaging "push" of the private key from boss using
ssh. The key is pushed from tbswap after nodes are imaged but before the
event system, and thus any user startup scripts, are started. We actually
use "pssh" (really "pscp") to scale a bit better, so YOU MUST HAVE THE
PSSH PACKAGE INSTALLED. So be sure to do a:

    pkg install -r Emulab pssh

on your boss node. See the new utils/pushrootkeys.in script for more.

The public key is distributed via the "tmcc localization" command which
was already designed to handle adding multiple public keys to root's
authorized_keys file on a node.

This approach should be backward compatible with old images. I BUMPED THE
VERSION NUMBER OF TMCD so that newer clients can also get back (via
rc.localize) a list of keys and the names of the files they should be stashed
in. This is used to allow us to pass along the SSL and SSH versions of the
public key so that they can be placed in /root/.ssl/<node>.pub and
/root/.ssh/id_rsa.pub respectively. Note that this step is not necessary for
inter-node ssh to work.

Also passed along is an indication of whether the returned key is encrypted.
This might be used in Round 2 if we securely implant a shared secret on every
node at imaging time and then use that to encrypt the ssh private key such
that we can return it via rc.localize. But the client side script currently
does not implement any decryption, so the client side would need to be changed
again in this future.

The per experiment root keypair mechanism has been exposed to the user via
old school NS experiments right now by adding a node "rootkey" method. To
export the private key to "nodeA" and the public key to "nodeB" do:

    $nodeA rootkey private 1
    $nodeB rootkey public 1

This enables an asymmetric relationship such that "nodeA" can ssh into
"nodeB" as root but not vice-versa. For a symmetric relationship you would do:

    $nodeA rootkey private 1
    $nodeB rootkey private 1
    $nodeA rootkey public 1
    $nodeB rootkey public 1

These user specifications will be overridden by hardwired Emulab restrictions.
The current restrictions are that we do *not* distribute a root pubkey to
tainted nodes (as it opens a path to root on a node where no one should be
root) or any keys to firewall nodes, virtnode hosts, delay nodes, subbosses,
storagehosts, etc. which are not really part of the user topology.

For more on how we got here and what might happen in Round 2, see:

    #302

operating in standalone mode (not part of a federation), which would be
the case for everyone that is not us. Further exercise would be to
automate portal setup when part of a federation. Not a big deal to add,
but lets checkpoint what I have done so far.

See emulab/emulab-devel issue #303. Ensure we have a controlled set of
pubkeys in root's .ssh/authorized_keys file when we create and load new
images. But allow for a user added key to survive node reboots if they
customize it within an experiment.

We want both to wind up in authorized_keys.

See clientside/tmcc/linux/docker/README.md for design notes.
See clientside/tmcc/linux/docker/dockerfiles/README.md for a description
of how we automatically Emulabize existing Docker images.

Also, this mostly fits within the existing vnodesetup path, but I did modify
mkvnode.pl to allow the libvnode backend to provide a vnodePoll wait
loop instead of the builtin vnodeState loop.

This allows other callers than rc.hostnames (i.e. docker clientside) to
generate the hostname list for an experiment.

Also remove use of local5.* in syslog.conf so we can turn around
and re-add it!

in blockstore-related VGs.

Right now, you have to decide globally and in advance, what disk types
are going to be included in blockstore pools. Then you set the sitevar
accordingly and then set the DB sysvol/nonsysvol/any node_type_features
to reflect the amount of storage available on just drives of that type.

This value is passed to clients via the otherwise unused PROTO field
of the blockstore line (when CMD=SLICE and CLASS=local), so this change
is backward compatible (OS images with older client code will ignore it
and just give you blockstores including all the devices).

So at Wisconsin, I set storage/local/disktype to "HDD-only" and tweak
the node_type_attributes '?+disk_any' and '?+disk_nonsysvol' to not
include the space for the 1 or 2 SSD drives in each machine. tmcd passes
the PROTO=HDD-only value and the client sees that and does not include
any SSD devices among the eligible devices from which to create the VG.

The hope is that ultimately, we could get rid of the sitevar and use the
PROTO field to select, per-blockstore, its type (only HDD, only SSD).
But that will require additional per node (type) assign features
differentiating the amount of each type available.

See emulab-devel issue 227 for details.

Also, on a "reset" clean out the correct BDB files. It has been
a long time since they used ".db" as the suffix. Now there are
".pag" and ".dir" files. We haven't noticed because we don't really
use the "reset" operation. The prepare script just removes
everything in /var/emulab/db.

Get rid of ELVIN_COMPAT and CONFIG_OPSVM from elabinelab land.
These options still exist throughout the install code, didn't touch that.

Normally, "pkg" will update itself to the latest version from FreeBSD
Central, but that version is now built with FreeBSD 10.3 and the binary
is incompatible with earlier 10.x versions.

So we go to heroic efforts to install the version from the pre-built
Emulab packages.

Support FreeBSD 10.3. We will need to be moving to this before long
as 10.2 EOLs in two days.

Support setup of "Emulab-aware" ZFS use in install scripts. Note that
the core support code was already done (WITHZFS, WITHAMD). Mostly this
involves changes to setup either amd (WITHAMD==1) or autofs (WITHAMD==0)
on the boss node and to NOT add mounts of /{users,groups,proj} to
/etc/fstab. We still need to add a section to the install documentation
about setting up a zpool for Emulab to use. There was also a fix to the
firstuser script which did not do the account setup correctly.

Support setup of ZFS in elabinelab. The elabinelab attributes CONFIG_ZFS
and CONFIG_AUTOFS are used to convey intent here. Currently they can only
be used in an "ops+fs" config (e.g., the standard boss and ops config,
NOT the seperate fs node config). It should work with either the physical
or virtual node setups:

* For the physical node setup, we actually use local blockstores in the
  ops node config: a SYSVOL blockstore for /usr/testbed and a tiny 1Mib
  NONSYSVOL blockstore. The latter blockstore is not actually used, we
  just make it to force setup of a ZFS zpool which we then use for the
  inner elab.

* For the virtual node setup, we just identify the virtual EXTRADISK
  intended for "/q" and create a zpool on that device.

I would like to change all physical elabinelab setups to use blockstors
rather than the current hacky mkextrafs usage. But that is a task for
another day.

Finally, a couple of random changes in elabinelab code: change the
CentOS image downloaded to CENTOS7-64-STD, increased the default sizes
of the EXTRADISKS used in the VM config.

For the case in which mkextrafs is used to create local homedirs/projdirs:

Look for the desired mount point (/local) in /etc/fstab and use that if
it exists (i.e., that FS was already setup by the blockstore system or a
previous mkextrafs).

Otherwise, look for /var/emulab/boot/extrafs which should contain info
left behind by the local blockstore setup code indicating a FS or unused
device to use. For an unused device, rc.storage will identify the largest
available device that is at least 10MB.

The prepare script now supports pre and post hooks.  It runs all hooks
in rc order, from the DYNRUNDIR/prepare.pre.d and BINDIR/prepare.pre.d
dirs (rc order in this case is the BSD order, or my version of it ---
any file prefixed with a number is run in numeric order; other files are
run sorted alphabetically following numeric files).  Post hooks are in
prepare.post.d, and are run at the end of prepare.

(DYNRUNDIR is always /var/run/emulab .  STATICRUNDIR is usually
/etc/emulab/run but could be /etc/testbed/run, depending on the
clientside installation.)

We now allow users to override our default interface configuration --
and if they do, and tell us about it by writing a file in either
$DYNRUNDIR or $STATICRUNDIR named interface-done-$mac , we will not
attempt to configure it, and will assume they have done it!  If they are
nice to us and write
  $iface $ipaddr $mac
into the file, we will parse that and put it into the @ifacemap and
%mac2iface structures in doboot().  We do *not* attempt to provide them
the ifconfig info in env vars or anything; they have to grok our
ifconfig file format, in all its potential glory.

We read the hosts.head file(s) from /etc, DYNRUNDIR, and STATICRUNDIR,
and prepend them to our Emulab hosts content.  Then, we append the
content of the hosts.tail file(s) from /etc, DYNRUNDIR, and STATICDIR
--- and that file becomes the new /etc/hosts file.

getmanifest() has become getrcmanifest() to avoid confusion with the
GENI manifest.  Also, it now supports local manifests embedded in the
filesystem from $DYNRUNDIR and $STATICRUNDIR (priority is manifest from
exp, then DYNRUNDIR, then STATICRUNDIR).  All manifests read and
applied.  Local manifests may also reference local files instead of blob
ids, of course.  It is important to support local manifests so that
experimenters can hook our services by default in the disk image.

call tc qdisc/class command lines. Had to implement a perl script to get
the current delay parameters cause parsing all that stuff in C is too
much of a pain, sorry Mike. Reshuffle a few things around since we are
now going to install delay-agent and rc.delayagent on linux.

Reset on local/remote blockstores ensures that there is no blockstore related
state left in the root filesystem (e.g., mounts in /etc/fstab, iSCSI config,
LVM/ZFS state). It does this in such a way that upon reboot, all the necessary
state is recreated.

What this means is that you should now be able to take an image of a node
that uses blockstores and have that image actually work on another node!
Previously, there could/would be leftover blockstore turds that would make
the new image fail to boot.

Of course, this won't work until the standard images are remade and will
then only work for those images or images derived from them.

This is when we are cleaning up to make an image. Seems like the correct
thing to do is process the startup scripts in reverse order like we do for
a "shutdown". Mostly this doesn't matter, but it does matter for the
blockstore cleanup where we need to process the remote BSes before the locals.

Reworked lookup/handling of linked interface objects for IP alias
vinterfaces.

I had added use of this feature to make startup pause until the name
server was responding to avoid transient failures in elabinelab with
an "inner control net". But this feature does not work with a firewall
as it tries to do the named wait before any firewall rules have been loaded.

Big changes a comin' to try to get us back on the supported path.

 * perl 5.14 -> 5.20
 * mysql 5.1 -> 5.5
 * php 5.4   -> 5.6
 * tcl 8.4   -> 8.6
 * number of vim patches up to 683.

Not everything tested yet, but getting there.

Specific changes:

 * New install/ports directory. New packages for FreeBSD 10.1 are version
   6.1. Cleaned up the ports' Makefiles getting rid of conditionals for
   all older versions. Also got rid of ports we don't use. Old ports tree
   is now install/oports.

 * Install script changes. Make sure /usr/bin/perl and /usr/local/bin/python
   links exist. Ports no longer make these but we use them in '#!'. Changes
   to mysql install and startup script--mysql has changed a LOT since we did
   the support in 4.x. Create syslog entry for named.log. Make sure php.conf
   loads the legacy "mysql" module rather than using "mysqli".

 * Elabinelab support. reflect new packages, remove all old packages
   (except perl) before installing new versions, install "extras" package,
   make sure sendmail cert get regenerated, make sure /usr/bin/perl link
   exists, make sure /usr/local/bin/python link exists.

 * Custom ports. otcl and xerces-c2 have both been removed from the ports
   tree as of Q2 2015. ipmitool-devel is a port for the latest version of
   ipmitool. The FreeBSD port is still a rev behind here. We need the
   newer version as it appears to make our SOL consoles more stable.

 * Random. Fixed prerender as neato output has changed again. Tweak to
   sslxmlrpc_server to reflect change in an underlying library. Tweak to
   db/libdb.py.in to turn on autocommit which matters now as mysql 5.5 will
   hang on a metadata lock otherwise. Remade eventsys perl/python stubs
   with SWIG 2.0. SWIG 1.3 did not produce working stubs for perl 5.20.

Specific un-changes:

 * Apache is still at 2.2. I lack the guts and skilz to upgrade to 2.4.

 * Xerces library is still at (now unsupported) 2.8. Assign will need
   changes before we can move to 3.x.

 * Python is still 2.7.

Thanks to Keith Sklower for all the work he did converting ports!

Yes, out of the blue and off the wall. But I got tired of trying to
guess what we had Linux and FreeBSD use. I was surprised to discover
that we were using UDP on Linux (which caused Clemson CloudLab to fail
because they have jumbo frames enabled on their control net switches
but ops had the MTU set to 1500).

Anyway, here it is. The default setting is UDP for backward compat.
We should probably set it to TCP nowadays. There is also an 'osdefault'
setting which says use the default setting on the client OS.