GitLab

Also add partial support for 11.2 MFS (just kernel right now,
binaries are still 10.3).

boss.

We try to emulate the standard Docker CLI's image handling.  Thus, if
user specifies an image like 'ubuntu', we turn that into
'ubuntu:latest'.  If user does not supply a registry host, we try
'registry.hub.docker.com'.  If they do not specify a registry host or
specify a registry host that is either registry-1.docker.io or
registry.hub.docker.com, and their image does not contain a /, we
prepend 'library/' to it (I *think* this is the right heuristic, but
it's inference).

For Dockerfiles, we must be able to download it, and it must contain a
line matching ^\s*FROM (i.e. a valid FROM instruction, which all
Dockerfiles must have).  We try to support DOS-mode textfiles too, but
only ASCII.

Might need to loosen these checks.

hardware address.
Fixed a egrep expression which was preventing the matching of said list.

(If the DOCKER-USER chain exists, we now can enable Docker's iptables
handling because its rules will no longer supplant ours.  But then, we
also have to permit the default network and our control network to reach
the outside world; that behavior seems to have changed somewhere along
the line.)

thirty seconds!

interface.

* In the CM we have always ignored the BW settings on a shared lan,
  since there is no way to set the properties in geni-lib for such a
  lan. There is the local hack I added (linkwide properties), but that
  was also ignored for shared vlans. Now I am looking to see if there is
  a bandwidth specification there, and using that. I assume we do not
  care about delay/loss since well, we never have before.

* But even so, the mapper was ignoring it too. But we also have the code
  that tries to not use 10G interfaces unless explicitly asked for a 10G
  link, and that is not in the shared vlan path. So ... I made a few
  changes, the worst that can happen is that I broke share vlans for
  everyone except this one case.

Docker containers may be (and default to, and in the shared host case,
must be) deprivileged; thus, they cannot mount devices, much less tell
the kernel (via iscsi userspace tools, etc) to make devices.

Therefore, we must setup any storage backing devices (temp LVs, iscsi
attachments) outside the container.  This commit makes that possible for
rc.storage and linux liblocstorage.  Basically, rc.storage now supports
(for the Linux liblocstorage and Docker) the -j vnodeid calling
convention; and if it's being called on behalf of a vnodeid, it uses
per-vnodeid fstab for any mounts, storage.conf for its state; etc.

I modified libvnode_docker to *not* create virtual networks for
remote blockstore links, because those are pinned to /30s, and thus I
have no client blockstore link address to place on a device in the root
context.  However, I (ab)used the existing Docker network setup for the
blockstore links, and that all happens the same as it used to; we just
no longer create the Docker virtual network nor attach the container to
it.

Finally, I modified tmcd dostorageconfig slightly to return
HOSTIP/HOSTMASK for remote blockstores; and now
libsetup::getstorageconfig will use HOSTIP in preference to its own
HOSTID->HOSTIP translation.  I had to do this so that libvnode_docker in
the root context would not have to go through the mess of translating
HOSTID on behalf of a vnode.

* I started out to add just deferred aggregates; those that are offline
  when starting an experiment (and marked in the apt_aggregates table as
  being deferable). When an aggregate is offline, we add an entry to the
  new apt_deferred_aggregates table, and periodically retry to start the
  missing slivers. In order to accomplish this, I split create_instance
  into two scripts, first part to create the instance in the DB, and the
  second (create_slivers) to create slivers for the instance. The daemon
  calls create_slivers for any instances in the deferred table, until
  all deferred aggregates are resolved.

  On the UI side, there are various changes to deal with allowing
  experiments to be partially create. For example used to wait till we
  have all the manifests until showing the topology. Now we show the
  topo on the first manifest, and then add them as they come in. Various
  parts of the UI had to change to deal with missing aggregates, I am
  sure I did not get them all.

* And then once I had that, I realized that "scheduled" experiments was
  an "easy" addition, its just a degenerate case of deferred. For this I
  added some new slots to the tables to hold the scheduled start time,
  and added a started stamp so we can distinguish between the time it
  was created and the time it was actually started. Lots of data.

  On the UI side, there is a new fourth step on the instantiate page to
  give the user a choice of immediate or scheduled start. I moved the
  experiment duration to this step. I was originally going to add a
  calendar choice for termination, but I did not want to change the
  existing 16 hour max duration policy, yet.

We now try to emulate any simple COPY <src> <dst> instructions via rsync
prior to image build.

This *does* mean that artifact builder scripts must be careful to create
all necessary dirs according to the base image semantics, because the
base image content is not there when we emulate the COPY instructions.
For instance, many of the modified Dockerfile-runit and
runit-artifacts.sh files depended on built runit packages being
installed into /tmp in the final image -- but they didn't create the
/tmp dir because the COPY instruction they used was running atop a
fully-populated base image that already had /tmp.  Thus, the
runit-artifacts.sh scripts had to be changed to create /tmp with the
proper permissions.