with cause and optionally freeze the user. "Cause" means you can paste
in a block of text that is emailed to the user.

an error in some situations. Also increase limit on XEN core from 4 to 12.

hopefully fixes the race.

We try to emulate the standard Docker CLI's image handling.  Thus, if
user specifies an image like 'ubuntu', we turn that into
'ubuntu:latest'.  If user does not supply a registry host, we try
'registry.hub.docker.com'.  If they do not specify a registry host or
specify a registry host that is either registry-1.docker.io or
registry.hub.docker.com, and their image does not contain a /, we
prepend 'library/' to it (I *think* this is the right heuristic, but
it's inference).

For Dockerfiles, we must be able to download it, and it must contain a
line matching ^\s*FROM (i.e. a valid FROM instruction, which all
Dockerfiles must have).  We try to support DOS-mode textfiles too, but
only ASCII.

Might need to loosen these checks.

* In the CM we have always ignored the BW settings on a shared lan,
  since there is no way to set the properties in geni-lib for such a
  lan. There is the local hack I added (linkwide properties), but that
  was also ignored for shared vlans. Now I am looking to see if there is
  a bandwidth specification there, and using that. I assume we do not
  care about delay/loss since well, we never have before.

* But even so, the mapper was ignoring it too. But we also have the code
  that tries to not use 10G interfaces unless explicitly asked for a 10G
  link, and that is not in the shared vlan path. So ... I made a few
  changes, the worst that can happen is that I broke share vlans for
  everyone except this one case.

process that hands off).

so that the SCS will accept it. Oh, and I learned that the SCS is
sensitive to the order of elements in the XML file! The link *must* be
after the nodes, or the SCS refuses to do anything with it. Sheesh.

as per issue #440.

based slices.

description in CreateImage();

* Add portal url to the existing emulab extension that tells the CM the
  CreateSliver() is coming from the Portal. Always send this info, not
  just for the Emulab Portal.

* Stash that info in the geni slice data structure so we can add links
  back to the portal status page for current slices.

* Add routines to generate a portal URL for the history entries, since
  we will not have those links for historical slices. Add links back to
  the portal on the showslice and slice history pages.

IPs on the lan and we fill them in, we put those IPs into the manifest,
but we also have to put the netmask in, since without that, the update
operation will get confused.

if the underlying Emulab experiment; return a busy error instead.

scheduler does not like duplicates.

for each node in the list. Also, add a new column to show what image
is running,

Portal.

on the new Start() path (which uses os_setup), since the experiment
state is ACTIVATING.

tell the CM that we want the type default OS, which might be just an
OSID. This avoids having to specify the OS or use setUseTypeDefaultImage
in the geni-lib script.

the new os_setup path in StartSliver(). This was causing a variety of
annoying emails to the tbops lists cause DeleteSliver and StartSliver()
not doing the proper cancel dance.

The docker VM server-side goo is mostly identical to Xen, with slightly
different handling for parent images.  We also support loading external
Docker images (i.e. those without a real imageid in our DB; in that
case, user has to set a specific stub image, and some extra per-vnode
metadata (a URI that points to a Docker registry/image repo/tag);
the Docker clientside handles the rest.

Emulab Docker images map to a Emulab imageid:version pretty seamlessly.
For instance, the Emulab `emulab-ops/docker-foo-bar:1` image would map
to `<local-registry-URI>/emulab-ops/emulab-ops/docker-foo-bar:1`; the
mapping is `<local-registry-URI>/pid/gid/imagename:version`.  Docker
repository names are lowercase-only, so we handle that for the user; but
I would prefer that users use lowercase Emulab imagenames for all Docker
images; that will help us.  That is not enforced in the code; it will
appear in the documentation, and we'll see.

Full Docker imaging relies on several other libraries
(https://gitlab.flux.utah.edu/emulab/pydockerauth,
https://gitlab.flux.utah.edu/emulab/docker-registry-py).  Each
Emulab-based cluster must currently run its own private registry to
support image loading/capture (note however that if capture is
unnecessary, users can use the external images path instead).  The
pydockerauth library is a JWT token server that runs out of boss's
Apache and implements authn/authz for the per-Emulab Docker registry
(probably running on ops, but could be anywhere) that stores images and
arbitrates upload/download access.  For instance, nodes in an experiment
securely pull images using their pid/eid eventkey; and the pydockerauth
emulab authz module knows what images the node is allowed to pull
(i.e. sched_reloads, the current image the node is running, etc).  Real
users can also pull images via user/pass, or bogus user/pass + Emulab
SSL cert.  GENI credential-based authn/z was way too much work, sadly.
There are other auth/z paths (i.e. for admins, temp tokens for secure
operations) as well.

As far as Docker image distribution in the federation, we use the same
model as for regular ndz images.  Remote images are pulled in to the
local cluster's Docker registry on-demand from their source cluster via
admin token auth (note that all clusters in the federation have
read-only access to the entire registries of any other cluster in the
federation, so they can pull images).  Emulab imageid handling is the
same as the existing ndz case.  For instance, image versions are lazily
imported, on-demand; local version numbers may not match the remote
image source cluster's version numbers.  This will potentially be a
bigger problem in the Docker universe; Docker users expect to be able to
reference any image version at any time anywhere.  But that is of course
handleable with some ex post facto synchronization flag day, at least
for the Docker images.

The big new thing supporting native Docker image usage is the guts of a
refactor of the utils/image* scripts into a new library, libimageops;
this is necessary to support Docker images, which are stored in their
own registry using their own custom protocols, so not amenable to our
file-based storage.  Note: the utils/image* scripts currently call out
to libimageops *only if* the image format is docker; all other images
continue on the old paths in utils/image*, which all still remain
intact, or minorly-changed to support libimageops.

libimageops->New is the factory-style mechanism to get a libimageops
that works for your image format or node type.  Once you have a
libimageops instance, you can invoke normal image logical operations
(CreateImage, ImageValidate, ImageRelease, et al).  I didn't do every
single operation (for instance, I haven't yet dealt with image_import
beyond essentially generalizing DownLoadImage by image format).
Finally, each libimageops is stateless; another design would have been
some statefulness for more complicated operations.   You will see that
CreateImage, for instance, is written in a helper-subclass style that
blurs some statefulness; however, it was the best match for the existing
body of code.  We can revisit that later if the current argument-passing
convention isn't loved.

There are a couple outstanding issues.  Part of the security model here
is that some utils/image* scripts are setuid, so direct libimageops
library calls are not possible from a non-setuid context for some
operations.  This is non-trivial to resolve, and might not be worthwhile
to resolve any time soon.  Also, some of the scripts write meaningful,
traditional content to stdout/stderr, and this creates a tension for
direct library calls that is not entirely resolved yet.  Not hard, just
only partly resolved.

Note that tbsetup/libimageops_ndz.pm.in is still incomplete; it needs
imagevalidate support.  Thus, I have not even featurized this yet; I
will get to that as I have cycles.

history.

directly (on the Cloudlab clusters); we were losing a lock out that
allowed DeleteSliver() to run while in the middle of a CreateSliver().
This was resulting in a lot of email about node failures since the nodes
were getting yanked out from underneath the CreateSliver(). From the
user perspective, this did not matter much, since they wanted the slice
gone, but it finally bothered me enough to look more closely.

that is too long.