1. 08 Aug, 2018 4 commits
    • David Johnson's avatar
      Add Docker container blockstore support. · 9bf09981
      David Johnson authored
      Docker containers may be (and default to, and in the shared host case,
      must be) deprivileged; thus, they cannot mount devices, much less tell
      the kernel (via iscsi userspace tools, etc) to make devices.
      
      Therefore, we must setup any storage backing devices (temp LVs, iscsi
      attachments) outside the container.  This commit makes that possible for
      rc.storage and linux liblocstorage.  Basically, rc.storage now supports
      (for the Linux liblocstorage and Docker) the -j vnodeid calling
      convention; and if it's being called on behalf of a vnodeid, it uses
      per-vnodeid fstab for any mounts, storage.conf for its state; etc.
      
      I modified libvnode_docker to *not* create virtual networks for
      remote blockstore links, because those are pinned to /30s, and thus I
      have no client blockstore link address to place on a device in the root
      context.  However, I (ab)used the existing Docker network setup for the
      blockstore links, and that all happens the same as it used to; we just
      no longer create the Docker virtual network nor attach the container to
      it.
      
      Finally, I modified tmcd dostorageconfig slightly to return
      HOSTIP/HOSTMASK for remote blockstores; and now
      libsetup::getstorageconfig will use HOSTIP in preference to its own
      HOSTID->HOSTIP translation.  I had to do this so that libvnode_docker in
      the root context would not have to go through the mess of translating
      HOSTID on behalf of a vnode.
      9bf09981
    • Leigh Stoller's avatar
      Left this out of previous commit. · ef517168
      Leigh Stoller authored
      ef517168
    • Leigh Stoller's avatar
      Big set of changes for deferred/scheduled/offline aggregates: · 6f17de73
      Leigh Stoller authored
      * I started out to add just deferred aggregates; those that are offline
        when starting an experiment (and marked in the apt_aggregates table as
        being deferable). When an aggregate is offline, we add an entry to the
        new apt_deferred_aggregates table, and periodically retry to start the
        missing slivers. In order to accomplish this, I split create_instance
        into two scripts, first part to create the instance in the DB, and the
        second (create_slivers) to create slivers for the instance. The daemon
        calls create_slivers for any instances in the deferred table, until
        all deferred aggregates are resolved.
      
        On the UI side, there are various changes to deal with allowing
        experiments to be partially create. For example used to wait till we
        have all the manifests until showing the topology. Now we show the
        topo on the first manifest, and then add them as they come in. Various
        parts of the UI had to change to deal with missing aggregates, I am
        sure I did not get them all.
      
      * And then once I had that, I realized that "scheduled" experiments was
        an "easy" addition, its just a degenerate case of deferred. For this I
        added some new slots to the tables to hold the scheduled start time,
        and added a started stamp so we can distinguish between the time it
        was created and the time it was actually started. Lots of data.
      
        On the UI side, there is a new fourth step on the instantiate page to
        give the user a choice of immediate or scheduled start. I moved the
        experiment duration to this step. I was originally going to add a
        calendar choice for termination, but I did not want to change the
        existing 16 hour max duration policy, yet.
      6f17de73
    • Leigh Stoller's avatar
      Add missing log file rolling. · cac4744f
      Leigh Stoller authored
      cac4744f
  2. 07 Aug, 2018 4 commits
  3. 06 Aug, 2018 2 commits
    • David Johnson's avatar
      Fix a couple minor Docker clientside bugs. · 2a9160f0
      David Johnson authored
      2a9160f0
    • David Johnson's avatar
      In docker image emulabization, attempt to combine COPY instructions. · 18361092
      David Johnson authored
      We now try to emulate any simple COPY <src> <dst> instructions via rsync
      prior to image build.
      
      This *does* mean that artifact builder scripts must be careful to create
      all necessary dirs according to the base image semantics, because the
      base image content is not there when we emulate the COPY instructions.
      For instance, many of the modified Dockerfile-runit and
      runit-artifacts.sh files depended on built runit packages being
      installed into /tmp in the final image -- but they didn't create the
      /tmp dir because the COPY instruction they used was running atop a
      fully-populated base image that already had /tmp.  Thus, the
      runit-artifacts.sh scripts had to be changed to create /tmp with the
      proper permissions.
      18361092
  4. 03 Aug, 2018 1 commit
  5. 02 Aug, 2018 2 commits
  6. 01 Aug, 2018 1 commit
  7. 31 Jul, 2018 2 commits
  8. 30 Jul, 2018 23 commits
  9. 27 Jul, 2018 1 commit