1. 06 Nov, 2018 2 commits
  2. 05 Nov, 2018 2 commits
    • Leigh Stoller's avatar
      Working Mellanox user alloc switch support (issue #445): · 95e7bded
      Leigh Stoller authored
      * The primary problem with the mellanox is that the install image does a
        kexec out of ONIE into Linux, spends 30+ minutes doing stuff, and then
        reboots. This throws the reload state machine out of whack cause we do
        not get a chance to send the RELOADDONE state. So ... some change to
        rc.testbed and rc.reload on the USB dongle: the ONIE MFS sends
        RELOADING and writes a flag file to the ONIE partition on the
        "disk" (not the usb). Then the kexec into MLNX, the install happens,
        and reboots. The next boot into ONIE sees the flag file, erases it and
        sends REDLOADDONE. Waits for a bit, and then continues on the normal
        path. This abuses stated in that there a whiny messages in the stated
        log file, but I am immune to stated whining.
      
      * Another item of note is that the switch DHCPs, but only to get the IP
        info, there is no ability to give it an initial config file like we
        can with the Dell switches. The main problem here is that the switch
        comes up with its default login/password which is obviously well known
        cause its in the manual. That means there is a window where the switch
        is vulnerable, but since we block the switches from the public side,
        this is not a serious problem. As soon as we can get in (sshd is
        running) we login and update the config with passwords, keys,
        etc.
      
      * Other changes to the machine dependent osload library module, I had
        done some of this before switching to the Dells way back when, but it
        needed to be updated/completed.
      95e7bded
    • Leigh Stoller's avatar
      Allow NTPSERVER override in the NS file. · 849432e1
      Leigh Stoller authored
      849432e1
  3. 30 Oct, 2018 2 commits
  4. 26 Oct, 2018 6 commits
  5. 25 Oct, 2018 3 commits
    • David Johnson's avatar
      Replace the Docker entrypoint/cmd/env implementation for augmented images. · a986a085
      David Johnson authored
      (Also, add support for user to change container entrypoint at runtime.
      Note also that the server side now stores the entrypoint/cmd/env
      attributes as base64url-encoded virt_node_attributes, so that we can
      just use the existing table_regex for those values.)
      
      We add a new runit service (/etc/service/dockerentrypoint) to
      clientside/tmcc/linux/docker/dockerfiles/common to handle the
      entrypoint/cmd/env/workingdir/user emulation.  From the comments:
      
        Docker's semantics for ENTRYPOINT/CMD vary depending on if those
        values are specified as arrays of string, or simple as single strings
        (which must be interpreted by /bin/sh -c).
      
        Handling all the quoting possibilities in the shell is a major pain.
        So, this script handles the basic stuff (in particular, sourcing env
        vars, because we want the shell to interpret them!) -- then execs our
        perl companion script (run.pl) to deal with the entrypoint/command
        files that libvnode_docker::emulabizeImage and
        libvnode_docker::vnodeCreate populated.
      
        libvnode_docker creates these single-line files in /etc/emulab/docker
        as either string:hexstr(<entrypoint-or-cmd-string>), or
        array:hexstr(a[0]),hexstr(a[1])... .  This allows us to preserve the
        original type of the image's entrypoint/cmd as well as the runtime
        entrypoint/cmd, and to preserve the exact bytes for the eventual final
        call to exec.
      
        The static files builtin to an emulabized image are
        /etc/emulab/docker/{entrypoint.image,cmd.image}, and those created
        dynamically at runtime if user changes the entrypoint or cmd are
        bind-mounted to /etc/emulab/docker{entrypoint.runtime,cmd.runtime}.
      
        Given the presence (or absence!) of those files, this script
        implements the emulation, based upon the content in those files.
      a986a085
    • David Johnson's avatar
      993e9f8c
    • David Johnson's avatar
      e48155a7
  6. 02 Oct, 2018 1 commit
    • David Johnson's avatar
      Force a regeneration of /etc/machine-id in prepare for systemd-based OSes. · b639e7f1
      David Johnson authored
      (Also link the dbus machine-id file to the one systemd will generate on
      the next boot.  This seems safe and correct.)
      
      Certain things (like systemd's dhcp client) use the machine-id as a seed
      for derived values.  For instance, systemd's dhcp client offers a
      ClientIdentifier in the new client style, and some servers will return
      the same address to *all* requesting clients, instead of returning only
      based on source MAC.  Can't have any of that confusion.
      b639e7f1
  7. 26 Sep, 2018 1 commit
  8. 04 Sep, 2018 1 commit
  9. 29 Aug, 2018 3 commits
  10. 24 Aug, 2018 2 commits
  11. 22 Aug, 2018 1 commit
  12. 21 Aug, 2018 3 commits
  13. 17 Aug, 2018 1 commit
  14. 16 Aug, 2018 3 commits
  15. 15 Aug, 2018 3 commits
  16. 14 Aug, 2018 1 commit
  17. 10 Aug, 2018 1 commit
  18. 08 Aug, 2018 1 commit
    • David Johnson's avatar
      Add Docker container blockstore support. · 9bf09981
      David Johnson authored
      Docker containers may be (and default to, and in the shared host case,
      must be) deprivileged; thus, they cannot mount devices, much less tell
      the kernel (via iscsi userspace tools, etc) to make devices.
      
      Therefore, we must setup any storage backing devices (temp LVs, iscsi
      attachments) outside the container.  This commit makes that possible for
      rc.storage and linux liblocstorage.  Basically, rc.storage now supports
      (for the Linux liblocstorage and Docker) the -j vnodeid calling
      convention; and if it's being called on behalf of a vnodeid, it uses
      per-vnodeid fstab for any mounts, storage.conf for its state; etc.
      
      I modified libvnode_docker to *not* create virtual networks for
      remote blockstore links, because those are pinned to /30s, and thus I
      have no client blockstore link address to place on a device in the root
      context.  However, I (ab)used the existing Docker network setup for the
      blockstore links, and that all happens the same as it used to; we just
      no longer create the Docker virtual network nor attach the container to
      it.
      
      Finally, I modified tmcd dostorageconfig slightly to return
      HOSTIP/HOSTMASK for remote blockstores; and now
      libsetup::getstorageconfig will use HOSTIP in preference to its own
      HOSTID->HOSTIP translation.  I had to do this so that libvnode_docker in
      the root context would not have to go through the mess of translating
      HOSTID on behalf of a vnode.
      9bf09981
  19. 07 Aug, 2018 1 commit
  20. 06 Aug, 2018 2 commits
    • David Johnson's avatar
      Fix a couple minor Docker clientside bugs. · 2a9160f0
      David Johnson authored
      2a9160f0
    • David Johnson's avatar
      In docker image emulabization, attempt to combine COPY instructions. · 18361092
      David Johnson authored
      We now try to emulate any simple COPY <src> <dst> instructions via rsync
      prior to image build.
      
      This *does* mean that artifact builder scripts must be careful to create
      all necessary dirs according to the base image semantics, because the
      base image content is not there when we emulate the COPY instructions.
      For instance, many of the modified Dockerfile-runit and
      runit-artifacts.sh files depended on built runit packages being
      installed into /tmp in the final image -- but they didn't create the
      /tmp dir because the COPY instruction they used was running atop a
      fully-populated base image that already had /tmp.  Thus, the
      runit-artifacts.sh scripts had to be changed to create /tmp with the
      proper permissions.
      18361092