1. 15 May, 2018 5 commits
    • Leigh B Stoller's avatar
      Changes for apache24 upgrade. · b94aa21e
      Leigh B Stoller authored
      b94aa21e
    • Mike Hibler's avatar
      Correct the list of images to Xen-ify. · a4636949
      Mike Hibler authored
      a4636949
    • Mike Hibler's avatar
      Another apache 2.4 fix. · 9771e08e
      Mike Hibler authored
      9771e08e
    • Leigh B Stoller's avatar
    • David Johnson's avatar
      Add a candidate systemd-networkd control net path for Ubuntu 18. · bbf4824b
      David Johnson authored
      Drop 99-emulab-networkd.rules into /etc/udev/rules.d,
      emulab-networkd@.service into /lib/systemd/system, and do
      `systemctl enable systemd-networkd.service`.  I would also do
      `ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf`; we don't
      want any part of systemd-resolved.
      
      This is the only way I could come up with to make systemd-networkd
      listen on all wired ethernet ifaces, and get it into the right state.
      systemd-networkd is like NetworkManager in some ways.  I don't know if
      it's better or worse.  The main problem for us is that it doesn't
      support hooks, and that it runs its own dhcp client (which is crappy and
      inflexible compared to dhclient).  We don't need anything super flexible
      though since we don't use custom options, and it has an option to ignore
      the lease lifetime and leave the iface up even if expiry happens before
      renewal can take place.
      
      Anyway, here's the flow: udev rules tell systemd to require a unit
      (emulab-networkd@$iface.service) for each physical wired ethernet
      device.  These files write runtime .network files into
      /run/systemd/network, which are read by systemd-networkd.service if it
      is enabled.  We just tell systemd-networkd to listen on them all, then
      we use its waiter script.  However, its wait script is broken and/or the
      documentation is contradictory, so we don't trust it to tell us which
      interface comes up; we invoke its networkctl client to tell us.  We then
      write the up iface to /run/emulab/cnet, rm -f the .network files for all
      ifaces other than the control net device (cause we don't want
      systemd-networkd to manage expt network devices), and restart
      systemd-networkd so it can appropriately report that it is "managing"
      the control net device, but not the expt devices.  It is very
      unfortunate that we have to restart systemd-networkd, but there is no
      other way to tell it about new configuration.  If you HUP it, it dies.
      These systemd folks must live in a box where all use cases look alike.
      
      Note that if the user overrides with their own .network scripts in
      /etc/systemd/network that Match any of the physical NICs, those take
      precedence over our per-interface scripts, which is what we want.
      
      Note also that to do something like move the real control net into a
      bridge, you can just remove or update the .network file in
      /run/systemd/network, and restart systemd-networkd.  It should re-dhcp
      on the bridge.  It does not remove NIC configuration at exit; it just
      "catches up" with any NIC address when it is restarted.
      
      I have not done all the dhclient-exit-hook normal goo (i.e. writing the
      control net controlif/IP into the right place, etc), but that should all
      be possible.
      
      Finally: I haven't yet decided that this is the path for Ubuntu 18.  We
      might well ignore this and fall back to the old /etc/network/interfaces
      ifupdown strategy.  At least there it is possible to use a real
      dhclient.
      bbf4824b
  2. 14 May, 2018 8 commits
  3. 12 May, 2018 1 commit
  4. 10 May, 2018 4 commits
    • Leigh B Stoller's avatar
    • David Johnson's avatar
      Rebuild the shadow package in docker emulabization of alpine linux. · 0526c21d
      David Johnson authored
      Alpine's shadow is like archlinux; it does not support uppercase
      chars in user/group names.  So, fix that along with the runit install.
      0526c21d
    • David Johnson's avatar
      Merge branch 'master' into 'master' · bc6429b3
      David Johnson authored
      Fixing errors in emulabizing Docker images
      
      See merge request !36
      bc6429b3
    • Elijah Grubb's avatar
      Fixing errors in emulabizing Docker images · b6e0952e
      Elijah Grubb authored
      Fixed issue with cmd and entrypoint values being set
      incorrectly
      
      Fixed issue with rcS init.d process negatively hurting
      our networking settings.
      
      Better handling of whitespace in Environment variable
      values.
      
      Squashed commit of the following:
      
      commit 9aff9b12aa6a60ebb66f10f0a044bf2288b492be
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 9 19:15:40 2018 -0600
      
          Handling environment variable values with whitespace
      
      commit c9a16bbd53c183d7f8259d878efb0be07bc8703c
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 9 17:07:43 2018 -0600
      
          Fixing racing issue and rcs issue
      
      commit 10bf4517c419fc82f89d084c4f45d60c05a67834
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 9 16:00:45 2018 -0600
      
          Setting cmd to be empty and entrypoint to be runit
      b6e0952e
  5. 09 May, 2018 8 commits
    • David Johnson's avatar
      Merge branch 'master' into 'master' · ccd50cbe
      David Johnson authored
      Fixing cmd/entrypoint issues and adding support for debian:sid
      
      See merge request !35
      ccd50cbe
    • Elijah Grubb's avatar
      Fixing bugs to support entrypoint/cmd from Docker · ff25cdf8
      Elijah Grubb authored
      Fixed issue involving alternate users being set in
      the image's Dockerfile.
      
      Fixed issue with accurate permissions to run runit service.
      
      Fixed issue with quotes to properly handle sh commands.
      
      Fixed issue where Docker images based on scratch are not in
      an array like other Docker images, so config attributes are
      preserved.
      ff25cdf8
    • Elijah Grubb's avatar
      Adding support for debian sid · 54dd7a77
      Elijah Grubb authored
      Modified the analyze script that if it sees a debian
      without a version, it should update it to debianS.
      Then added a symlink that points debianS to debian9.
      This with some other changes David made should result
      in functioning support of debian:sid in Docker
      
      Squashed commit of the following:
      
      commit 7af0b39cbcd9059f1af35e2b778b3879019aa06e
      Merge: dff5441da 9b7e1a39
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 9 13:07:22 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit dff5441da8d938ec2b27290b3fdd3ceba56fc502
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 9 11:24:14 2018 -0600
      
          Adding support for debian sid
      
      commit 5b4e102b77aae312dc26cf54c0760af59211f974
      Merge: ef1f1102b 34aa0d23
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 19:18:13 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit ef1f1102bee796ad83c51b01a319c226518ff503
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 16:42:12 2018 -0600
      
          Removed ubuntu18 references
      
      commit ecded73c3bb6340e6bed0032f602a98397577d0c
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 16:40:59 2018 -0600
      
          Adding missing suppress flags for install
      
      commit 4235897954d59c1c473b71c34a4c666f2079d009
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 14:49:10 2018 -0600
      
          Fixing bad fd issue with libssl install
      
      commit 40528b133f7ab3b23d1e533b36bcf41d406c5f90
      Merge: a8061efea e468cc49
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 12:58:40 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit a8061efea8f9a6762ad628ebb8071c5397cd4f8f
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 09:53:55 2018 -0600
      
          Update path reference to ubuntu18
      
      commit 4305ad261edd5be0714c9b000d8a3919fddc4300
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 09:42:10 2018 -0600
      
          Adding support for ubuntu 18.04 lts
      
      commit e21c64a378025ee2e970ff731a30479eac8efd8e
      Merge: 9ed6c0e55 642d15f8
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Tue May 8 08:41:09 2018 -0600
      
          Merge remote-tracking branch 'upstream/master' into entrypoint-bug-fix
      
      commit 9ed6c0e55c48aa8dfe74d4d2823a185a59d281a0
      Merge: 7fdb8351b ee3694f4
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 22:27:03 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit 7fdb8351b35bd4a02c725ba9757a2ed5cb5a6e6e
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 17:56:16 2018 -0600
      
          Reorganizing user account support for cmd/entrypoint
      
      commit 608afa03b154e0e0d70c9994765ab6bdb8f4eed1
      Merge: 44c4ca770 faf2368f
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 17:10:22 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit 44c4ca77017bf6733db619e55192133f45977156
      Merge: fe063cc22 186c6b7b
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 16:19:47 2018 -0600
      
          Merge branch 'master' into entrypoint-bug-fix
      
      commit fe063cc22def6a08e468f882e6d4a34f66b9bb48
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 16:17:31 2018 -0600
      
          Persist environment variables
      
      commit 8c8e206d04b6d07932fc5fe9519735cdabd7bed0
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Mon May 7 15:55:00 2018 -0600
      
          Adding support for dockerfiles that use alt users
      
      commit d6082d11bcce75a99c7fe9ec1bf2f46970ca4f0d
      Merge: f5a5b0e4c d633f6a2
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Fri May 4 10:54:00 2018 -0600
      
          Merge remote-tracking branch 'origin/master' into entrypoint-bug-fix
      
      commit f5a5b0e4ca6c31a395e3f909e8c107b7f3bdb58e
      Merge: 6ef666cd2 0bf7e0b3
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Fri May 4 00:25:20 2018 -0600
      
          Resolved libvnode_docker conflict
      
      commit 6ef666cd220327845174fa8d0d4897755d7c8786
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Thu May 3 18:27:48 2018 -0600
      
          Removing debugging statements
      
      commit 72336705c35493849496e1529ea39e8c944e45ec
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Thu May 3 18:06:24 2018 -0600
      
          Fixed issue where os images aren't in an array
      
      commit ab48322165fb02701ac25acbfb23a4da0e99ddf2
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 2 23:49:39 2018 -0600
      
          more debugging
      
      commit 88e7a74f3f4054ab3b1e87d5640cd4afda498efa
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 2 21:02:35 2018 -0600
      
          some debugging work
      
      commit cb107aaaf1f7431c32b4b34575f0a67fecf3e07e
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 2 17:55:08 2018 -0600
      
          Fixing permission issue
      
      commit 55f07ba52a8e7998413896a0d0fec4d79cf68b0d
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 2 09:20:23 2018 -0600
      
          Forgot semicolons are a thing in perl
      
      commit d667575e474f25012a4d5ac8177506af5dd08f2d
      Author: Elijah Grubb <u0894728@utah.edu>
      Date:   Wed May 2 08:45:22 2018 -0600
      
          Fixing quotes bug
      54dd7a77
    • David Johnson's avatar
    • David Johnson's avatar
    • David Johnson's avatar
    • David Johnson's avatar
      When emulabizing docker images, do not trust certain base image values. · bff8f548
      David Johnson authored
      Particularly, override Entrypoint and User.
      bff8f548
    • David Johnson's avatar
  6. 08 May, 2018 6 commits
    • David Johnson's avatar
      Fix a nasty docker/mkvnode.pl race inspired by bootvnodes/vnodesetup. · e468cc49
      David Johnson authored
      This is probably true for Xen too, but in some cases, the
      vnodesetup early-release hackwaitandexit timeout of 30 seconds
      causes a race condition.  Normally, the first node sets up
      significant network state, and sometimes flips MAC addresses
      around from interface to interface -- OR puts a physical interface
      into a bridge, then changes the bridge's MAC address.  There is a
      short window of time where both the bridge and the new member
      interface share a MAC address -- and if the tmcc ifconfig assembly
      process for vnodes following the first vnode resolves
      the wrong device's MAC address and uses that to flesh out the
      ifconfig info, the vnodesetup will be in a world of hurt (i.e., you
      might see an attempt to make a vlan device out of a vlan device).
      The chance of this happening is miniscule, but I've seen it.
      
      So, at least for docker for now, we protect the first vnode against
      the 30-second timeout in vnodesetup hackwaitandexit, and we wait for the
      actual running file to be written, or error.
      
      This is probably applicable to any linux mkvnode.pl path, but I suppose
      it would have been another hundred thousand vnode creates before I saw
      it again.
      e468cc49
    • David Johnson's avatar
      Do not fail iptables rules gen on name resolution failure. · a9827417
      David Johnson authored
      Under high load, of course we can have DNS problems.  However, perl
      seems to get stuck on retry; it's like the nak gets cached (which would
      be extremely odd, but can't argue with the evidence).
      
      Anyway, if resolution continues to fail, give up and feed the name to
      iptables, and let it try :).
      a9827417
    • David Johnson's avatar
    • David Johnson's avatar
      533e1714
    • David Johnson's avatar
      Minor debug message fix. · 7515dc50
      David Johnson authored
      7515dc50
    • David Johnson's avatar
      Do not run ddjikstra while holding the global lock in docker clientside. · ee3694f4
      David Johnson authored
      (All we need to do while holding the global lock is allocated IFBs; the
      generation of routing scripts and traffic shaping scripts is both
      unlikely to fail and potentially slow due to running djikstra.  So, also
      let the vnode early release prior to those things, immediately after IFB
      allocation.)
      ee3694f4
  7. 07 May, 2018 8 commits