1. 04 Mar, 2017 2 commits
  2. 27 Feb, 2017 3 commits
  3. 25 Feb, 2017 1 commit
    • Mike Hibler's avatar
      Fix for local homedirs getting left as owned by root. · 2beb1824
      Mike Hibler authored
      See emulab-devel issue 227 for details.
      Also, on a "reset" clean out the correct BDB files. It has been
      a long time since they used ".db" as the suffix. Now there are
      ".pag" and ".dir" files. We haven't noticed because we don't really
      use the "reset" operation. The prepare script just removes
      everything in /var/emulab/db.
  4. 18 Feb, 2017 1 commit
  5. 17 Feb, 2017 1 commit
  6. 16 Feb, 2017 4 commits
    • Mike Hibler's avatar
      More robustness improvements for FreeBSD vnodes. · aa9bc39b
      Mike Hibler authored
      Not sure how I got headed down this path, but here I am:
       * replace use of "ps" and "grep" with, wait for it..."pgrep"!
       * explicitly specify type=vif so we don't wind up with the extra,
         vifN.M-emu backend interface that gets left laying around,
       * add -F option to "xl shutdown" which is needed for HVMs else
         shutdown will fail and the domain won't go away (qemu left behind)
         and FBSD filesystem can be messed up,
       * Use "hd" instead of "sd" to avoid emulated SCSI driver which has
         caused me grief in the past (though it should never actually get
         used due to PVHVM config of kernel).
    • Mike Hibler's avatar
      Lint fixes from Jon. · 67141c59
      Mike Hibler authored
    • Mike Hibler's avatar
    • Mike Hibler's avatar
      Add some code to the Xen option to detect console device changes. · 59f04842
      Mike Hibler authored
      Whenever an HVM domU reboots, the pty used for the console changes
      and capture would blissfully keep using the old abandoned pty. Since
      HVM is currently only used for FreeBSD vnodes, nobody but Mike ever
      xenstore has a watch mechanism where you can be informed if any key
      changes value. This would have been really convenient to use...if I
      was using the native xenstore C API. But I didn't want to drag in
      dependencies on Xen libraries, so I do all the xen stuff using the
      command line tools. So now I popen() xenstore_watch on the console
      key and add that file descriptor to the select loop. See the comments
      for further lamentation.
      Also, cleaned up the (GCC detected) lint while I was in there!
  7. 15 Feb, 2017 2 commits
  8. 10 Feb, 2017 1 commit
    • Mike Hibler's avatar
      It is Cleanup Friday! · f624f158
      Mike Hibler authored
      Get rid of ELVIN_COMPAT and CONFIG_OPSVM from elabinelab land.
      These options still exist throughout the install code, didn't touch that.
  9. 09 Feb, 2017 2 commits
  10. 08 Feb, 2017 1 commit
  11. 04 Feb, 2017 1 commit
  12. 03 Feb, 2017 1 commit
    • David Johnson's avatar
      Fix route saving in Linux firewall code to preserve default route. · 044c33bd
      David Johnson authored
      Our firewall rules rely on the ability to resolve names.  Thus, the
      manipulation we do on the firewall node to move the control net from a
      NIC to a bridge with the NIC inside it must save all the routes, too.
      If this code used to do that properly for Ubuntu 10, it certainly does
      not for Ubuntu 16, for one of potentially multiple reasons.  So I wrote
      some fun code to do its very best to save all routes related to the
      control net device (including the default route) in such a way as to 1)
      obtain *all* routes before tearing *any* down any; and 2) to ensure that
      the transformed routes can be safely restored.  (This will then work for
      any testbed with an obtuse control net setup in which the default route
      is insufficient to reach boss, not that that will ever happen.)  This
      code does its thing on both the enable/disable paths, of course.  Here's
      the relevant details from the code:
          This is very, very messy.  We have to save the
          existing routes for $pdev, but in an order that they
          can be restored without failing, before we move
          $pdev into br0.  Then the restored routes must be
          rewritten in terms of br0 instead of $pdev.
          Finally, we have to collect these routes prior to 1)
          assigning $pdev's IP to br0, and 2) moving $pdev
          into br0, and 3) prior to *deleting* any routes.
          All these conditions seem necessary to me.
          ip route show does not necessarily display routes in
          restoreable order.  In particular, the thing that
          bites is that the default route is (now, as of
          Ubuntu16, at least) displayed *prior to* the scope
          link (broadcast route) for a device).  So, we first
          harvest the scope link routes for $pdev (i.e., the
          natural ones that come into being as a side affect
          of assiging an IP address to a link); then we
          harvest all other routes associated with $pdev; then
          we remove all routes associated with $pdev.  Then we
          do the mucking about with bridge membership and
          transfer IP from $pdev to bridge, and finally,
          restore the routes we saved (modified to br0, of
  13. 02 Feb, 2017 1 commit
  14. 01 Feb, 2017 1 commit
  15. 31 Jan, 2017 3 commits
    • Mike Hibler's avatar
      Sort out issues w.r.t. unlimited bandwidth and dynamic bandwidth. · 0126a71d
      Mike Hibler authored
      Setting bandwidth==0 in either the sitevars or via subboss_attributes
      will now get properly translated to no bandwidth cap in frisbeed.
      Previously it was ignored. But I still do not recommend this setting
      as slow clients can get swamped.
      Allow setting of dynamic BW control along with unlimited bandwidth.
      Likewise not recommended (though marginally better than the above).
    • Mike Hibler's avatar
      Change the interpretation of the loadinfo HEARTBEAT value. · 2797b4b6
      Mike Hibler authored
      Zero means disable, anything non-zero means "let the server decide".
    • Mike Hibler's avatar
      More tweaks to frisbee heartbeat code. · df78b7ef
      Mike Hibler authored
      Make sure server doesn't exit as long as it is getting heartbeats
      from known clients. We used to exit when we stopped getting requests,
      however clients often finish their network activity long before they
      are actually done.
      Emulab event now reports Mebibytes rather than bytes. It is accurate
      enough and avoids perl bigints in the receiver(s).
  16. 30 Jan, 2017 2 commits
  17. 25 Jan, 2017 2 commits
  18. 19 Jan, 2017 2 commits
    • Mike Hibler's avatar
      Output event key/val pairs as strings. · cd6188a3
      Mike Hibler authored
      Our SWIG generated perl wrappers cannot handle the out pointer
      params of event_notification_get_int*.
    • Mike Hibler's avatar
      Redo the frisbee heartbeat code again. · 5407463e
      Mike Hibler authored
      My "shortcut" to enable a heartbeat via a client-side command line
      proved to be untenable. There are just too many places where we fire
      off the client and getting the right heartbeat interval value to all
      those places would have been...challenging.
      So back to the original plan of having a server-side command line
      option and letting the server tell the client when/what to report.
      This limits the changes to just the frisbee master server, in particular
      I now just have to get the value to master server instances running
      on the subbosses (not done yet, just hardwiring a value for now).
      All this said, I still had to modify the various places we invoke
      the frisbee client to add an option to enable the heartbeat, but at
      least I didn't need to know a specific value.
  19. 18 Jan, 2017 1 commit
    • Mike Hibler's avatar
      Add the ability to "report" for proxied nodes. · 3665ce07
      Mike Hibler authored
      Not even 48 hours old and I have already had to change it...
      Forgot about vnode hosts that proxy on behalf of their vnodes.
      For those, we want to be reporting the actual client identity and
      not the physical host's. So add a "who" field to the report message
      to explicitly identify who requested the data. Previously we were
      just using the IP of the sender of the report message to identify
      the client.
      Note that this is NOT backward compatible with yesterday's version.
      Since I know everywhere that version was running, I could just
      eliminate them and pretend that version never existed. Dead men
      tell no tales.
  20. 17 Jan, 2017 3 commits
    • Mike Hibler's avatar
      Get in sync with freebsd9 version. · 0ae9f4e1
      Mike Hibler authored
    • Mike Hibler's avatar
      Implement heartbeat/status reports in Frisbee. · 2be46ba4
      Mike Hibler authored
      There are three pieces here, a change to the frisbee protocol itself, an
      Emulab event component to get status back to the portal, and the surrounding
      infrastructure to make it all work.
      Frisbee heartbeat messages:
      Added a new message type to the frisbee protocol, "Progress". In theory it
      operates by having the server send a multicast progress request to its clients
      which includes an interval at which to report (or "just once") and an
      indication of what to report (nothing, progress summary, or full stats). The
      client then sends unicast "fire and forget" UDP replies according to that
      schedule. However, I took a shortcut for the moment and just added a command
      line option to the client to tell it to report a summary at the indicated
      interval (-H <interval>).  So the server never sends requests.
      This is implemented in the client by a fourth thread since I wanted it to
      operate independent of packet reception (which would cause clients to report
      in a highly synchronized fashion due to multicast). The server instance just
      logs progress reports into its log.
      This protocol addition should be fully backward compatible as both client and
      server ignore (but log) unknown messages.
      Emulab progress report events:
      When this is compiled in (-DEMULAB_EVENTS) and turned on (-E <server>), the
      frisbee server instances will send a FRISBEEPROGRESS event to the indicated
      event server for every progress report it receives (in addition to logging the
      events to its own log). Right now it will create an event with key/value pairs
      for the information in a client summary reply:
      TSTAMP is the client's time at which it sends the event. Could be used by the
      received to determine latency of the report if it cared (and if it assumed
      that the clocks are in sync). We don't care about this.
      SEQUENCE is the report number. Again, could be used by the receiver, in this
      case to detect loss, if it cared. We don't.
      CHUNKS_RECV is complete chunks that the client has received from the network.
      CHUNKS_DECOMP is chunks decompressed by the client.  BYTES_WRITTEN is bytes
      written to disk by the client.
      Any of the three can be used by the event receiver as an indication of life
      and/or progress. However, only the last would be a reasonable indicator of
      time remaining since it is the last (and slowest) phase of imaging. To
      estimate time remaining we could compare that value to the amount of
      uncompressed data that is in the image. This makes the sketchy assumptions
      that time for writes to the disk are uniform and that the number and distance
      of seeks is uniform, but it is better than a sharp stick in the eye.
      Emulab infrastructure:
      There is a new sitevar "images/frisbee/heartbeat" which can be set to a
      non-zero value to tell the frisbee MFS to fire off frisbee with -H <value>
      and thus make reports. The default value of zero means to not make reports.
      The tmcd "loadinfo" command sends this through via the HEARTBEAT=<value>
    • Mike Hibler's avatar
      Clean up various mfs targets. · 746fc74a
      Mike Hibler authored
  21. 12 Jan, 2017 1 commit
  22. 11 Jan, 2017 1 commit
  23. 10 Jan, 2017 2 commits
  24. 09 Jan, 2017 1 commit