1. 21 Mar, 2016 1 commit
  2. 15 Jul, 2015 1 commit
  3. 14 Jul, 2015 1 commit
    • David Johnson's avatar
      Mirror libosload properly for image/os load in libosload_new. · 5882f0a2
      David Johnson authored
      libosload_new wasn't accepting the "new" types of ways to specify
      one or more images to load on nodes, so manual os_load wasn't
      working.  I also mirrored the usedefault and usecurrent stuff from
      libosload into libosload_new; will make it easier in the future.
      5882f0a2
  4. 23 Jun, 2015 1 commit
  5. 15 May, 2015 1 commit
    • Leigh Stoller's avatar
      Directory based image paths. · 3a21f39e
      Leigh Stoller authored
      Soon, we will have images with both full images and deltas, for the same
      image version. To make this possible, the image path will now be a
      directory instead of a file, and all of the versions (ndz,sig,sha1,delta)
      files will reside in the directory.
      
      A new config variable IMAGEDIRECTORIES turns this on, there is also a check
      for the ImageDiretories feature. This is applied only when a brand new
      image is created; a clone version of the image inherits the path it started
      with. Yes, you can have a mix of directory based and file based image
      descriptors.
      
      When it is time to convert all images over, there is a script called
      imagetodir that will go through all image descriptors, create the
      directory, move/rename all the files, and update the descriptors.
      Ultimately, we will not support file based image paths.
      
      I also added versioning to the image metadata descriptors so that going
      forward, old clients can handle a descriptor from a new server.
      3a21f39e
  6. 01 May, 2015 1 commit
  7. 11 Nov, 2014 1 commit
    • Mike Hibler's avatar
      Fix a bug when expanding the list of images implied by delta images. · a62de08a
      Mike Hibler authored
      I was expanding a global list in a loop for every node. So for each
      node, I was finding all the delta images in the ever-growing list and
      adding their dependencies (again!) making the list even larger. In an
      experiment loading a two-level delta image on 8 nodes, the list included
      40+ copies of the same three images to load by the time we got to the last
      node. However, no node attempted to load all those images because tmcd
      exceeded its reply buffer size on the "loadinfo" call and would not
      return anything. Of course, by then we had computed a max wait time
      based on image.max_wait * 45 so the experiment suffered a slow, lingering
      death even though the nodes were not doing anything.
      
      Beware, I do not know if I got the "access key" code right for remote
      nodes. Not even sure if we use that path anymore. I attempted to fix it
      in libosload, I did not even try in libosload_new.
      a62de08a
  8. 20 Oct, 2014 2 commits
  9. 29 Sep, 2014 1 commit
  10. 18 Jul, 2014 1 commit
  11. 17 Mar, 2014 2 commits
    • Kirk Webb's avatar
      Refactor taintstate code and move final taint updates to stated. · 662972cd
      Kirk Webb authored
      Can't do the untainting for all cases in libosload*.  The untainting
      is now hooked into stated, where we catch the nodes as they send
      along their "RELOADDONE" events to update their taint state according
      to the final state of their partitions.
      662972cd
    • Kirk Webb's avatar
      Add taint state tracking for OSes and Nodes. · 1de4e516
      Kirk Webb authored
      Emulab can now propagate OS taint traits on to nodes that load these OSes.
      The primary reason for doing this is for loading images which
      require special treatment of the node.  For example, an OS that has
      proprietary software, and which will be used as an appliance (blackbox)
      can be marked (tainted) as such.  Code that manages user accounts on such
      OSes, along with other side channel providers (console, node admin, image
      creation) can key off of these taint states to prevent or alter access.
      
      Taint states are defined as SQL sets in the 'os_info' and 'nodes' tables,
      kept in the 'taint_states' column in both.  Currently these sets are comprised
      of the following entries:
      
      * usermode: OS/node should only allow user level access (not root)
      * blackbox: OS/node should allow no direct interaction via shell, console, etc.
      * dangerous: OS image may contain malicious software.
      
      Taint states are inherited by a node from OSes it loads during the OS load
      process.  Similarly, they are cleared from nodes as these OSes are removed.
      Any taint state applied to a node will currently enforce disk zeroing.
      
      No other tools/subsystems consider the taint states currently, but that will
      change soon.
      
      Setting taint states for an OS has to be done via SQL presently.
      1de4e516
  12. 26 Apr, 2013 1 commit
    • David Johnson's avatar
      Fix suddenly-appearing bug in libosload_new dynamic module loading code. · f8434aad
      David Johnson authored
      Not sure why this change was needed, but it wasn't working anymore.
      Maybe this is a perl 5.10 -> 5.12 thing; how I was doing an eval of a
      require to load an libosload_X module was no longer working.  The new
      way is better anyway, but even after looking through the perlXdelta
      pages, I didn't see a reason for this.
      
      Oh well.
      f8434aad
  13. 29 Mar, 2013 1 commit
  14. 17 Nov, 2012 1 commit
    • Mike Hibler's avatar
      More PRObE inspired improvements to the swapin path. · 5a2810f2
      Mike Hibler authored
      Replace an exec of the perl os_select script with a call to the OSSelect()
      node method. This cut in half the time spent in the DB setup for each node.
      Note that this change had already been made to libosload_new.
      
      Reworked the code that setup the partitions table entries. We were potentially
      updating each DB row for each image loaded. Now we just work out all the changes
      in a perl data struct and make one set of DB changes at the end. The code is
      more comprehensible now as well (I hope!)
      
      Finally, disable the "swapinfo" stuff which was the first step in doing stateful
      swapout of disk state. That code never got finished.
      5a2810f2
  15. 25 Sep, 2012 1 commit
  16. 24 Sep, 2012 2 commits
    • Mike Hibler's avatar
      Add check in "wedged" code to verify that the node is not already reloading. · 45ec3557
      Mike Hibler authored
      Due to a race with collecting events, it looks like some events will still
      slip through the crack and we might wind up having missed a transition after
      five minutes. If we see that we are already in RELOADING (the state transition
      we are looking for) when we would declare the node wedged, then fake the
      transition and continue.
      
      I suspect this would not happen if I just looped on event_poll til there
      were no more events, but I am afraid of letting that loop go unbounded.
      So til I gather more data, lets go with this hack check.
      45ec3557
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  17. 23 Sep, 2012 1 commit
  18. 23 Aug, 2012 2 commits
  19. 25 Jul, 2012 3 commits
  20. 11 Aug, 2011 1 commit
    • Mike Hibler's avatar
      Initial support for loading Windows7 .wim images via WinPE/ImageX. · ac711ea5
      Mike Hibler authored
      1. Support for "one-shot" PXE booting ala the one-shot osid. Switches to
         pxelinux to boot WinPE and then switch back after done. Painful now
         because we have to HUP dhcpd everytime we change the PXE path, but we
         may be able to fix this in the future by going all-pxelinux-all-the-time.
      
      2. Added pxe_select, analogous to os_select, for changing the pxe_boot_path
         including the one time path.
      
      3. Added the WIMRELOAD state machine to shepherd a node through the process.
         Still has some rough edges and may need refining.
      ac711ea5
  21. 09 May, 2011 1 commit
  22. 18 Feb, 2011 1 commit
  23. 09 Feb, 2011 2 commits
  24. 18 Jan, 2011 1 commit
  25. 22 Dec, 2010 1 commit
    • David Johnson's avatar
      Misc fixes. · 12005979
      David Johnson authored
      File::stat doesn't work; CORE::stat does, so switch out File::stat.
      
      Bugfixes for access_keys stuff -- syntax!
      
      Add pcRemote stub class; just inherits from pc class.
      12005979
  26. 20 Dec, 2010 1 commit
  27. 11 Nov, 2010 1 commit
  28. 09 Nov, 2010 1 commit
  29. 08 Nov, 2010 1 commit
    • David Johnson's avatar
      If we don't flash the switch, restore its osid/partition info. · 7f5fa6bf
      David Johnson authored
      If a switch reload fails, and we're sure we didn't flash the
      switch, then revert the changes to osid info in the nodes table
      and the partitions table entries.  This way, we don't lie about
      what's actually on the switch.  Otherwise, there could be a
      problem on a subsequent reload -- a user could request something
      that failed to load last time, but Emulab thinks that's what's
      really on the switch, so it doesn't reload... but that wasn't
      really what was installed on the switch.  So, this will help
      us stay in synch.
      
      Also remove all the os_select script invocations in favor of
      using the Node->OSSelect method instead.  Save a fork, save a
      life.
      7f5fa6bf
  30. 04 Nov, 2010 1 commit
    • David Johnson's avatar
      libosload rewrite and HP Procurve switch support. · 7cfa7aa6
      David Johnson authored
      libosload_new contains the new libosload.  It provides the same
      API as the original libosload, but the functionality has been
      split out into well-defined methods that can be overridden by
      per-node type subclasses in small chunks.  This is all protected
      by the NewOsload feature.
      
      libosload_switch.pm.in and libossetup_switch.pm.in contain the HP
      switch support, which requires the new libosload and new
      libossetup.  libossetup_switch doesn't do anything too radical, but
      it does support running a node it is "lighting up" through multiple
      operations -- i.e., first a RELOAD, then a RECONFIG.  This was
      necessary because switches require a config to be pushed to them
      (well, they do in the way I elected to support them), and they
      cannot fetch configuration in the normal Emulab way via tmcd.
      
      Eventually, that multi-op support will be factored out into
      libossetup more generically so that other node types which require
      push-based configuration can leverage it.
      
      libosload_switch contains the HP switch loading code.  It's built
      around expect-like interaction on the serial console.  It supports
      xmodem and tftp uploads, but tries to only flash the switch lazily.
      Thus, if the switch is already loaded with an image and a user
      (or the reload daemon, or anybody) reloads it with the same image,
      it won't actually get reloaded.  You can now force this reload
      with os_load.
      
      libosload_switch also contains reconfig code (i.e., to wipe
      switch configs and recreate based on which experiment it is in).
      libossetup_switch directly invokes this instead of going through
      libosload, since configuring is not really part of loading.  It
      is just convenient since configuration also happens over the
      serial console, and all the osload code for the switch can be
      reused.
      
      Also, of course, there are miscellaneous changes to support the
      new libosload in os_load itself.
      7cfa7aa6