1. 22 Jan, 2018 1 commit
  2. 12 Jul, 2017 1 commit
  3. 31 May, 2017 1 commit
  4. 01 Feb, 2017 1 commit
    • Leigh Stoller's avatar
      Checkpoint two changes: · bd9613cc
      Leigh Stoller authored
      1. Using frisbee events in libosload_new as a replacement for
         statically (hand waved) maxwait times for image loading. When frisbee
         is generating events, we use those to determined if progress is being
         made.
      
      2. Convert the CM to using the libosload_new library directly (like
         os_setup does). This is conditional on the NewOsload feature being
         attached to the geniuser. Otherwise, we go through the old path.
      bd9613cc
  5. 06 Oct, 2016 1 commit
  6. 04 Apr, 2016 2 commits
  7. 28 Mar, 2016 1 commit
  8. 24 Mar, 2016 1 commit
  9. 21 Mar, 2016 3 commits
  10. 14 Mar, 2016 1 commit
  11. 21 Aug, 2015 1 commit
  12. 22 May, 2015 1 commit
  13. 15 May, 2015 1 commit
    • Leigh Stoller's avatar
      Directory based image paths. · 3a21f39e
      Leigh Stoller authored
      Soon, we will have images with both full images and deltas, for the same
      image version. To make this possible, the image path will now be a
      directory instead of a file, and all of the versions (ndz,sig,sha1,delta)
      files will reside in the directory.
      
      A new config variable IMAGEDIRECTORIES turns this on, there is also a check
      for the ImageDiretories feature. This is applied only when a brand new
      image is created; a clone version of the image inherits the path it started
      with. Yes, you can have a mix of directory based and file based image
      descriptors.
      
      When it is time to convert all images over, there is a script called
      imagetodir that will go through all image descriptors, create the
      directory, move/rename all the files, and update the descriptors.
      Ultimately, we will not support file based image paths.
      
      I also added versioning to the image metadata descriptors so that going
      forward, old clients can handle a descriptor from a new server.
      3a21f39e
  14. 11 Nov, 2014 1 commit
    • Mike Hibler's avatar
      Fix a bug when expanding the list of images implied by delta images. · a62de08a
      Mike Hibler authored
      I was expanding a global list in a loop for every node. So for each
      node, I was finding all the delta images in the ever-growing list and
      adding their dependencies (again!) making the list even larger. In an
      experiment loading a two-level delta image on 8 nodes, the list included
      40+ copies of the same three images to load by the time we got to the last
      node. However, no node attempted to load all those images because tmcd
      exceeded its reply buffer size on the "loadinfo" call and would not
      return anything. Of course, by then we had computed a max wait time
      based on image.max_wait * 45 so the experiment suffered a slow, lingering
      death even though the nodes were not doing anything.
      
      Beware, I do not know if I got the "access key" code right for remote
      nodes. Not even sure if we use that path anymore. I attempted to fix it
      in libosload, I did not even try in libosload_new.
      a62de08a
  15. 20 Oct, 2014 2 commits
  16. 18 Jul, 2014 1 commit
  17. 16 Jul, 2014 1 commit
  18. 14 Jul, 2014 1 commit
  19. 02 Jul, 2014 1 commit
  20. 01 Jul, 2014 1 commit
  21. 12 May, 2014 1 commit
    • Leigh Stoller's avatar
      Fix for loading an image on a remoteded pg node. This is a kludge, the · 15dce279
      Leigh Stoller authored
      notion of "dedicated" is currently a type specific attribute, but we
      also have "shared" nodes running on "dedicated" nodes, which messes
      everything up. I am not inclined to fix the underlying problem since
      Utah is the only site that uses this stuff, and these nodes are slowly
      dying out anyway.
      15dce279
  22. 26 Mar, 2014 1 commit
  23. 17 Mar, 2014 2 commits
    • Kirk Webb's avatar
      Refactor taintstate code and move final taint updates to stated. · 662972cd
      Kirk Webb authored
      Can't do the untainting for all cases in libosload*.  The untainting
      is now hooked into stated, where we catch the nodes as they send
      along their "RELOADDONE" events to update their taint state according
      to the final state of their partitions.
      662972cd
    • Kirk Webb's avatar
      Add taint state tracking for OSes and Nodes. · 1de4e516
      Kirk Webb authored
      Emulab can now propagate OS taint traits on to nodes that load these OSes.
      The primary reason for doing this is for loading images which
      require special treatment of the node.  For example, an OS that has
      proprietary software, and which will be used as an appliance (blackbox)
      can be marked (tainted) as such.  Code that manages user accounts on such
      OSes, along with other side channel providers (console, node admin, image
      creation) can key off of these taint states to prevent or alter access.
      
      Taint states are defined as SQL sets in the 'os_info' and 'nodes' tables,
      kept in the 'taint_states' column in both.  Currently these sets are comprised
      of the following entries:
      
      * usermode: OS/node should only allow user level access (not root)
      * blackbox: OS/node should allow no direct interaction via shell, console, etc.
      * dangerous: OS image may contain malicious software.
      
      Taint states are inherited by a node from OSes it loads during the OS load
      process.  Similarly, they are cleared from nodes as these OSes are removed.
      Any taint state applied to a node will currently enforce disk zeroing.
      
      No other tools/subsystems consider the taint states currently, but that will
      change soon.
      
      Setting taint states for an OS has to be done via SQL presently.
      1de4e516
  24. 28 Feb, 2013 1 commit
  25. 17 Nov, 2012 1 commit
    • Mike Hibler's avatar
      More PRObE inspired improvements to the swapin path. · 5a2810f2
      Mike Hibler authored
      Replace an exec of the perl os_select script with a call to the OSSelect()
      node method. This cut in half the time spent in the DB setup for each node.
      Note that this change had already been made to libosload_new.
      
      Reworked the code that setup the partitions table entries. We were potentially
      updating each DB row for each image loaded. Now we just work out all the changes
      in a perl data struct and make one set of DB changes at the end. The code is
      more comprehensible now as well (I hope!)
      
      Finally, disable the "swapinfo" stuff which was the first step in doing stateful
      swapout of disk state. That code never got finished.
      5a2810f2
  26. 25 Sep, 2012 1 commit
  27. 24 Sep, 2012 2 commits
    • Mike Hibler's avatar
      Add check in "wedged" code to verify that the node is not already reloading. · 45ec3557
      Mike Hibler authored
      Due to a race with collecting events, it looks like some events will still
      slip through the crack and we might wind up having missed a transition after
      five minutes. If we see that we are already in RELOADING (the state transition
      we are looking for) when we would declare the node wedged, then fake the
      transition and continue.
      
      I suspect this would not happen if I just looped on event_poll til there
      were no more events, but I am afraid of letting that loop go unbounded.
      So til I gather more data, lets go with this hack check.
      45ec3557
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  28. 23 Sep, 2012 1 commit
  29. 23 Aug, 2012 2 commits
  30. 25 Jul, 2012 3 commits
  31. 01 May, 2012 1 commit
    • Mike Hibler's avatar
      Fix libosload to properly get the size of images it cannot read directly. · 014b0f04
      Mike Hibler authored
      If a project image has been "exported" with grantimage, os_load may not
      be able to stat it due to unix permissions. So have os_load make a query
      to the local frisbee master server to get attributes for the image. The
      master server already knows how to deal with these exported images.
      This query also works for an inner boss determining the size of an image
      it has not yet downloaded and thus subsumes that case.
      014b0f04