1. 10 Jun, 2016 1 commit
  2. 22 Mar, 2016 2 commits
  3. 21 Mar, 2016 1 commit
    • Leigh Stoller's avatar
      New (test) version of image_import that can import the entire image history · aef647a6
      Leigh Stoller authored
      from the server, keeping it in sync with the server as new versions of the
      image are added. Also handles importing deltas if the metadata says there
      is a delta.
      
      Note that downloading the image files is still lazy; we will not import all
      15 versions of an image unless they actually are needed.
      
      Lots of work still do. This is a bit of a nightmare cause of client/server
      (backward) compatibility issues wrt provenance/noprovenance and
      deltas/nodeltas. I might change my mind and say the hell with
      compatibility!
      
      Along these same lines, there is an issue of what to do when a site that is
      running with provenance turned on, gets this new code. Up to now, the
      client and server never tried to stay in sync, but now they have to (cause
      of deltas), and so the client image descriptors have to be upgraded. That
      will be a hassle too.
      aef647a6
  4. 11 Feb, 2016 1 commit
  5. 15 Oct, 2015 1 commit
    • Leigh Stoller's avatar
      Hmm, we do not add nonlocal users to groups on ops, so doing the image · 1c0d1a2d
      Leigh Stoller authored
      import as the updater does not work in this case, so use the creator which
      is always going to have local groups (since it is geniuser or a real
      user). I need to think about why we do not do this, it dates back to the
      original stuff we did for the Probe cluster, but we are now using nonlocal
      users and projects n a different way for the Cloudlab portal.
      1c0d1a2d
  6. 12 Oct, 2015 1 commit
    • Leigh Stoller's avatar
      When triggering an update at another cluster, send along the urn of the · cf12fb1d
      Leigh Stoller authored
      user doing the image snapshot. At the image origin cluster, use this URN
      to set the updater urn for the new image version, and if that urn is fot
      a real local user, also set the updater,updater_idx accordingly. Lastly,
      the image import *must* be done in the context of a user in the project
      of the image, so fall back to using the image creator.
      cf12fb1d
  7. 21 Aug, 2015 1 commit
  8. 07 Jul, 2015 1 commit
  9. 19 Jun, 2015 1 commit
    • Leigh Stoller's avatar
      New support for importing image backed datasets from other clusters. This · 613d90dd
      Leigh Stoller authored
      is just like importing images (by using a url instead of a urn), which
      makes sense since image backed datasets are just images with a flag set.
      
      Key differences:
      
      1. You cannot snapshot a new version of the dataset on a cluster it has
         been imported to. The snapshot has to be done where the dataset was
         created initially. This is slightly inconvenient and will perhaps
         confuse users, but it is far less confusing that then datasets getting
         out of sync.
      
      2. No image versioning of datasets. We can add that later if we want to.
      613d90dd
  10. 18 May, 2015 1 commit
  11. 15 May, 2015 2 commits
    • Leigh Stoller's avatar
      Fix syntax error. · f2f298fb
      Leigh Stoller authored
      f2f298fb
    • Leigh Stoller's avatar
      Directory based image paths. · 3a21f39e
      Leigh Stoller authored
      Soon, we will have images with both full images and deltas, for the same
      image version. To make this possible, the image path will now be a
      directory instead of a file, and all of the versions (ndz,sig,sha1,delta)
      files will reside in the directory.
      
      A new config variable IMAGEDIRECTORIES turns this on, there is also a check
      for the ImageDiretories feature. This is applied only when a brand new
      image is created; a clone version of the image inherits the path it started
      with. Yes, you can have a mix of directory based and file based image
      descriptors.
      
      When it is time to convert all images over, there is a script called
      imagetodir that will go through all image descriptors, create the
      directory, move/rename all the files, and update the descriptors.
      Ultimately, we will not support file based image paths.
      
      I also added versioning to the image metadata descriptors so that going
      forward, old clients can handle a descriptor from a new server.
      3a21f39e
  12. 07 May, 2015 1 commit
  13. 24 Apr, 2015 2 commits
  14. 01 Apr, 2015 1 commit
  15. 17 Mar, 2015 1 commit
  16. 10 Mar, 2015 1 commit
  17. 30 Jan, 2015 1 commit
  18. 22 Jan, 2015 1 commit
  19. 21 Jan, 2015 1 commit
  20. 07 Jan, 2015 1 commit
  21. 02 Dec, 2014 1 commit
  22. 05 Nov, 2014 1 commit
  23. 04 Nov, 2014 1 commit
  24. 08 Oct, 2014 1 commit
  25. 12 Sep, 2014 1 commit
  26. 25 Aug, 2014 1 commit
  27. 09 May, 2014 1 commit
    • Mike Hibler's avatar
      New imagevalidate tool for printing/checking/updating image metadata. · 0bb906f4
      Mike Hibler authored
      This should be run whenever an image is created or updated and possibly
      periodically over existing images. It makes sure that various image
      metadata fields are up to date:
      
       * hash: the SHA1 hash of the image. This field has been around for
         awhile and was previously maintained by "imagehash".
      
       * size: the size of the image file.
      
       * range: the sector range covered by the uncompressed image data.
      
       * mtime: modification time of the image. This is the "updated"
         datetime field in the DB. Its intent was always to track the update
         time of the image, but it wasn't always exact (create-image would
         update this with the current time at the start of the image capture
         process).
      
      Documentation? Umm...the usage message is comprehensive!
      It sports a variety of useful options, but the basics are:
      
       * imagevalidate -p <image> ...
          Print current DB metadata for indicated images. <image> can either
          be a <pid>/<imagename> string or the numeric imageid.
      
       * imagevalidate <image> ...
          Check the mtime, size, hash, and image range of the image file and
          compare them to the values in the DB. Whine for ones which are out
          of date.
      
       * imagevalidate -u <image> ...
          Compare and then update DB metadata fields that are out of date.
      
      Fixed a variety of scripts that either used imagehash or computed the
      SHA1 hash directly to now use imagevalidate.
      0bb906f4
  28. 17 May, 2013 1 commit
  29. 14 May, 2013 1 commit
  30. 28 Feb, 2013 1 commit
  31. 08 Feb, 2013 1 commit
  32. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  33. 04 Sep, 2012 1 commit
    • Leigh Stoller's avatar
      Add image import utilities. · e468f885
      Leigh Stoller authored
      image_setup is run from tbprerun to verify and create image
      descriptors, and then later from tbswap to actually download
      and verify the image (ndz) file.
      
      image_import does the actual work for a specific image (url).
      e468f885