1. 18 Jul, 2014 1 commit
  2. 17 Jul, 2014 1 commit
    • Mike Hibler's avatar
      Add support for auto-detecting when to create a delta image. · bc7c6e40
      Mike Hibler authored
      Uses the new-ish imagezip -P option. But first we had to get that option
      to imagezip. So the key-value interface between server and client now has
      an IZOPTS key where you can pass options to imagezip (This is now used to
      pass the -N that we have always specified as well).
      
      Also changed imagezip to print out a less ambiguous message when the option
      is in use, if will clearly say that it is full or delta. This matters because
      we have to parse the imagezip output on the server to figure out what kind
      of image it is!
      bc7c6e40
  3. 09 Jul, 2014 3 commits
  4. 03 Jul, 2014 1 commit
    • Mike Hibler's avatar
      Client-side script for creating versioned images. · ec664cd2
      Mike Hibler authored
      This is the replacement for create-image. Rather than puzzle out how to
      maintain backward-compat syntax while allowing all the new features in
      create-image, I just made a new script. When we don't care about backward
      compat anymore, we can just rename the script.
      
      Anyway, this script takes a lot more parameters (see the comment at the
      top) allow creation of full or delta images and uploading via NFS or
      frisbee. Since delta images require a signature file, there is new code
      to handle downloading (and uploading) these files. And since their size
      is measured in 10s of MB, possibly too big for the default MFS, there
      is code to create a temporary new MFS to hold them. Currently that MFS
      is 64MB which is big enough for our current generation of images (16GB FS)
      and will work on the pc600/pc850s.
      
      Still need to redo the boss-side of image creation (create_image, not
      to be confused with create-image...yeah, I know).
      ec664cd2
  5. 15 Nov, 2012 1 commit
  6. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  7. 20 Mar, 2012 1 commit
  8. 16 Mar, 2012 1 commit
  9. 15 Mar, 2012 1 commit
  10. 05 Mar, 2012 1 commit
  11. 21 Nov, 2011 1 commit
  12. 16 Nov, 2011 1 commit
  13. 24 Oct, 2011 1 commit
  14. 13 Oct, 2011 1 commit
    • Mike Hibler's avatar
      Nits. · 22405760
      Mike Hibler authored
      Remove another (harmless) ref to ext4fs. Fix comment for NTFS.
      22405760
  15. 11 Oct, 2011 4 commits
    • Ryan Jackson's avatar
      Merge imagezip ext4 code into extfs code · b8d79ac6
      Ryan Jackson authored
      Merge the ext4 support for imagezip into the extfs code.  There's no
      real reason to keep it separate, since ext4 is backward-compatible
      with ext3 and ext2.
      
      All of the macro definitions in extfs.c were changed from EXT2_* or
      EXT3_* to EXT4_*.  The primary reason for this is that while ext4 is
      backward compatible, the data structures have been extended and some
      of the definitions needed to change to handle them.  Some things that
      were constant before (block group descriptor sizes, for example) are
      now dynamic and must be calculated from other fields in the
      superblock.
      
      Since imagezip and frisbee don't support 64-bit block numbers, ext4
      filesystems that are larger than 2 TB (assuming 512-byte sectors) are
      not supported.  These can be detected by examining the
      feature_incompat field of the superblock.  Imagezip will fail if the
      user tries to compress one of these filesystems.
      b8d79ac6
    • Mike Hibler's avatar
      Switch to using version 2.0.0 of the libntfs library for Windows images. · 5d115d2c
      Mike Hibler authored
      I ran about 50 Windows images through this to "verify" that it produces the
      same results as the old 1.7.1 library.
      5d115d2c
    • Mike Hibler's avatar
      Add -Z option to zero free space included in the image due to -F. · 5a02dd31
      Mike Hibler authored
      Previously, this "internal free space" was whatever garbage happened to
      be on the disk we were imaging. By zeroing, we eliminate any leakage of
      information from the source disk and also allow the image to compress just
      a tad (1-4%) better.
      
      Why isn't this the default? Eh...no good reason, other than that this whole
      squish-out-small-free-ranges-to-allow-longer-writes optimization should be
      done by the client and should not be encoded in the image.
      5a02dd31
    • Mike Hibler's avatar
      Frisbee Master Server support for image_permissions table. · 64b3c003
      Mike Hibler authored
      To finish what Leigh started. Note that the master server currently only
      does node (IP) based authentication so "user" permissions in the
      image_permissions table are applied based on the uid of the swapper of
      the experiment that the contacting node is a part of.
      64b3c003
  16. 10 Oct, 2011 1 commit
  17. 08 Oct, 2011 3 commits
    • Mike Hibler's avatar
      Adjust the set of unix gids used for a download server. · b207680c
      Mike Hibler authored
      [ this is a re-commit of a faulty earlier commit. ]
      
      When downloading an image, start the frisbeed process with the minimum set of
      gids necessary to access the image. This includes the unix gid of the
      project that the image is in and, optionally, the unix gid of the project
      subgroup if the image is part of one.
      
      Previously, we just use the gid set of the uid of the swapper of the
      experiment. Not only was this excessive, but it might also not include the
      gids needed in the case of a "global" image that is not in the world-readable
      /usr/testbed/images directory.
      b207680c
    • Mike Hibler's avatar
      Revert "Adjust the set of unix gids used for a download server." · baba7478
      Mike Hibler authored
      This reverts commit fc89eb38.
      
      Checked in a bunch of crap that was unrelated.
      baba7478
    • Mike Hibler's avatar
      Adjust the set of unix gids used for a download server. · fc89eb38
      Mike Hibler authored
      When downloading an image, start the frisbeed process with the minimum set of
      gids necessary to access the image. This includes the unix gid of the
      project that the image is in and, optionally, the unix gid of the project
      subgroup if the image is part of one.
      
      Previously, we just use the gid set of the uid of the swapper of the
      experiment. Not only was this excessive, but it might also not include the
      gids needed in the case of a "global" image that is not in the world-readable
      /usr/testbed/images directory.
      fc89eb38
  18. 29 Sep, 2011 1 commit
  19. 28 Sep, 2011 4 commits
    • Mike Hibler's avatar
      Update with a couple of items that came up recently. · 1f54baef
      Mike Hibler authored
      Also update the DONE status on a few things.
      1f54baef
    • Mike Hibler's avatar
      Add client '-f' option for using O_DIRECT open mode for the output device. · 46a62612
      Mike Hibler authored
      On Linux, device IO goes through the buffer cache by default. This makes
      frisbee run really fast...until it closes the output device. Then it sits
      for minutes while it flushes disk data out of the cache. This is
      technically okay, but wasteful, since frisbee allocates its own memory
      for caching disk write data. By using direct IO on the output device,
      writes do not go through the cache.
      
      Aren't two caches better than one? No. They can compete for memory and it
      just causes an extra data copy. Frisbee is faster when using O_DIRECT.
      
      Note that this change is complicated somewhat because Linux requires that
      the IO buffer for an O_DIRECT opened file be sector aligned. So we play
      some games to do this.
      
      This should have no effect on FreeBSD where device writes don't go through
      the buffer cache.
      46a62612
    • Mike Hibler's avatar
      Avoid a pre-mature close of the output fd. · 179c189f
      Mike Hibler authored
      Wait til all other threads have joined.
      179c189f
    • Mike Hibler's avatar
      Lint and fixes to TRACE code. · c5d15a6b
      Mike Hibler authored
      c5d15a6b
  20. 22 Sep, 2011 1 commit
  21. 12 Sep, 2011 1 commit
  22. 30 Aug, 2011 2 commits
  23. 26 Aug, 2011 3 commits
  24. 25 Aug, 2011 3 commits
  25. 23 Aug, 2011 1 commit