1. 12 May, 2015 2 commits
    • Mike Hibler's avatar
      Add -M option to set maximum sector range. · 63fe5e30
      Mike Hibler authored
      In cases where code does not compute it. This is only useful
      for some assertion checking, so the option may not stick around.
      63fe5e30
    • Mike Hibler's avatar
      Working well enough that I can start doing some serious testing. · bf3b4dfc
      Mike Hibler authored
      To that end, added a new "imagerezip" program which reads in an image
      and writes it back out. Right now, this is just to test that reading
      and writing images via the new library produces the same result as the
      "classic" tools. But it might be useful in the future to repack images
      with a different compression level or a different signature (hash function
      or blocksize).
      
      Also checked in a hack to imageunzip I did for Xing (-I option) to just
      produce a stream of the uncompressed image data (it ignores skips, aka
      it doesn't skip on the output device to put the data in the correct place).
      
      Unfortunately, I cannot just compare raw images because even trying to
      replicate the chunk packing strategy of the original code, the new library
      packs a bit more into each chunk. Not sure why, but anyway, I have to
      compare old/new images by dumping their contents and by checking the
      generated signatures.
      bf3b4dfc
  2. 04 May, 2015 1 commit
  3. 24 Feb, 2015 1 commit
    • Mike Hibler's avatar
      Teach imageunzip and frisbee about GPT. · 315910bd
      Mike Hibler authored
      Now the -s option works as well as the -D option to set the partition
      type. We can now properly save and restore both partition and whole
      disk images. Well, once I change the scripts...
      315910bd
  4. 19 Feb, 2015 2 commits
  5. 03 Feb, 2015 1 commit
  6. 02 Feb, 2015 1 commit
  7. 19 Dec, 2014 1 commit
  8. 18 Dec, 2014 1 commit
    • Mike Hibler's avatar
      More tweaks. · af8e9149
      Mike Hibler authored
      Mostly letting GPT or MBR decide how big the disk is and let it initially
      decide on areas (outside of partitions) that can be skipped.
      af8e9149
  9. 17 Dec, 2014 1 commit
  10. 16 Dec, 2014 1 commit
  11. 15 Apr, 2014 1 commit
  12. 10 Feb, 2014 1 commit
  13. 06 Feb, 2014 1 commit
    • Mike Hibler's avatar
      Make sure we don't do a partial sector write when using O_DIRECT. · 7e260e9a
      Mike Hibler authored
      Will cause a write error. This would only happen for the final range of
      an image created with -f. We now pad out any partial last sector if direct
      IO is specified (-f on imageunzip/frisbee) and write that to the file.
      
      If the padding causes any problems in the future, we can always get rid of
      it by truncating the file to the correct size when we are done...I think.
      7e260e9a
  14. 07 Feb, 2013 1 commit
  15. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  16. 08 Aug, 2012 1 commit
    • Mike Hibler's avatar
      Bug Fix: fix incorrect usage of "sizeof(blockhdr_t)". · 9f5bed29
      Mike Hibler authored
      The size of the block header differs depending on the version of the image.
      sizeof(blockhdr_t) gives the size of the current version of the header when
      the binary was built (V4 currently). This only mattered in two places:
      
      In imagezip, it would have prevented us from creating a backward compatible
      V1 image. Probably nobody has done this in 10 years.
      
      In imageunzip, it could prevent relocations from being processed. This only
      affected recent builds (V4 format) when attempting to unzip an older
      single-partition FreeBSD image (FBSD5 or before). This is unlikely to occur
      outside of Utah and only very rarely here.
      9f5bed29
  17. 22 Jun, 2012 1 commit
  18. 20 Mar, 2012 1 commit
  19. 05 Mar, 2012 1 commit
  20. 28 Sep, 2011 2 commits
    • Mike Hibler's avatar
      Add client '-f' option for using O_DIRECT open mode for the output device. · 46a62612
      Mike Hibler authored
      On Linux, device IO goes through the buffer cache by default. This makes
      frisbee run really fast...until it closes the output device. Then it sits
      for minutes while it flushes disk data out of the cache. This is
      technically okay, but wasteful, since frisbee allocates its own memory
      for caching disk write data. By using direct IO on the output device,
      writes do not go through the cache.
      
      Aren't two caches better than one? No. They can compete for memory and it
      just causes an extra data copy. Frisbee is faster when using O_DIRECT.
      
      Note that this change is complicated somewhat because Linux requires that
      the IO buffer for an O_DIRECT opened file be sector aligned. So we play
      some games to do this.
      
      This should have no effect on FreeBSD where device writes don't go through
      the buffer cache.
      46a62612
    • Mike Hibler's avatar
      Avoid a pre-mature close of the output fd. · 179c189f
      Mike Hibler authored
      Wait til all other threads have joined.
      179c189f
  21. 30 Aug, 2011 1 commit
  22. 25 Aug, 2011 1 commit
  23. 13 Jul, 2011 1 commit
  24. 02 Jun, 2011 1 commit
  25. 31 May, 2011 1 commit
    • Mike Hibler's avatar
      Added compat option and assorted cleanups. · d8de4d6a
      Mike Hibler authored
      Added WITH_V3COMPAT to make sure that we can still generated V3 images
      (for other sites) if checksum/encryption is not used.
      
      Try to clean up the command line options some. Be more consistent by putting
      generated uuid into a file instead of just spitting it out on stderr. Make
      sure that if the decryption specifies signing and/or encryption that we
      require the image to have that info. Add some more assertions. (Re)distinguish
      checksums from signed-checksums.
      d8de4d6a
  26. 29 Oct, 2010 1 commit
    • Mike Hibler's avatar
      Improve regular (non-image) file transfer via frisbee. · 78f3f8d3
      Mike Hibler authored
      Basically, make it possible to transfer a non imagezip image.  Previously
      you had to wrap a regular file as an image in order to transfer it.  The
      big hang up was that the frisbee protocol could only transfer files that
      were a multiple of 1MB (the chunk size).
      
      This commit changes the frisbee protocol slightly to allow transfer of
      non-1MB-multiple files.  The protocol change was to add a new JOIN message
      that returns the size of the file in bytes rather than in blocks.  This
      allows the client to know that the file in question is not a multiple of 1MB
      and allows it to request the correct partial number of blocks for the
      final chunk and to extract the correct amount of data from the final 1K block
      (that block is still padded to 1K by the server).  For the server side, the
      request mostly allows it to do some sanity checking.  The fact that the
      server is started with a file that is not a multiple of 1MB is what triggers
      it to know about partial chunks.  The sanity checking is that the server will
      not acknowledge clients that attempt to join with a version 1 JOIN message,
      since nothing good would come of that pairing.
      
      On the client side, frisbee must be invoked with the -N (nodecompress) option
      in order to issue a v2 JOIN.  See the comment in the code for the rationale,
      but it is largely a backward compat feature.
      
      While I was changing the JOIN message, I added a couple of other future
      features.  One is that by passing back a 64-bit value for the size of the
      image in bytes, we can feed bigger images.  However there is still much to
      be done to realize this.  The other was to add blocksize/chunksize fields
      in the message so that the server/client can negotiate the transfer parameters,
      e.g., 1024 blocks of 1024 bytes vs. 256 blocks of 8192 bytes, the latter being
      for "jumbo" packets on a Gb ethernet.  But there is still more to be done to
      get this working too.
      78f3f8d3
  27. 28 Oct, 2010 1 commit
  28. 20 Jul, 2010 1 commit
  29. 06 Jul, 2010 1 commit
  30. 02 Jul, 2010 3 commits
  31. 17 Jun, 2010 5 commits