1. 12 May, 2015 3 commits
  2. 17 Dec, 2014 1 commit
  3. 27 Jun, 2014 1 commit
    • Mike Hibler's avatar
      Resurrect/enhance delta image code. Another multi-day "one hour hack"! · 49d2f488
      Mike Hibler authored
      Resurrect: get the basic signature matching code working again.
      
      Enhance: add -U option to have imagezip update (or create) the
      signature file. Previously, the signature file was created off-line
      on boss with the imagehash command (that would be Mike's "imagehash"
      (/usr/testbed/bin/imagehash) and not Leigh's "imagehash"
      (/usr/testbed/sbin/imagehash)). Creating it as we create the image
      makes a lot of sense...except for how we do it. We actually read and
      create the hashes as a separate pass before we re-read, compress, and
      create the image--so we read the disk twice. [This is primarily because we
      are mooching off of the existing hash checking code (-H option). Doing
      this right will require re-writing The Big Loop which makes a single
      pass through the data, simultaneously dealing with disk IO, allocated
      ranges, and compression blocks all of which have different size/alignment
      criteria. But I digress...] Anyway, reading the disk data twice sucks,
      but at least it is on the client and not on boss. The take away is:
      don't create your images on pc600s.
      
      Note that -U will always create a signature file for the complete disk
      or partition even when you are creating a delta image (i.e., when combined
      with -H).
      
      Enhance: add "-P <pct>" option, used with -H, which tells imagezip that if
      a resulting delta image would be over <pct> percent the size (where size
      is number of uncompressed sectors in the image) of a full image, then just
      create a full image instead. So "-P 50" says if it would be over half the
      size, "-P 200" says if it was over twice the size, etc. If you always want
      a delta image to be produced, use -H without -P. If you always want a full
      image, don't use -H.
      
      This is part 2 of supporting images. Part 1 is the DB and user interface
      changes that Leigh is working on. Part 3 is next up and involves modifying
      the image creation MFS to download and use signatures along with the
      new imagezip when creating images. Stay tuned.
      49d2f488
  4. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  5. 20 Mar, 2012 1 commit
  6. 31 May, 2011 1 commit
    • Mike Hibler's avatar
      Added compat option and assorted cleanups. · d8de4d6a
      Mike Hibler authored
      Added WITH_V3COMPAT to make sure that we can still generated V3 images
      (for other sites) if checksum/encryption is not used.
      
      Try to clean up the command line options some. Be more consistent by putting
      generated uuid into a file instead of just spitting it out on stderr. Make
      sure that if the decryption specifies signing and/or encryption that we
      require the image to have that info. Add some more assertions. (Re)distinguish
      checksums from signed-checksums.
      d8de4d6a
  7. 06 Jul, 2010 1 commit
  8. 02 Jul, 2010 2 commits
  9. 17 Jun, 2010 8 commits
  10. 28 Sep, 2009 1 commit
    • Mike Hibler's avatar
      A variety of stop-gap changes: · f0885af0
      Mike Hibler authored
       * lint: compile cleanly under newer GCC and on 64-bit OS
       * rename SUBBLOCKSIZE to CHUNKSIZE since that is what it is
       * collect some more stats, primarily for frisbee event collection code
       * add stupid hacky bubble-sort workaround but don't define it by default
         cuz it is stupid...and hacky...but it allows mike to make an image of
         his home machine before he dies of old age
       * tweaks to libntfs version support
      f0885af0
  11. 11 Sep, 2009 1 commit
    • Mike Hibler's avatar
      Add '-f' "filemode" option to imagezip for creating image files from regular · 3b61ce12
      Mike Hibler authored
      files (as opposed to from disk partitions).  This is almost identical to
      rawmode, but also knows how to handle inputs that are not a multiple of the
      sector size.  Imagezip will pad out the final sector, but uses a special
      relocation entry (SHORTSECTOR) to tell imageunzip/frisbee how many bytes
      to write in that final sector.
      
      This is "pretty much" backward compatible.  If an old version of imageunzip
      encounters the unknown new relocation entry, it will ignore it, resulting
      in imageunzip just writing out the additional pad bytes to the output file.
      This may or may not screw over the resulting output file.
      
      Also fixed some lint found by GCC 4.2 on FBSD7.
      3b61ce12
  12. 12 Mar, 2004 1 commit
    • Mike Hibler's avatar
      Random imagezip hacking: · 27d76461
      Mike Hibler authored
      1. Make imagezip (and imageunzip) understand LILO bootblocks.
         Imagezip will create relocation information for all the absolute
         blocks that LILO embeds in the partition.  Imageunzip will correctly
         relocate those locations when laying down the new image.  Thus
         Linux slice images can be layed down anywhere on the disk (as can
         BSD images).
      
      2. Because #1 is incompatible with the old imageunzip, I bumped the
         imagezip format version number (to 3) on images that contain relocations
         (i.e., slice images).  Make sure you have a new imageunzip to unzip
         new slice images.  Whole disk images are still created as version 2
         since they contain no relocs and thus are no different than before.
      
      3. Added -N option to imagezip to create slice images without any
         relocations.  [ We could create our Emulab images this way, since we
         don't have any mechanism for, or reason to, let the user choose which
         partition to load an image in.  Hence our slice images always wind up
         at the same offset on the disk and relocations are unnecessary. ]
         Note that there is no significant expense in either time or space for
         relocations, this option was more of a debugging thing.
      27d76461
  13. 14 Jun, 2003 1 commit
  14. 17 Jan, 2003 1 commit
  15. 14 Jan, 2003 1 commit
  16. 11 Dec, 2002 1 commit
    • Mike Hibler's avatar
      all: · a6a648e9
      Mike Hibler authored
      	add -Wall to CFLAGS and clean up lint
      	update the TODO file
      	explicitly size the header fields (e.g., int32_t not int)
      
      imagezip:
      	Version 2.
      
      	Adds two ints to the header to help track free space.  Each chunk
      	now has a first and last sector number which can describe any free
      	block before or after the data contained in the chunk.  This is
      	needed in order to properly zero all free space when laying down
      	an image.  In practice: the first chunk describes any free space
      	before the first allocated range and any free space after its
      	contained ranges and before the first allocated range in the second
      	chunk.  Every other chunk then describes just free space following
      	itself (since the previous chunk has already described the space
      	before this chunk).  The point being, we only describe each free
      	range once.
      
      	Added "relocation" information.  Relocation entries go in the chunk
      	header along with region descriptors.  This allows us to identify
      	chunks of data which need to be absolute disk blocks instead of
      	offsets from the containing partition.  This is now used for BSD-slice
      	partition tables which contain absolute disk blocks.  We can now
      	create an image in one slice and reload it into another slice.
      
      	Allow zlib compression level 0 (no compression).  This might be
      	useful on machines that have slow CPUs: do just FS-compression and
      	transfer the image elsewhere faster where it could be re-zipped
      	with regular compression.
      
      	Fix goof.  Previously we were not saving any DOS partition with
      	an unrecognized type.  We should be naively compressing it instead.
      	This is what we now do.  We continue to skip partitions of type 0
      	("unused").
      
      	mikeism: add handler for SIGINFO (^T) to report progress of
      	a zip-age.
      
      	Added everybody's favorite "dots" mode for reporting progress.
      
      	Eliminate some excess copies left over from the conversion from
      	write-every-little-piece to buffer-up-a-full-chunk-and-then-write.
      
      	Eliminaged the special case handling of no skips (ranges) in
      	compress_image by creating a single allocated range describing
      	the whole disk/partition in this case.
      
      	For NTFS, make the behavior of calling missing unicode routines
      	be to return an error rather than exit.  These calls happen,
      	but their failing doesn't seem to be fatal.
      
      	Lots of typical mike-pissing on everything else.
      
      imageunzip:
      
      	Modify to handle both V1 and V2 images.
      
      	In slice mode, make sure we don't write past the bounds of
      	the slice.  ES&D if we try.
      
      	Make output to unseekable devices work again (broken when
      	pwrite was added)
      
      	Add debug -F (Frisbee) option to randomize the presentation of
      	chunks to the unzip/write threads.  Used to simulate frisbee.
      
      	Add "-T DOS-type" option to tell imageunzip, when in slice mode,
      	to set the type of the slice in the DOS partition table.
      	This is useful if you are dropping say a BSD filesystem into
      	an unused slice, you don't have to go back later and set this
      	with fdisk.  Considered making this info part of the image
      	itself (recorded by imagezip when creating a slice image),
      	but decided against it.
      
      	writezero takes an off_t for the size, we can be asked to write
      	many gigabytes of zero at the end of a disk.
      
      	Turn off dots mode by default.  Ya wanna see spots?  Ya gotta
      	turn it on!
      
      	Lots of typical mike-pissing on everything else.
      
      imagedump:
      
      	New tool for checking/dumping the structure of an image and
      	reporting stats about it.
      a6a648e9
  17. 20 Apr, 2001 1 commit
  18. 01 Mar, 2001 1 commit