1. 19 Apr, 2004 1 commit
  2. 07 Apr, 2004 2 commits
  3. 12 Mar, 2004 1 commit
    • Mike Hibler's avatar
      Random imagezip hacking: · 27d76461
      Mike Hibler authored
      1. Make imagezip (and imageunzip) understand LILO bootblocks.
         Imagezip will create relocation information for all the absolute
         blocks that LILO embeds in the partition.  Imageunzip will correctly
         relocate those locations when laying down the new image.  Thus
         Linux slice images can be layed down anywhere on the disk (as can
         BSD images).
      
      2. Because #1 is incompatible with the old imageunzip, I bumped the
         imagezip format version number (to 3) on images that contain relocations
         (i.e., slice images).  Make sure you have a new imageunzip to unzip
         new slice images.  Whole disk images are still created as version 2
         since they contain no relocs and thus are no different than before.
      
      3. Added -N option to imagezip to create slice images without any
         relocations.  [ We could create our Emulab images this way, since we
         don't have any mechanism for, or reason to, let the user choose which
         partition to load an image in.  Hence our slice images always wind up
         at the same offset on the disk and relocations are unnecessary. ]
         Note that there is no significant expense in either time or space for
         relocations, this option was more of a debugging thing.
      27d76461
  4. 08 Mar, 2004 1 commit
  5. 24 Oct, 2003 1 commit
    • Mike Hibler's avatar
      Introducing imagezip redux! · 0063d2c4
      Mike Hibler authored
      Ok, not really any fundamental changes, mostly a source reorganization,
      but some new features as well:
      
      1. Split off FS specific code for all FS types into subdirectories.
         imagezip.c was just getting unwieldy.  You can now (in theory, I
         haven't enabled this in any easy way) build a version of imagezip
         with only the FSes you care about.
      
      2. As part of the above, localize the filesystem specific headers in
         the subdirs instead of relying on the installed system headers.
         This allows easy building of, for example, BSD FS support under
         Linux.
      
      3. Hacked up the GNUmakefiles some so that imagezip with all FS support
         will build (in the Emulab context) under Linux and FreeBSD 5.1 as
         well as FreeBSD 4.x.  Still a hack, for Linux you need to first do:
      	setenv LINUX 1
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         and for FreeBSD 5.x:
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         these trigger ifdefs in some of the Makefiles to DTRT.  Imagezip
         and frisbee still need a standalone configure...
      
      4. Added support for UFS2 (aka FreeBSD 5.x default filesystems).
      
      5. Added "-S dos-type" option to imagezip.  This is the general form
         of the -b, -l, and -n options to force imagezip to treat the device
         as though it contains a filesystem of the indicated type.
      0063d2c4
  6. 17 Oct, 2003 1 commit
  7. 03 Oct, 2003 1 commit
    • Mike Hibler's avatar
      First cut at Windows FAT support. With code from the FreeBSD fsck_msdosfs · a706c6fb
      Mike Hibler authored
      program, it only took a couple of hours.  Heavily tested: it didn't core dump
      examining my 20GB FAT32 partition, ship it!
      
      Actually, I did imagezip/imageunzip a FAT12 DOS floppy.  Since imagezip
      files are a minimum of 1MB (the chunk size), it is probably not practical
      for saving 1.4MB floppies :-)
      
      Also, updated the man page.
      a706c6fb
  8. 02 Oct, 2003 2 commits
    • Mike Hibler's avatar
      Fix bug in computation of max input sector when there are extended partitions. · 4b6a7925
      Mike Hibler authored
      Changes to "info" option
      4b6a7925
    • Mike Hibler's avatar
      "Support for extended partition tables is easy!" he says... · 4301b8be
      Mike Hibler authored
      "Piece of cake!" he says...
      "15 minutes" he says...
      
      Well, a day later, we can now deal with extended partition tables.
      It was almost easy.  The offsets in an extended partition table are
      relative to the location of that table...mostly.  The exception is if
      an extended partition table contains an entry for another extended table.
      Then the offset provided is relative to the *first* extended partition
      table found, not the current one.
      
      This was also complicated by the NTFS library code which needs to be
      fed a device special file corresponding to the filesystem partition.
      So if I am cruising alone creating a whole disk image using /dev/ad0
      and I hit a partition containing an NTFS, I have to figure out the
      BSD name for that particular partition: "hmm..I'm looking at partition 3?
      That would be /dev/ad0s3".  Its easy for the first 4 partitions, but for
      extended partitions, I had to figure out how BSD mapped those partitions
      into device names.  Once I became aware of the DIOCGSLICEINFO ioctl, this
      was easy as it gave back all the info needed.
      4301b8be
  9. 03 Sep, 2003 1 commit
  10. 02 Jul, 2003 1 commit
    • Leigh B. Stoller's avatar
      Remove some Linux FS hardwired restrictions and assumptions about the · 2ddfb904
      Leigh B. Stoller authored
      block size. I was only allowing 4K block size FS's, and assumed that
      the group descriptor map was in the second block (not true when the
      blocksize if 1K), and that the number of blocks in a descriptor as 8
      times the block size. Also moved some sanity checks around, and
      cleaned up a couple of seeks. Now works on the baleep image, which is
      where we first noticed the problem.
      2ddfb904
  11. 23 Jun, 2003 1 commit
    • Mike Hibler's avatar
      A couple of new options inspired by home use: · 39b1f04c
      Mike Hibler authored
      1. From the "Thank you DOS for all your stupid limits" department
         we have an enhancement to the -I option to ignore a slice.
         The slice number can be followed by a BSD partition letter
         (e.g., "-I1g") to ignore a BSD partition within a slice.
         Ditto for the -R options which forces raw compression.
      
      2. From the "Gee, I shoulda taken that grinding noise more seriously"
         department we have the -X option to force imagezip to try e(X)tremely
         hard (yeah, yeah, its the only letter I could come up with) to read
         the data off the disk, even faking (zero) data if a read fails after
         10 tries.
      39b1f04c
  12. 14 Jun, 2003 1 commit
  13. 09 Apr, 2003 1 commit
    • Mike Hibler's avatar
      Frisbee general: · 9e55b0b1
      Mike Hibler authored
      1. Implement PREQUEST message which passes a bit map of desired blocks.
         We still use the REQUEST message (start block + number of blocks) for
         full chunk requests as that is more efficient.  This message also
         includes a flag indicating whether it is a retry of a request we
         originally made or not.  This gives the server more accurate loss info.
      
      2. More stats and tracing goo.
      
      
      Frisbee client:
      
      1. Add 'C' and 'W' command line options to specify amount of memory
         for chunk buffers (network buffering) and for write buffers (disk
         buffering).  The Emulab frisbee startup script uses these to partition
         up all the available memory on a machine.  Previously we were just
         using a fixed ~128MB even though our machines have 256 or 512MB of
         memory.  Also add the 'M' option which specifies the overall memory,
         internally dividing it up between chunk buffers and write buffers.
      
      2. Add 'S' command line option to explicitly specify the server.  This
         allows us to make a feeb......
      9e55b0b1
  14. 31 Mar, 2003 1 commit
  15. 21 Mar, 2003 1 commit
  16. 25 Jan, 2003 1 commit
  17. 21 Jan, 2003 1 commit
    • Mike Hibler's avatar
      Add option to vary the minimum free range size. · bc67baa3
      Mike Hibler authored
      This replaces a couple of ad-hoc checks in the FreeBSD and Linux filesystem
      code.  The idea is that we record a free range only if it is over a certain
      length.  This has the effect of combining multiple short allocated ranges
      into a single range, making for more efficient disk writes in the unzipper.
      bc67baa3
  18. 17 Jan, 2003 1 commit
  19. 17 Dec, 2002 2 commits
  20. 13 Dec, 2002 1 commit
  21. 11 Dec, 2002 1 commit
    • Mike Hibler's avatar
      all: · a6a648e9
      Mike Hibler authored
      	add -Wall to CFLAGS and clean up lint
      	update the TODO file
      	explicitly size the header fields (e.g., int32_t not int)
      
      imagezip:
      	Version 2.
      
      	Adds two ints to the header to help track free space.  Each chunk
      	now has a first and last sector number which can describe any free
      	block before or after the data contained in the chunk.  This is
      	needed in order to properly zero all free space when laying down
      	an image.  In practice: the first chunk describes any free space
      	before the first allocated range and any free space after its
      	contained ranges and before the first allocated range in the second
      	chunk.  Every other chunk then describes just free space following
      	itself (since the previous chunk has already described the space
      	before this chunk).  The point being, we only describe each free
      	range once.
      
      	Added "relocation" information.  Relocation entries go in the chunk
      	header along with region descriptors.  This allows us to identify
      	chunks of data which need to be absolute disk blocks inst...
      a6a648e9
  22. 26 Nov, 2002 1 commit
    • Mike Hibler's avatar
      Commit of USENIX driven improvements: · dd7a05ee
      Mike Hibler authored
      1. The big one is from Leigh: multithreading the unzipper so that
         decompression overlaps with disk IO.  Also added the '-n' "no threads"
         option for comparison purposes.
      
      2. Eh...guess that's it for imagezip/unzip.  Oh wait, mike fixed a comment!
      dd7a05ee
  23. 15 Nov, 2002 1 commit
  24. 04 Nov, 2002 1 commit
  25. 09 Oct, 2002 1 commit
  26. 03 Sep, 2002 1 commit
  27. 13 Jul, 2002 1 commit
  28. 12 Jul, 2002 1 commit
  29. 07 Jul, 2002 1 commit
  30. 02 Jul, 2002 1 commit
    • Leigh B. Stoller's avatar
      Convert imagezip to optionally using stdout instead of a file by giving · ff17c390
      Leigh B. Stoller authored
      it "-" for the output filename. This means we cannot seek, and so the
      little bit of stuff we do for netdisk cannot be done, but so what.
      It also means that output errors are fatal, and so no reason to retry
      them (like we do for writes to an NFS file).
      
      Convert imageunzip to optionally using stdin instead of a file, by
      giving it "-" for the input name. I added an 1MB chunk buffer to,
      which reads are buffered into. I also changed unified the interface
      from the the Frisbee client, which got rid of some ifdefs.
      ff17c390
  31. 19 Jun, 2002 1 commit
    • Leigh B. Stoller's avatar
      Fixes! The new NFS server appears to be more asynchronous, and errors · 8c3c41f4
      Leigh B. Stoller authored
      caused by hupping mountd are not getting propogated back until later
      writes. Must do an fsync to catch the error. Too many fsyncs slows
      down imagezip by a factor of three though, so I reorged things so that
      we create the 1MB chunks all in memory, and then write out each one
      out as finished. Much better; only a 10% slowdown.
      8c3c41f4
  32. 25 Mar, 2002 1 commit
  33. 07 Jan, 2002 1 commit
  34. 02 Sep, 2001 1 commit
  35. 01 Aug, 2001 1 commit
  36. 25 Jul, 2001 1 commit
  37. 27 Jun, 2001 1 commit