1. 23 Mar, 2003 1 commit
  2. 21 Mar, 2003 1 commit
  3. 20 Mar, 2003 2 commits
    • Kirk Webb's avatar
      afabc26d
    • Kirk Webb's avatar
      Looks big and scary, and probably is. This checkin includes all the · 47962f80
      Kirk Webb authored
      cruft needed to compile ntfs support into imagezip, including the i18n
      unicode library
      routines from FBSD 5.0.  Most of the magic is in the makefiles, but there
      is a patch for the ntfsprogs package we're using to get NTFS support.
      (patch contributed by Russ).
      
      Note that the ntfsprogs tarball is located in emulab webspace (like TG).
      
      Don't worry!  You still have to pass --enable-windows to configure in order to
      pull any of this in, although it should compile nicely from here on out.
      
      Russ will commit his imagezip changes (hiber, and page file skipping) soon.
      47962f80
  4. 25 Jan, 2003 1 commit
  5. 21 Jan, 2003 2 commits
    • Mike Hibler's avatar
      Add option to vary the minimum free range size. · bc67baa3
      Mike Hibler authored
      This replaces a couple of ad-hoc checks in the FreeBSD and Linux filesystem
      code.  The idea is that we record a free range only if it is over a certain
      length.  This has the effect of combining multiple short allocated ranges
      into a single range, making for more efficient disk writes in the unzipper.
      bc67baa3
    • Mike Hibler's avatar
      summarize free/allocated range info · d380d13a
      Mike Hibler authored
      recognize OpenBSD reloc type
      d380d13a
  6. 17 Jan, 2003 1 commit
  7. 16 Jan, 2003 3 commits
  8. 14 Jan, 2003 1 commit
  9. 10 Jan, 2003 2 commits
  10. 09 Jan, 2003 1 commit
  11. 08 Jan, 2003 2 commits
  12. 06 Jan, 2003 2 commits
  13. 03 Jan, 2003 1 commit
  14. 18 Dec, 2002 2 commits
  15. 17 Dec, 2002 2 commits
  16. 13 Dec, 2002 1 commit
  17. 12 Dec, 2002 3 commits
  18. 11 Dec, 2002 2 commits
    • Mike Hibler's avatar
      all: · a6a648e9
      Mike Hibler authored
      	add -Wall to CFLAGS and clean up lint
      	update the TODO file
      	explicitly size the header fields (e.g., int32_t not int)
      
      imagezip:
      	Version 2.
      
      	Adds two ints to the header to help track free space.  Each chunk
      	now has a first and last sector number which can describe any free
      	block before or after the data contained in the chunk.  This is
      	needed in order to properly zero all free space when laying down
      	an image.  In practice: the first chunk describes any free space
      	before the first allocated range and any free space after its
      	contained ranges and before the first allocated range in the second
      	chunk.  Every other chunk then describes just free space following
      	itself (since the previous chunk has already described the space
      	before this chunk).  The point being, we only describe each free
      	range once.
      
      	Added "relocation" information.  Relocation entries go in the chunk
      	header along with region descriptors.  This allows us to identify
      	chunks of data which need to be absolute disk blocks instead of
      	offsets from the containing partition.  This is now used for BSD-slice
      	partition tables which contain absolute disk blocks.  We can now
      	create an image in one slice and reload it into another slice.
      
      	Allow zlib compression level 0 (no compression).  This might be
      	useful on machines that have slow CPUs: do just FS-compression and
      	transfer the image elsewhere faster where it could be re-zipped
      	with regular compression.
      
      	Fix goof.  Previously we were not saving any DOS partition with
      	an unrecognized type.  We should be naively compressing it instead.
      	This is what we now do.  We continue to skip partitions of type 0
      	("unused").
      
      	mikeism: add handler for SIGINFO (^T) to report progress of
      	a zip-age.
      
      	Added everybody's favorite "dots" mode for reporting progress.
      
      	Eliminate some excess copies left over from the conversion from
      	write-every-little-piece to buffer-up-a-full-chunk-and-then-write.
      
      	Eliminaged the special case handling of no skips (ranges) in
      	compress_image by creating a single allocated range describing
      	the whole disk/partition in this case.
      
      	For NTFS, make the behavior of calling missing unicode routines
      	be to return an error rather than exit.  These calls happen,
      	but their failing doesn't seem to be fatal.
      
      	Lots of typical mike-pissing on everything else.
      
      imageunzip:
      
      	Modify to handle both V1 and V2 images.
      
      	In slice mode, make sure we don't write past the bounds of
      	the slice.  ES&D if we try.
      
      	Make output to unseekable devices work again (broken when
      	pwrite was added)
      
      	Add debug -F (Frisbee) option to randomize the presentation of
      	chunks to the unzip/write threads.  Used to simulate frisbee.
      
      	Add "-T DOS-type" option to tell imageunzip, when in slice mode,
      	to set the type of the slice in the DOS partition table.
      	This is useful if you are dropping say a BSD filesystem into
      	an unused slice, you don't have to go back later and set this
      	with fdisk.  Considered making this info part of the image
      	itself (recorded by imagezip when creating a slice image),
      	but decided against it.
      
      	writezero takes an off_t for the size, we can be asked to write
      	many gigabytes of zero at the end of a disk.
      
      	Turn off dots mode by default.  Ya wanna see spots?  Ya gotta
      	turn it on!
      
      	Lots of typical mike-pissing on everything else.
      
      imagedump:
      
      	New tool for checking/dumping the structure of an image and
      	reporting stats about it.
      a6a648e9
    • Mike Hibler's avatar
      Server: back to using a condvar since they seem to be fixed. · 2e77122f
      Mike Hibler authored
      Server: make file readsize independent of burstsize (previously
      readsize had to be a divisor of burstsize).  A subtle side-effect
      is that the dynamic burst rate is recalcluated at the conslusion
      of every burst instead of after every readsize count of blocks has
      been sent (less than a burst)  This just seems to be more logical.
      
      Client: add "-T DOS-type" option to tell frisbee, when in slice
      mode, to set the type of the slice in the DOS partition table.
      This is useful if you are dropping say a BSD filesystem into
      an unused slice, you don't have to go back later and set this
      with fdisk.  Considered making this info part of the image
      itself (recorded by imagezip when creating a slice image),
      but decided against it.
      2e77122f
  19. 06 Dec, 2002 1 commit
  20. 02 Dec, 2002 1 commit
  21. 27 Nov, 2002 1 commit
  22. 26 Nov, 2002 4 commits
    • Mike Hibler's avatar
      Remove hokey loss rate code · 1f897669
      Mike Hibler authored
      1f897669
    • Mike Hibler's avatar
      Commit of USENIX driven improvements: · 2ff95cee
      Mike Hibler authored
      1. Client: add "NAK avoidance."  We track our (and others, via snooping) block
         requests and avoid making re-requests unless it has been "long enough."
      
      2. Server: more aggressive merging of requests in the work queue.  For every
         new request, look for any overlap with an existing entry.
      
      3. Server: from Leigh: first cut at dynamic rate adjustment.  Can be enabled
         with -D option.
      
      4. Both: change a lot of the magic constants into runtime variables so that
         they can be adjusted on the command line or via the event interface (see
         below).
      
      5. Add code to do basic validatation of incoming packets.
      
      6. Client: randomization of block request order is now optional.
      
      7. Client: startup delay is optional and specified via a parameter N which
         says "randomly delay between 0 and N seconds before attempting to join."
      
      8. Both: add a new LEAVE message which reports back all the client stats to
         the server (which logs them).
      
      9. Both: attempt to comment some of the magic values in decls.h.
      
      10. Both: add cheezy hack to fake packet loss.  Disabled by default, see
         the GNUmakefile.  This code is coming out right after I archive it with
         this commit.
      
      11. Add tracing code.  Frisbee server/client will record a number of
         interesting events in a memory buffer and dump them at the end.  Not
         compiled in by default, see the GNUmakefile (NEVENTS) for turning this on.
      
      12. Not to be confused with the events above, also added testbed event system
         code so that frisbee clients can be remotely controlled.  This is a hack
         for measurement purposes (it requires a special rc.frisbee in the frisbee
         MFS).  Allows changing of all sorts of parameters as well as implementing
         a crude form of identification allowing you to start only a subset of
         clients.  Interface is via tevc with commands like:
      	tevc -e testbed,frisbee now frisbee start maxclients=5 readahead=5
      	tevc -e testbed,frisbee now frisbee stop exitstatus=42
         Again, this is not compiled in by default as it makes the client about
         4x bigger.  See the GNUmakefile for turning it on.
      2ff95cee
    • Mike Hibler's avatar
      Forgot this one... · e416a8cc
      Mike Hibler authored
      e416a8cc
    • Mike Hibler's avatar
      Commit of USENIX driven improvements: · dd7a05ee
      Mike Hibler authored
      1. The big one is from Leigh: multithreading the unzipper so that
         decompression overlaps with disk IO.  Also added the '-n' "no threads"
         option for comparison purposes.
      
      2. Eh...guess that's it for imagezip/unzip.  Oh wait, mike fixed a comment!
      dd7a05ee
  23. 23 Nov, 2002 1 commit
  24. 17 Nov, 2002 1 commit
  25. 15 Nov, 2002 1 commit