1. 19 Mar, 2005 1 commit
  2. 17 Mar, 2005 1 commit
  3. 16 Mar, 2005 1 commit
    • Mike Hibler's avatar
      Really important stuff: · a8ef625f
      Mike Hibler authored
      Unified the 'dot' handling (status printing) of frisbee and imagezip.
      They now both report the number of chunks remaining along with the dots.
      Also put out a periodic splat for every GB of uncompressed data we write.
      This is useful when you are zero-filling, since otherwise it appears that
      frisbee has hung when it is really just zeroing the last unused 100GB of
      your disk.
      a8ef625f
  4. 14 Mar, 2005 1 commit
  5. 07 Mar, 2005 1 commit
  6. 06 Mar, 2005 1 commit
  7. 27 Jan, 2005 1 commit
  8. 10 Jan, 2005 1 commit
    • Mike Hibler's avatar
      Experimental support for Sid's checkpointer. · e753d3e9
      Mike Hibler authored
      The checkpoint driver can return a list of blocks that have been modified
      since its startup.  If you use "imagezip -C /dev/shd0 -s 1 /dev/ad0 foo.ndz"
      imagezip will make an ioctl call on the shd device to get the modified
      blocks it needs to save (rather than parsing the filesystem).  It then
      creates an image using those blocks.
      
      The end result should be an "incremental" image that can be put down on
      the disk by frisbee after the original image has been loaded.
      
      This is all a theory right now (including the ioctl :-) but should get
      tested RSN.
      e753d3e9
  9. 03 Jan, 2005 1 commit
  10. 15 Nov, 2004 1 commit
    • Mike Hibler's avatar
      Move a little closer to being pedantic ansi: · ebc4c413
      Mike Hibler authored
      - add some missing casts
      - avoid void * arithmetic
      - change "q" printf formats to "ll"
      
      Still not there yet by any means.  Note that the version of the NTFS library
      we use won't compile with "-pedantic -ansi" so those are not part of the
      default CFLAGS.
      ebc4c413
  11. 12 Nov, 2004 2 commits
  12. 28 Oct, 2004 1 commit
    • Mike Hibler's avatar
      Minor tweaks from a one-day binge of performance analysis. · 1a76e634
      Mike Hibler authored
      The only meaningful change was to insert a sched_yield() in the frisbee
      decompressor path.  Apparently, the decompressor can run long enough to
      cause the incoming socket buffer to overflow.  I was under the assumption
      that the decompressor would not run much longer than a single time slice
      (0.001 seconds, about 8 packets) before its priority would force it to
      be context switched.  But it was running much longer than that!  Forcing
      a periodic yield seems to have taken care of this.
      
      One other cause of retransmitted blocks that I saw was where the server
      was taking a long time to read data from a file (up to 0.25 seconds).
      This would stall the clients and force them to rerequest blocks (which
      they do after about 0.10 seconds).  We can improve on this by splitting
      the file reading off to a seperate thread.
      
      Most other changes are related to the event logging code.
      1a76e634
  13. 25 Oct, 2004 1 commit
  14. 19 Oct, 2004 1 commit
  15. 15 Oct, 2004 1 commit
  16. 11 Sep, 2004 1 commit
  17. 14 Jul, 2004 1 commit
    • Mike Hibler's avatar
      Add a "raw" hash type ("-D raw"). Doesn't compute and compare hashes vs. · c6c4a847
      Mike Hibler authored
      a signature file, rather it compares the actual data in the image file vs.
      what is on a disk.  Mostly a debugging option, trying to find the source
      of a disk corruption problem when loading images with FBSD 4.10.
      
      Teach it about relocations, and (optionally) have it not compute hashes
      for blocks which involve relocations, since those areas will likely always
      differ in the image and disk.  The default behavior is to have it look at
      everything as before, use -N to have it skip relocation areas.
      c6c4a847
  18. 24 Jun, 2004 1 commit
    • Mike Hibler's avatar
      Improve the client-side install. With these changes, it should now be · 976133e4
      Mike Hibler authored
      possible to:
      
      	gmake client
      	sudo gmake client-install
      
      on a FBSD4, FBSD5, RHL7.3, and RHL9.0 client node.
      
      There are still some dependencies that are not explicit and which would
      prevent a build/install from working on a "clean" OS.  Two that I know of are:
      you must install our version of the elvin libraries and you must install boost.
      976133e4
  19. 27 May, 2004 1 commit
  20. 26 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Cleanup Makefiles: · 297019fb
      Mike Hibler authored
      1. "make clean" will just remove stuff built in the process of a regular build
      2. "make distclean" will also clean out configure generated files.
      
      This is how it was always supposed to be, there was just some bitrot.
      297019fb
  21. 20 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Improve the client-install. You can now do a "make client-install" from · 361ee691
      Mike Hibler authored
      the top level.  This will build all the necessary binaries and then install
      them.  This works on FBSD4 and RHL7.3.  It still doesn't work on FBSD5
      (newer compiler that no longer supports a style of use of _FUNCTION_ in the
      event lib) or RHL9 (event lib needs SSL lib which has a bad dependency
      on Kerberos).  Notes:
      
      - requires that elvin libraries be installed on nodes (they are) to build
        event agents, requires linuxthreads be installed on FBSD (it is now) to
        build imagezip (which is installed, but is not strictly necessary)
      
      - installed event-agents and other binaries are stripped
      
      - added a few missing files to the source tree for bsd (healthd.conf)
        and linux (healthd.conf, rc.local)
      
      - the only thing that doesn't get rebuilt in /usr/local/etc/emulab is
        healthd, I couldn't quickly find how it gets built
      
      - uses a scaled down version of libtb with no DB functions (since mysql
        isn't installed on nodes).  N.B. DO NOT DO A CLIENT INSTALL FROM YOUR
        REGULAR OBJ TREE OR ELSE YOU MAY WIND UP WITH A NEUTERED VERSION OF
        libtb.a!
      
      The build-as-well-as-install semantics are counter to the regular install
      targets, but this is what we gotta do for now.  Once the TB source builds
      under Linux and newer BSDs, we could undo this and just require that people
      do a regular "make" followed by "make client-install"  OTOH, there should
      be no reason to require installation of mysql and other server-side packages
      just to build clients (or make them sit through the compilation of assign),
      so maybe we will keep the client build special.
      361ee691
  22. 19 Apr, 2004 1 commit
  23. 07 Apr, 2004 4 commits
  24. 22 Mar, 2004 1 commit
  25. 12 Mar, 2004 1 commit
    • Mike Hibler's avatar
      Random imagezip hacking: · 27d76461
      Mike Hibler authored
      1. Make imagezip (and imageunzip) understand LILO bootblocks.
         Imagezip will create relocation information for all the absolute
         blocks that LILO embeds in the partition.  Imageunzip will correctly
         relocate those locations when laying down the new image.  Thus
         Linux slice images can be layed down anywhere on the disk (as can
         BSD images).
      
      2. Because #1 is incompatible with the old imageunzip, I bumped the
         imagezip format version number (to 3) on images that contain relocations
         (i.e., slice images).  Make sure you have a new imageunzip to unzip
         new slice images.  Whole disk images are still created as version 2
         since they contain no relocs and thus are no different than before.
      
      3. Added -N option to imagezip to create slice images without any
         relocations.  [ We could create our Emulab images this way, since we
         don't have any mechanism for, or reason to, let the user choose which
         partition to load an image in.  Hence our slice images always wind up
         at the same offset on the disk and relocations are unnecessary. ]
         Note that there is no significant expense in either time or space for
         relocations, this option was more of a debugging thing.
      27d76461
  26. 08 Mar, 2004 1 commit
  27. 05 Mar, 2004 2 commits
  28. 17 Nov, 2003 1 commit
  29. 24 Oct, 2003 4 commits
    • Mike Hibler's avatar
      Make clean target work correctly too! · 30d21c3b
      Mike Hibler authored
      30d21c3b
    • Mike Hibler's avatar
    • Mike Hibler's avatar
      Introducing imagezip redux! · 0063d2c4
      Mike Hibler authored
      Ok, not really any fundamental changes, mostly a source reorganization,
      but some new features as well:
      
      1. Split off FS specific code for all FS types into subdirectories.
         imagezip.c was just getting unwieldy.  You can now (in theory, I
         haven't enabled this in any easy way) build a version of imagezip
         with only the FSes you care about.
      
      2. As part of the above, localize the filesystem specific headers in
         the subdirs instead of relying on the installed system headers.
         This allows easy building of, for example, BSD FS support under
         Linux.
      
      3. Hacked up the GNUmakefiles some so that imagezip with all FS support
         will build (in the Emulab context) under Linux and FreeBSD 5.1 as
         well as FreeBSD 4.x.  Still a hack, for Linux you need to first do:
      	setenv LINUX 1
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         and for FreeBSD 5.x:
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         these trigger ifdefs in some of the Makefiles to DTRT.  Imagezip
         and frisbee still need a standalone configure...
      
      4. Added support for UFS2 (aka FreeBSD 5.x default filesystems).
      
      5. Added "-S dos-type" option to imagezip.  This is the general form
         of the -b, -l, and -n options to force imagezip to treat the device
         as though it contains a filesystem of the indicated type.
      0063d2c4
    • Mike Hibler's avatar
      Added -i (ignore v1 features) support to imagedump. Prevents it · ccf23ccb
      Mike Hibler authored
      from looking at the embedded (and deprecated) blockindex field in
      the headers.  Now imagedump won't complain if you run it on a
      piece of a split image.
      ccf23ccb
  30. 23 Oct, 2003 1 commit
  31. 17 Oct, 2003 1 commit
  32. 03 Oct, 2003 1 commit
    • Mike Hibler's avatar
      First cut at Windows FAT support. With code from the FreeBSD fsck_msdosfs · a706c6fb
      Mike Hibler authored
      program, it only took a couple of hours.  Heavily tested: it didn't core dump
      examining my 20GB FAT32 partition, ship it!
      
      Actually, I did imagezip/imageunzip a FAT12 DOS floppy.  Since imagezip
      files are a minimum of 1MB (the chunk size), it is probably not practical
      for saving 1.4MB floppies :-)
      
      Also, updated the man page.
      a706c6fb