1. 14 Jun, 2003 7 commits
  2. 13 Jun, 2003 1 commit
  3. 28 May, 2003 1 commit
  4. 12 May, 2003 1 commit
  5. 02 May, 2003 1 commit
  6. 30 Apr, 2003 1 commit
  7. 29 Apr, 2003 1 commit
  8. 28 Apr, 2003 1 commit
  9. 24 Apr, 2003 1 commit
  10. 18 Apr, 2003 2 commits
    • Mike Hibler's avatar
      Combine the chunk buffer inprogress and ready fields into a single state · aa078bd8
      Mike Hibler authored
      field.  This is NOT just an aesthetic change, it fixes a once-in-a-blue-
      moon race (that happened just yesterday) where the disk write thread
      cleared ready and then inprogress as two seperate assignments and got
      context switched after the first.  The result was that the net thread
      saw a chunk that was inprogress even though all blocks had been received
      and it issued a zero-block request.  This triggered an assert in the server.
      Note that if we were prissy we would protect chunk buffer accesses with
      a mutex, but that is overkill for this single piece of shared state that
      can be protected via the "reasonable" atomicity of a write instruction.
    • Mike Hibler's avatar
      make 0-length request just a warning and not fatal · 29ff81b5
      Mike Hibler authored
      get the client tracking code right this time!
  11. 11 Apr, 2003 1 commit
  12. 09 Apr, 2003 3 commits
    • Mike Hibler's avatar
      Minor cleanup · 00a3925f
      Mike Hibler authored
    • Mike Hibler's avatar
      Once again remove the hacky/hokey LOSSRATE code, · 8f349249
      Mike Hibler authored
      hopefully never to return again!
    • Mike Hibler's avatar
      Frisbee general: · 9e55b0b1
      Mike Hibler authored
      1. Implement PREQUEST message which passes a bit map of desired blocks.
         We still use the REQUEST message (start block + number of blocks) for
         full chunk requests as that is more efficient.  This message also
         includes a flag indicating whether it is a retry of a request we
         originally made or not.  This gives the server more accurate loss info.
      2. More stats and tracing goo.
      Frisbee client:
      1. Add 'C' and 'W' command line options to specify amount of memory
         for chunk buffers (network buffering) and for write buffers (disk
         buffering).  The Emulab frisbee startup script uses these to partition
         up all the available memory on a machine.  Previously we were just
         using a fixed ~128MB even though our machines have 256 or 512MB of
         memory.  Also add the 'M' option which specifies the overall memory,
         internally dividing it up between chunk buffers and write buffers.
      2. Add 'S' command line option to explicitly specify the server.  This
         allows us to make a feeb...um, "lightweight" authentication check
         on incoming messages.
      3. Use the common BlockMap data struct to track which pieces of a chunk
         we have received.  This is easily inverted to make PREQUESTS and it is
         also smaller than the older byte-per-block technique.
      4. Allow partial request-ahead.  Previously, we only issued request-ahead
         if there were enough empty chunk buffers for a maximum (2) request-ahead.
      Frisbee server:
      1. Use BlockMap for workQ elements.  An easy way to allow a complete merge
         of incoming requests with existing ones.
      2. Check for overlap of incoming requests with the request currently
         being serviced.  This happens surprisingly often.
      3. Dubious: burst gap becomes burst interval.  The latter takes into
         account the time required to read data, etc., in other words, we now
         have variable-sized gaps and put out bursts at specific times rather
         than having fixed gaps and putting out bursts at variable times.
         This gives us more accurate pacing over shorter time periods.  I
         thought this might be important for dynamic pacing.
      4. Add 'W' command line option to specify a target bandwidth.  Frisbeed
         will use this to calculate a burst size/interval.
      5. Rewrote the dynamic pacing code.  It is now easily as bad as before
         if not worse.  But it does have fewer magic constants!  Needs to be
         redone by someone who understands the TCP-friendly rate equation.
      1. add 'R' option to specify one or more partitions for which to force
         raw (naive) compression even if the FS format is understood.  Useful
         for benchmarking.
      2. add 'D' option to allow "dangerous" writes.  In this mode, we don't
         do the fsync's or retries of failed writes.  Overrides the hack we put
         in for NFS.  Use this if writing to a local filesystem (or /dev/null).
      3. Eliminate an extra copy of every chunk header.
      1. Eliminate extra copy of decompressed data that we were doing between
         the single decompression buffer and the disk buffers.  Helps on slow
         machines (like gatech's 300Mhz machines with 66MHz memory bus).
      2. Allow dynamic number of variable-sized write buffers.  Total memory
         not to exceed the writebufmem limit.  Previously we had a small number
         of fixed-size (256K) buffers.
      3. Add debugging 'C' option to just compute a single CRC of the decompressed
         image.  Back-ported to older imageunzip and used to make sure my write
         buffer changes were correct.  Maybe handy for similar massive changes
         in the future.
  13. 31 Mar, 2003 1 commit
  14. 23 Mar, 2003 1 commit
  15. 21 Mar, 2003 1 commit
  16. 20 Mar, 2003 2 commits
    • Kirk Webb's avatar
    • Kirk Webb's avatar
      Looks big and scary, and probably is. This checkin includes all the · 47962f80
      Kirk Webb authored
      cruft needed to compile ntfs support into imagezip, including the i18n
      unicode library
      routines from FBSD 5.0.  Most of the magic is in the makefiles, but there
      is a patch for the ntfsprogs package we're using to get NTFS support.
      (patch contributed by Russ).
      Note that the ntfsprogs tarball is located in emulab webspace (like TG).
      Don't worry!  You still have to pass --enable-windows to configure in order to
      pull any of this in, although it should compile nicely from here on out.
      Russ will commit his imagezip changes (hiber, and page file skipping) soon.
  17. 25 Jan, 2003 1 commit
  18. 21 Jan, 2003 2 commits
    • Mike Hibler's avatar
      Add option to vary the minimum free range size. · bc67baa3
      Mike Hibler authored
      This replaces a couple of ad-hoc checks in the FreeBSD and Linux filesystem
      code.  The idea is that we record a free range only if it is over a certain
      length.  This has the effect of combining multiple short allocated ranges
      into a single range, making for more efficient disk writes in the unzipper.
    • Mike Hibler's avatar
      summarize free/allocated range info · d380d13a
      Mike Hibler authored
      recognize OpenBSD reloc type
  19. 17 Jan, 2003 1 commit
  20. 16 Jan, 2003 3 commits
  21. 14 Jan, 2003 1 commit
  22. 10 Jan, 2003 2 commits
  23. 09 Jan, 2003 1 commit
  24. 08 Jan, 2003 2 commits
  25. 06 Jan, 2003 1 commit