• Mike Hibler's avatar
    Frisbee general: · 9e55b0b1
    Mike Hibler authored
    1. Implement PREQUEST message which passes a bit map of desired blocks.
       We still use the REQUEST message (start block + number of blocks) for
       full chunk requests as that is more efficient.  This message also
       includes a flag indicating whether it is a retry of a request we
       originally made or not.  This gives the server more accurate loss info.
    2. More stats and tracing goo.
    Frisbee client:
    1. Add 'C' and 'W' command line options to specify amount of memory
       for chunk buffers (network buffering) and for write buffers (disk
       buffering).  The Emulab frisbee startup script uses these to partition
       up all the available memory on a machine.  Previously we were just
       using a fixed ~128MB even though our machines have 256 or 512MB of
       memory.  Also add the 'M' option which specifies the overall memory,
       internally dividing it up between chunk buffers and write buffers.
    2. Add 'S' command line option to explicitly specify the server.  This
       allows us to make a feeb...um, "lightweight" authentication check
       on incoming messages.
    3. Use the common BlockMap data struct to track which pieces of a chunk
       we have received.  This is easily inverted to make PREQUESTS and it is
       also smaller than the older byte-per-block technique.
    4. Allow partial request-ahead.  Previously, we only issued request-ahead
       if there were enough empty chunk buffers for a maximum (2) request-ahead.
    Frisbee server:
    1. Use BlockMap for workQ elements.  An easy way to allow a complete merge
       of incoming requests with existing ones.
    2. Check for overlap of incoming requests with the request currently
       being serviced.  This happens surprisingly often.
    3. Dubious: burst gap becomes burst interval.  The latter takes into
       account the time required to read data, etc., in other words, we now
       have variable-sized gaps and put out bursts at specific times rather
       than having fixed gaps and putting out bursts at variable times.
       This gives us more accurate pacing over shorter time periods.  I
       thought this might be important for dynamic pacing.
    4. Add 'W' command line option to specify a target bandwidth.  Frisbeed
       will use this to calculate a burst size/interval.
    5. Rewrote the dynamic pacing code.  It is now easily as bad as before
       if not worse.  But it does have fewer magic constants!  Needs to be
       redone by someone who understands the TCP-friendly rate equation.
    1. add 'R' option to specify one or more partitions for which to force
       raw (naive) compression even if the FS format is understood.  Useful
       for benchmarking.
    2. add 'D' option to allow "dangerous" writes.  In this mode, we don't
       do the fsync's or retries of failed writes.  Overrides the hack we put
       in for NFS.  Use this if writing to a local filesystem (or /dev/null).
    3. Eliminate an extra copy of every chunk header.
    1. Eliminate extra copy of decompressed data that we were doing between
       the single decompression buffer and the disk buffers.  Helps on slow
       machines (like gatech's 300Mhz machines with 66MHz memory bus).
    2. Allow dynamic number of variable-sized write buffers.  Total memory
       not to exceed the writebufmem limit.  Previously we had a small number
       of fixed-size (256K) buffers.
    3. Add debugging 'C' option to just compute a single CRC of the decompressed
       image.  Back-ported to older imageunzip and used to make sure my write
       buffer changes were correct.  Maybe handy for similar massive changes
       in the future.
event.c 8.33 KB