• Mike Hibler's avatar
    Frisbee general: · 9e55b0b1
    Mike Hibler authored
    1. Implement PREQUEST message which passes a bit map of desired blocks.
       We still use the REQUEST message (start block + number of blocks) for
       full chunk requests as that is more efficient.  This message also
       includes a flag indicating whether it is a retry of a request we
       originally made or not.  This gives the server more accurate loss info.
    
    2. More stats and tracing goo.
    
    
    Frisbee client:
    
    1. Add 'C' and 'W' command line options to specify amount of memory
       for chunk buffers (network buffering) and for write buffers (disk
       buffering).  The Emulab frisbee startup script uses these to partition
       up all the available memory on a machine.  Previously we were just
       using a fixed ~128MB even though our machines have 256 or 512MB of
       memory.  Also add the 'M' option which specifies the overall memory,
       internally dividing it up between chunk buffers and write buffers.
    
    2. Add 'S' command line option to explicitly specify the server.  This
       allows us to make a feeb...um, "lightweight" authentication check
       on incoming messages.
    
    3. Use the common BlockMap data struct to track which pieces of a chunk
       we have received.  This is easily inverted to make PREQUESTS and it is
       also smaller than the older byte-per-block technique.
    
    4. Allow partial request-ahead.  Previously, we only issued request-ahead
       if there were enough empty chunk buffers for a maximum (2) request-ahead.
    
    Frisbee server:
    
    1. Use BlockMap for workQ elements.  An easy way to allow a complete merge
       of incoming requests with existing ones.
    
    2. Check for overlap of incoming requests with the request currently
       being serviced.  This happens surprisingly often.
    
    3. Dubious: burst gap becomes burst interval.  The latter takes into
       account the time required to read data, etc., in other words, we now
       have variable-sized gaps and put out bursts at specific times rather
       than having fixed gaps and putting out bursts at variable times.
       This gives us more accurate pacing over shorter time periods.  I
       thought this might be important for dynamic pacing.
    
    4. Add 'W' command line option to specify a target bandwidth.  Frisbeed
       will use this to calculate a burst size/interval.
    
    5. Rewrote the dynamic pacing code.  It is now easily as bad as before
       if not worse.  But it does have fewer magic constants!  Needs to be
       redone by someone who understands the TCP-friendly rate equation.
    
    Imagezip:
    
    1. add 'R' option to specify one or more partitions for which to force
       raw (naive) compression even if the FS format is understood.  Useful
       for benchmarking.
    
    2. add 'D' option to allow "dangerous" writes.  In this mode, we don't
       do the fsync's or retries of failed writes.  Overrides the hack we put
       in for NFS.  Use this if writing to a local filesystem (or /dev/null).
    
    3. Eliminate an extra copy of every chunk header.
    
    Imageunzip:
    
    1. Eliminate extra copy of decompressed data that we were doing between
       the single decompression buffer and the disk buffers.  Helps on slow
       machines (like gatech's 300Mhz machines with 66MHz memory bus).
    
    2. Allow dynamic number of variable-sized write buffers.  Total memory
       not to exceed the writebufmem limit.  Previously we had a small number
       of fixed-size (256K) buffers.
    
    3. Add debugging 'C' option to just compute a single CRC of the decompressed
       image.  Back-ported to older imageunzip and used to make sure my write
       buffer changes were correct.  Maybe handy for similar massive changes
       in the future.
    9e55b0b1
Name
Last commit
Last update
account Loading commit data...
apache Loading commit data...
assign Loading commit data...
autoconf Loading commit data...
capture Loading commit data...
cdrom Loading commit data...
db Loading commit data...
dhcpd Loading commit data...
discvr Loading commit data...
doc Loading commit data...
event Loading commit data...
hw_config Loading commit data...
install Loading commit data...
ipod Loading commit data...
lib Loading commit data...
os Loading commit data...
patches Loading commit data...
pxe Loading commit data...
rc.d Loading commit data...
rpms Loading commit data...
security Loading commit data...
sensors Loading commit data...
sql Loading commit data...
ssl Loading commit data...
sysadmin Loading commit data...
tbsetup Loading commit data...
testsuite Loading commit data...
tip Loading commit data...
tmcd Loading commit data...
tools Loading commit data...
utils Loading commit data...
vis Loading commit data...
www Loading commit data...
xmlrpc Loading commit data...
BUGS Loading commit data...
GNUmakefile.in Loading commit data...
GNUmakerules Loading commit data...
Makeconf.in Loading commit data...
PROJECTS Loading commit data...
README Loading commit data...
RESTRICTED-RIGHTS Loading commit data...
TODO Loading commit data...
config.h.in Loading commit data...
configure Loading commit data...
configure.in Loading commit data...
defs-barb-emulab Loading commit data...
defs-barb-mini Loading commit data...
defs-calfeld-emulab Loading commit data...
defs-default Loading commit data...
defs-example Loading commit data...
defs-kwebb-emulab Loading commit data...
defs-mini Loading commit data...
defs-newbold-emulab Loading commit data...
defs-newbold-macdb Loading commit data...
defs-newbold-mini Loading commit data...
defs-rchriste-emulab Loading commit data...
defs-ricci-emulab Loading commit data...
defs-ricci-mini Loading commit data...
defs-shash-emulab Loading commit data...
defs-shash-mini Loading commit data...
defs-stoller-emulab Loading commit data...
defs-stoller-home Loading commit data...
defs-stoller-mini Loading commit data...
defs-uky Loading commit data...
defs-wide Loading commit data...