1. 28 Sep, 2011 3 commits
    • Mike Hibler's avatar
      Add client '-f' option for using O_DIRECT open mode for the output device. · 46a62612
      Mike Hibler authored
      On Linux, device IO goes through the buffer cache by default. This makes
      frisbee run really fast...until it closes the output device. Then it sits
      for minutes while it flushes disk data out of the cache. This is
      technically okay, but wasteful, since frisbee allocates its own memory
      for caching disk write data. By using direct IO on the output device,
      writes do not go through the cache.
      
      Aren't two caches better than one? No. They can compete for memory and it
      just causes an extra data copy. Frisbee is faster when using O_DIRECT.
      
      Note that this change is complicated somewhat because Linux requires that
      the IO buffer for an O_DIRECT opened file be sector aligned. So we play
      some games to do this.
      
      This should have no effect on FreeBSD where device writes don't go through
      the buffer cache.
      46a62612
    • Mike Hibler's avatar
      Avoid a pre-mature close of the output fd. · 179c189f
      Mike Hibler authored
      Wait til all other threads have joined.
      179c189f
    • Mike Hibler's avatar
      Lint and fixes to TRACE code. · c5d15a6b
      Mike Hibler authored
      c5d15a6b
  2. 22 Sep, 2011 1 commit
  3. 12 Sep, 2011 1 commit
  4. 30 Aug, 2011 2 commits
  5. 26 Aug, 2011 3 commits
  6. 25 Aug, 2011 3 commits
  7. 23 Aug, 2011 1 commit
  8. 16 Aug, 2011 2 commits
  9. 27 Jul, 2011 1 commit
  10. 19 Jul, 2011 2 commits
  11. 13 Jul, 2011 1 commit
  12. 06 Jul, 2011 1 commit
    • Mike Hibler's avatar
      Add -f option for reporting hashes. · bfaa986c
      Mike Hibler authored
      This forces imagehash to hash pieces on aligned sector boundaries rather
      than just doing it from the start of each chunk. Result now is that there
      may be a partial hash piece at the beginning of each range and another
      partial piece at the end. Ideally, we would avoid those partial pieces,
      but we are only using this as a point of comparison right now.
      bfaa986c
  13. 01 Jul, 2011 1 commit
  14. 30 Jun, 2011 1 commit
    • Mike Hibler's avatar
      Make sure upload/download server run with proper group list. · bf97b597
      Mike Hibler authored
      Attempts to load an image from a subgroup of a project didn't work because
      I was only setting the primary group ID based on the node's experiment's
      project. Now we use setgroups() to establish all groups that the swapper
      of the experiment belongs to.
      bf97b597
  15. 29 Jun, 2011 1 commit
  16. 22 Jun, 2011 2 commits
  17. 15 Jun, 2011 1 commit
    • Ryan Jackson's avatar
      Change rindex() to strrchr() · 4c75d2cf
      Ryan Jackson authored
      Default builds of uClibc (used for the Linux MFS) don't include
      support for index() and rindex() (gone from the POSIX standard as of
      POSIX.1-2008).  I've fixed the uClibc config to include them now, but
      I need to build frisbee for an existing build of the Linux MFS with
      the old uClibc.
      4c75d2cf
  18. 02 Jun, 2011 1 commit
  19. 01 Jun, 2011 1 commit
  20. 31 May, 2011 2 commits
    • Mike Hibler's avatar
      Added compat option and assorted cleanups. · d8de4d6a
      Mike Hibler authored
      Added WITH_V3COMPAT to make sure that we can still generated V3 images
      (for other sites) if checksum/encryption is not used.
      
      Try to clean up the command line options some. Be more consistent by putting
      generated uuid into a file instead of just spitting it out on stderr. Make
      sure that if the decryption specifies signing and/or encryption that we
      require the image to have that info. Add some more assertions. (Re)distinguish
      checksums from signed-checksums.
      d8de4d6a
    • Mike Hibler's avatar
      3bc2d561
  21. 27 May, 2011 3 commits
  22. 18 May, 2011 3 commits
    • Mike Hibler's avatar
    • Mike Hibler's avatar
      Lint. · b0b994a8
      Mike Hibler authored
      b0b994a8
    • Mike Hibler's avatar
      Support image PUT (aka, "upload") and assorted minor changes. · 77dbad39
      Mike Hibler authored
      1. Support for PUT.
      
      The big change is support for uploading via the master server, based heavily
      on the prototype that Grant did. Currently only host-based (IP-based)
      authentication is done as is the case with download. Grant's SSL-based
      authentication code is "integrated" but has not even been compiled in.
      
      The PUT protocol allows for assorted gewgaws, like specifying a maximum size,
      setting a timeout value, returning size and signature info, etc.
      
      There is a new, awkwardly-named client utility "frisupload" which, like the
      download client, takes an "image ID" as an argument and requests to upload
      (PUT) that image via the master server. As with download, the image ID can
      be either of the form "<pid>/<emulab-image-name>", to upload/update an actual
      Emulab image or it can start with a "/" in which case it is considered to
      be a pathname on the server.
      
      On the server side, the master server takes PUT requests, verifies permission
      to upload the image, fires up a separate instance of an upload daemon (with
      the even catchier moniker "frisuploadd"), and returns the unicast addr/port
      info to the client which then begins the upload. The master server also acts
      as a traffic cop to make sure that downloads and uploads (or uploads and
      uploads) don't overlap.
      
      This has been integrated into the Emulab "create image" process in a
      backward-compatible way (i.e., so old admin MFSes will continue to work).
      Boy, was that fun. One not-so-desirable effect of this integration is that
      images now traverse our network twice, once to upload from node to boss and
      once for boss to write out the image file across NFS to ops. This is not
      really something that should be "fixed" in frisbee, it is only "undesirable"
      because we have a crappy NFS server.
      
      What has NOT been done includes: support of hierarchical PUT operations
      (we don't need it for either the elabinelab or subboss case), support for
      uploading standard images stored on boss (we really want something better
      than host-based authentication here), and the aforementioned support of
      SSL-based authentication.
      
      2. Other tidbits that got mixed in with PUT support:
      
      Added two new site variables:
          images/frisbee/maxrate_std
          images/frisbee/maxrate_usr
      which replace the hardwired (in mfrisbeed and frisbeelauncher before that)
      bandwidth limits for image download. mfrisbeed reads these (and the
      images/create/* variables) when it starts up or receives a HUP signal.
      These could be read from the DB on every GET/PUT, but they really don't change
      much and I needed something to test the reread-the-config-on-a-HUP code!
      
      Fixed avoidance of "problematic multicast addresses" so it would actually
      work as intended.
      
      Lots of internal "refactoring" to make up for things I did wrong the first
      time and to give the general impression that "Wow, Mike did a LOT!"
      77dbad39
  23. 04 May, 2011 2 commits
  24. 07 Apr, 2011 1 commit