1. 25 May, 2011 2 commits
  2. 18 May, 2011 1 commit
    • Mike Hibler's avatar
      Support image PUT (aka, "upload") and assorted minor changes. · 77dbad39
      Mike Hibler authored
      1. Support for PUT.
      
      The big change is support for uploading via the master server, based heavily
      on the prototype that Grant did. Currently only host-based (IP-based)
      authentication is done as is the case with download. Grant's SSL-based
      authentication code is "integrated" but has not even been compiled in.
      
      The PUT protocol allows for assorted gewgaws, like specifying a maximum size,
      setting a timeout value, returning size and signature info, etc.
      
      There is a new, awkwardly-named client utility "frisupload" which, like the
      download client, takes an "image ID" as an argument and requests to upload
      (PUT) that image via the master server. As with download, the image ID can
      be either of the form "<pid>/<emulab-image-name>", to upload/update an actual
      Emulab image or it can start with a "/" in which case it is considered to
      be a pathname on the server.
      
      On the server side, the master server takes PUT requests, verifies permission
      to upload the image, fires up a separate instance of an upload daemon (with
      the even catchier moniker "frisuploadd"), and returns the unicast addr/port
      info to the client which then begins the upload. The master server also acts
      as a traffic cop to make sure that downloads and uploads (or uploads and
      uploads) don't overlap.
      
      This has been integrated into the Emulab "create image" process in a
      backward-compatible way (i.e., so old admin MFSes will continue to work).
      Boy, was that fun. One not-so-desirable effect of this integration is that
      images now traverse our network twice, once to upload from node to boss and
      once for boss to write out the image file across NFS to ops. This is not
      really something that should be "fixed" in frisbee, it is only "undesirable"
      because we have a crappy NFS server.
      
      What has NOT been done includes: support of hierarchical PUT operations
      (we don't need it for either the elabinelab or subboss case), support for
      uploading standard images stored on boss (we really want something better
      than host-based authentication here), and the aforementioned support of
      SSL-based authentication.
      
      2. Other tidbits that got mixed in with PUT support:
      
      Added two new site variables:
          images/frisbee/maxrate_std
          images/frisbee/maxrate_usr
      which replace the hardwired (in mfrisbeed and frisbeelauncher before that)
      bandwidth limits for image download. mfrisbeed reads these (and the
      images/create/* variables) when it starts up or receives a HUP signal.
      These could be read from the DB on every GET/PUT, but they really don't change
      much and I needed something to test the reread-the-config-on-a-HUP code!
      
      Fixed avoidance of "problematic multicast addresses" so it would actually
      work as intended.
      
      Lots of internal "refactoring" to make up for things I did wrong the first
      time and to give the general impression that "Wow, Mike did a LOT!"
      77dbad39
  3. 13 May, 2011 1 commit
  4. 06 May, 2011 1 commit
  5. 05 May, 2011 1 commit
  6. 20 Apr, 2011 1 commit
    • Leigh B Stoller's avatar
      Changes our ssh key/account handling in RedeemTicket() and · 03c2107c
      Leigh B Stoller authored
      CreateSliver(), to handle multiple accounts.  This somewhat reflects
      the Geni AM API for keys, which allows the client to specify multiple
      users, each with a set of ssh keys.
      
      The keys argument to the CM now looks like the following (note that
      the old format is still accepted and will be for a while).
      
      [{'urn'   => 'urn:blabla'
        'login' => 'dopey',
        'keys'  => [ list of keys like before ]},
       {'login' => "leebee",
        'keys'  => [ list of keys ... ]}];
      
      Key Points:
      
      1. You can supply a urn or a login or both. Typically, it is going to
         be the result of getkeys() at the PG SA, and so it will include
         both.
      
      2. If a login is provided, use that. Otherwise use the id from the urn.
      
      3. No matter what, verify that the token is valid for Emulab an uid
         (standard 8 char unix login that is good on just about any unix
         variant), and transform it if not.
      
      4. For now, getkeys() at the SA will continue to return the old format
         (unless you supply version=2 argument) since we do not want to
         default to a keylist that most CMs will barf on.
      
      5. I have modified the AM code to transform the Geni AM version of the
         "users" argument into the above structure. Bottom line here, is
         that users of the AM interface will not actually need to do
         anything, although now multiple users are actually supported
         instead of ignored.
      
      Still to be done are the changes to the login services structure in
      the manifest. We have yet to settle on what these changes will look
      like, but since people generally supply valid login ids, you probably
      will not need this, since no transformation will take place.
      03c2107c
  7. 12 Apr, 2011 2 commits
  8. 11 Apr, 2011 5 commits
  9. 01 Apr, 2011 1 commit
  10. 31 Mar, 2011 1 commit
  11. 18 Mar, 2011 1 commit
  12. 10 Mar, 2011 1 commit
  13. 09 Mar, 2011 1 commit
  14. 14 Feb, 2011 1 commit
  15. 03 Feb, 2011 1 commit
  16. 11 Jan, 2011 1 commit
  17. 04 Jan, 2011 2 commits
  18. 29 Dec, 2010 1 commit
  19. 28 Dec, 2010 1 commit
  20. 15 Dec, 2010 2 commits
  21. 07 Dec, 2010 2 commits
  22. 17 Nov, 2010 2 commits
  23. 16 Nov, 2010 2 commits
    • Kevin Atkinson's avatar
      Add support for all node "tb-set-tarfiles". · a0d0c95e
      Kevin Atkinson authored
      "tb-set-tarfiles" is like "tb-set-node-tarfiles" except that it
      distributes the tarfile to all nodes rather than just one and that it
      uses frisbee to distribute the file.
      
      These changes involved 1) refactoring frisbee info from images table
      into a new table, frisbee_blobs, 2) a new experiment_blobs table, and
      3) a new tmcd command so the node knows how to get the files from the
      server.
      
      The changes where designed to be general purpose enough to eventually
      support:
        1) Distributing arbitrary files (not just tarfiles) to nodes
        2) Perform arbitrary actions on those files
        3) Use arbitrary methods to get the files
      
      As such the tmcd line is as follows:
        URL=* ACTION=*
      
      where URL is currently:
        frisbee.mcast://<ADDR>/<FILE>
      for example
        frisbee.mcast://234.16.184.192:18092/users/kevina/home-dir.tar.gz
      and when we get around to using a master Frisbee server it could be
        frisbee://*
      or it could be a file://, http://, etc.
      
      and ACTION is currently:
        unpack:<LOCATION>
      for example
        unpackt:/users
      with future syntax to be determined.
      a0d0c95e
    • Jonathon Duerig's avatar
      57935897
  24. 09 Nov, 2010 5 commits
  25. 25 Oct, 2010 1 commit