Skip to content
  • Mike Hibler's avatar
    Working download path: inner-node <-> inner-boss <-> sub-boss <-> real-boss · f1016e75
    Mike Hibler authored
    Who says Mike can't do N-level hierarchy for N > 2!
    
    The different levels are configured like:
    
    1. Running the master server on real-boss.  Just use the Emulab config (-C),
       which will authenticate by caller IP using info from the DB:
          mfrisbeed -C emulab
    
    2. Running the master server on sub-boss.  Use the default (null) config
       (authentication allows GETs only from within the default image cache
       directory), tell it where to cache images (-I), name real boss as the
       parent server (-S), pass calling client's context (IP) to parent for
       authentication (-A) (since the subboss itself has no experiment context
       and must serve all experiments), run in "mirror mode" (-M) where we are
       strictly a read-only cache for images from our parent, and allow
       redirection of sub-boss clients to the real-boss when sub-boss is itself
       downloading the image for the first time (-R):
          mfrisbeed -S boss.emulab.net -I /z/image_cache -A -M -R
    
    3. Running the master server on an inner-boss.  Use the Emulab config (-C),
       use sub-boss as our parent server (-S):
          mfrisbeed -C emulab -S pc404.emulab.net
    
    An inner elab node would make a request from the inner-boss (-S) to download
    an image with the given ID imageid (-F), trying again every 30 seconds (-B)
    if the server reports "try again" (i.e., it is downloading the image from
    its parent):
       frisbee -S boss -B 30 -F emulab-ops/FEDORA10-STD /dev/da0
    
    Note the differences between the sub-boss config (2) and the inner-boss
    config (3); both are proxying, but in different ways:
    
     * Authentication: inner-boss can use its own identity when downloading
       images on behalf of its nodes since the inner-boss is in the same
       experiment as those nodes and has the same privileges as seen by the
       outer elab. Sub-boss is in a more privileged position where its clients
       may be in mulitple experiments, hence it must pass on the client's
       authentication info (IP) to its parent.
    
     * Caching: sub-boss is just a read-only copy of the images on real
       boss; the state of these images should always match what is on boss.
       Hence it always checks in with boss to make sure that an image, even
       if cached locally, is up to date. Inner-boss is more of a COW image
       repository, since inner nodes can create new images and modify existing
       ones independent of the real Emulab; images are only downloaded if they
       don't exist locally and do exist in the outer elab.
    
     * Redirection: one optimization Ryan did in the current sub-boss setup
       was to allow clients of the sub-boss frisbee to be redirected to the
       real boss when the sub-boss itself is downloading an image for the
       first time. Hence, while sub-boss is downloading an image, its clients
       will download at the same time from the same multicast stream.  Once
       sub-boss has the image, clients will just download from it as usual.
       Conceptual this could work for inner-boss, but since we do NAT for all
       inner nodes using inner-boss as the gateway, it would be a convoluted
       setup if it could even be made to work (e.g., do we even pass MC traffic?
       can we do it without rewriting the port used by the client?)
    f1016e75