1. 30 Oct, 2003 1 commit
  2. 27 Oct, 2003 3 commits
  3. 24 Oct, 2003 2 commits
    • Robert Ricci's avatar
      Commit the stuff necessary to copy out new plab rootballs, versions of · d12f9b61
      Robert Ricci authored
      which had been hanging around in my home directory for a while.
      
      There are a few new things in plab/etc/netbed_files that set up a
      directory of the same name in @prefix@. This will get rsync'ed with
      netbed_files/ on each planetlab node.
      log/  - just needs to exist for the httpd server
      sbin/ - contains thttpd, and scripts to manipulate it
      www/  - the directory served by thttpd. Contains symlinks to the 'real'
              location of the rootballs (etc/plab)
      
      I've committed a binary of thttpd - this is simply because it'd be a
      PITA to compile a Linux binary for every devel tree, etc.
      
      PLAB_ROOTBALL has now become a configure options. The idea is that we
      will keep the latest version number in configure.in, but you can
      override it in your defs
      file. This way, we don't have to update every defs file when there's a new
      version, but people can still play around with their own version if they want.
      
      The two scripts that interact with the plab nodes skip ones that are
      down. They ssh in as 'utah1', meaning that one of us who has access to
      that account needs to run them, so that they can have access to our
      keys. We can put boss's public key (or something) out there to remove
      this requirement.
      
      plabdist runs an rsync between @prefix@/etc/plab/netbed_files and a
      file of the same name on the planetlab nodes. It's intended to be run
      from the main install tree - the local rsync directory is not normally
      set up in devel trees. It runs in parallel, but is limited to 4 to
      avoid beating up boss too much. Takes about 1:40 with the current set
      of plab nodes (took > 10 minutes doing one at a time).
      
      plabhttpd (re)starts the mini web server on all plab nodes
      d12f9b61
    • Mike Hibler's avatar
      84bf73a5
  4. 23 Oct, 2003 1 commit
  5. 21 Oct, 2003 1 commit
  6. 20 Oct, 2003 1 commit
  7. 17 Oct, 2003 1 commit
  8. 16 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Brave new world of tmcc client side caching. The goal is to reduce the · 2b72f2c9
      Leigh B. Stoller authored
      number of connections to tmcd, and the resulting number of DB queries.
      Currently thats about 24 per node when it boots. Each vnode adds
      another 24 or so. The new approach is to use the "fullconfig" command,
      which dumps the entire config in one shot, saving about 20 of those
      connections. We still need to do the status/state commands for real of
      course. When a node boots, it requests the fullconfig; the client side
      takes this fullconfig, and dumps the individual sections to
      /var/emulab/boot/tmcc/section_name. Subsequent requests first look for
      it locally in the above named files, falling back to real tmcc if none
      exists. The update command also refreshes the cache.
      
      Tested for jails and plab node vservers as well.
      2b72f2c9
  9. 15 Oct, 2003 5 commits
  10. 14 Oct, 2003 1 commit
  11. 09 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Reorg of two aspects of node update. · 2641af4d
      Leigh B. Stoller authored
      * install-rpm, install-tarfile, spewrpmtar.php3, spewrpmtar.in: Pumped
        up even more! The db file we store in /var/db now records both the
        timestamp (of the file, or if remote the install time) and the MD5
        of the file that was installed. Locally, we can get this info when
        accessing the file via NFS (copymode on or off). Remote, we use wget
        to get the file, and so pass the timestamp along in the URL request,
        and let spewrpmtar.in determine if the file has changed. If the
        timestamp it gets is >= to the timestamp of the file, an error code
        of 304 (Not Modifed) is returned. Otherwise the file is returned.
      
        If the timestamps are different (remote, server sends back an actual
        file), the MD5 of the file is compared against the value stored. If
        they are equal, update the timestamp in the db file to avoid
        repeated MD5s (or server downloads) in the future. If the MD5 is
        different, then reinstall the tarball or rpm, and update the db file
        with the new timestamp and MD5. Presto, we have auto update capability!
      
        Caveat: I pass along the old MD5 in the URL, but it is currently
        ignored. I do not know if doing the MD5 on the server is a good
        idea, but obviously it is easy to add later. At the moment it
        happens on the node, which means wasted bandwidth when the timestamp
        has changed, but the file has not (probably not something that will
        happen in typical usage).
      
        Caveat: The timestamp used on remote nodes is the time the tarfile
        is installed (GM time of course). We could arrange to return the
        timestamp of the local file back to the node, but that would mean
        complicating the protocol (or using an http header) and I was not in
        the mood for that. In typical usage, I do not think that people will
        be changing tarfiles and rpms so rapidly that this will make a
        difference, but if it does, we can change it.
      
      * node_update.in, client side watchdog, and various web pages:
        Deflated node_update, removing all of the older ssh code. We now
        assume that all nodes will auto update on a periodic basis, via the
        watchdog that runs on all client nodes, including plab nodes.
      
        Changed the permission check to look for new UPDATE permission (used
        to be UPDATEACCOUNT). As before, it requires local_root or better.
        The reason for this is that node_update now implies more than just
        updating the accounts/mounts. The web pages have been changed to
        explain that in addition to mounts/accounts, rpms and tarfiles will
        also be updated. At the moment, this is still tied to a single
        variable (update_accounts) in the nodes table, but as Kirk requested
        at the meeting, it will probably be nice to split these out in the
        future.
      
        Added the ability to node_update a single node in an experiment (in
        addition to all nodes option on the showexp page). This has been
        added to the shownode webpage menu options.
      
        Changed locking code to use the newer wrapper states, and to move
        the experiment to RUNNING_LOCKED until the update completes. This is
        to prevent mayhem in the rest of the system (which could be dealt
        with, but is not worth the trouble; people have to wait until their
        initiated update is complete, before they can swap out the
        experiment).
      
        Added "short" mode to shownode routine, equiv to the recently added
        short mode for showexp. I use this on the confirmation page for
        updating a single node, giving the user a couple of pertinent (feel
        good) facts before they comfirm.
      2641af4d
  12. 07 Oct, 2003 2 commits
  13. 06 Oct, 2003 2 commits
    • Leigh B. Stoller's avatar
      Quickie change to experiment with client side caching; Added a new · 9fb9ef8f
      Leigh B. Stoller authored
      "fullconfig" command to cycle through the list of existing commands
      and spit out a section for each. Not all of them of course; just a
      subset that makes sense. I did not that mounts are a bit of a problem
      cause of the USESFS argument. Not sure what to do yet.
      9fb9ef8f
    • Leigh B. Stoller's avatar
      * New libtmcc.pm module that encapsulates the tmcc interface. Most of the · 434a472a
      Leigh B. Stoller authored
        code that was in libsetup has moved into this library, and underwent a
        giant cleaning and pumping up. The interface from your typical perl
        script now looks like this:
      
        use libtmcc;
      
        if (tmcc(TMCCCMD_STATUS, "optional arguments", \@tmccresults) < 0) {
            warn("*** WARNING: Could not get status from server!\n");
            return -1;
        }
        foreach my $me (@tmccresults) {
      	print "bite $me";
        }
      
        The arguments and results are optional values. There is a fourth optional
        value that is a hash of config options (basically converted to command
        line switches passed to tmcc). For example, to set the timeout on an
        individual call, pass a fourth argument like:
      
      	("timeout" => 5)
      
        There is also a way to set global options so that all subsequent tmcc
        calls are affected:
      
      	configtmcc("timeout", 5);
      
        I'll probably clean this up a bit to avoid the direct strings.
      
        The result list is a list of strings. Since we are trending away from
        using tmcc to transfer large amounts of data, I think this is okay.
      
      * A new tmcc.pl which does little more than load libtmcc and use it.
        This will become the new tmcc, with the existing C version becoming a
        backend binary for it.
      
      * All of the perl scripts in tmcd have been changed to use the new
        library. I left the few uses of tmcc in shell scripts alone since they
        were of the simple variety (mostly "state" command).
      
      * And again, if you have read this far, you will learn why I bothered with
        all this. Well, the existing code was really bad and it was getting out
        of control. Sort of like a squid that was getting harder to control as
        its rotting tenticles slithered into more and more scripts. Anyway ...
      
        More important, my goal is to use the libtmcc library to add caching.  I
        have not worked out the details yet, but I am envisioning a configuration
        file, perhaps generated initially by tmcd, of all of the config
        values. If the library finds that file, it sucks the info out of the file
        instead of going to tmcd. Eventually, this config file would be generated
        as part of experiment swapping and stored in the DB, but thats a longer
        term project, and perhaps orthogonal (how we fill the cache is not as
        important as adding the ability to use a cache, right?).
      
        Note that certain operations (like "state" and "ready") are flagged by
        the library to always bypass the "cache".
      434a472a
  14. 02 Oct, 2003 2 commits
    • Mike Hibler's avatar
      Add yet another route to jails: an interface route for the real control net. · 9fd95b1a
      Mike Hibler authored
      Previously, if you accessed a phys node by its real IP address (155.101.132.N)
      from inside a jail, it used the default route going to the router and back.
      Now it will put it out directly on the control net.
      
      The main reason for this is so that minibed vnodes can reach their boss and
      ops nodes.  Vnodes in mini don't have a default route that works, so we needed
      a way to get to boss/ops.
      
      Note that gated will remove this route when it starts up.  Thus, vnodes will
      revert to using the default route to get to the real control net.  On mainbed,
      this will still work (hop through the router as before).  On mini, it won't
      work at all.
      
      Moral: don't use session routing and vnodes on mini.
      9fd95b1a
    • Leigh B. Stoller's avatar
      Change ixp config to get the host side (gateway) of the pci/eth link · 82e2ec73
      Leigh B. Stoller authored
      from the interfaces table of the parent node. This coincides with
      Rob's changes to stash the IP there, and arrange for it to go into
      DNS.
      82e2ec73
  15. 30 Sep, 2003 2 commits
  16. 25 Sep, 2003 2 commits
  17. 24 Sep, 2003 4 commits
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      Convert install-rpm/install-tarfile to use the web server instead of · 0a259517
      Leigh B. Stoller authored
      tmcd (which is bad, since tying up the tmcd threads blocks all nodes
      in the testbed). The old functionality is left in tmcd for now.
      
      On the server side, a new web page (www/spewrpmtar.php3) receives a
      request for a file, along with the nodeid (pcXXX) making the request,
      and the secret key that is generated for each new experiment and
      transfered to the node via tmcd. If the key matches, the operation is
      handed off to tbsetup/spewrpmtar.in which verifies that the file is in
      the list of rpm/tar files for that node, and then spits it out to
      stdout. The web page uses fpassthru() to send the file out to the
      client. The client is using wget, and is required to use https (the
      web page checks).
      
      At present, the external script is run as the creator of the
      experiment, and gid of the experiment. Perhaps this is not a good
      idea. In any event, the file must be in the list of rpm/tarfiles,
      either owned by the experiment creator or with a group of the
      experiment, and the file must reside in either /proj or /groups.
      I use the realpath() function to make sure there are no symlink tricks
      pointing to outside those filesystems. I use the standard NFS read goo to
      prevent transient mount problems that we all know and love.
      0a259517
    • Mike Hibler's avatar
      install of start_if.em0 · 00038164
      Mike Hibler authored
      00038164
    • Mike Hibler's avatar
      another startup file for aero · 6b5af092
      Mike Hibler authored
      6b5af092
  18. 23 Sep, 2003 4 commits
  19. 22 Sep, 2003 1 commit
  20. 19 Sep, 2003 1 commit
  21. 18 Sep, 2003 2 commits
    • Kirk Webb's avatar
      Oops, forgot to commit our little sh script that gets copied over and run · 78710371
      Kirk Webb authored
      first on plab nodes to fix up sudo.
      78710371
    • Kirk Webb's avatar
      · abc9fa4b
      Kirk Webb authored
      * New GNUmakefile that produces the rootball - not linked in from
        the main TB build.
      
        - You can (almost) completely build the plab rootball just from the
          testbed source tree.  The one wrinkle is that you must copy the
          Linux tmcc binary into the tmcd/plab object dir since there is no
          great way to build it automatically, and storing it in CVS is not
          a good idea since its likely to change.
      
      * Removed the suid RPM, no longer needed - added suidperl binary
        since I couldn't think of a better place to put it.
      abc9fa4b