1. 24 Oct, 2003 2 commits
    • Robert Ricci's avatar
      Commit the stuff necessary to copy out new plab rootballs, versions of · d12f9b61
      Robert Ricci authored
      which had been hanging around in my home directory for a while.
      
      There are a few new things in plab/etc/netbed_files that set up a
      directory of the same name in @prefix@. This will get rsync'ed with
      netbed_files/ on each planetlab node.
      log/  - just needs to exist for the httpd server
      sbin/ - contains thttpd, and scripts to manipulate it
      www/  - the directory served by thttpd. Contains symlinks to the 'real'
              location of the rootballs (etc/plab)
      
      I've committed a binary of thttpd - this is simply because it'd be a
      PITA to compile a Linux binary for every devel tree, etc.
      
      PLAB_ROOTBALL has now become a configure options. The idea is that we
      will keep the latest version number in configure.in, but you can
      override it in your defs
      file. This way, we don't have to update every defs file when there's a new
      version, but people can still play around with their own version if they want.
      
      The two scripts that interact with the plab nodes skip ones that are
      down. They ssh in as 'utah1', meaning that one of us who has access to
      that account needs to run them, so that they can have access to our
      keys. We can put boss's public key (or something) out there to remove
      this requirement.
      
      plabdist runs an rsync between @prefix@/etc/plab/netbed_files and a
      file of the same name on the planetlab nodes. It's intended to be run
      from the main install tree - the local rsync directory is not normally
      set up in devel trees. It runs in parallel, but is limited to 4 to
      avoid beating up boss too much. Takes about 1:40 with the current set
      of plab nodes (took > 10 minutes doing one at a time).
      
      plabhttpd (re)starts the mini web server on all plab nodes
      d12f9b61
    • Mike Hibler's avatar
      f77e9fc4
  2. 23 Oct, 2003 2 commits
  3. 16 Oct, 2003 1 commit
  4. 15 Oct, 2003 1 commit
    • Mike Hibler's avatar
      Uniform syslog'ing. Change everything I could find to use a syslog facility · cc6d6fa7
      Mike Hibler authored
      as defined in the defs-* file (e.g. "TBLOGFACIL=local2").  The default is
      "local5" which is what we are setup to use so you shouldn't need to mess
      with your defs- file!
      
      perl scripts just get this value configured in when configure is run.
      C programs get the value in two ways.  For programs that are intimate with
      the testbed infrastructure, and include "config.h", they just get it from
      that file.  For programs that we sometimes use outside the Emulab build
      environment (e.g., frisbee, capture) and that don't include config.h,
      the value is set via a "-DLOG_TESTBED=..." in the GNUmakefile build line.
      If the value isn't set, it defaults to what it used to be (usually LOG_USER).
      
      Still to do: healthd, hmcd (whose build doesn't seem to be completely
      integrated) and plabdaemon.in (since its icky python :-)
      cc6d6fa7
  5. 06 Oct, 2003 1 commit
  6. 30 Sep, 2003 1 commit
  7. 24 Sep, 2003 2 commits
    • Leigh Stoller's avatar
      Commit my daemon to monitor the status of plab physnodes in hwdown, · 59c5d5bb
      Leigh Stoller authored
      trying to bring them back from the dead periodically by trying to
      instantiate a vserver/vnode on them, and then tearing it down. If we
      can do that, then the node is usable, and it gets moved back into the
      normal holding experiment so that ptopgen will add it to ptop files.
      
      This deamon is not turned on yet; waiting for other little bits and
      pieces to be done.
      
      There is an equiv change in os_setup that moves physnodes into hwdown
      when a setup on a vnode fails.
      
      Lbs
      59c5d5bb
    • Leigh Stoller's avatar
      Convert install-rpm/install-tarfile to use the web server instead of · 0a259517
      Leigh Stoller authored
      tmcd (which is bad, since tying up the tmcd threads blocks all nodes
      in the testbed). The old functionality is left in tmcd for now.
      
      On the server side, a new web page (www/spewrpmtar.php3) receives a
      request for a file, along with the nodeid (pcXXX) making the request,
      and the secret key that is generated for each new experiment and
      transfered to the node via tmcd. If the key matches, the operation is
      handed off to tbsetup/spewrpmtar.in which verifies that the file is in
      the list of rpm/tar files for that node, and then spits it out to
      stdout. The web page uses fpassthru() to send the file out to the
      client. The client is using wget, and is required to use https (the
      web page checks).
      
      At present, the external script is run as the creator of the
      experiment, and gid of the experiment. Perhaps this is not a good
      idea. In any event, the file must be in the list of rpm/tarfiles,
      either owned by the experiment creator or with a group of the
      experiment, and the file must reside in either /proj or /groups.
      I use the realpath() function to make sure there are no symlink tricks
      pointing to outside those filesystems. I use the standard NFS read goo to
      prevent transient mount problems that we all know and love.
      0a259517
  8. 23 Sep, 2003 1 commit
  9. 18 Sep, 2003 1 commit
  10. 12 Sep, 2003 1 commit
  11. 04 Sep, 2003 1 commit
  12. 03 Sep, 2003 1 commit
  13. 22 Aug, 2003 1 commit
  14. 05 Aug, 2003 1 commit
    • Leigh Stoller's avatar
      The rest of the sync server additions: · 212cc781
      Leigh Stoller authored
      * Parser: Added new tb command to set the name of the sync server:
      
      	tb-set-sync-server <node>
      
        This initializes the sync_server slot of the experiment entry to the
        *vname* of the node that should run the sync server for that
        experiment. In other words, the sync server is per-experiment, runs
        on a node in the experiment, and the user gets to chose which node
        it runs on.
      
      * tmcd and client side setup. Added new syncserver command which
        returns the name of the syncserver and whether the requesting node
        is the lucky one to run the daemon:
      
          SYNCSERVER SERVER='nodeG.syncserver.testbed.emulab.net' ISSERVER=1
      
        The name of the syncserver is written to /var/emulab/boot/syncserver
        on the nodes so that clients can easily figure out where the server
        is.
      
        Aside: The ready bits are now ignored (no DB accesses are made) for
        virtual nodes; they are forced to use the new sync server.
      
      * New os/syncd directory containing the daemon and the client. The
        daemon is pretty simple. It waits for TCP (and UDP, although that
        path is not complete yet) connections, and reads in a little
        structure that gives the name of the "barrier" to wait for, and an
        optional count of clients in the group (this would be used by the
        "master" who initializes barriers for clients). The socket is saved
        (no reply is made, so the client is blocked) until the count reaches
        zero. Then all clients are released by writting back to the
        sockets, and the sockets are closed. Obviously, the number of
        clients is limited by the numbed of FDs (open sockets), hence the
        need for a UDP variant, but that will take more work.
      
        The client has a simple command line interface:
      
          usage: emulab-sync [options]
          -n <name>         Optional barrier name; must be less than 64 bytes long
          -d                Turn on debugging
          -s server         Specify a sync server to connect to
          -p portnum        Specify a port number to connect to
          -i count          Initialize named barrier to count waiters
          -u                Use UDP instead of TCP
      
          The client figures out the server by looking for the file created
          above by libsetup (/var/emulab/boot/syncserver). If you do not
          specify a barrier "name", it uses an internal default. Yes, the
          server can handle multiple barriers (differently named of course)
          at once (non-overlapping clients obviously).
      
          Clients can wait before a barrier in "initialized." The count on
          the barrier just goes negative until someone initializes the
          barrier using the -i option, which increments the count by the
          count. Therefore, the master does not have to arrange to get there
          "first." As an example, consider a master and one client:
      
      	nodeA> /usr/local/etc/emulab/emulab-sync -n mybarrier
      	nodeB> /usr/local/etc/emulab/emulab-sync -n mybarrier -i 1
      
          Node A waits until Node B initializes the barrier (gives it a
          count).  The count is the number of *waiters*, not including the
          master. The master is also blocked until all of the waiters have
          checked in.
      
          I have not made an provision for timeouts or crashed clients. Lets
          see how it goes.
      212cc781
  15. 28 Jul, 2003 1 commit
  16. 21 Jul, 2003 1 commit
  17. 16 Jul, 2003 1 commit
  18. 30 Jun, 2003 1 commit
    • Leigh Stoller's avatar
      Make the new parser live on mini. New parser ssh'es over to ops to · 2202fc5a
      Leigh Stoller authored
      do the actual parse. The parser now spits out XML instead of DB
      queries, and the wrapper on boss converts that to DB insertions after
      verification. There are some makefile changes as well to install the
      new parser on ops via NFS, since otherwise the parser could
      intolerably out of date on ops!
      2202fc5a
  19. 20 Jun, 2003 1 commit
  20. 16 Jun, 2003 1 commit
  21. 30 May, 2003 1 commit
  22. 13 May, 2003 1 commit
  23. 22 Apr, 2003 1 commit
  24. 21 Apr, 2003 1 commit
    • Robert Ricci's avatar
      New script: switchmac . Lists all MACs that have been learned by all · 5f8fea31
      Robert Ricci authored
      the experimental switches. The idea is to be able to auto-detect
      where a node has been plugged in, so that we fill out the wires table
      without any manual intervention! This is a step towards being able
      to automate the adding of nodes.
      
      Has a runtime linear in the number of VLANs on the experimental
      switches, so it should run pretty fast on a new testbed, but can
      be kinda slow on, say, ours.
      5f8fea31
  25. 18 Apr, 2003 1 commit
  26. 17 Apr, 2003 1 commit
    • Chad Barb's avatar
      · 95c08b45
      Chad Barb authored
      Added "check sitevars" to build, so we can be sure
      other sites have all of the requisite site variables
      when they install.
      
      Some of it is new; some of it is modified versions
      of Rob's schemacheck stuff.
      95c08b45
  27. 16 Apr, 2003 1 commit
    • Leigh Stoller's avatar
      Add support for idleswapping an experiment as the creator of the · ff5a57de
      Leigh Stoller authored
      experiment, rather than as an administrator, which presents group
      permission problems when the experiment is in a subgroup (requires two
      additional group, whereas suexec adds only one group). That aside, the
      correct approach is to run the swap as the creator. To do that, must
      flip to the user (from the admin person) in the backend using the new
      idleswap script, and then run the normal swapexp. Add new option to
      swapexp (-i) which changes the email slightly to make it clear that
      the experiment was idleswapped, and so that the From: is tbops not the
      user (again, to make it more clear).
      ff5a57de
  28. 15 Apr, 2003 1 commit
  29. 11 Apr, 2003 1 commit
    • Chad Barb's avatar
      · a0248828
      Chad Barb authored
      Added site variables ('sitevars').
      These are stored in the sitevariables database table.
      Each one has a name, a description (NOT OPTIONAL!), a
      default value, and a current value.
      If the current value is NULL, the default value is used.
      
      Soon, a mechanism will be added to the install process to
      ensure all needed site variables exist before installing;
      more on that when it is committed.
      
       - Added 'editsitevars.php3' page, accessable to admins
         via the 'Edit Site Variables' menu option.
      
       - Added 'setsitevar' script,
         an interface for listing, viewing in detail, and setting
         site variables.
      
       - Web interface now uses 'web/nologins' and 'web/message'
         instead of one-off database tables.
      
      NOTE that setting a variable to the default value and
      setting a variable to a value which is string-identical
      are NOT the same thing.
      (This doesn't matter yet, but when we push default values out to
       remote sites as part of our install, it will.)
      a0248828
  30. 10 Apr, 2003 2 commits
  31. 09 Apr, 2003 1 commit
    • Mac Newbold's avatar
      Add new script to notify users when they are over their quota. Runs from · 38310b4e
      Mac Newbold authored
      cron daily, on the fs node (ops) as root. Uses lots of configure variables
      so that it does the right thing in any installation, without any
      customization.
      
      One possible caveat regarding quotas: If FS_NODE != USERNODE, they don't
      have a login on the fs machine. So checking their quota won't work unless
      the rpc.rquotad(8) daemon is running on FS_NODE, which we currently don't
      do.
      
      In order to do this right, I had to add a new configure var,
      FS_WITH_QUOTAS, that has a space separated list of file systems that have
      quotas enabled. (Ie the default is 'FS_WITH_QUOTAS="/q /users"'.) It
      doesn't have any default, since I couldn't come up with a reasonable one.
      All the defs files have been updated appropriately to define this new
      variable.
      38310b4e
  32. 08 Apr, 2003 1 commit
    • Mac Newbold's avatar
      Remove idlecheck. Add idletimes and idlemail (and webidlemail). · fd4eaa8d
      Mac Newbold authored
      (webidlecheck was removed in a separate commit.)
      
      idletimes is the new commandline tool for checking idleness. It shows a
      table like this:
      
      PID          EID          Nodes  HrsIdle     Last Activity     Act. type
      ------------------------------------------------------------------------
      AVQ          Tunnel           4   15.18   2003-04-07 21:18:20  tty
      pces         toy              1    3.92   2003-04-08 08:34:02  tty
      Spinglass    fawaz            1   13.76   2003-04-07 22:43:48  tty
      TempleSCTP   FTP-REAL         1   26.40   2003-04-07 10:05:18  tty,net
      testbed      bvclass          4   24.58   2003-04-07 11:54:13  net
      testbed      ltest            1   14.37   2003-04-07 22:07:02  tty
      testbed      ron-image        1  501.88   2003-03-18 13:36:28  cpu,ext,tty,net
      testbed      Tone             1  501.88   2003-03-18 13:36:30  cpu,ext,tty,net
      
      By default it only shows things that are over an idleness threshold. It
      also has a mode to show all expts. Also can show idleness/activity by node
      instead of by expt, which has a slightly different table. (s/Nodes/NodeID/)
      
      idlemail isn't finished yet, but I don't want it to hold up idletimes and
      the other changes any longer. It doesn't get used by anything yet, of
      course.
      fd4eaa8d
  33. 04 Apr, 2003 1 commit
    • Chad Barb's avatar
      · 1f12fbc9
      Chad Barb authored
      ... forgot to run autoconf.
      1f12fbc9
  34. 03 Apr, 2003 1 commit
  35. 02 Apr, 2003 1 commit
    • Mac Newbold's avatar
      Massive reworking of our structure for defs files. · 09eb5852
      Mac Newbold authored
      Before:
      The main defs file (ie for configure) had an entry for WWWDEFS that
      pointed to a <@WWWDEFS@>-defs.php3 file in the www/ directory. The www
      defs file loaded some values about web pages, URLs, and some web
      configuration parameters.
      
      Problem:
      Anything that was only in the www defs file was not accessible in the rest
      of the universe (ie perl, C, and any other non-web-page scripts). For
      instance, you couldn't have a perl script send an email to a user with a
      link to the web site.
      
      Solution:
      Nuke all the www-defs files, move any important values into the main
      configure, and change the web defs infrastructure to respect that. This
      also meant adding about 3 lines each to all of the configure defs files.
      (There really are about 10 new values you can change in your defs file,
      but in almost all cases, the default values are the right thing.)
      
      Upgrading:
      External sites will need to move a few variables from their www-defs file
      into their configure defs file. The example file should make it pretty
      obvious. They may also want to customize some of the other vars that are
      mentioned in configure.in and www/defs.php3.in .
      09eb5852
  36. 20 Mar, 2003 1 commit