1. 10 Jan, 2007 1 commit
  2. 20 Oct, 2006 1 commit
    • Mike Hibler's avatar
      Wow, this should make me look important! · afa5e919
      Mike Hibler authored
      Two-day boondoggle to support "/scratch", an optional large, shared filesystem
      for users.  To do this, I needed to find all the instances where /proj is used
      and behave accordingly.  The boondoggle part was the decision to gather up all
      the hardwired instances of shared directory names ("/proj", "/users", etc.)
      so that they are set in a common place (via unexposed configure variables).
      This is a boondoggle because:
      
      1. I didn't change the client-side scripts.  They need a different mechanism
         (e.g., tmcd) to get the info, configure is the wrong way.
      
      2. Even if I had done #1 it is likely--no, certain--that something would
         fail if you tried to rename "/proj" to be "/mike".  These names are just
         too ingrained.
      
      3. We may not even use "/scratch" as it turns out.
      
      Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
      To maintain my illusion in the future you should:
      
      1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
         et.al. functions where possible.  If not possible, make sure they run
         through configure and use @PROJROOT_DIR@, etc.
      
      2. Use the configure method for python, C, php and other languages.
      
      3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
         you should call to determine if an NS, template parameter, tarball or
         other file are in "an acceptable location."  Use these functions where
         possible.  They know about the optional "scratch" filesystem.  Note that
         the perl function is over-engineered to handles cases that don't occur
         in nature.
      afa5e919
  3. 25 Oct, 2005 1 commit
  4. 23 Aug, 2005 1 commit
  5. 31 May, 2005 1 commit
  6. 12 Feb, 2005 1 commit
    • Kirk Webb's avatar
      · f14bfb3e
      Kirk Webb authored
      If WINSUPPORT isn't set, don't try generating/updating the samba shares.
      f14bfb3e
  7. 11 Feb, 2005 1 commit
    • Kirk Webb's avatar
      · 1d2ddefb
      Kirk Webb authored
      Added dynamic samba share setup to exports_setup.
      
      Share names are constructed as follows:
      
      Path			Sharename
      ----------------------------------
      /proj/<pid>		proj-<pid>
      /group/<pid>/<eid>	<pid>-<eid>
      /users/<uid>		<uid>
      
      Share namespace collisions are detected, and mail is sent in the
      (unlikely) event that this occurs (and exports_setup bails out).
      1d2ddefb
  8. 27 Oct, 2004 1 commit
  9. 26 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Added config option DISABLE_EXPORTS_SETUP for sites without proper control · 56acaee5
      Mike Hibler authored
      of their file server (aero)
      
      Extend DISABLE_NAMED_SETUP: when set, we don't set a nodes hostname to
      <name>.<eid>.<pid>.<domain> since that won't resolve.  Just stick with
      pc<XXX>.<domain> in those cases.  The various sethostname* scripts are
      now .in so that they get preprocessed to check for the option.
      56acaee5
  10. 25 Feb, 2004 3 commits
  11. 05 Dec, 2003 2 commits
  12. 04 Dec, 2003 1 commit
  13. 26 Aug, 2003 1 commit
  14. 25 Jul, 2003 1 commit
  15. 28 Apr, 2003 2 commits
    • Leigh Stoller's avatar
      Change 2002 to 2003 on any file that was modified in 2003, but di not · 9e26c63a
      Leigh Stoller authored
      get an updated copyrights message.
      9e26c63a
    • Leigh Stoller's avatar
      Add support for new {user,group,project,experiment}_stats tables. · 5e5508bf
      Leigh Stoller authored
      The first three are aggregate tables, while the experiment stats table
      gets a record for each new experiment, and is updated when an
      experiment is swapped in/out/modify or terminated. Look at the table
      to see what is tracked. Once the experiment_stats record is updated,
      the aggregate tables are updated as necessary. There are a bunch of
      ugly changes to assign_wrapper to get the stats. Note that pnodes is
      not incremented until an experiment sucessfully swaps in. This is in
      leu of getting status codes; I'm not tracking failed operations yet,
      nor creating the log file that Jay wants. I'll do that in the next
      round of changes when we see how useful these numbers are.
      
      Most of the changes are to create/delete table entries where
      appropriate, and to display the records. Display is only under admin
      mode, and the display is raw; just a dump of the assoc tables in php.
      The last 100 experiment stats records are available via the Experiment
      List page, using the "Stats" show option at the top. Bad place, but
      will do for now.
      5e5508bf
  16. 29 Jan, 2003 1 commit
    • Mac Newbold's avatar
      Save a bunch of totally redundant db queries, and speed things up. · 55865301
      Mac Newbold authored
      Order the query for the nodes by pid/gid, then save the info from the last
      node. If the node I'm doing has the same pid/gid as the last one, reuse
      the info without grabbing it from the database again and recalculating
      everything. Now it will do one db query for each pid/gid that has an expt
      swapped in, instead of once per node in use. That cuts the number of db
      queries by a factor of (avg. # of nodes in use per proj), which is often
      greater than 10.
      55865301
  17. 03 Oct, 2002 1 commit
  18. 14 Aug, 2002 1 commit
  19. 13 Aug, 2002 1 commit
  20. 07 Jul, 2002 1 commit
  21. 05 Jun, 2002 1 commit
    • Leigh Stoller's avatar
      Changes to sshtb. Remove sshremote, and convert sshtb into a perl · 231fc2b1
      Leigh Stoller authored
      script that checks the database to see if local or remote. The problem
      with this is that the ssh syntax makes it hard to determine the host
      name by inspection. Would need to parse all the ssh args (bad idea),
      ot work backwards and try to figure out the difference between the
      command (which is not a string but a sequence of args) and the host
      and the preceeding ssh args. Hell with that! Changed sshtb to require
      a specific -host argument. Read the args and look for it. Error out of
      not found, to catch improper usage.
      
      The moral of this update: "sshtb [ssh args] -host <host> [more args ...]
      231fc2b1
  22. 05 May, 2002 1 commit
  23. 22 Apr, 2002 1 commit
  24. 05 Mar, 2002 1 commit
    • Leigh Stoller's avatar
      A wide ranging set of event system changes: · 0318cc22
      Leigh Stoller authored
      assign_wrapper.in: Hack in a change that ensures a delay node is
      created for any link on which an event is posted (up,down,modify),
      no matter what its initial parameters are. ie: If a link is created
      with no delay, but there is an event that adds a delay later, then we
      must drop in a delay node. Same for up/down on a link. We do this in
      the delay node. I am reasonably confident that this change is fine for
      duplex links, but I am less sure of the effect on lans!
      
      eventsys_control.in: Checkpoint latest changes. Add "replay" option,
      which right now just stops and starts the event scheduler so that it
      reloads the entire event list. Add check for existing experiment, and
      that the experiment is either active or swapping (do not want to start
      a scheduler for a swapped out experiment!). Add check to see if there
      are any events, and skip startup if there are not events in the DB.
      Lastly, get very serious about preventing more than one scheduler from
      being started, either by accident or intentionally. My protocol is to
      lock the table, grab and set the pid to -pid, test the pid for a
      positive value, and if positive, send the scheduler a kill(TERM) so
      that it can cleanup, clear the pid to zero in the DB, and exit. This
      approach ensures that we do not try to send a kill to a pid that is no
      longer active or owned by the user (this last part is not really
      necessary cause of how pids are reused, but it was easy to add so why
      not).
      
      exports_setup.in: Trivial change to make it easier to turn this on
      temporarily in devel trees.
      named_setup.in: Ditto.
      
      node_reboot.in: Add call to TBdbfork() in child cause of apparent DB
      connection problems across forks. In the child, set the eventstatus
      for the node to REBOOT if successful (not this event status stuff is
      temporary, will be recast in next set of revisions).
      
      GNUmakefile:  Add new controlling program, eventsys_control.
      power.in:     Ditto previous comment about REBOOT.
      os_setup.in:  Non event system cleanups.
      tbend.in:     Add DB cleanup of the new virt_trafgens and eventlist tables.
      tbprerun.in:  Ditto.
      tbreport.in:  Print out the event list in a pretty print format.
      tbswapin.in:  Add call to start the event system. Also a big fix; move
                    the named script up above the os_setup so that the named
                    tables have been updated by the time the first node
                    reboots. I noticed that nodes were failing on gethostbyname().
      tbswapout.in: Add call to stop the event system.
      0318cc22
  25. 12 Feb, 2002 1 commit
  26. 01 Feb, 2002 1 commit
    • Mike Hibler's avatar
      Yet another attempt to Get It Right based on our latest half-assed · 609c547e
      Mike Hibler authored
      understanding of how mountd operates.
      
      Things that should be fixed:
      
      1. It iterates over every node calculating what directories are exported,
         what FSes they are on, etc.  Most of that work only needs to happen
         per experiment.
      
      2. The algorithm to determine what FS a directory is on is a hack.  I just
         take the first component of the path provided.  As long as we mount all
         our FSes at the top level and configure with canonical pathes, this is
         ok.  Other solutions require calling out to the ops node to get actual
         mount info.
      
      3. Once shared experiments are revived, the code to determine exported
         directories will need to change.  The algorithm for computing the
         exports lines should still be correct.
      609c547e
  27. 18 Jan, 2002 4 commits
  28. 10 Jan, 2002 1 commit
  29. 08 Jan, 2002 1 commit
  30. 27 Dec, 2001 1 commit
    • Leigh Stoller's avatar
      Another set of group changes. As discussed in email and meetings, · 8404af03
      Leigh Stoller authored
      group directories are now created in a different tree than the
      the project directory so that they can be exported independently of
      the project tree to the nodes in a group experiment. The tree is
      routed at /groups on boss/users and on nodes.
      
      1. mkgroup,rmgroup,mkproj - Minor changes to reflect new group
         directory location (/groups). We leave a symlink in the old spot to
         maintain compatability, and to reduce the number of different
         directories that a person needs to worry about. So, when a group is
         made, you get a real directory /groups/pid/gid, and a symlink
         /proj/pid/groups/gid that points to the former.
      
      2. tmcd/tmcd.c - Minor change to add the additional group directory mount
         in the mounts command. Only done when pid!=gid for the experiment.
      
      3. tmcd/libsetup.pm and friends - Minor changes to fix the fact that
         mkdir does not create subdirs along the way unless the -p option is
         specified. Needed to create the local directory for the mounts
         returned by tmcd for group dirs. Pushed them out to the sup trees,
         although 6.2 images older than the most recent one are not going to
         work right. No one is using those images though, and we should just
         flush the sup trees.
      
      4. exports_setup.in - Ah, the crux of the issue. I really dislike NFS
         at this point. The original idea was to export a third set of
         directories to nodes that were part of a group experiment. Those
         nodes would get /groups/pid/gid exported, and /proj/pid read-only.
         Well, no such luck. On users, /groups and /proj are both really on
         /q, and the old restriction of mountd not allowing an IP to
         specified more than once on the right hand side for any FS, reared
         its ugly head again. As far as mountd is concerned, /q/groups and
         /q/proj are the same thing, and so it bombed when I tried to export
         them on different lines, since that meant an IP was repeated twice.
         So, I reworked exports_setup, and now for any node that is part of
         a group experiment, it gets this:
      
      	/q/proj/pid /q/groups/pid/gid -maproot=root 155.101.132.26
      
         which at least allows the individual group dirs to be protected
         from each other, but does not allow /proj/pid to be exported read
         only. Sigh.
      8404af03
  31. 06 Nov, 2001 1 commit
  32. 16 Oct, 2001 2 commits