1. 16 Dec, 2013 1 commit
    • Mike Hibler's avatar
      Changes to support non-racy NFS exports. · 4ddc74c2
      Mike Hibler authored
      As of FreeBSD 9, the race condition in mountd has been addressed.
      We automatically (based on the OS running) add the option to mountd (-S)
      as part of ops-install. There is also a configuration variable, NFSRACY,
      that we auto-detect and set in elabinelab setup. For regular installs,
      you need to set this manually in the defs-* file. Note that the setting
      in the defs file does not affect very much. It is passed to the client-side
      which uses it in one copy function (used as a fall back for getting the
      topology file).
      
      There are still a couple of places in the Emulab code where handling
      the race condition is hardwired into the code (most notably, imagezip).
      These remain.
      4ddc74c2
  2. 06 Jun, 2013 2 commits
  3. 10 Dec, 2012 1 commit
  4. 19 Jun, 2012 1 commit
    • Mike Hibler's avatar
      Make frisbee more directly IGMP (v2) aware. · 66e07584
      Mike Hibler authored
      Add "-Q <interval>" option to the master server to allow it to act as an
      IGMP V2 querier in environment where there is otherwise not one. It does
      essentially what the perl-based querier (code.google.com/p/perl-igmp-querier/)
      does, sending out a v2 membership query at the specified interval.
      
      This eliminates the need to run mrouted in some environments (e.g., elabinelab)
      just to issue IGMP queries. As a result, all the boss-install and elabinelab
      setup related to using mrouted to perform this function has been removed.
      The elabinelab CONFIG_MROUTED option has been changed to CONFIG_QUERIER
      (the former is still recognized and mapped to the latter). The undocumented
      defs-* variable NEEDMROUTED has been changed to NEEDMCQUERIER (the former
      still exists in install/installvars.pm.in but is always set to 0) to more
      accurately reflect the variable's purpose. If NEEDMCQUERIER is set, then
      the mfrisbeed startup script is modified to add the "-Q 30" option.
      
      The implementation of the client and server "-K <interval>" keep-alive option
      has been changed to directly send IGMP v2 membership reports containing the
      associated MC address.
      
      Note that the -K options have always been a hack to work-around assorted
      IGMP-related misconfigurations and incompatibilities, and really should
      only be used as a last resort. As implemented, they could cause the host
      machine to be pruned out of other MC groups at the nearest switch since
      they only report membership in the frisbee MC group. With the master server
      acting as an IGMP querier, instances of the frisbee server on that host
      should no longer need to do keep alives. We still have one case where it
      is needed on the client-side: a FreeBSD 8.x or later host connected to an
      IGMPv2-only switch. It appears that the IGMPv3 implementation added in
      FreeBSD 8.x always sends v3 reports, even when the default is configured
      (via sysctl or even recompiling the kernel) as v2.
      66e07584
  5. 30 Apr, 2012 1 commit
    • Mike Hibler's avatar
      More minor cleanups exposed by FBSD9 install. · ffb52cd0
      Mike Hibler authored
      Bogus param list for AddPerExperimentSwitchStack; can't have scalar args
      after a list arg.
      
      Missing mailing list TBERRORSEMAIL in defs-elabinelab. Caused redundant
      "testbed-ops" addresses in /etc/aliases.
      
      Weed out said redundant mailing list names when creating aliases during
      ops-install.
      
      named.reload script has gone away. Use rndc instead.
      ffb52cd0
  6. 11 Apr, 2012 1 commit
  7. 27 Mar, 2012 1 commit
  8. 29 Feb, 2012 1 commit
  9. 25 Feb, 2012 1 commit
  10. 10 Aug, 2011 1 commit
  11. 17 May, 2011 1 commit
    • Mike Hibler's avatar
      Add machinery to automatically setup mrouted on the control network. · 3ba294d9
      Mike Hibler authored
      This is specifically intended for elabinelab right now where configurations
      with an "inner control network" need a multicast router for inner frisbee
      to work. There is (an undocumented) NEEDMROUTED defs variable that controls
      whether the machinery is triggered or not. It is not documented because, though
      in theory this would work for a real boss setup, it has not been tested and
      each install would probably need a custom mrouted.conf file.
      
      For an inner elab, this variable is automatically set in defs-elabinelab
      if there is an inner control network. The result is that the mrouted port
      will be installed, it will be enabled in rc.conf, and a stub mrouted.conf
      file is created (to force mrouted to ignore the real control network).
      3ba294d9
  12. 28 Mar, 2011 1 commit
    • Leigh Stoller's avatar
      Begin the transition away from the ancient Mysql.pm module to the more · 5030b44d
      Leigh Stoller authored
      current and maintained DBI::mysql module. A couple of things make this
      a little more work then you might think.
      
      Mysql exports a slightly different API then DBI, both at the DB *and*
      the statement level. The former required some restructuring of
      emdbi.pm, partly cause we want external sites to continue using Mysql
      for a while longer. So, emdbi suppports both interfaces, via the
      configure variable TBUSEDBI.
      
      I also took the opportunity to also scrap the existing fork()
      detection code and redo it in an easier to understand manner.
      Actually, I had no idea what the previous code was trying to do, so it
      was easier to just get rid of it, rather then try to make it work for
      the DBI API.
      
      There are also API differences in the "statement" class, but
      fortunately this can be hidden by wrapping the statement class with a
      wrapper that adds the routines we need to avoid making silly changes
      to 1000s of queries. They are all trivial little things since mostly
      its a matter of naming (numrows --> rows).
      
      I also changed the library we use on ops (db/libtbdb.pm.in) to use
      DBI, but in this case I just switched it over. Seemed like overkill to
      worry about supporting both APIs on ops. If it works it works, and so
      far it does. 
      
      Lastly, the following modules still use Mysql directly. They all need
      to be changed, but none of these are on the critical path to swapin
      and swapout, so they can change later.
      
      db/dumperrorlog.proxy.in
      db/showgraph.in
      db/sitevarscheck.in
      bgmon/find-asymmetric
      db/pelab_opspush.proxy.in
      slothd/sdisrunning.in
      utils/export_tables.in
      utils/setbuildinfo.in
      pelab/bgmon/libpelabdb.pm
      pelab/dbmonitor/libtbdb.pm
      5030b44d
  13. 20 Oct, 2010 1 commit
    • Mike Hibler's avatar
      Support for no shared filesystem (unsupport for shared filesystem?) and · c1c1bce2
      Mike Hibler authored
      (eventual) support for NFS servers without race conditions!
      
      This means no NFS between nodes and ops/fs. There are still NFS mounts of
      ops on boss however.
      
      Added new defs-* variable NOSHAREDFS, which when set non-zero will disable
      the export of NFS filesystems to nodes.  Involved lots of little changes:
      
       * /users, /proj, and /share filesystems are not exported to nodes.
      
       * Returned mount info now includes an FSTYPE key which will be set to "LOCAL"
         if NOSHAREDFS is in effect (by default it is set to "NFS-RACY"; more on
         this later).  In the case where it is set to LOCAL, the other mount lines
         no longer contain REMOTE=foo settings.  Because of this change,
         THE TMCD VERSION NUMBER HAS BEEN BUMPED TO 32.
      
       * The client rc.mounts script will now create local versions of /users/*,
         /proj/<pid>, and /share when FSTYPE=LOCAL.  It first runs mkextrafs to
         create a large partition for these, since someday we will likely want
         to pre-populate these with a non-trivial amount of data.  Right now,
         the only thing that is put in the user's homedir is the standard dotfiles
         for the OS and the Emulab authorized_keys file (so you can login).
      
       * Linktest had to be modified to fetch the various results files (via
         loghole) rather than just assuming they were in /proj.  And also changed
         to invoke tevc with the local copy of the event key so it won't try to
         read it over NFS.
      
       * create_image was modified to ssh to the node and run the imagezip
         command, capturing the output of ssh.  This is controlled via the "-s"
         option which defaults to on for a NOSHAREDFS system, but can also be
         used on a normal system.
      
       * elabinelab's can be configured with/without a shared FS via the
         CONFIG_SHAREDFS attribute (note polarity change) which defaults to 1.
      
      Another new defs-* variable, NFSRACY, will some day allow you to specify
      (by setting to 0) that your NFS server does NOT have the nefarious mountd
      race condition when changing /etc/exports.  Currently, this defaults to 1
      since all versions of FreeBSD supported as an "fs" node have this "feature."
      Rumor has it that FreeBSD 8 does not have this problem nor, presumably,
      would a Linux NFS server.
      
      The only use of this variable right now is to set the FSTYPE returned by the
      tmcd "mounts" call, which in turn is used by one client script, rc.topomap
      (via a libsetup function) to determine whether it should try copying
      the topo file multiple times.
      
      Random: add python2.6 to list of python's checked for in configure.
      Random: resync defs-example-privatecnet with defs-example.
      Random: did a little code-pissin here and there.
      c1c1bce2
  14. 19 Oct, 2010 1 commit
    • Mike Hibler's avatar
      Elabinelab fixes. · f32313ce
      Mike Hibler authored
      Need to enable/disable the SelfLoader based on the version of perl installed.
      Other minor nits.
      f32313ce
  15. 09 Feb, 2010 2 commits
  16. 23 Apr, 2007 1 commit
  17. 14 Dec, 2006 1 commit
  18. 20 Oct, 2006 1 commit
    • Mike Hibler's avatar
      Wow, this should make me look important! · afa5e919
      Mike Hibler authored
      Two-day boondoggle to support "/scratch", an optional large, shared filesystem
      for users.  To do this, I needed to find all the instances where /proj is used
      and behave accordingly.  The boondoggle part was the decision to gather up all
      the hardwired instances of shared directory names ("/proj", "/users", etc.)
      so that they are set in a common place (via unexposed configure variables).
      This is a boondoggle because:
      
      1. I didn't change the client-side scripts.  They need a different mechanism
         (e.g., tmcd) to get the info, configure is the wrong way.
      
      2. Even if I had done #1 it is likely--no, certain--that something would
         fail if you tried to rename "/proj" to be "/mike".  These names are just
         too ingrained.
      
      3. We may not even use "/scratch" as it turns out.
      
      Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
      To maintain my illusion in the future you should:
      
      1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
         et.al. functions where possible.  If not possible, make sure they run
         through configure and use @PROJROOT_DIR@, etc.
      
      2. Use the configure method for python, C, php and other languages.
      
      3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
         you should call to determine if an NS, template parameter, tarball or
         other file are in "an acceptable location."  Use these functions where
         possible.  They know about the optional "scratch" filesystem.  Note that
         the perl function is over-engineered to handles cases that don't occur
         in nature.
      afa5e919
  19. 05 Jul, 2006 1 commit
  20. 09 Jun, 2006 1 commit
  21. 05 Jun, 2006 1 commit
  22. 01 Jun, 2006 1 commit
  23. 06 Mar, 2006 1 commit
  24. 26 Feb, 2006 1 commit
  25. 24 Feb, 2006 1 commit
    • Leigh Stoller's avatar
      Two main changes: · be69c6e4
      Leigh Stoller authored
      * Add mailman to the install process. This includes the package
        install, all the setup needed in ops-install, and the little patches
        we apply locally.
      
        Note: Only under 6.0: no point in going backwards since its a pain
        in the ass to build these packages for the 4.10 image.
      
      * Push more of the package install back down into the ops/boss install
        scripts. This improves testing within those scripts, and avoids
        duplication. There are some changes in rc.mkelab that will temp
        mount the package dir on /packages for the benefit of the scripts.
      be69c6e4
  26. 12 Dec, 2005 1 commit
  27. 14 Sep, 2005 1 commit
    • Mike Hibler's avatar
      Changes related to allowing seperate 'fs' (file server) node. · c53d5827
      Mike Hibler authored
      Entailed new instructions for manual setup as well as integration into
      elabinelab framework.  First, the manual path:
      
      setup.txt, setup-boss.txt, setup-ops.txt and new setup-fs.txt:
          Updated to reflect potential for separate fs node.  The org here
          is a little dicey and could be confusing with ops+fs vs. ops and fs.
          Has not been field tested yet.
      
      */GNUmakefile.in: new fs-install target.
      
      configure, configure.in, defs-*:
          Somewhat unrelated, make min uid/gid to use be a defs setting.
          Also add config of fs-install.in script.
      
      boss-install.in, ops-install.in and new fs-install.in:
          Handle distinct fs node.  If you have one, fs-install is run before
          ops-install.  All scripts rely on the defs file settings of FSNODE
          and USERNODE to determine if the fs node is seperate.
      
      utils/checkquota.in:
          Just return "ok" if quotas are not used (i.e., if defs file FS_WITH_QUOTA
          string is null.
      
      install/ports/emulab-fs:
          Meta port for fs node specific stuff.  Also a patch for the samba port
          Makefile so it doesn't drag in CUPs, etc.  Note that the current samba
          port Makefile has this change, I am just backporting to our version.
      
      Elabinelab specific changes:
      
      elabinelab-withfs.ns:
          NS fragment used in conjunction with
      	tb-elab-in-elab-topology "withfs"
          to setup inner-elab with fs node.
      
      elabinelab.ns:
          The hard work on the boss side.  Recognize seperate-fs config and handle
          running of rc.mkelab on that node.  fs setup happens before ops setup.
      
      rc.mkelab:
          The hard work on the client side.  Recognize FsNode setup as well as
          differentiate ops+fs from ops setup.
      
      Related stuff either not part of the repo or checked in previously:
          emulab-fs package
      c53d5827
  28. 05 May, 2005 1 commit
  29. 21 Dec, 2004 1 commit
  30. 15 Nov, 2004 1 commit
  31. 12 Nov, 2004 1 commit
  32. 10 Nov, 2004 1 commit
  33. 01 Nov, 2004 1 commit
  34. 29 Oct, 2004 1 commit
  35. 07 Oct, 2004 1 commit
  36. 04 Oct, 2004 1 commit
    • Leigh Stoller's avatar
      Add: · 29ca84ab
      Leigh Stoller authored
      	SFSSUPPORT=0
      	DISABLE_NSE=1
      
      cause I do not want to deal with these yet (if ever) in inner elab.
      29ca84ab
  37. 30 Sep, 2004 1 commit
  38. 23 Sep, 2004 1 commit