1. 25 Sep, 2014 1 commit
  2. 15 May, 2014 1 commit
  3. 17 Mar, 2014 1 commit
  4. 24 Jan, 2014 1 commit
  5. 11 Dec, 2013 1 commit
  6. 17 Sep, 2013 2 commits
  7. 18 Mar, 2013 1 commit
  8. 10 Jan, 2013 1 commit
  9. 30 Oct, 2012 1 commit
    • Mike Hibler's avatar
      Remaining infrastructure for control network "ARP lockdown". · 4b5e17b0
      Mike Hibler authored
      It works like this. Certain nodes that are on the node control net
      (right now just subbosses, but ops coming soon) can set static ARP entries
      for the nodes they serve. This raises the bar for (but does not eliminate
      the possibility of) nodes spoofing servers. Currently this is only for
      FreeBSD.
      
      When such a server boots, it will early on run /etc/rc.d/arplock.sh
      which will in turn run /usr/local/etc/emulab/fixarpinfo. fixarpinfo
      asks boss via an SSL tmcc call for "arpinfo" (using SSL ensures that the
      info coming back is really from boss). Tmcd on boss returns such arpinfo
      as appropriate for the node (subboss, ops, fs, etc.) along with the type
      of lockdown being done. The script uses this info to update the ARP
      cache on the machine, adding, removing, or making permanent entries
      as appropriate.
      
      fixarpinfo is intended to be called not just at boot, but also whenever
      we might need to update the ARP info on a server. The only other use right
      now is in subboss_dhcpd_makeconf which is called whenever DHCP info may
      need to be changed on a subboss (we hook this because a call to this script
      might also indicate a change in the set of nodes served by the subboss).
      In the future, fixarpinfo might be called from the newnode path (for ops/fs,
      when a node is added to the testbed), the deletenode path, or maybe from
      the watchdog (if we started locking down arp entries on experiment nodes)
      
      The type of the lockdown is controlled by a sitevar on boss,
      general/arplockdown, which can be set to 'none', 'static' or 'staticonly'.
      'none' means do nothing, 'static' means just create static arp entries
      for the given nodes but continue to dynamically arp for others, and
      'staticonly' means use only this set of static arp entries and disable
      dynamic arp on the control net interface. The last implies that the server
      will only be able to talk to the set of nodes for which it got ARP info.
      
      As mentioned, tmcd is responsible for returning the correct set of arp
      info for a given request. The logic currently is:
      
       * Only return ARP info to nodes which are on the CONTROL_NETWORK.
         If the requester is elsewhere (e.g., Utah's boss and ops are currently
         segregated on different IP subnets) then this whole infrastructure
         does not apply and nothing is returned.
      
       * If the requester is a subboss, return info for all other servers that
         are on the node control network as well as for the set of nodes
         which the subboss serves.
      
       * If the requester is an ops or fs node, again return info for all
         other servers and info for all testnodes or virtnodes whose control
         net IP is on the node control net.
      
       * Otherwise, return nothing.
      
      One final note is that the ARP info for servers such as boss/ops/fs or
      the gateway router is not readily available in most Emulab instances
      since those machines are not in the DB nodes or interfaces tables.
      Eventually we will fix that, but for now the info must come from new
      site variables. To help initially populate those variables, I added
      the utils/update_sitevars script which attempts to determine which
      servers are on the node control net and gathers the appropriate IP and
      MAC info from them.
      4b5e17b0
  10. 11 Oct, 2012 1 commit
  11. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  12. 25 Jul, 2012 1 commit
  13. 28 Mar, 2011 1 commit
    • Leigh B Stoller's avatar
      Begin the transition away from the ancient Mysql.pm module to the more · 5030b44d
      Leigh B Stoller authored
      current and maintained DBI::mysql module. A couple of things make this
      a little more work then you might think.
      
      Mysql exports a slightly different API then DBI, both at the DB *and*
      the statement level. The former required some restructuring of
      emdbi.pm, partly cause we want external sites to continue using Mysql
      for a while longer. So, emdbi suppports both interfaces, via the
      configure variable TBUSEDBI.
      
      I also took the opportunity to also scrap the existing fork()
      detection code and redo it in an easier to understand manner.
      Actually, I had no idea what the previous code was trying to do, so it
      was easier to just get rid of it, rather then try to make it work for
      the DBI API.
      
      There are also API differences in the "statement" class, but
      fortunately this can be hidden by wrapping the statement class with a
      wrapper that adds the routines we need to avoid making silly changes
      to 1000s of queries. They are all trivial little things since mostly
      its a matter of naming (numrows --> rows).
      
      I also changed the library we use on ops (db/libtbdb.pm.in) to use
      DBI, but in this case I just switched it over. Seemed like overkill to
      worry about supporting both APIs on ops. If it works it works, and so
      far it does. 
      
      Lastly, the following modules still use Mysql directly. They all need
      to be changed, but none of these are on the critical path to swapin
      and swapout, so they can change later.
      
      db/dumperrorlog.proxy.in
      db/showgraph.in
      db/sitevarscheck.in
      bgmon/find-asymmetric
      db/pelab_opspush.proxy.in
      slothd/sdisrunning.in
      utils/export_tables.in
      utils/setbuildinfo.in
      pelab/bgmon/libpelabdb.pm
      pelab/dbmonitor/libtbdb.pm
      5030b44d
  14. 16 Mar, 2011 1 commit
  15. 25 Oct, 2010 1 commit
    • Leigh B Stoller's avatar
      New module, called Emulab Features. The basic usage (see tbswap) is: · 1d430992
      Leigh B Stoller authored
      use EmulabFeatures;
      
      if (EmulabFeatures->FeatureEnabled("NewMapper", $user, $group, $experiment)) {
         # Do something
      }
      else {
         # Do something else.
      }
      
      where $user, $group, and $experiment is the current Emulab user, group, and
      experiment the script is operating as. Any of them can be undef. Note that
      features can easily be globally enabled or disabled (bypassing user/group
      check). See below.
      
      There are two scripts to deal with features. The easy one is the script to
      grant (or revoke) feature usage to a particular user or group or experiment:
      
      boss> wap grantfeature -u stoller NewMapper
      boss> wap grantfeature -p geni NewMapper
      boss> wap grantfeature -e geni,myexp NewMapper
      
      Add -r to revoke the feature.
      
      The other script is for managing features. To create a new feature:
      
      boss> wap emulabfeature create NewFeature 'A pithy description'
      
      which adds the feature to the emulab_features DB table. Use "delete"
      to remove a feature from the DB.
      
      You can globally enable and disable features for all users/groups (the
      user/group checks are bypassed). Global disable overrides global
      enable. There are actually two different flags. Lots of rope, I mean
      flexibility.
      
      boss> wap emulabfeature enable NewFeature 1
      boss> wap emulabfeature enable NewFeature 0
      
      boss> wap emulabfeature disable NewFeature 1
      boss> wap emulabfeature disable NewFeature 0
      
      To display a list of all features and associated settings:
      
      boss> wap emulabfeature list
      
      To show the details (including the users and groups) of a specific
      feature:
      
      boss> wap emulabfeature show NewFeature
      
      Oh, if a test is made in the code for a feature, and that feature is
      not in the emulab_features table (as might be the case on other
      Emulab's), the feature is "disabled".
      1d430992
  16. 11 Oct, 2010 1 commit
    • Leigh B Stoller's avatar
      Work on an optimization to the perl code. Maybe you have noticed, but · 92f83e48
      Leigh B Stoller authored
      starting any one of our scripts can take a second or two. That time is
      spent including and compiling 10000s of thousands of lines of perl
      code, both from our libraries and from the perl libraries.
      
      Mostly this is just a maintenance thing; we just never thought about
      it much and we have a lot more code these days.
      
      So I have done two things.
      
      1) I have used SelfLoader() on some of our biggest perl modules.
         SelfLoader delays compilation until code is used. This is not as
         good as AutoLoader() though, and so I did it with just a few 
         modules (the biggest ones).
      
      2) Mostly I reorganized things:
      
        a) Split libdb into an EmulabConstants module and all the rest of
           the code, which is slowly getting phased out.
      
        b) Move little things around to avoid including libdb or Experiment
           (the biggest files).
      
        c) Change "use foo" in many places to a "require foo" in the
           function that actually uses that module. This was really a big
           win cause we have dozens of cases where we would include a
           module, but use it in only one place and typically not all.
      
      Most things are now starting up in 1/3 the time. I am hoping this will
      help to reduce the load spiking we see on boss, and also help with the
      upcoming Geni tutorial (which kill boss last time).
      92f83e48
  17. 23 Jun, 2010 1 commit
  18. 11 Jun, 2010 1 commit
  19. 09 Jun, 2010 1 commit
  20. 04 May, 2010 1 commit
  21. 15 Apr, 2010 1 commit
  22. 10 Nov, 2009 1 commit
  23. 22 Sep, 2009 1 commit
  24. 02 Mar, 2009 2 commits
    • Leigh B. Stoller's avatar
      A bunch of changes for a "standalone" clearinghouse. Presently this · 60f04310
      Leigh B. Stoller authored
      its really a hugely stripped down Emulab boss install, using a very
      short version of install/boss-install to get a few things into place.
      
      I refactored a few things in both the protogeni code and the Emulab
      code, and whacked a bunch of makefiles and configure stuff. The result
      is that we only need to install about 10-12 files from the Emulab
      code, plus the protogeni code. Quite manageable, if you don't mind
      that it requires FreeBSD 6.X ... Still, I think it satisfies the
      requirement that we have a packaged clearinghouse that can be run
      standalone from a running Emulab site.
      60f04310
    • Leigh B. Stoller's avatar
  25. 06 Feb, 2009 1 commit
    • Leigh B. Stoller's avatar
      Start at an attempt to bring some sanity to the virt topology tables. · 68b193a5
      Leigh B. Stoller authored
      So far this wraps the virtual tables in objects, and I have changed
      xmlconvert already, but that needs more testing. Eventually I want to
      provide some simple generators like addnode and addlink so that I can
      use form the Protogeni code to build up a virtual topology from an
      rspec. Not sure how that will go.
      68b193a5
  26. 23 Jan, 2009 1 commit
  27. 21 Jan, 2009 1 commit
  28. 12 Jan, 2009 1 commit
  29. 07 May, 2008 1 commit
  30. 24 Apr, 2008 2 commits
  31. 11 Feb, 2008 1 commit
    • Leigh B. Stoller's avatar
      Initial checkpoint of a script that will check the stats tables · 8ab3e170
      Leigh B. Stoller authored
      nightly, and make repairs for the common things I see happening.  The
      current version repairs most of the errors that have crept in since
      the Epoch, and then rebuilds the summary stats table (user_stats,
      group_stats, project_stats). My intent is to add a "daily_stats" table
      as well since scanning the testbed_stats and experiment_resources
      table to generate range data is getting pretty slow as more records
      enter the system. We can also use a daily_stats table to generate
      graphs on the fly as Jay requested a few weeks ago.
      
      Not done yet, hope to return to it later this week.
      8ab3e170
  32. 14 Jan, 2008 1 commit
  33. 06 Nov, 2007 1 commit
    • Leigh B. Stoller's avatar
      This started out as a simple change to turn the datastore into a CVS · c1cff09b
      Leigh B. Stoller authored
      sandbox, and that I did. It falls back to the older archive when
      the template is older then CVS repos.
      
      But along the way I got annoyed with the fact that template instantiation
      does not provide a logfile to the web interface. The reason is that
      the current logfile stuff is very experiment centric; there has to be an
      experiment and an attached logfile. An instance does not have an experiment
      until really late in the game so the code was just not bothering.
      
      Anyway, I've started to generalize the logfile stuff with a new table
      and the approach that a logfile is named by a random key, and if you
      know the key you can look at the logfile in the web (since without an
      experiment it is hard to do permission checks unless we make logfiles
      uid/gid owned, and I did not want to do that.
      c1cff09b
  34. 11 Jul, 2007 1 commit
  35. 30 May, 2007 1 commit
  36. 07 May, 2007 1 commit
    • Leigh B. Stoller's avatar
      Mostly this commit is the switch from SVN archives to ZIP archives. · 55d1bb6e
      Leigh B. Stoller authored
      Other stuff leaked in too ...
      
      I did separate out a lot of tbsetup/libArchive into db/Archive, and
      whats left in libArchive.pm will eventually move over into the
      Template library.
      
      Note that I have dropped archiving of plain experiments; this is not
      really owrth it outside the workbench context, and it just wastes
      space and makes a lot if stuff painful in the web interface.
      55d1bb6e
  37. 13 Mar, 2007 1 commit