1. 18 Jan, 2007 4 commits
  2. 25 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      Makefile Whacking! Try to deal with the problem caused by the delay · 7590f9c5
      Leigh Stoller authored
      between when something is installed and when post-install runs. Short
      of a global lock (which we probably need anyway someday), my solution
      is this. In your makefiles, add these variables before the line that
      has the include of $(TESTBED_SRCDIR)/GNUmakerules:
      
      	SETUID_BIN_SCRIPTS   =
      	SETUID_SBIN_SCRIPTS  =
      
      I have added three new rules to GNUmakerules that look like this:
      
      	$(addprefix $(SBINDIR)/, $(SETUID_SBIN_SCRIPTS)): $(SBINDIR)/%: %
      		echo "Installing (setuid) $<"
      		-mkdir -p $(INSTALL_SBINDIR)
      		$(SUDO) $(INSTALL) -o root -m 4755 $< $@
      
      Yep, your eyes ain't lying to you; use sudo to run the target so that
      install does the right thing (which is that the old file is not
      replaced until the new one has the proper attributes on it).
      
      Note that post-install is still needed for the initial install, but
      should no longer be needed for day to day installs since all that other
      stuff post-install does is mkdir/chmod on directories.
      7590f9c5
  3. 24 Oct, 2006 1 commit
  4. 18 Oct, 2006 1 commit
  5. 16 Oct, 2006 1 commit
  6. 08 Aug, 2006 1 commit
  7. 03 Aug, 2006 1 commit
    • Leigh Stoller's avatar
      Support for capturing the trace data that is stored in the pcal files · 4ce9c421
      Leigh Stoller authored
      into per-experiment databases on ops. Additional support for reconsituting
      those databases back into temporary databases on ops, for post processing.
      
      * This revision relies on the "snort" port (/usr/ports/security/snort)
        to read the pcap files and load them into a database. The schema is
        probably not ideal, but its better then nothing. See the file
        ops:/usr/local/share/examples/snort/create_mysql for the schema.
      
      * For simplicity, I have hooked into loghole, which already had all
        the code for downloading the trace data. I added some new methods to
        the XMLRPC server for loghole to use, to get the users DB password
        and the name of the per-experiment database. There is a new slot in
        the traces table that indicates that the trace should be snorted to
        its DB. In case you forgot, at the end of a run or when the instance
        is swapped out, loghole is run to download the trace data.
      
      * For reconsituting, there are lots of additions to opsdb_control and
        opsdb_control.proxy to create "temporary" databases and load them
        from a dump file that is stored in the archive. I've added a button
        to the Template Record page, inappropriately called "Analyze" since
        right now all it does is reconsitute the trace data into a DB on
        ops.
      
        Currently, the only indication of what has been done (the name of
        the DBs created on ops) is the log email that the user gets. A
        future project is tell the user this info in the web interface.
      
      * To turn on database capturing of trace data, do this in your NS
        file:
      
      	set link0 ...
      	$link0 trace
      	$link0 trace_snaplen 128
      	$link0 trace_db 1
      
         the increase in snaplen is optional, but a good idea if you want
         snort to undertand more then just ip headers.
      
      * Also some changes to the parser to allow plain experiments to take
        advantage of all this stuff. To simple get yourself a per-experiment
        DB, put this in your NS file:
      
      	tb-set-dpdb 1
      
        however, anytime you turn trace_db on for a link or lan, you
        automatically get a per-experiment DB.
      
      * To capture the trace data to the DB, you can run loghole by hand:
      
      	loghole sync -s
      
        the -s option turns on the "post-process" phase of loghole.
      4ce9c421
  8. 28 Jul, 2006 1 commit
    • Leigh Stoller's avatar
      Add a "Create Template from Instance" ability. Basically, you can · a651da71
      Leigh Stoller authored
      create a new template (well, really a modify) from the current
      swapped in experiment. This allows you to create a template, swap in
      an instance, modify the datastore in the instance (which is a copy
      of the datastore in the template), and then create a new template
      using the datastore and nsfile from the instance. This is a new menu
      item on the showexp page for the instance.
      
      Also in this commit are fixes and improvements to the new navagation
      bar that I recently added.
      a651da71
  9. 26 Jul, 2006 1 commit
  10. 11 Jul, 2006 1 commit
  11. 21 Jun, 2006 1 commit
  12. 30 May, 2006 1 commit
    • Leigh Stoller's avatar
      Add an export option to the record listing. A new button on the Template · 2cfe4630
      Leigh Stoller authored
      Record page lets you export the contents of the archive that corresponds
      to that record, along with an XML file that describes the various DB bits
      for the template and instance.
      
      This is just a first cut so that Mike can start playing around. Subject to
      change, I'm sure.
      
      The archive is dumped to /proj/$pid/exports/$guid/$vers/$exptidx, which
      is basically the last commit of the instance when it was terminated.
      
      The xml file is called export.xml and is placed in the top level directory
      of the above directory. The file is created with XML::Simple, and a typical
      XML file might look like:
      
      <instance>
        <bindings>
          <name>NodeCount</name>
          <description>Number of nodes!</description>
          <value>1</value>
        </bindings>
        <bindings>
          <name>OS</name>
          <description></description>
          <value>RHL90-STD</value>
        </bindings>
        <bindings>
          <name>ScriptArgs</name>
          <description></description>
          <value>-b</value>
        </bindings>
        <eid>NewOne-V2</eid>
        <guid>10149/2</guid>
        <metadata>
          <name>M1</name>
          <guid>10162/1</guid>
          <value>Some metadata</value>
        </metadata>
        <pid>testbed</pid>
        <runs>
          <name>1</name>
          <archive_tag>T20060526-082533-172_endexp</archive_tag>
          <description></description>
          <exptidx>110</exptidx>
          <idx>1</idx>
          <runid>NewOne-V2</runid>
          <start_time>2006-05-26 08:23:02</start_time>
          <stop_time>2006-05-26 08:25:16</stop_time>
        </runs>
        <uid>stoller</uid>
      </instance>
      2cfe4630
  13. 15 May, 2006 1 commit
    • Mike Hibler's avatar
      Initial "Inner Plab" support. In your NS file, you declare one node: · 9512772e
      Mike Hibler authored
      tb-set-node-plab-role $plc plc
      
      to make it the PLC node.  Then any number of other nodes are declared as:
      
      tb-set-node-plab-role $plab1 node
      
      to make them inner plab nodes.  Unlike elabinelab, there is no magic
      "tb-plab-in-elab" command which implies the topology, you put all the
      plab nodes in a LAN or whatever yourself.  This may or may not be a good idea.
      
      Anyway, these NS commands set DB state in virt_nodes and reserved much like
      elabinelab.  During swapin, the dhcpd.conf file is rewritten so that
      inner plab nodes have their "filename" set to "pxelinux.0" and their
      "next-server" set to the designated PLC node.  The PLC node will then be
      loaded/booted before anything is done to the inner-plab nodes.  After
      it comes up, the inner plab nodes are rebooted and declared as up.
      There is a new tmcd command "eplabconfig" (suggestions for a new name
      welcom!), which returns info like:
      
          NAME=plc ROLE=plc IP=155.98.36.3 MAC=00d0b713f57d
          NAME=plab1 ROLE=node IP=155.98.36.10 MAC=0002b3877a4f
          NAME=plab2 ROLE=node IP=155.98.36.34 MAC=00d0b7141057
      
      to just the PLC node (returns nothing to any other node).
      
      The implications of this setup are:
      
       * The PLC node must act as a TFTP server as we have discussed in the past.
         The TMCC info above is hopefully enough to configure pxelinux, if not
         we can change it.
      
       * The PLC node is responsible for loading the disks of inner plab nodes.
         This is implied by the setup, where we change the dhcpd.conf file before
         doing anything to the inner nodes.  Thus, once the inner nodes are
         rebooted, they will be talking pxelinux with PLC, and not to boss.
         This step is dubious, as we could no doubt load the disks faster than
         whatever plab uses can.  But it simplified the setup (and is more
         realistic!).  The alternative, which is something that might be useful
         anyway, is to introduce a "state" after which nodes have been reloaded
         but before they are rebooted.  With that, we can reload the plab nodes
         and then change the dhcpd.conf file so when they reboot they start
         talking to the PLC.
      9512772e
  14. 12 May, 2006 1 commit
    • Leigh Stoller's avatar
      Redo the entire template library. I've been meaning to use perl · 78503406
      Leigh Stoller authored
      "object" and this was a good opportunity to see if they are useful and
      easy enough to use. Yep they are; the code is much cleaner with many
      fewer utility functions to get at stuff. I recommend this approach
      from now on.
      
      The problem is the php side, which ends up duplicating some stuff, but
      in the old style. This is not so bad for the template code since I
      have made it a point not to do anything but display functions in php;
      all modifications are handled in the backend.
      78503406
  15. 05 May, 2006 1 commit
  16. 30 Mar, 2006 1 commit
  17. 28 Mar, 2006 1 commit
  18. 22 Feb, 2006 1 commit
  19. 07 Feb, 2006 1 commit
  20. 26 Jan, 2006 1 commit
    • Kevin Atkinson's avatar
      · 05015359
      Kevin Atkinson authored
      Merged in changes from tblog-2-branch:
      
                Move parts of libtblog into libtblog_simple.  Libtblog simple
                provided the basic logging functions but doesn't touch anything.
                Moreover including libtblog_simple doesn't automatically start
                the logging subsystem.  It also doesn't have testbed dependencies
                which mean 1) it can be used in the core testbed libraries (such
                as libdb, libtestbed) without introducing a circular dependency
                and 2) can be used independently.
      
                Reworked DBFatal and DBWarn to use tblog.  It will still email
                testbed-ops, however.
      
                Make use of the "cause" field to determine the cause of the bug.
                In particular tblog_find_error will look at the value of this
                field and report the "cause".  In the future different actions
                can be taken based on the ultimate "cause" of the bug, such as if
                testbed-ops should be notified.
      
                Change format of Error Message reported by libtblog.  As per the
                email "Format or Error Messages" ro testbed-dev.
      
                Have libtblog use its own Database handle to avoid problems with
                locked tables.
      
                Also set DBCONN_MAXTRIES to 3 for most important queries.  For
                queries that are not important don't send mail on error.
      05015359
  21. 23 Jan, 2006 1 commit
    • Timothy Stack's avatar
      · add602df
      Timothy Stack authored
      Parse the NS file with the real NS parser so we can make sure linktest is
      doing the "right" thing.
      
      	* configure, configure.in: Add tbsetup/nsverify files.
      
      	* tbsetup/GNUmakefile.in: Add nsverify subdir.
      
      	* tbsetup/tbprerun.in: Run verify-ns on the experiments NS file.
      
      	* tbsetup/ns2ir/nstb_compat.tcl: Bring up-to-date with the current
      	world.
      
      	* tbsetup/nsverify/GNUmakefile.in: Makefile.
      
      	* tbsetup/nsverify/ns-2.27.patch: Patch file for NS version 2.27.
      
      	* tbsetup/nsverify/nstbparse.in: Wrapper for the NS parser.
      
      	* tbsetup/nsverify/tb_compat.tcl: Different version of
      	tb_compat.tcl that is used to verify linktest parameters.
      
      	* tbsetup/nsverify/verify-ns.in: Script that runs on boss and
      	verifies that the testbed parser worked correctly.
      
      	* tbsetup/ns2ir/parse-ns.in, tbsetup/ns2ir/parse.proxy.in: Tweaked
      	a bit so parse.proxy can be used to run the regular NS parser in
      	addition to the testbed one.
      add602df
  22. 05 Jan, 2006 1 commit
  23. 02 Jan, 2006 1 commit
    • Timothy Stack's avatar
      · bd20dd17
      Timothy Stack authored
      First cut at a daemon that does regular checkups of the testbed
      hardware/software.
      
      	* configure, configure.in: Add tbsetup/checkup directory.
      
      	* db/audit.in: Add a listing of stuck checkups.
      
      	* install/boss-install.in: Add 'elabckup' user.
      
      	* rc.d/3.testbed.sh.in: Startup the checkup_daemon.
      
      	* sql/database-create.sql, sql/database-migrate.txt: Add the
      	checkups tables.
      
      	* tbsetup/GNUmakefile.in: Descend into the checkup directory.
      
      	* tbsetup/checkup: The checkup daemon, man page, and
      	  associated scripts.
      
      	* tbsetup/ptopgen.in: Add a feature with a value of 0.9 to
      	  prereserved nodes to keep them from being allocated unless
      	  they're really wanted.
      
      	* utils/firstuser.in: Add some other options so the script can be
      	  used to create other pseudo users.
      bd20dd17
  24. 19 Dec, 2005 1 commit
    • Kevin Atkinson's avatar
      · 45f997fd
      Kevin Atkinson authored
      Updates to to Error Logging API Code.
      
      You should start seeing much better error messages coming from my
      system.  Errors coming from parse.proxy and assign (the two most
      frequent sources of errors) should now be concise and to the point.
      Errors coming from libosload/libreboot (the next most frequent source
      of errors) should now also be much better, but not perfect.  Getting
      perfect errors will likely a rework of how errors are handled in
      libosload/libreboot, just adding tberror/tbwarn/tbnotice calls is not
      enough.  I can do this at a latter date if necessary.
      
      A few minor database changes.
      
      Some changes to the API.  A few bug fixes. Lots of tberror/tbwarn/tbnotice
      added to scripts.
      
      Since assign is a C program, and at this time my API is perl only, I wrote a
      second wrapper around assign, assign_wrapper2.  When assign fails errors are
      now parsed in assign_wrapper2, sent to stderr and logged.  This means that
      RunAssign() just returns when assign fails rather than echoing some of
      assign.log output and then quiting.  The output to the activity log remains
      unchanged.
      
      Since "parse.proxy" is run from ops I couldn't use my API in it, even though
      it is a perl program.  Instead I parse the errors coming form it in
      parse-ns.
      45f997fd
  25. 15 Dec, 2005 1 commit
  26. 28 Nov, 2005 1 commit
  27. 17 Nov, 2005 1 commit
    • Mike Hibler's avatar
      1. Beef up "admin mode" support. · 4ec701e7
      Mike Hibler authored
      * Add libadminmfs.pm with routines for entering/exiting and executing
        commands in, the admin MFS.  Node admin and firewall swapout (see
        below) now use this, the image creation process does not yet.
      
      * Add swapout time hooks for running an admin mode process, likely to
        be used to collect swapout time state.  Currently controlled globally
        by two new sitevars.
      
      * Modified node_admin to use the library and added a "-c <command>"
        option to have nodes go into admin mode and run a command.  I don't
        really expect this to be useful, it was just a testing vehicle for
        the library.
      
      2. Improved the swapout process for firewalled experiments.  Largely
         just generalized what we already did for paniced experiments.
         At swapout, firewalled nodes are:
      
         - powered off
         - set to boot into admin mode and run a disk zapper
         - powered on
      
        The swapout process then waits for all nodes to successfully complete
        disk zapage, at which point the nodes are nfree'ed as usual.  Any
        failure of the above process, marks the experiment as panic'ed (to
        ensure that we are involved in cleanup) and sends mail to testbed-ops
        describing the state of the nodes.
      
      3. Added the aforementioned disk zapper, a little C program in the MFS
         which zeroes out the MBR and partition boot blocks (but not the MBR
         partition table or FS superblocks).  This is added insurance that if
         a node somehow gets diverted after being nfree'd but before getting
         the disk reloaded (e.g., goes to hwdown), that we cannot accidentally
         boot from the disk.  This program gets installed in the admin MFS.
      
      4. Related to firewalls, modified swapin to use the new documented
         "snmpit -N" to get the firewall VLAN number rather than parsing the
         output that was a side-effect of VLAN creation.
      4ec701e7
  28. 04 Nov, 2005 1 commit
  29. 20 Oct, 2005 1 commit
    • Kirk Webb's avatar
      · 5326988f
      Kirk Webb authored
      New node_attributes facility and table.
      
      Auxiliary node attributes, such as service tag #, BIOS version, etc., are
      should now be placed into the node_attributes table.  This can be accomplished
      by either using the node_attributes command line tool, or by using the
      modnodeattributes_form.php3 form (not linked in anywhere yet, but will be
      in a moment).  Attribute names and values are checked for sanity using
      table_regex entries.  Also note that I started with the nodecontrol stuff
      as a template.
      
      The command line tool and web form (which simply calls the command line tool
      to actually do the modifications) can add, delete, and/or remove attributes.
      
      Finally, note that the bios_version column has been moved from the nodes
      table to the node_attributes table.  The Node Information page will show
      the list of current attributes at the bottom of the info table.
      5326988f
  30. 19 Sep, 2005 1 commit
    • Leigh Stoller's avatar
      Move all modification of the group_membership table to the backend, · cfba1ac7
      Leigh Stoller authored
      into a single new script call modgroups. Usage:
      
      	modgroups [-a pid:gid:trust[,pid:gid:trust]...]
                        [-m pid:gid:trust[,pid:gid:trust]...]
                        [-r pid:gid[,pid:gid]...] user
      
      So, -a to add groups, -r to remove groups, and -m to modify the trust
      value for a member of a group.
      
      The reason for doing this is that previously, we had no idea in the
      backend what group changes actually happened; we just knew what the
      current groups are. This make it hard to add and remove users from
      mailing lists, chat server buddy lists, etc. This is cleaner ...
      cfba1ac7
  31. 14 Sep, 2005 1 commit
    • Mike Hibler's avatar
      Changes related to allowing seperate 'fs' (file server) node. · c53d5827
      Mike Hibler authored
      Entailed new instructions for manual setup as well as integration into
      elabinelab framework.  First, the manual path:
      
      setup.txt, setup-boss.txt, setup-ops.txt and new setup-fs.txt:
          Updated to reflect potential for separate fs node.  The org here
          is a little dicey and could be confusing with ops+fs vs. ops and fs.
          Has not been field tested yet.
      
      */GNUmakefile.in: new fs-install target.
      
      configure, configure.in, defs-*:
          Somewhat unrelated, make min uid/gid to use be a defs setting.
          Also add config of fs-install.in script.
      
      boss-install.in, ops-install.in and new fs-install.in:
          Handle distinct fs node.  If you have one, fs-install is run before
          ops-install.  All scripts rely on the defs file settings of FSNODE
          and USERNODE to determine if the fs node is seperate.
      
      utils/checkquota.in:
          Just return "ok" if quotas are not used (i.e., if defs file FS_WITH_QUOTA
          string is null.
      
      install/ports/emulab-fs:
          Meta port for fs node specific stuff.  Also a patch for the samba port
          Makefile so it doesn't drag in CUPs, etc.  Note that the current samba
          port Makefile has this change, I am just backporting to our version.
      
      Elabinelab specific changes:
      
      elabinelab-withfs.ns:
          NS fragment used in conjunction with
      	tb-elab-in-elab-topology "withfs"
          to setup inner-elab with fs node.
      
      elabinelab.ns:
          The hard work on the boss side.  Recognize seperate-fs config and handle
          running of rc.mkelab on that node.  fs setup happens before ops setup.
      
      rc.mkelab:
          The hard work on the client side.  Recognize FsNode setup as well as
          differentiate ops+fs from ops setup.
      
      Related stuff either not part of the repo or checked in previously:
          emulab-fs package
      c53d5827
  32. 13 Jun, 2005 1 commit
    • Timothy Stack's avatar
      · 5e43a771
      Timothy Stack authored
      Initial checkin of a "repositioning" daemon that moves robots back to
      their pens on swapout.
      
      	* configure, configure.in: Add tbsetup/repos_daemon.
      
      	* db/libdb.pm.in: Add constants for the
      	repositionpending/repositioning experiments.
      
      	* db/nfree.in: When freeing garcias, send them to
      	repositionpending instead of reloadpending.
      
      	* event/sched/event-sched.c: Deal with the rare case of no
      	SIMULATOR object being in the agent list for an experiment.
      
      	* robots/emc/emcd.c, robots/emc/locpiper.in: Fix some typos.
      
      	* robots/rmcd/masterController.h, robots/rmcd/masterController.c,
      	robots/rmcd/obstacles.h, robots/rmcd/obstacles.c: Ignore dynamic
      	obstacles that are far away and remove dynamic obstacles where the
      	robot is inside the natural obstacle area.
      
      	* sql/database-create.sql, sql/database-migrate.txt: Add a
      	reposition_status table that tracks the status of robots that are
      	being moved back to their pens.
      
      	* tbsetup/GNUmakefile.in: Install the repos_daemon script.
      
      	* tbsetup/reload_daemon.in: Move robots to the repositionpending
      	experiment, if they haven't already reached their pen.
      
      	* tbsetup/repos_daemon.in: Daemon that takes care of seeing robots
      	back to their pens after they are freed from an experiment.
      5e43a771
  33. 26 May, 2005 1 commit
  34. 18 Mar, 2005 1 commit
    • Mike Hibler's avatar
      Add whack-on-lan power cycle module. · d9d8a58e
      Mike Hibler authored
      To enable WhOL, you need to add outlets table entries for nodes which are
      whol-enabled.  The power_id encodes the interface on boss to use:
      
      +---------+-----------+--------+----------------+
      | node_id | power_id  | outlet | last_power     |
      +---------+-----------+--------+----------------+
      | pcwf6   | whol-fxp0 |      0 | 20050318152119 |
      +---------+-----------+--------+----------------+
      
      You then need interfaces and wires table entries for that interface on
      boss, so that snmpit works with syntax like "boss:1".  This is probably not
      really needed once the VLAN has been setup.
      
      You need a magic VLAN called "WhOL", I used VLAN 999.  Add it to all the
      switches and trunks.  Put boss's port in it.  It will remain there, enabled,
      forever.
      
      In the interfaces table entry for every interface that supports WhOL,
      you need to set the 'whol' field to 1.
      d9d8a58e
  35. 04 Feb, 2005 1 commit
    • Kirk Webb's avatar
      · 79387881
      Kirk Webb authored
      Support for reloading garcias.
      
      Currently this is a total hack; we simply rsync the stargate as it is freed
      from the experiment.  However, until we unravel os_load dependancies between
      nodes and subnodes (motes and stargates in this case), doing this through
      the appropriate setup channels won't work.
      
      The tbrsync script borrows much of its infrastructure from Rob's tbuisp
      script.
      79387881
  36. 24 Jan, 2005 1 commit
    • Timothy Stack's avatar
      · 3c1a5bad
      Timothy Stack authored
      Robot related stuff: power via e-mail, client-install fixups, checking
      coords against camera boundaries.
      
      	* configure, configure.in: Add tbsetup/power_mail.pm to the list
      	of template files.
      
      	* doc/cross-compiling.txt: More stargate notes.
      
      	* event/sched/rpc.cc: Updates for the addition of the cameras
      	table.
      
      	* robots/GNUmakefile.in, robots/emc/GNUmakefile.in,
      	robots/mtp/GNUmakefile.in, robots/rmcd/GNUmakefile.in,
      	robots/tbsetdest/GNUmakefile.in, robots/vmcd/GNUmakefile.in:
      	client-install fixups.
      
      	* tbsetup/GNUmakefile.in: Add power_mail.pm.
      
      	* tbsetup/os_setup.in: Don't skip reboot of robots anymore.
      
      	* tbsetup/power.in: Add special case for a power_id of "mail",
      	which calls into the power_mail.pm backend.
      
      	* tbsetup/power_mail.pm.in: E-mail backend for power, it sends an
      	e-mail to tbops and waits for the outlets.last_power value to be
      	updated from the power.php3 web page.
      
      	* tbsetup/ns2ir/parse-ns.in: Add the contents of the cameras table
      	to the TBCOMPAT namespace.
      
      	* tbsetup/ns2ir/sim.tcl.in: More checking of "setdest" inputs.
      
      	* tbsetup/ns2ir/topography.tcl: Update the checkdest method to
      	check destination points against the camera list.
      
      	* www/powertime.php3: Webpage used to update the last power time
      	for nodes.
      
      	* www/shownode.php3: Add "Update Power Time" menu button.
      3c1a5bad
  37. 13 Jan, 2005 1 commit