1. 05 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      More work on "recording" template events. · e9607a77
      Leigh Stoller authored
      * New version of template_record just for ops, since so much is
        different about ops, not bothering to maintain a single version.
      
      * Various fixes to how the recorded events are stored and reconstituted.
        The big fix is to wrap them in a sequence to that they get fired
        properly (waiting for completion of previous event in recording).
      
      * New buttons to Pause and Continue event time, which is used when
        adding recorded events. This allows users to pause time while they
        "think" so when an event is recorded, the thinking time is not actually
        in the timeline. Eventually hope to figure this out automatically, but
        that will take some real, uh, thinking.
      
      * Add a new event editor (linked off the template page) that allows
        you to delete and change the recordings. Note that you can only edit
        the events at the template level; you cannot edit the events of an
        instance (swapped in experiment), and you can only edit the recorded
        events, not any other events. Not sure its useful to be able to do
        either of these yet, but probably not too hard to add at some point.
      e9607a77
  2. 13 Sep, 2006 1 commit
  3. 05 Sep, 2006 1 commit
    • Leigh Stoller's avatar
      A bunch of template changes resulting from meetings last week. · 087dbfff
      Leigh Stoller authored
      * Add XMLRPC interface for template swapin,stoprun,startrun,swapout and
        add the appropriate wrappers to the script_wrapper on ops.
      
      * Allow parameter descriptions in NS files. This is probably not in its
        final form since its a bit confusing as to what has priority; something
        in the NS file or a metadata item. Anyway, you can do this in your NS
        file:
      
      	$ns define-template-parameter GUID "0/0" "The GUID to be analyzed"
      
        The rules are currently that the NS file description has priority and
        is copied to child templates, unless the user has modified a description
        via the web interface, in which case the NS file description is ignored.
        I know, sounds awful, but for the most part people are going to use the
        NS file anyway.
      
      * Add "clear" option when starting a new experiment run; the per
        experiment DB at the logholes are cleared. Note that this is *not* the
        default behaviour; you have to either check the checkbox on the web form
        or use the -c option to the script wrapper, or clear=yes if talking
        directly to the XMLRPC server.
      
      * Fix up how email is generated for template_swapin and template_create,
        so that Kevin can debug tblog/tbreport stuff, but also so that we maintain
        mail logs as before. I have made some improvements to libaudit so as to
        centralize the mail goo, and avoid duplicating all that stuff.
      
      * Minor fixes to the program agent so that the new environment strings are
        sent before the program agent exits and reloads them!
      
      * Other minor little things.
      087dbfff
  4. 14 Aug, 2006 1 commit
    • Leigh Stoller's avatar
      Checkpoint my dynamic event stuff, crude as it is. The idea for this first · 9d021a07
      Leigh Stoller authored
      draft is that the user will at the end of an experiment run, log into one
      of his nodes and perform some analysis which is intended to be repeated at
      the end of the next run, and in future instantiations of the template.
      
      A new table called experiment_template_events holds the dynamic events for
      the template. Right now I am supporting just program events, but it will be
      easy to support arbitrary events later. As an absurd example:
      
      	node6> /usr/local/bin/template_analyze ~/data_analyze arg arg ...
      
      The user is currently responsible for making sure the output goes into a
      file in the archive. I plan to make the template_analyze wrapper handle
      that automatically later, but for now what you really want is to invoke a
      script that encapsulates that, redirecting output to $ARCHIVE (this
      variable is installed in the environment template_analyze.
      
      The wrapper script will save the current time, and then run the program.
      If the program terminates with a zero exit status, it will ssh over to ops
      and invoke an xmlrpc routine to tell boss to add a program event to both
      the eventlist for the current instance, and to the template_eventlist for
      future instances. The time of the event is the relative start time that was
      saved above (remember, each experiment run replays the event stream from
      time zero).
      
      For the future, we want to allow this to be done on ops as well, but
      that will take more infrastructure, to run "program agents" on ops.
      
      It would be nice to install the ssl xmlrpc client side on our images so
      that we do not have to ssh to ops to invoke the client.
      9d021a07
  5. 08 Aug, 2006 1 commit
  6. 07 Aug, 2006 1 commit
  7. 03 Aug, 2006 1 commit
    • Leigh Stoller's avatar
      Support for capturing the trace data that is stored in the pcal files · 4ce9c421
      Leigh Stoller authored
      into per-experiment databases on ops. Additional support for reconsituting
      those databases back into temporary databases on ops, for post processing.
      
      * This revision relies on the "snort" port (/usr/ports/security/snort)
        to read the pcap files and load them into a database. The schema is
        probably not ideal, but its better then nothing. See the file
        ops:/usr/local/share/examples/snort/create_mysql for the schema.
      
      * For simplicity, I have hooked into loghole, which already had all
        the code for downloading the trace data. I added some new methods to
        the XMLRPC server for loghole to use, to get the users DB password
        and the name of the per-experiment database. There is a new slot in
        the traces table that indicates that the trace should be snorted to
        its DB. In case you forgot, at the end of a run or when the instance
        is swapped out, loghole is run to download the trace data.
      
      * For reconsituting, there are lots of additions to opsdb_control and
        opsdb_control.proxy to create "temporary" databases and load them
        from a dump file that is stored in the archive. I've added a button
        to the Template Record page, inappropriately called "Analyze" since
        right now all it does is reconsitute the trace data into a DB on
        ops.
      
        Currently, the only indication of what has been done (the name of
        the DBs created on ops) is the log email that the user gets. A
        future project is tell the user this info in the web interface.
      
      * To turn on database capturing of trace data, do this in your NS
        file:
      
      	set link0 ...
      	$link0 trace
      	$link0 trace_snaplen 128
      	$link0 trace_db 1
      
         the increase in snaplen is optional, but a good idea if you want
         snort to undertand more then just ip headers.
      
      * Also some changes to the parser to allow plain experiments to take
        advantage of all this stuff. To simple get yourself a per-experiment
        DB, put this in your NS file:
      
      	tb-set-dpdb 1
      
        however, anytime you turn trace_db on for a link or lan, you
        automatically get a per-experiment DB.
      
      * To capture the trace data to the DB, you can run loghole by hand:
      
      	loghole sync -s
      
        the -s option turns on the "post-process" phase of loghole.
      4ce9c421
  8. 28 Jul, 2006 1 commit
    • Leigh Stoller's avatar
      Add a "Create Template from Instance" ability. Basically, you can · a651da71
      Leigh Stoller authored
      create a new template (well, really a modify) from the current
      swapped in experiment. This allows you to create a template, swap in
      an instance, modify the datastore in the instance (which is a copy
      of the datastore in the template), and then create a new template
      using the datastore and nsfile from the instance. This is a new menu
      item on the showexp page for the instance.
      
      Also in this commit are fixes and improvements to the new navagation
      bar that I recently added.
      a651da71
  9. 26 Jul, 2006 1 commit
  10. 20 Jul, 2006 2 commits
    • Leigh Stoller's avatar
      Minor tweak to previous revision. · b1bdca1a
      Leigh Stoller authored
      b1bdca1a
    • Leigh Stoller's avatar
      This start out as: · 5dfa4a47
      Leigh Stoller authored
      * add an "active" flag to the template record, which will be used by
        the user to indicate what templates he wants listed (rather then the
        roots). Basically, the current working templates, rather then a big
        graph.
      
      But I never actually finished that cause it sorta morphed into:
      
      * Added a vis_graphs table to cache the last generated visualization
        rendering in the database so that we do not have to wait so long for:
      
      * Add new buttons to showexp and template_show pages, to display in the
        same page either the settings (current view), the NS file, or the
        visualization (along with zoom in/out buttons).
      
      And now I can go back to that "active" thing I mentioned up above ...
      5dfa4a47
  11. 10 Jul, 2006 1 commit
  12. 24 May, 2006 1 commit
  13. 16 May, 2006 1 commit
  14. 12 May, 2006 1 commit
    • Leigh Stoller's avatar
      Redo the entire template library. I've been meaning to use perl · 78503406
      Leigh Stoller authored
      "object" and this was a good opportunity to see if they are useful and
      easy enough to use. Yep they are; the code is much cleaner with many
      fewer utility functions to get at stuff. I recommend this approach
      from now on.
      
      The problem is the php side, which ends up duplicating some stuff, but
      in the old style. This is not so bad for the template code since I
      have made it a point not to do anything but display functions in php;
      all modifications are handled in the backend.
      78503406
  15. 05 May, 2006 1 commit
  16. 31 Mar, 2006 1 commit
    • Leigh Stoller's avatar
      More experiment run stuff. Also the NS file parsing support that was · 74bf8d1d
      Leigh Stoller authored
      needed (reparsing of NS file with current bindings).
      
      Mostly I decided that I made a huge mistake in using the exptidx as the
      main cross index. I started to change that, which took a bunch of time.
      Still more tables to fix up, but I think all of this code is gonna need
      a major reorg once I am no longer under the gun to get it done.
      74bf8d1d
  17. 30 Mar, 2006 1 commit
  18. 29 Mar, 2006 1 commit
  19. 28 Mar, 2006 1 commit
  20. 21 Mar, 2006 1 commit
    • Kevin Atkinson's avatar
      · d258dde6
      Kevin Atkinson authored
      Changed format of email sent to user on errors.  The error will now
      appear instead of the generic message when I am confident it is
      accurate.  The subject line will also change to reflect the cause of
      an error.
      
      Avoid sending mail to testbed-ops during failed swap related evenets
      in some cases.  It will instead be sent to a new mailing list
      testbed-errors.
      
      Added a new row in the experiment info table "Last Error:" which
      states the cause of the error, and links to a new page displaying the
      error.
      
      Made some assign/assign_wrapper errors more informative.
      
      The error (as determined by tblog) is now stored in the database in a
      more structured fashion.  This inlcudes adding a column for the session
      (in the log table) to testbed_stats to link eash swap event with the
      logs and possible the error.
      
      Other changes to the database, see sql/database-migrate.txt
      d258dde6
  21. 13 Feb, 2006 1 commit
  22. 05 Jan, 2006 2 commits
  23. 27 Dec, 2005 1 commit
  24. 15 Dec, 2005 1 commit
  25. 30 Nov, 2005 1 commit
  26. 17 Nov, 2005 1 commit
  27. 09 Nov, 2005 1 commit
  28. 01 Nov, 2005 1 commit
  29. 14 Oct, 2005 1 commit
    • Mike Hibler's avatar
      Mike's yearly PHP hacking: · 39b1477e
      Mike Hibler authored
       * Add a "showclass" param to shownodes so that the
         node listing will include only nodes of the indicated class.
         Prepend "no-" to exclude nodes of a particular class.  So:
              showclass=pc,mote
         would show just pcs and motes.
      
       * Now the real reason I added the above: change the hwdown experiment
         node list to include "showclass=no-pcplabphys" by default so I don't have
         to flip through 600+ dead plab nodes.  Change the URL to "showclass=all"
         if you want to see plab nodes in hwdown.
      39b1477e
  30. 22 Jun, 2005 1 commit
    • Leigh Stoller's avatar
      Added my simplistic link tracing and monitoring. Example usage and · 7942119e
      Leigh Stoller authored
      some details can be found in the advanced tutorial that I wrote up.
      See this link:
      
      http://www.emulab.net/tutorial/docwrapper.php3?docname=advanced.html#Tracing
      
      The basic idea is that each virt_lan entry gets a couple of new slots
      describing the type of tracing that is desired.
      
        traced tinyint(1) default '0',
        trace_type enum('header','packet','monitor') NOT NULL default 'header',
        trace_expr tinytext,
        trace_snaplen int(11) NOT NULL default '0',
        trace_endnode tinyint(1) NOT NULL default '0',
      
      There is a new physical table called "traces" that is a little bit
      like the current delays table. A new tmcd command returns the trace
      configuration to the client nodes (tmcd/common/config/rc.trace).
      
      The delays table got a new boolean called "noshaping" that tells the
      delay node to bridge, but not set up any pipes. This allows us to
      capture traffic at the delay node, but without much less overhead on
      the packets.
      
      The pcapper got bloated up to do packet capture and more event stuff.
      I also had to add some mutex locking around calls into the pcap
      library and around malloc, since the current setup used linuxthreads,
      which is not compatable with the standard libc_r library. I was
      getting all kinds of memory corruption, and I am sure that if someone
      breathes on the pcapper again, it will break in some new way.
      7942119e
  31. 31 May, 2005 1 commit
    • Leigh Stoller's avatar
      Pass around pid,eid (through the applet) so that the backend knows · 30667bc5
      Leigh Stoller authored
      what locpiper to connect to. Locpiper now binds a dynamic port, and
      stores that into the DB, and the php script expects to get that pid,eid
      back.
      
      Changed the showexp page so that pid,eid are passed to the robotrack
      top level page.
      
      Changed the robotrack page so that if it does not get a pid,eid, it looks
      for an experiment running (a locpiper). If none is running, let the applet
      start of course, but there will be an error dialog when it starts. You can
      still play with the applet of course. If there is more then one experiment
      running, then just pick first one. At some point I might add a page to ask
      which experiment, but its low priority.
      30667bc5
  32. 11 May, 2005 1 commit
  33. 09 Mar, 2005 1 commit
  34. 15 Feb, 2005 1 commit
  35. 09 Feb, 2005 1 commit
    • Timothy Stack's avatar
      · ea206d40
      Timothy Stack authored
      Robot-related improvements based on feedback from Sid:
      
      	* event/sched/event-sched.h, event/sched/event-sched.c,
      	event/sched/rpc.h, event/sched/rpc.cc: Send a notification of the
      	start of event time and timelines.  Add a '-r' option that forces
      	it to use the default rpc path (so dan can use a custom
      	scheduler/rmcd/etc on ops).
      
      	* robots/GNUmakefile.in, robots/emc/GNUmakefile.in,
      	robots/mtp/GNUmakefile.in, robots/primotion/GNUmakefile.in,
      	robots/rmcd/GNUmakefile.in, robots/vmcd/GNUmakefile.in,
      	robots/tbsetdest/tbsetdest.cc: Add a control-install target.
      
      	* robots/primotion/garcia-pilot.cc: Take out some experimental
      	code.
      
      	* robots/primotion/dashboard.hh, robots/primotion/dashboard.cc,
      	robots/primotion/pilotClient.cc, robots/primotion/wheelManager.hh,
      	robots/primotion/wheelManager.cc: When doing a COMMAND_STOP, make
      	sure we only send one update packet, an idle or an abort.  Also
      	send back the actual distance travelled/pivoted.
      
      	* robots/tbsetdest/tbsetdest.cc: Don't output setdest event's with
      	a speed of zero.
      
      	* tbsetup/eventsys_control.in: When replaying/stopping, clear the
      	start of event time for the experiment.
      
      	* tbsetup/power_mail.pm.in: Decrease the threshold used to tell if
      	a node has been power cycled "recently".
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/sim.tcl.in: Bah, backout
      	last change, orientation is supposed to be in degrees at these
      	points.  Also, when automatically picking the sync server, don't
      	use nodes that are subnodes.
      
      	* tmcd/common/bootsubnodes: Send an ISUP for motes.
      
      	* www/moteleds.php3: Show the mote node names instead of the
      	robots.
      
      	* www/powertime.php3: Add option to mark nodes as powered off.
      
      	* www/robotmap.php3: Display the elapsed time for the overall
      	event time and for each active timeline.
      
      	* www/showexp.php3: Only display the blinky lights menu item when
      	there are motes.
      
      	* www/shownode.php3: Change "Update Power Time" -> "Update Power
      	State".
      
      	* www/tutorial/mobilewireless.php3: Fix some nits.
      
      	* xmlrpc/emulabserver.py.in: Add event_time_start method for
      	storing/getting/clearing the start of event time for an
      	experiment/timeline.
      ea206d40
  36. 04 Feb, 2005 1 commit
    • Timothy Stack's avatar
      · 6f545cf0
      Timothy Stack authored
      Some more robot integration.
      
      	* event/lib/event.h, event/lib/event.c: Add some
      	event_notification creation functions that get used in
      	event-sched.
      
      	* event/sched/event-sched.c, event/sched/rpc.h,
      	event/sched/rpc.cc: Sync the start of event time with the
      	robots reaching their initial positions.  This is done by
      	creating a master event-sequence that takes care of sending
      	the SETDEST events and then starting the Simulator timeline.
      
      	* mote/tbuisp.in: Use node_reboot instead of ssh'ing in.
      
      	* robots/GNUmakefile.in: Don't build tbsetdest if ulsshxmlrpcpp is
      	not available.
      
      	* robots/emc/loclistener.in: Clear the destination values for a
      	node when it reaches its destination.
      
      	* robots/primotion/garcia-pilot.cc,
      	robots/primotion/pilotClient.cc: Some cleanup and debugging.
      
      	* robots/primotion/wheelManager.cc: Check the rear sensors for
      	obstructions and then decide which way to pivot.
      
      	* robots/rmcd/pilotConnection.h, robots/rmcd/pilotConnection.c,
      	robots/rmcd/rclip.h: Even more tweaking.
      
      	* robots/tbsetdest/tbsetdest.cc: Don't generate points that are
      	outside the camera bounds or inside an obstacle.
      
      	* robots/vmcd/visionTrack.c, robots/vmcd/vmcd.h,
      	robots/vmcd/vmcd.c:  Add more debugging output.
      
      	* tbsetup/os_setup.in: Removed the robot hack used when deciding
      	which nodes to reconfig/reboot.  Added a robot hack to avoid
      	rebooting a robot whose mote is being os_load'd, since it would
      	interrupt tbuisp which does the reboot anyways.  Also fixed a
      	small typo.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/sim.tcl.in: Oops, forgot
      	to convert degrees to radians.
      
      	* tbsetup/ns2ir/topography.tcl: When checking for node destination
      	points in obstacles, include the implicit exclusion zone.
      
      	* tmcd/common/bootsubnodes: Add empty "mote" case.
      
      	* tmcd/linux-sg/GNUmakefile.in: Make some of the /etc subdirs when
      	doing the install.
      
      	* tmcd/linux-sg/rc.stargate: Start garcia-pilot.
      
      	* vis/floormap.in: Add options for showing the camera bounds,
      	obstacle exclusion zones, and displaying vnames instead of pnames.
      
      	* www/ledpipe.php3: Finally figured out how to use a socket
      	instead of popening a perl script.
      
      	* www/robotmap.php3: Add checkboxes for displaying/not displaying
      	the camera bounds and obstacle exclusion zones.  Add a legend
      	showing what actual vs. destination points are.  Pass pid/eid
      	through to vis/floormap if it is given.
      
      	* www/showexp.php3: Add a "Robot Map" link to experiments that
      	have allocated garcias.
      
      	* www/floormap/map_legend_node.gif,
      	www/floormap/map_legend_node_dst.gif: Icons used by the robot map
      	legend.
      
      	* www/floormap/robots-4.jpg: Added an obstacle around the entryway
      	so people are slightly less likely to trip over the robots.  Added
      	a coordinate system legend to the top left corner.
      
      	* www/tutorial/mobilewireless.php3: Add links to David's movie of
      	the robot making its way around the pillar.
      
      	* www/tutorial/robot_anim.gif: A nifty gifanim clip of the robot
      	movie.
      
      	* xmlrpc/emulabserver.py.in: Pull the camera data from the DB,
      	instead of returning hardcoded stuff.
      6f545cf0
  37. 12 Jan, 2005 1 commit
  38. 16 Dec, 2004 1 commit