1. 08 Aug, 2006 1 commit
  2. 03 Aug, 2006 1 commit
    • Leigh B. Stoller's avatar
      Support for capturing the trace data that is stored in the pcal files · 4ce9c421
      Leigh B. Stoller authored
      into per-experiment databases on ops. Additional support for reconsituting
      those databases back into temporary databases on ops, for post processing.
      
      * This revision relies on the "snort" port (/usr/ports/security/snort)
        to read the pcap files and load them into a database. The schema is
        probably not ideal, but its better then nothing. See the file
        ops:/usr/local/share/examples/snort/create_mysql for the schema.
      
      * For simplicity, I have hooked into loghole, which already had all
        the code for downloading the trace data. I added some new methods to
        the XMLRPC server for loghole to use, to get the users DB password
        and the name of the per-experiment database. There is a new slot in
        the traces table that indicates that the trace should be snorted to
        its DB. In case you forgot, at the end of a run or when the instance
        is swapped out, loghole is run to download the trace data.
      
      * For reconsituting, there are lots of additions to opsdb_control and
        opsdb_control.proxy to create "temporary" databases and load them
        from a dump file that is stored in the archive. I've added a button
        to the Template Record page, inappropriately called "Analyze" since
        right now all it does is reconsitute the trace data into a DB on
        ops.
      
        Currently, the only indication of what has been done (the name of
        the DBs created on ops) is the log email that the user gets. A
        future project is tell the user this info in the web interface.
      
      * To turn on database capturing of trace data, do this in your NS
        file:
      
      	set link0 ...
      	$link0 trace
      	$link0 trace_snaplen 128
      	$link0 trace_db 1
      
         the increase in snaplen is optional, but a good idea if you want
         snort to undertand more then just ip headers.
      
      * Also some changes to the parser to allow plain experiments to take
        advantage of all this stuff. To simple get yourself a per-experiment
        DB, put this in your NS file:
      
      	tb-set-dpdb 1
      
        however, anytime you turn trace_db on for a link or lan, you
        automatically get a per-experiment DB.
      
      * To capture the trace data to the DB, you can run loghole by hand:
      
      	loghole sync -s
      
        the -s option turns on the "post-process" phase of loghole.
      4ce9c421
  3. 27 Jul, 2006 1 commit
  4. 20 Jul, 2006 2 commits
    • Leigh B. Stoller's avatar
      Minor change to previous revision; need the right number of 0s after · 1a564e6d
      Leigh B. Stoller authored
      the decimal point in decimal slots.
      1a564e6d
    • Leigh B. Stoller's avatar
      This start out as: · 5dfa4a47
      Leigh B. Stoller authored
      * add an "active" flag to the template record, which will be used by
        the user to indicate what templates he wants listed (rather then the
        roots). Basically, the current working templates, rather then a big
        graph.
      
      But I never actually finished that cause it sorta morphed into:
      
      * Added a vis_graphs table to cache the last generated visualization
        rendering in the database so that we do not have to wait so long for:
      
      * Add new buttons to showexp and template_show pages, to display in the
        same page either the settings (current view), the NS file, or the
        visualization (along with zoom in/out buttons).
      
      And now I can go back to that "active" thing I mentioned up above ...
      5dfa4a47
  5. 11 Jul, 2006 1 commit
    • Leigh B. Stoller's avatar
      Add new table, node_type_attributes, which will eventually encompass · 9bcb29bb
      Leigh B. Stoller authored
      most of what is in the node_types table (and node_types_auxtypes).
      The goal is to make most of the type information free form key/value
      pairs so that type information is more extensible.
      
      This commit simply creates the table and populates it with initial
      info using init_nodeattrs.pl. I expect this script to change, but it
      can be rerun later so no big deal.
      9bcb29bb
  6. 05 Jul, 2006 2 commits
    • Kevin Atkinson's avatar
      · 183040de
      Kevin Atkinson authored
      Many changes to tblog code.  Database update needed:
      
      1) Added summary of failed nodes is os_setup.  The cause of the error is now
      classified as "user" if it is only user images that failed and the user
      image failed on every pc of a particular type.  Otherwise I leave the cause
      as "unknown" since it is really hard to tell what the real cause is.
      
      2) Raised the confidence threshold for most errors so that they will appear
      on the top.
      
      3) Added a special error when an experiment is canceled.  The cause is
      "canceled" and testbed-ops won't see these errors.
      
      4) Fixed a bug in assign_wrapper where it will incorrectly report "This
      experiment cannot be instantiated on this testbed..." when really the user
      canceled the swapin.
      
      5) Fixed a bug where os_setup errors where being incorrectly reported as
      assign errors.  This happens when os_setup fails for some reason and
      tbswap tries again, but the second time around there are not enough nodes.
      So the last error is coming from assign even though the true cause of the
      error is due to failed nodes.  The fix for this involved added a new column
      to the log table, "attempt", which will be 1 for the first attempt and then
      incremented for each new attempt.  tblog_find_error will then simply ignore
      any errors with "attempt > 1".
      
      6) Also fixed a potential problem when there is an error during the cleanup
      phase by adding another column "cleanup".  tblog_find_error will
      also ignore any errors with the cleanup bit set.
      183040de
    • Leigh B. Stoller's avatar
      488bf732
  7. 03 Jul, 2006 1 commit
    • Mike Hibler's avatar
      Framework for supporting 802.1q tagged VLANs as a form of multiplexed link. · 3f1c15e2
      Mike Hibler authored
      Actually, most of the changes here were just to generalize the "virtual
      interface" state in the DB.  Other than the client-side scripts, there
      is very little specific here specific to tagged VLANs.
      
      In fact, you cannot specify "vlan" as a type yet as we haven't done the
      snmpit support for setting up the switches.
      
      For more info see bas:~mike/flux/doc/testbed-virtinterfaces.txt, which I
      will integrate into the knowledge base and the Emulab doc at some point.
      3f1c15e2
  8. 26 Jun, 2006 1 commit
  9. 14 Jun, 2006 2 commits
    • Leigh B. Stoller's avatar
      The template "datastore" ... · fe9aa6a4
      Leigh B. Stoller authored
      Each template has a datastore, which is really just a subdirectory that can
      be populated with files, and committed to the subversion archive.  Note,
      the datastore os specific to the template itself. The Template Archive link
      on the Show Template page takes you to the subdirectory, which by
      convention I am calling "datastore".
      
      The directory actually lives in /proj/pid/exp/eid/TGUID-VERS ... but that
      path is printed out for you on the archive page.
      
      Anyway, put stuff in the datastore directory, and then commit the template
      archive so there is a tag associated with it.
      
      When an instance is created, a checkout of the datastore is placed in the
      experiment directory (/proj/pid/eid/exp/template_datastore). The current
      tag (from above) is stored with the instance so that we can later recreate
      the enviroment for the instance, say for rerun.
      
      Tarfiles and rpms in the datastore can be referenced as xxx://foo.rpm (in
      your NS file).  tarfiles_setup transforms those when the instance is
      swapped in, sorta like it does other URLs, only it does not actually fetch
      them, just need to rewrite the paths so they reference datastore.
      
      The program agent gets another environment variable so you can refer to the
      datastore without hardwiring paths ($DATASTORE). Eventually I want to move
      the checkout someplace else, but it was easy to drop it into the experiment
      directory for now.
      fe9aa6a4
    • Leigh B. Stoller's avatar
      Changes to allow users to be logged in from more then one browser. · b452cfd4
      Leigh B. Stoller authored
      The adminon/off bit is now kept in the login table instead of the
      users table so that you can be reddot on a per-browser (login) basis.
      b452cfd4
  10. 07 Jun, 2006 1 commit
  11. 01 Jun, 2006 5 commits
  12. 22 May, 2006 1 commit
  13. 18 May, 2006 1 commit
    • Leigh B. Stoller's avatar
      Address a number of issues raised by Mike. The most notable: · 4d663bb8
      Leigh B. Stoller authored
      * Rework metadata to be somewhat more general so that they can be used for
        things like parameter descriptions.
      
      * My own little part of javascript hell ... add a highlight around the
        "current" template in the template graph. Sheesh!
      
      * Add more tooltips (using the tooltip library). When you mouseover the
        metadata or descriptions, the formatted text is shown in a tooltip,
        which is handy when the valus is truncated cause its too long for the
        tables.
      4d663bb8
  14. 11 May, 2006 1 commit
    • Mike Hibler's avatar
      Initial support for plab-in-elab: · 088c8e74
      Mike Hibler authored
      * setup some fields that parallel the
        elab-in-elab ones, for modifying the dhcpd.conf file.
      
      * tb-set-node-plab-role to set a node as either 'plc' or 'node'
        used to set the DB role field above
      088c8e74
  15. 09 May, 2006 2 commits
  16. 08 May, 2006 1 commit
    • Kevin Atkinson's avatar
      · 95f529d3
      Kevin Atkinson authored
      Refactor "log" table to move some stuff into a new table.
      95f529d3
  17. 05 May, 2006 1 commit
  18. 26 Apr, 2006 1 commit
  19. 18 Apr, 2006 1 commit
  20. 14 Apr, 2006 2 commits
  21. 13 Apr, 2006 1 commit
  22. 10 Apr, 2006 1 commit
    • Mike Hibler's avatar
      Its back! · fabfd1eb
      Mike Hibler authored
      node_types pxe_boot_path is going to be needed after all, so it has
      made a triumphant return.
      fabfd1eb
  23. 07 Apr, 2006 1 commit
  24. 04 Apr, 2006 1 commit
  25. 01 Apr, 2006 1 commit
  26. 31 Mar, 2006 1 commit
    • Leigh B. Stoller's avatar
      More experiment run stuff. Also the NS file parsing support that was · 74bf8d1d
      Leigh B. Stoller authored
      needed (reparsing of NS file with current bindings).
      
      Mostly I decided that I made a huge mistake in using the exptidx as the
      main cross index. I started to change that, which took a bunch of time.
      Still more tables to fix up, but I think all of this code is gonna need
      a major reorg once I am no longer under the gun to get it done.
      74bf8d1d
  27. 29 Mar, 2006 2 commits
  28. 28 Mar, 2006 2 commits
  29. 21 Mar, 2006 1 commit
    • Kevin Atkinson's avatar
      · d258dde6
      Kevin Atkinson authored
      Changed format of email sent to user on errors.  The error will now
      appear instead of the generic message when I am confident it is
      accurate.  The subject line will also change to reflect the cause of
      an error.
      
      Avoid sending mail to testbed-ops during failed swap related evenets
      in some cases.  It will instead be sent to a new mailing list
      testbed-errors.
      
      Added a new row in the experiment info table "Last Error:" which
      states the cause of the error, and links to a new page displaying the
      error.
      
      Made some assign/assign_wrapper errors more informative.
      
      The error (as determined by tblog) is now stored in the database in a
      more structured fashion.  This inlcudes adding a column for the session
      (in the log table) to testbed_stats to link eash swap event with the
      logs and possible the error.
      
      Other changes to the database, see sql/database-migrate.txt
      d258dde6