1. 06 Aug, 2007 1 commit
  2. 02 Aug, 2007 1 commit
  3. 28 May, 2007 1 commit
    • Leigh B. Stoller's avatar
      Implement two requests by Eric: · e12ea1e2
      Leigh B. Stoller authored
      * The checkout directory now includes just the datastore directory and
        tbdata/nsfile.ns. This removes a bunch of clutter.
      
      * You can now run template_commit in any subdir of a checkout; the
        python wrapper will crawl upwards looking for the .template file.
      
        Note that I removed the option that allowed you to do a commit from
        the template tree in /proj/pid/templates since that directory will
        ultimately go away (or at least hide from view), and cause it
        conflicted with the new option and I didn't want to make things any
        messier for no reason.
      e12ea1e2
  4. 07 May, 2007 1 commit
    • Leigh B. Stoller's avatar
      Mostly this commit is the switch from SVN archives to ZIP archives. · 55d1bb6e
      Leigh B. Stoller authored
      Other stuff leaked in too ...
      
      I did separate out a lot of tbsetup/libArchive into db/Archive, and
      whats left in libArchive.pm will eventually move over into the
      Template library.
      
      Note that I have dropped archiving of plain experiments; this is not
      really owrth it outside the workbench context, and it just wastes
      space and makes a lot if stuff painful in the web interface.
      55d1bb6e
  5. 10 Apr, 2007 1 commit
  6. 08 Dec, 2006 1 commit
    • Leigh B. Stoller's avatar
      As discussed in meetings and email ... this commit changes what is · b898a8cc
      Leigh B. Stoller authored
      archived.  Rather then a special "archive" directory in the experiment
      directory, we know archive the entire experiment directory.
      
      This change should be backwards compatable, but let me know if not.
      
      Note that the nsdata directory is gone; the nsfile comes from the
      tbdata, but I know place a copy in nsfile.ns so that the name is well
      known.
      b898a8cc
  7. 18 Oct, 2006 1 commit
  8. 06 Oct, 2006 1 commit
  9. 10 Aug, 2006 1 commit
    • Leigh B. Stoller's avatar
      Okay, now we can view graphs from the historical data (template record). · 0c1b1a23
      Leigh B. Stoller authored
      A couple of things to note:
      
      * When requesting a graph, we have to have a checkout of the archive
        (the DB dump file) so that we can create a temporary DB with the data.
        This is done on demand, and the DB is left in place since its a
        fairly time consuming operation to do the checkout and the dbload.
        I do not delete the DBs though; we will need to age them out as needed.
      
      * Even so, when returning to a page we end up getting the graphs
        again, and that still takes more time then I like to wait. Perhaps
        add a refresh button so that the user has to force a redraw. Might
        need to add a time/date stamp to the graph.
      0c1b1a23
  10. 03 Aug, 2006 1 commit
    • Leigh B. Stoller's avatar
      Support for capturing the trace data that is stored in the pcal files · 4ce9c421
      Leigh B. Stoller authored
      into per-experiment databases on ops. Additional support for reconsituting
      those databases back into temporary databases on ops, for post processing.
      
      * This revision relies on the "snort" port (/usr/ports/security/snort)
        to read the pcap files and load them into a database. The schema is
        probably not ideal, but its better then nothing. See the file
        ops:/usr/local/share/examples/snort/create_mysql for the schema.
      
      * For simplicity, I have hooked into loghole, which already had all
        the code for downloading the trace data. I added some new methods to
        the XMLRPC server for loghole to use, to get the users DB password
        and the name of the per-experiment database. There is a new slot in
        the traces table that indicates that the trace should be snorted to
        its DB. In case you forgot, at the end of a run or when the instance
        is swapped out, loghole is run to download the trace data.
      
      * For reconsituting, there are lots of additions to opsdb_control and
        opsdb_control.proxy to create "temporary" databases and load them
        from a dump file that is stored in the archive. I've added a button
        to the Template Record page, inappropriately called "Analyze" since
        right now all it does is reconsitute the trace data into a DB on
        ops.
      
        Currently, the only indication of what has been done (the name of
        the DBs created on ops) is the log email that the user gets. A
        future project is tell the user this info in the web interface.
      
      * To turn on database capturing of trace data, do this in your NS
        file:
      
      	set link0 ...
      	$link0 trace
      	$link0 trace_snaplen 128
      	$link0 trace_db 1
      
         the increase in snaplen is optional, but a good idea if you want
         snort to undertand more then just ip headers.
      
      * Also some changes to the parser to allow plain experiments to take
        advantage of all this stuff. To simple get yourself a per-experiment
        DB, put this in your NS file:
      
      	tb-set-dpdb 1
      
        however, anytime you turn trace_db on for a link or lan, you
        automatically get a per-experiment DB.
      
      * To capture the trace data to the DB, you can run loghole by hand:
      
      	loghole sync -s
      
        the -s option turns on the "post-process" phase of loghole.
      4ce9c421
  11. 02 Jun, 2006 1 commit
  12. 31 May, 2006 1 commit
  13. 30 May, 2006 1 commit
    • Leigh B. Stoller's avatar
      Add an export option to the record listing. A new button on the Template · 2cfe4630
      Leigh B. Stoller authored
      Record page lets you export the contents of the archive that corresponds
      to that record, along with an XML file that describes the various DB bits
      for the template and instance.
      
      This is just a first cut so that Mike can start playing around. Subject to
      change, I'm sure.
      
      The archive is dumped to /proj/$pid/exports/$guid/$vers/$exptidx, which
      is basically the last commit of the instance when it was terminated.
      
      The xml file is called export.xml and is placed in the top level directory
      of the above directory. The file is created with XML::Simple, and a typical
      XML file might look like:
      
      <instance>
        <bindings>
          <name>NodeCount</name>
          <description>Number of nodes!</description>
          <value>1</value>
        </bindings>
        <bindings>
          <name>OS</name>
          <description></description>
          <value>RHL90-STD</value>
        </bindings>
        <bindings>
          <name>ScriptArgs</name>
          <description></description>
          <value>-b</value>
        </bindings>
        <eid>NewOne-V2</eid>
        <guid>10149/2</guid>
        <metadata>
          <name>M1</name>
          <guid>10162/1</guid>
          <value>Some metadata</value>
        </metadata>
        <pid>testbed</pid>
        <runs>
          <name>1</name>
          <archive_tag>T20060526-082533-172_endexp</archive_tag>
          <description></description>
          <exptidx>110</exptidx>
          <idx>1</idx>
          <runid>NewOne-V2</runid>
          <start_time>2006-05-26 08:23:02</start_time>
          <stop_time>2006-05-26 08:25:16</stop_time>
        </runs>
        <uid>stoller</uid>
      </instance>
      2cfe4630