1. 16 Feb, 2007 1 commit
  2. 15 Feb, 2007 1 commit
  3. 12 Feb, 2007 2 commits
    • Leigh Stoller's avatar
      Minor bug fix; show bindings only once! · 790084f6
      Leigh Stoller authored
      790084f6
    • Leigh Stoller's avatar
      * Replace the argument processing code in all pages. Currently we rely on · 48acc8e3
      Leigh Stoller authored
        register_globals=1 to turn POST/GET/COOKIES arguments in local variables.
        This is known to be a terrible security risk, and we keep saying we are
        going to fix it, and now I am. In order to accomplish this on a
        transitional basis (since I don't want the entire web interface to stop
        working while I debug it), and because the code just needs the cleanup, I
        am doing it like this: Each page will sport new declarations at the top:
      
      	RequiredPageArguments("experiment", PAGEARG_EXPERIMENT,
                                    "template",   PAGEARG_TEMPLATE,
                                    "instance",   PAGEARG_INSTANCE,
                                    "metadata",   PAGEARG_METADATA,
                                    "osinfo",     PAGEARG_OSINFO,
                                    "image",      PAGEARG_IMAGE,
                                    "project",    PAGEARG_PROJECT,
                                    "group",      PAGEARG_GROUP,
                                    "user",       PAGEARG_USER,
      			      "node",       PAGEARG_NODE,
      			      "yesno",      PAGEARG_BOOLEAN,
      			      "message",    PAGEARG_STRING,
      			      "age",        PAGEARG_INTEGER,
                                    "cost",       PAGEARG_NUMERIC,
                                    "formfields", PAGEARG_ARRAY,
                                    "unknown",    PAGEARG_ANYTHING);
      
      	OptionalPageArguments("canceled", PAGEARG_BOOLEAN);
      
        The first token in each pair is the name of the global variable to
        set, and the second token is the type. So, for "experiment" we look at
        the URL for a pid/eid or exptidx, etc, sanity check them (safe for a
        DB query), and then try to find that experiment in the DB. If it maps
        to an experiment, set global variable $experiment to the object. Since
        its a required argument, produce an error if not supplied. Similar
        treatment for optional arguments, with the obvious difference.
      
        The goal is to have ALL argument processing in one place, consistent,
        and correct. I've found numerous places where we leak unchecked
        arguments into queries. It also cuts out a lot of duplicated code.
      
      * To make the above easier to deal with, I've been replacing lots of
        hardcoded URLS in the code of the form:
      
      	foo.php3?pid=$pid&eid=$eid ...
      
        with
      
              CreateURL("foo", $experiment)
      
        which creates and returns the neccessary url string, by looking at
        the type of its arguments (experiment, template, instance, etc.)
      
        Eventually plan to replace them all so that URL handling throughout
        the code is all defined in one place (all the new URL code is in
        url_defs.php).
      
      * I have cranked up error reporting to tell me anytime a variable is
        used before it is initialized, plus a bunch of other stuff that PHP
        deems improper. Think of it like -Wall ... and boy we get a lot of
        warnings.  A very large percentage of the diffs are to fix all these
        warnings.
      
        The warnings are currently going to /usr/testbed/log/php-errors.log,
        and I'll be adding a script to capture them each night and mail them
        to tbops. This file also gets errors (this will be a change for
        developers; rather then seeing errors and warnings dumped in the
        middle of web pages, they will go to this file instead).
      
      * Major refactoring of the code. More objects (nodes, images, osids).
        Moving tons of queries into the objects in the hopes of someday
        getting to a point where we can split the web interface onto a
        different server.  Lots of general cleanup.
      48acc8e3
  4. 18 Jan, 2007 1 commit
  5. 20 Dec, 2006 1 commit
  6. 05 Dec, 2006 1 commit
    • Kevin Atkinson's avatar
      · 784c702e
      Kevin Atkinson authored
      When a run is over changed:
        Template: 10424/23 (Current Run: bt-R54)
      to
        Template: 10424/23 (Last Run: bt-R54)
      Rather Than displaying
        Template: 10424/23 (Current Run:)
      
      Also changed "Instance Bindings" to "Last Run Bindings" since that makes
      much more sense, at least to me. (FS # 128:
      https://users.emulab.net/flyspray/?do=details&id=128)
      The function to show Instance Bindings still exists so it can easily be
      changed back.
      784c702e
  7. 09 Nov, 2006 1 commit
  8. 25 Oct, 2006 1 commit
  9. 18 Oct, 2006 1 commit
  10. 09 Oct, 2006 1 commit
  11. 08 Oct, 2006 1 commit
  12. 05 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      More work on "recording" template events. · e9607a77
      Leigh Stoller authored
      * New version of template_record just for ops, since so much is
        different about ops, not bothering to maintain a single version.
      
      * Various fixes to how the recorded events are stored and reconstituted.
        The big fix is to wrap them in a sequence to that they get fired
        properly (waiting for completion of previous event in recording).
      
      * New buttons to Pause and Continue event time, which is used when
        adding recorded events. This allows users to pause time while they
        "think" so when an event is recorded, the thinking time is not actually
        in the timeline. Eventually hope to figure this out automatically, but
        that will take some real, uh, thinking.
      
      * Add a new event editor (linked off the template page) that allows
        you to delete and change the recordings. Note that you can only edit
        the events at the template level; you cannot edit the events of an
        instance (swapped in experiment), and you can only edit the recorded
        events, not any other events. Not sure its useful to be able to do
        either of these yet, but probably not too hard to add at some point.
      e9607a77
  13. 13 Sep, 2006 1 commit
  14. 05 Sep, 2006 1 commit
    • Leigh Stoller's avatar
      A bunch of template changes resulting from meetings last week. · 087dbfff
      Leigh Stoller authored
      * Add XMLRPC interface for template swapin,stoprun,startrun,swapout and
        add the appropriate wrappers to the script_wrapper on ops.
      
      * Allow parameter descriptions in NS files. This is probably not in its
        final form since its a bit confusing as to what has priority; something
        in the NS file or a metadata item. Anyway, you can do this in your NS
        file:
      
      	$ns define-template-parameter GUID "0/0" "The GUID to be analyzed"
      
        The rules are currently that the NS file description has priority and
        is copied to child templates, unless the user has modified a description
        via the web interface, in which case the NS file description is ignored.
        I know, sounds awful, but for the most part people are going to use the
        NS file anyway.
      
      * Add "clear" option when starting a new experiment run; the per
        experiment DB at the logholes are cleared. Note that this is *not* the
        default behaviour; you have to either check the checkbox on the web form
        or use the -c option to the script wrapper, or clear=yes if talking
        directly to the XMLRPC server.
      
      * Fix up how email is generated for template_swapin and template_create,
        so that Kevin can debug tblog/tbreport stuff, but also so that we maintain
        mail logs as before. I have made some improvements to libaudit so as to
        centralize the mail goo, and avoid duplicating all that stuff.
      
      * Minor fixes to the program agent so that the new environment strings are
        sent before the program agent exits and reloads them!
      
      * Other minor little things.
      087dbfff
  15. 14 Aug, 2006 1 commit
    • Leigh Stoller's avatar
      Checkpoint my dynamic event stuff, crude as it is. The idea for this first · 9d021a07
      Leigh Stoller authored
      draft is that the user will at the end of an experiment run, log into one
      of his nodes and perform some analysis which is intended to be repeated at
      the end of the next run, and in future instantiations of the template.
      
      A new table called experiment_template_events holds the dynamic events for
      the template. Right now I am supporting just program events, but it will be
      easy to support arbitrary events later. As an absurd example:
      
      	node6> /usr/local/bin/template_analyze ~/data_analyze arg arg ...
      
      The user is currently responsible for making sure the output goes into a
      file in the archive. I plan to make the template_analyze wrapper handle
      that automatically later, but for now what you really want is to invoke a
      script that encapsulates that, redirecting output to $ARCHIVE (this
      variable is installed in the environment template_analyze.
      
      The wrapper script will save the current time, and then run the program.
      If the program terminates with a zero exit status, it will ssh over to ops
      and invoke an xmlrpc routine to tell boss to add a program event to both
      the eventlist for the current instance, and to the template_eventlist for
      future instances. The time of the event is the relative start time that was
      saved above (remember, each experiment run replays the event stream from
      time zero).
      
      For the future, we want to allow this to be done on ops as well, but
      that will take more infrastructure, to run "program agents" on ops.
      
      It would be nice to install the ssl xmlrpc client side on our images so
      that we do not have to ssh to ops to invoke the client.
      9d021a07
  16. 08 Aug, 2006 1 commit
  17. 07 Aug, 2006 1 commit
  18. 03 Aug, 2006 1 commit
    • Leigh Stoller's avatar
      Support for capturing the trace data that is stored in the pcal files · 4ce9c421
      Leigh Stoller authored
      into per-experiment databases on ops. Additional support for reconsituting
      those databases back into temporary databases on ops, for post processing.
      
      * This revision relies on the "snort" port (/usr/ports/security/snort)
        to read the pcap files and load them into a database. The schema is
        probably not ideal, but its better then nothing. See the file
        ops:/usr/local/share/examples/snort/create_mysql for the schema.
      
      * For simplicity, I have hooked into loghole, which already had all
        the code for downloading the trace data. I added some new methods to
        the XMLRPC server for loghole to use, to get the users DB password
        and the name of the per-experiment database. There is a new slot in
        the traces table that indicates that the trace should be snorted to
        its DB. In case you forgot, at the end of a run or when the instance
        is swapped out, loghole is run to download the trace data.
      
      * For reconsituting, there are lots of additions to opsdb_control and
        opsdb_control.proxy to create "temporary" databases and load them
        from a dump file that is stored in the archive. I've added a button
        to the Template Record page, inappropriately called "Analyze" since
        right now all it does is reconsitute the trace data into a DB on
        ops.
      
        Currently, the only indication of what has been done (the name of
        the DBs created on ops) is the log email that the user gets. A
        future project is tell the user this info in the web interface.
      
      * To turn on database capturing of trace data, do this in your NS
        file:
      
      	set link0 ...
      	$link0 trace
      	$link0 trace_snaplen 128
      	$link0 trace_db 1
      
         the increase in snaplen is optional, but a good idea if you want
         snort to undertand more then just ip headers.
      
      * Also some changes to the parser to allow plain experiments to take
        advantage of all this stuff. To simple get yourself a per-experiment
        DB, put this in your NS file:
      
      	tb-set-dpdb 1
      
        however, anytime you turn trace_db on for a link or lan, you
        automatically get a per-experiment DB.
      
      * To capture the trace data to the DB, you can run loghole by hand:
      
      	loghole sync -s
      
        the -s option turns on the "post-process" phase of loghole.
      4ce9c421
  19. 28 Jul, 2006 1 commit
    • Leigh Stoller's avatar
      Add a "Create Template from Instance" ability. Basically, you can · a651da71
      Leigh Stoller authored
      create a new template (well, really a modify) from the current
      swapped in experiment. This allows you to create a template, swap in
      an instance, modify the datastore in the instance (which is a copy
      of the datastore in the template), and then create a new template
      using the datastore and nsfile from the instance. This is a new menu
      item on the showexp page for the instance.
      
      Also in this commit are fixes and improvements to the new navagation
      bar that I recently added.
      a651da71
  20. 26 Jul, 2006 1 commit
  21. 20 Jul, 2006 2 commits
    • Leigh Stoller's avatar
      Minor tweak to previous revision. · b1bdca1a
      Leigh Stoller authored
      b1bdca1a
    • Leigh Stoller's avatar
      This start out as: · 5dfa4a47
      Leigh Stoller authored
      * add an "active" flag to the template record, which will be used by
        the user to indicate what templates he wants listed (rather then the
        roots). Basically, the current working templates, rather then a big
        graph.
      
      But I never actually finished that cause it sorta morphed into:
      
      * Added a vis_graphs table to cache the last generated visualization
        rendering in the database so that we do not have to wait so long for:
      
      * Add new buttons to showexp and template_show pages, to display in the
        same page either the settings (current view), the NS file, or the
        visualization (along with zoom in/out buttons).
      
      And now I can go back to that "active" thing I mentioned up above ...
      5dfa4a47
  22. 10 Jul, 2006 1 commit
  23. 24 May, 2006 1 commit
  24. 16 May, 2006 1 commit
  25. 12 May, 2006 1 commit
    • Leigh Stoller's avatar
      Redo the entire template library. I've been meaning to use perl · 78503406
      Leigh Stoller authored
      "object" and this was a good opportunity to see if they are useful and
      easy enough to use. Yep they are; the code is much cleaner with many
      fewer utility functions to get at stuff. I recommend this approach
      from now on.
      
      The problem is the php side, which ends up duplicating some stuff, but
      in the old style. This is not so bad for the template code since I
      have made it a point not to do anything but display functions in php;
      all modifications are handled in the backend.
      78503406
  26. 05 May, 2006 1 commit
  27. 31 Mar, 2006 1 commit
    • Leigh Stoller's avatar
      More experiment run stuff. Also the NS file parsing support that was · 74bf8d1d
      Leigh Stoller authored
      needed (reparsing of NS file with current bindings).
      
      Mostly I decided that I made a huge mistake in using the exptidx as the
      main cross index. I started to change that, which took a bunch of time.
      Still more tables to fix up, but I think all of this code is gonna need
      a major reorg once I am no longer under the gun to get it done.
      74bf8d1d
  28. 30 Mar, 2006 1 commit
  29. 29 Mar, 2006 1 commit
  30. 28 Mar, 2006 1 commit
  31. 21 Mar, 2006 1 commit
    • Kevin Atkinson's avatar
      Changed format of email sent to user on errors. The error will now · d258dde6
      Kevin Atkinson authored
      appear instead of the generic message when I am confident it is
      accurate.  The subject line will also change to reflect the cause of
      an error.
      
      Avoid sending mail to testbed-ops during failed swap related evenets
      in some cases.  It will instead be sent to a new mailing list
      testbed-errors.
      
      Added a new row in the experiment info table "Last Error:" which
      states the cause of the error, and links to a new page displaying the
      error.
      
      Made some assign/assign_wrapper errors more informative.
      
      The error (as determined by tblog) is now stored in the database in a
      more structured fashion.  This inlcudes adding a column for the session
      (in the log table) to testbed_stats to link eash swap event with the
      logs and possible the error.
      
      Other changes to the database, see sql/database-migrate.txt
      d258dde6
  32. 13 Feb, 2006 1 commit
  33. 05 Jan, 2006 2 commits
  34. 27 Dec, 2005 1 commit
  35. 15 Dec, 2005 1 commit
  36. 30 Nov, 2005 1 commit
  37. 17 Nov, 2005 1 commit