1. 25 Feb, 2004 1 commit
    • Kirk Webb's avatar
      Kirk takes the weed whacker to the plab code. This is the first pass result. · ae2eec76
      Kirk Webb authored
      I'll come along for a closer cut in the future.
      
      * Modularized the plab communications 'adaptor' interface and moved the
        dslice- and PLC-specific code into their own modules.
      
      * Wrote an API definition README
      
      * Separated out generic routines from libplab into their own library modules
        (libtestbed.py and libdb.py)
      
      Functionally, not much has changed - this was just a massive re-org with some
      other cleanup.  Should be much easier to code up new PLAB interfaces as the
      plab folks flail around in their attempt to standardize on something.
      
      XXX: may want to re-think where the generic library modules should go.  If
      more python code enters Elab, we'll probably want to move 'em to more standard
      locations.
      
      This isn't the end of the cleanup - I would eventually like to go back and
      rethink the class structures, beef up the comments, and extend the API.
      ae2eec76
  2. 23 Feb, 2004 1 commit
  3. 16 Feb, 2004 2 commits
  4. 11 Feb, 2004 3 commits
  5. 10 Feb, 2004 1 commit
  6. 09 Feb, 2004 1 commit
  7. 07 Feb, 2004 1 commit
  8. 06 Feb, 2004 1 commit
  9. 03 Feb, 2004 1 commit
  10. 29 Jan, 2004 1 commit
  11. 16 Jan, 2004 4 commits
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      Restore Mac's fix to the fieldcheck code, but with additional fixes so · a81eeeb4
      Leigh B. Stoller authored
      that it does the right thing. Existing bug, but was uncovered by the
      previous cleanups.
      a81eeeb4
    • Leigh B. Stoller's avatar
      Back part of Mac's last revision since it breaks the parser. · 56139163
      Leigh B. Stoller authored
      No idea, just making it work.
      56139163
    • Mac Newbold's avatar
      libdb changes: · 78ad260c
      Mac Newbold authored
      - add functions to recursively dump hashes and arrays into a string
        suitable for printing as debugging output (great for data structures)
      
      - add three new trigger strings
      
      - add 'use strict', do corresponding cleanup
      
      stated changes:
      
      - move special-cased stuff in handleEvent for PXEBOOTING and BOOTING into
        triggers (PXEBOOTING, BOOTING, and CHECKGENISUP)
      
      - clarify (via comments) the existing kinds of triggers and which ones run
        when, and add a new kind (global "any-mode" triggers). We already had
        per-node mode-specific, per-node any-mode, and global mode-specific
        triggers. Now you can have a trigger that is good for any mode in a
        given state, that can be overridden on a mode-specific basis. This is
        great for PXEBOOTING, BOOTING, and ISUP, since they each have a trigger
        list that should be run regardless of what mode you're in. Now they only
        require 3 entries instead of 3*N that have to be maintained per mode.
      
           # A note about triggers:
           #
           # "per-node" triggers only affect their specific node in a
           # particular mode/state, and are run first of all. "global"
           # triggers are triggers for a given mode/state that affect all
           # nodes, and are run after any per-node triggers. "Any-mode"
           # triggers are tied to a state, and occur in that state in any
           # mode. The any-mode triggers are over-ridden by global triggers,
           # and if an "Any-mode" trigger for state XYZ exists as well as a
           # global trigger for mode FOOBAR state XYZ, then when I arrive in
           # XYZ any per-node triggers will be run. Then, if I'm in mode
           # FOOBAR, only the global trigger will run. If I'm in any other
           # mode, only the any-mode trigger will run.
      
           # (our "*" is stored as $TBANYMODE)
           # Per-node triggers have a specific node_id
           # Global triggers have "*" as the node_id
           # Any-mode triggers have "*" as the mode, and can be global or per-node
      
        The updated table looks like this in the accompanying change to
        database-fill.sql:
      
      +---------+----------+------------+-----------------------+
      | node_id | op_mode  | state      | trigger               |
      +---------+----------+------------+-----------------------+
      | *       | *        | BOOTING    | BOOTING, CHECKGENISUP |
      | *       | *        | ISUP       | RESET                 |
      | *       | *        | PXEBOOTING | PXEBOOT               |
      | *       | RELOAD   | RELOADDONE | RESET, RELOADDONE     |
      | *       | ALWAYSUP | SHUTDOWN   | ISUP                  |
      +---------+----------+------------+-----------------------+
      
      - I also cleaned up the functions that add, get, and delete triggers.
        Before, the get function didn't include global triggers. Now it does,
        and has an option to just get the per-node triggers. Add and delete are
        still just per-node, of course.
      
      - Also found and fixed some little bugs while I was in there. (global
        triggers not taking a list,
      
      These changes are me getting ready to re-add all the changes I made months
      ago in order to do a before-and-after experiment for my thesis. Between
      now and the end of next week I'll be working on taking before numbers,
      patching stated with the changes, and getting after numbers.
      
      The problems I'm trying to replicate are the problems and slowdowns we
      used to get when os_{load,setup} would reboot a node, thinking it had
      timed out, when it really didn't know whether it was making progress or
      not. The fix includes making os_{load,setup} depend on stated to watch for
      progress and timeouts, and do any appropriate retries. Part of that is the
      StateWait stuff, that lets programs watch for events easily, and the
      node_reboot-with-events stuff that puts stated in control of nodes as they
      reboot.
      78ad260c
  12. 15 Jan, 2004 1 commit
  13. 13 Jan, 2004 1 commit
  14. 12 Jan, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add more node state machine constants. · 899be6af
      Leigh B. Stoller authored
      Add constants for the osids describing the FreeBSD and Frisbee MFSs.
      Complete redo of TBBooWhat to match the changes in bootinfo. Look
      there for description of new boot protocol (how TBBooWhat now works).
      899be6af
  15. 09 Jan, 2004 1 commit
    • Shashi Guruprasad's avatar
      During a simparse, virt_agents and eventlist need to be cleaned up · 467cb150
      Shashi Guruprasad authored
      but not other virt_* tables. Thats coz the above two tables are not
      truly virtual since they contain the vnode to which the event should
      be sent to. My previous patch has been to use replace instead of
      insert. Unfortunately, the tables get messed up with the same agent
      having multiple entries where some of the entries were left behind
      from the previous swapin.
      467cb150
  16. 31 Dec, 2003 2 commits
  17. 30 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      If xmlconvert is called after an nseparse, we need to use replace coz · 08a40103
      Shashi Guruprasad authored
      some of the tables such as virt_agents and eventlist are not truly
      virtual tables. That is coz they contain the vnode field which is
      the same as the vname field in the reserved table. For simulated
      nodes, the mapping may change across swapins and the event may have
      to be delivered to a different simhost. Of course this problem
      isn't there for jail vnodes or pc vnodes.
      08a40103
  18. 28 Dec, 2003 3 commits
  19. 25 Dec, 2003 1 commit
  20. 23 Dec, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add new state PXEWAIT to be explained later ... · 22ee2594
      Leigh B. Stoller authored
      Change ptopgen to look at the eventstate of a node; a node is not considered
      free unless it is ISUP or PXEWAIT.
      
      Add TBAvailablePCs() to libdb, removing the corresponding code from
      assign_wrapper. This routine does the equiv of ptopgen, returning the
      number of PCs that are available for use (looking at eventstate).
      
      Change TBFreePCs in the web interface accordingly.
      
      The above changes correspond to an upcoming change in stated.
      22ee2594
  21. 21 Dec, 2003 1 commit
  22. 18 Dec, 2003 1 commit
    • Leigh B. Stoller's avatar
      First try at solving the problem of validating user input for the · 8dbead16
      Leigh B. Stoller authored
      zillions of DB fields that we have to set. My solution was to add a
      meta table that describes what is a legal value for each table/slot
      for which we take from user input. The table looks like this right
      now, but is likely to adapt as we get more experience with this
      approach (or it might get tossed if it turns out to be a pain in the
      ass!).
      
      	CREATE TABLE table_regex (
      	  table_name varchar(64) NOT NULL default '',
      	  column_name varchar(64) NOT NULL default '',
      	  column_type enum('text','int','float') default NULL,
      	  check_type enum('regex','function','redirect') default NULL,
      	  check tinytext NOT NULL,
      	  min int(11) NOT NULL default '0',
      	  max int(11) NOT NULL default '0',
      	  comment tinytext,
      	  UNIQUE KEY table_name (table_name,column_name)
      	) TYPE=MyISAM;
      
      Entries in this table look like this:
      
      	('virt_nodes','vname','text','regex','^[-\\w]+$',1,32,NULL);
      
      Which says that the vname slot of the virt_nodes table (which we trust the
      user to give us in some form) is a text field to be checked with the given
      regex (perlre of course), and that the min/max length of the text field is
      1 and 32 chars respectively.
      
      Now, you wouldn't want to write the same regex over and over, and since we
      use the same fields in many tables (like pid, eid, vname, etc) there is an
      option to redirect to another entry (recursively). So, for "PID" I do this:
      
              ('eventlist','pid','text','redirect','projects:pid',0,0,NULL);
      
      which redirects to:
      
      	('projects','pid','text','regex','^[a-zA-Z][-\\w]+$',2,12,NULL);
      
      And, for many fields you just want to describe generically what could go
      into it. For that I have defined some default fields. For example, a user
      description:
      
              ('experiment,'usr_name','text','redirect','default:tinytext',0,0,NULL);
      
      which redirects to:
      
      	('default','tinytext','text','regex','^[\\040-\\176]*$',0,256,NULL);
      
      and this says that a tinytext (in our little corner of the database
      universe) field can have printable characters (but not a newline), and
      since its a tinytext field, its maxlen is 256 chars.
      
      You also have integer fields, but these are little more irksome in the
      details.
      
      	('default','tinyint,'int,'regex','^[\\d]+$',-128,127,NULL);
      
      and you would use this anyplace you do not care about the min/max values
      being something specific in the tinyint range. The range for a float is of
      course stated as an integer, and thats kinda bogus, but we do not have many
      floats, and they generally do not take on specific values anyway.
      
      A note about the min/max fields and redirecting. If the initial entry has
      non-zero min/max fields, those are the min mac fields used. Otherwise they
      come from the default. So for example, you can do this:
      
          ('experiments','mem_usage','int','redirect','default:tinyint',0,5,NULL);
      
      So, you can redirect to the standard "tinyint" regular expression, but you
      still get to define min/max for the specific field.
      
      Isn't this is really neat and really obtuse too? Sure, you can say it.
      
      Anyway, xmlconvert now sends all of its input through these checks (its
      all wrapped up in library calls), and if a slot does not have an entry, it
      throws an error so that we are forced to define entries for new slots as we
      add them.
      
      In the web page, I have changed all of the public pages (login, join
      project, new project, and a couple of others) to also use these checks.
      As with the perl code, its all wrapped up in a library. Lots more code
      needs to be changed of course, but this is a start.
      8dbead16
  23. 17 Dec, 2003 1 commit
  24. 16 Dec, 2003 2 commits
    • Robert Ricci's avatar
      e979e409
    • Shashi Guruprasad's avatar
      Distributed NSE changes. In other words, simulation resources are · d266bd71
      Shashi Guruprasad authored
      now mapped to more than one PC if required. The simnode_capacity
      column in the node_types table determines how many sim nodes can
      be packed on one PC. The packing factor can also be controlled via
      tb-set-colocate-factor to be smaller than simnode_capacity.
      
      - No frontend code changes. To summarize:
        $ns make-simulated {
          ...
        }
        is still the easy way to put a whole bunch of Tcl code to be
        in simulation.
        One unrelated fix in the frontend code is to fix the
        xmlencode() function which prior to this would knock off
        newlines from columns in the XML output. This affected
        nseconfigs since it is one of the few columns with embedded
        newlines. Also changed the event type and event object type
        in traffic.tcl from TRAFGEN/MODIFY to NSE/NSEEVENT.
      
      - More Tcl code in a new directory tbsetup/nseparse
        -> Runs on ops similar to the main parser. This is invoked
           from assign_wrapper in the end if there are simnodes
        -> Partitions the Tcl code into multiple Tcl specifications
           and updates the nseconfigs table via xmlconvert
        -> Comes with a lot of caveats. Arbitrary Tcl code such as user
           specified objects or procedures will not be re-generated. For
           example, if a user wanted a procedure to be included in Tcl
           code for all partitions, there is no way for code in nseparse
           to do that. Besides that, it needs to be tested more thoroughly.
      
      - xmlconvert has a new option -s. When invoked with this option,
        the experiments table is not allowed to be modified. Also,
        virtual tables are just updated (as opposed to deleting
        all rows in the first invocation before inserting new rows)
      
      - nse.patch has all the IP address related changes committed in
        iversion 1.11 + 2 other changes. 1) MTU discovery support in
        the ICMP agent 2) "$ns rlink" mechanism for sim node to real
        node links
      
      - nseinput.tcl includes several client side changes to add IP
        routes in NSE and the kernel routing table for packets crossing
        pnodes. Also made the parsing of tmcc command output more robust
        to new changes. Other client side changes in libsetup.pm and other
        scripts to run nse, are also in this commit
      
      - Besides the expected changes in assign_wrapper for simulated nodes,
        the interfaces and veth_interfaces tables are updated with
        routing table identifiers (rtabid). The tmcd changes are already
        committed. This field is used only by sim hosts on the client side.
        Of course, they can be used by jails as well if desired.
      d266bd71
  25. 15 Dec, 2003 2 commits
  26. 11 Dec, 2003 2 commits
  27. 10 Dec, 2003 1 commit
  28. 08 Dec, 2003 1 commit