1. 15 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      Distributed NSE changes. In other words, simulation resources are · d266bd71
      Shashi Guruprasad authored
      now mapped to more than one PC if required. The simnode_capacity
      column in the node_types table determines how many sim nodes can
      be packed on one PC. The packing factor can also be controlled via
      tb-set-colocate-factor to be smaller than simnode_capacity.
      
      - No frontend code changes. To summarize:
        $ns make-simulated {
          ...
        }
        is still the easy way to put a whole bunch of Tcl code to be
        in simulation.
        One unrelated fix in the frontend code is to fix the
        xmlencode() function which prior to this would knock off
        newlines from columns in the XML output. This affected
        nseconfigs since it is one of the few columns with embedded
        newlines. Also changed the event type and event object type
        in traffic.tcl from TRAFGEN/MODIFY to NSE/NSEEVENT.
      
      - More Tcl code in a new directory tbsetup/nseparse
        -> Runs on ops similar to the main parser. This is invoked
           from assign_wrapper in the end if there are simnodes
        -> Partitions the Tcl code into multiple Tcl specifications
           and updates the nseconfigs table via xmlconvert
        -> Comes with a lot of caveats. Arbitrary Tcl code such as user
           specified objects or procedures will not be re-generated. For
           example, if a user wanted a procedure to be included in Tcl
           code for all partitions, there is no way for code in nseparse
           to do that. Besides that, it needs to be tested more thoroughly.
      
      - xmlconvert has a new option -s. When invoked with this option,
        the experiments table is not allowed to be modified. Also,
        virtual tables are just updated (as opposed to deleting
        all rows in the first invocation before inserting new rows)
      
      - nse.patch has all the IP address related changes committed in
        iversion 1.11 + 2 other changes. 1) MTU discovery support in
        the ICMP agent 2) "$ns rlink" mechanism for sim node to real
        node links
      
      - nseinput.tcl includes several client side changes to add IP
        routes in NSE and the kernel routing table for packets crossing
        pnodes. Also made the parsing of tmcc command output more robust
        to new changes. Other client side changes in libsetup.pm and other
        scripts to run nse, are also in this commit
      
      - Besides the expected changes in assign_wrapper for simulated nodes,
        the interfaces and veth_interfaces tables are updated with
        routing table identifiers (rtabid). The tmcd changes are already
        committed. This field is used only by sim hosts on the client side.
        Of course, they can be used by jails as well if desired.
      d266bd71
  2. 10 Dec, 2003 3 commits
  3. 03 Dec, 2003 1 commit
  4. 01 Dec, 2003 2 commits
    • Leigh B. Stoller's avatar
      If the user does not specify a sync server, do it for them by picking · 581c0389
      Leigh B. Stoller authored
      the first node in the list. Useful for the linktest stuff, and the
      sync server is harmless.
      581c0389
    • Robert Ricci's avatar
      New scripts: tarfiles_setup, fetchtar.proxy, and webtarfiles_setup . · c0c6547c
      Robert Ricci authored
      The idea is to give us hooks for grabbing experimenters' tarballs (and
      RPMs) from locations other than files on ops. Mainly, to remove
      another dependance on users having shells on ops.
      
      tarfiles_setup supports fetching files from http and ftp URLs right
      now, through wget. It places them into the experiment directory, so
      that they'll go away when the experiment is terminated, and the rest
      of the chain (ie. downloading to clients and os_setup's checks)
      remains unchaged.  It is now tarfiles_setup's job to copy tarballs and
      RPMs from the virt_nodes table to the nodes table for allocated nodes.
      This way, it can translate URLs into the local filenames it
      constructs. It get invoked from tbswap.
      
      Does the actual fetching over on ops, running as the user, with
      fetchtar.proxy.
      
      Should be idempotent, so we should be able to give the user a button
      to run webtarfiles_setup (none exists yet) yet to 'freshen' their
      tarballs. (We'd also have to somehow let the experiment's nodes know
      they need to re-fetch their tarballs.)
      
      One funny side effect of this is that the separator in
      virt_nodes.tarfiles is now ';' instead of ':' like nodes.tarballs,
      since we can now put URLs in the former. Making these consistent is a
      project for another day.
      c0c6547c
  5. 20 Nov, 2003 1 commit
  6. 16 Nov, 2003 1 commit
  7. 12 Nov, 2003 1 commit
    • Leigh B. Stoller's avatar
      Changes/Fixes to allow Shashi to spit NS files into the DB from the · fb2a8122
      Leigh B. Stoller authored
      parser:
      
      * Change xmlconvert to use XML::Parser instead of the ad-hoc stuff I
        wrote. This was easier then trying to deal with multiline fields or
        all the special chars. XML::Parser does all that for me. Also make
        sure all fields are either properly quoted (with DBQuoteMeta) or
        regex'ed before we stick them into the DB. For the future, we need
        to add table/slot specific regular expressions.
      
      * Also add Shashi's fixes to the parser; properly escape XML special
        characters, as well as cleanup of my original spitxml code.
      fb2a8122
  8. 05 Nov, 2003 1 commit
    • Leigh B. Stoller's avatar
      Frontend and parser portion of two event system changes: · 091a0b62
      Leigh B. Stoller authored
      * Generate a shared secret key for the event system. This key is
        stored into the DB, and passed to the node via tmcd. It is also
        stashed into a file in the experiment directory (can be accessed
        only by the project/group members). The key is used to attach a
        HMAC (hashed message authentication) to each event, which is checked
        by the receivers to ensure that the event is not bogus. More details
        on this later when I commit the event library/client changes.
      
      * Added "virt_programs" table to store info about each program object
        defined by the user. The intent is to no longer send the command
        string in the event, but to fix it in the DB, and transfer it via
        tmcd. This removes our "remote execution facility" which was always
        a bad idea (we have ssh for that, and that is a lot more secure then
        the event system!).
      
        Note that for the time being we need to continue send the command in
        the event because of old images, but the new images will now ignore
        that part of the event.
      091a0b62
  9. 18 Oct, 2003 1 commit
  10. 02 Oct, 2003 3 commits
  11. 26 Sep, 2003 2 commits
    • Robert Ricci's avatar
      New for subnodes: the host of a subnode now automatically gets a · 4e11b844
      Robert Ricci authored
      desire for 'hosts-<type>', where <type> is the type of its child node.
      This helps assign, because it can now limit the number of places to
      try assigning the host, and it means that we can give the hosts this
      feature, so that they don't get used for other purposes. For example,
      we can give the IXP-hosting nodes the feature 'hosts-ixp-bv' with
      weight 1, and they will never get used for anything but IXP hosting.
      
      This means that the node_type_features (or just node_features) table
      must now have hosts-<type> entires to work correctly.
      4e11b844
    • Robert Ricci's avatar
      Support for vnode desires in the frontend. In an NS file, you can now · 2a23edce
      Robert Ricci authored
      put:
      $node add-desire desire weight
      
      This will end up in the virt_node_desires table. assign_wrapper now
      puts the desires from this table into the top file.
      2a23edce
  12. 18 Sep, 2003 1 commit
  13. 17 Sep, 2003 1 commit
  14. 15 Sep, 2003 2 commits
  15. 11 Sep, 2003 3 commits
  16. 04 Sep, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add variable netmask support to the parser. You can now do this in · 7d524fde
      Leigh B. Stoller authored
      your NS file:
      
      	tb-set-netmask $lan0  "255.255.240.0"
      	tb-set-netmask $link0 "255.255.255.248"
      	tb-set-netmask $link1 "255.255.255.240"
      
      Yep, more rope for the user to hang herself with. Notes:
      
      * You are restricted to 255.255.XXX.XXX. I did not see a reason to
        allow the user that much rope.
      
      * get_subnet can handle 10 or 192.168 addresses so that other sites
        can continue to operate without changing to 10 addresses, although
        they will still be able to change the netmask.
      
      * I've changed the handling for widearea networks to use 192.168, but
        I force the netmask to 255.255.255.248 so that we are not restricted
        to just 255 networks (not that it really matters). To avoid possible
        confusion, the user is not allowed to choose their own IPs for
        widearea networks, and I actually set them to 1.1.x.x, and then
        patch it up later. This is to avoid conflict with existing
        experiments where people may have used tb-set-ip in their NS files.
      
      * There are tmcd and staticroutes and image changes that are required
        to make this all work right!
      7d524fde
  17. 12 Aug, 2003 1 commit
  18. 09 Aug, 2003 1 commit
    • Leigh B. Stoller's avatar
      Support for subnodes: · d053d84e
      Leigh B. Stoller authored
      	set myixp [$ns node]
      	set ixphost [$ns node]
      	tb-set-hardware $myixp ixp-bveil
      	tb-fix-node $myixp $ixphost
      	tb-set-node-os $ixphost RHL-STD
      
      This gives you a handle on the host node so you can treat it like a
      normal testbed node, including setting the OSID.
      
      There are associated assign_wrapper changes, but I won't check those
      in till I get back from vacation; not enough people around this coming
      week in case I screwed something up.
      d053d84e
  19. 07 Aug, 2003 1 commit
  20. 06 Aug, 2003 1 commit
    • Leigh B. Stoller's avatar
      Two new functions: · 27cc5adc
      Leigh B. Stoller authored
      * A node method to create a startup routine:
      
      	$node start-command "$mydir/batchcmd.sh >& $mydir/b1.log"
      
      * And some old style syntactic pepper:
      
      	tb-set-node-startcommand $nodeb1 "$mydir/batchcmd.sh >& $mydir/b1.log"
      
      The latter is intended to replace the exiting tb-set-node-startup
      function. The new implementation of the above is to use a program
      object that starts at time 0.
      27cc5adc
  21. 05 Aug, 2003 1 commit
    • Leigh B. Stoller's avatar
      The rest of the sync server additions: · 212cc781
      Leigh B. Stoller authored
      * Parser: Added new tb command to set the name of the sync server:
      
      	tb-set-sync-server <node>
      
        This initializes the sync_server slot of the experiment entry to the
        *vname* of the node that should run the sync server for that
        experiment. In other words, the sync server is per-experiment, runs
        on a node in the experiment, and the user gets to chose which node
        it runs on.
      
      * tmcd and client side setup. Added new syncserver command which
        returns the name of the syncserver and whether the requesting node
        is the lucky one to run the daemon:
      
          SYNCSERVER SERVER='nodeG.syncserver.testbed.emulab.net' ISSERVER=1
      
        The name of the syncserver is written to /var/emulab/boot/syncserver
        on the nodes so that clients can easily figure out where the server
        is.
      
        Aside: The ready bits are now ignored (no DB accesses are made) for
        virtual nodes; they are forced to use the new sync server.
      
      * New os/syncd directory containing the daemon and the client. The
        daemon is pretty simple. It waits for TCP (and UDP, although that
        path is not complete yet) connections, and reads in a little
        structure that gives the name of the "barrier" to wait for, and an
        optional count of clients in the group (this would be used by the
        "master" who initializes barriers for clients). The socket is saved
        (no reply is made, so the client is blocked) until the count reaches
        zero. Then all clients are released by writting back to the
        sockets, and the sockets are closed. Obviously, the number of
        clients is limited by the numbed of FDs (open sockets), hence the
        need for a UDP variant, but that will take more work.
      
        The client has a simple command line interface:
      
          usage: emulab-sync [options]
          -n <name>         Optional barrier name; must be less than 64 bytes long
          -d                Turn on debugging
          -s server         Specify a sync server to connect to
          -p portnum        Specify a port number to connect to
          -i count          Initialize named barrier to count waiters
          -u                Use UDP instead of TCP
      
          The client figures out the server by looking for the file created
          above by libsetup (/var/emulab/boot/syncserver). If you do not
          specify a barrier "name", it uses an internal default. Yes, the
          server can handle multiple barriers (differently named of course)
          at once (non-overlapping clients obviously).
      
          Clients can wait before a barrier in "initialized." The count on
          the barrier just goes negative until someone initializes the
          barrier using the -i option, which increments the count by the
          count. Therefore, the master does not have to arrange to get there
          "first." As an example, consider a master and one client:
      
      	nodeA> /usr/local/etc/emulab/emulab-sync -n mybarrier
      	nodeB> /usr/local/etc/emulab/emulab-sync -n mybarrier -i 1
      
          Node A waits until Node B initializes the barrier (gives it a
          count).  The count is the number of *waiters*, not including the
          master. The master is also blocked until all of the waiters have
          checked in.
      
          I have not made an provision for timeouts or crashed clients. Lets
          see how it goes.
      212cc781
  22. 01 Aug, 2003 1 commit
  23. 30 Jul, 2003 1 commit
  24. 14 Jul, 2003 1 commit
  25. 10 Jul, 2003 1 commit
    • Leigh B. Stoller's avatar
      Two sets of minor changes. · 7d0f59e6
      Leigh B. Stoller authored
      * In the parser, make -n (impotent) and -a (anonymous) more
        independent. Used to be that -n required -a, but that makes the
        preparse less useful, since it cannot catch project related errors
        (like bad osids, or node type permissions), and so the user does not
        get that until the email message later. Thats so annoying, even Mike
        whined about it.
      
        Note that impotent mode is sorta misnamed now, since the parse never
        operates on the DB. Rather, impotent mode now skips doing the XML
        output phase (still aptly named updateDB!).
      
      * Add -p (pass) option. I added this for my script that was parsing
        all the old NS files to get renderings. In this case, I do not want
        -n or -a; I want to upload the results into the DB, but the project
        related checks are obviously going to fail since I was doing it
        inside the testbed project. So, -p turns on some of the anon checks,
        and later might be used to turn certain features that are no longer
        supported, since all we really care about is the toplology (some NS
        files failed on old features and syntax).
      
      Upon reflection I think these three options could probably be rolled
      into just two, by cleaning up the current impotent and anonymous
      flags.
      7d0f59e6
  26. 03 Jul, 2003 1 commit
  27. 02 Jul, 2003 2 commits
  28. 01 Jul, 2003 1 commit
  29. 30 Jun, 2003 1 commit
    • Leigh B. Stoller's avatar
      Make the new parser live on mini. New parser ssh'es over to ops to · 2202fc5a
      Leigh B. Stoller authored
      do the actual parse. The parser now spits out XML instead of DB
      queries, and the wrapper on boss converts that to DB insertions after
      verification. There are some makefile changes as well to install the
      new parser on ops via NFS, since otherwise the parser could
      intolerably out of date on ops!
      2202fc5a
  30. 25 Jun, 2003 1 commit
    • Shashi Guruprasad's avatar
      Fixed the traffic generator TCL statement ordering problem. · f4c667ea
      Shashi Guruprasad authored
      Now, an agent connect statement can come before application attach-agent
      which complies with NS. In addition, the order of
      "$ns attach-agent <node> <agent>" and "$ns connect <agent1> <agent2>"
      can by anything now. Reversing the order in NS causes problems for this though.
      However, in emulab's case, we don't need that order. Most of these checks
      are now moved to updatedb reducing some clutter in traffic.tcl
      
      One thing that NS supports and it would be difficult for us is the order
      of application attach-agent and "$ns at" statements. In NS, its ok to
      flip the order. In emulab, its still not. Thats coz "$ns at" statements
      need to know what vnode is the event going to. Of course, this can be
      fixed by making "$ns at" storing the eventstring as is and only parsing
      during the updatedb/$ns run phase. Seems like its not worth changing now.
      f4c667ea