1. 03 Feb, 2004 1 commit
  2. 30 Jan, 2004 2 commits
  3. 29 Jan, 2004 1 commit
    • Shashi Guruprasad's avatar
      Fixes the bug unearthed by Tim. When a link is created between a jail · 8e236ecc
      Shashi Guruprasad authored
      vnode and a PC node, under normal circumstances, a real physical link
      is used. However, a user can use tb-set-multiplexed on the link that
      allows assign to map multiple vlinks over the same physical link.
      This is fine when the PC is running FBSD-STD which can use veth
      interfaces. Not so if it is running Linux or a non-standard BSD.
      This fix forces EMULATED links and VETH devices when the PC is running
      FBSD-STD and turns those attributes off otherwise so that assign
      will not give a mapping that cannot be realized on the client side.
      For the gory details of this problem, look at testbed-ops email thread
      "pcvm route problem" dated 29th Jan 2004.
      
      A problem still exists in the non-BSD router + jail vnodes in a delayed
      LAN. VLANs table is not getting the correct entries. A fix will come
      soon. Follow the testbed-ops email thread "fix for the pcvm route problem"
      dated 29th Jan 2004 for the gory details.
      8e236ecc
  4. 21 Jan, 2004 1 commit
  5. 12 Jan, 2004 1 commit
  6. 09 Jan, 2004 1 commit
    • Shashi Guruprasad's avatar
      Changes to do auto re-swap of expts with simnodes when an nse on a simhost · 375f87c1
      Shashi Guruprasad authored
      (or more than one simhost) is unable to keep up with real-time. It includes
      changes to assign_wrapper to handle swap modify for simnodes, the simple
      algorithm in nseswap that bumps up the nodeweight of simnodes being hosted
      on a simhost that reports "can't keep up with real-time" (aka nse violation),
      ptopgen and sim.tcl to prefer nodes that already have the FBSD-NSE image.
      Also, changes to other files to send out NSESWAP event.
      
      One unrelated change: We now have per-swapin .top files and assign.log
      files along with .ptop files. This helps in debugging across multiple
      swapins since files remain in the form of
      <pid>-<eid>-<process_id>.{top,ptop} and assign-<pid>-<eid>-<process_id>.log
      Also useful for archiving.
      375f87c1
  7. 06 Jan, 2004 1 commit
    • Robert Ricci's avatar
      Fix dynamic pclasses, and make them the default for virtual nodes · c7e2d29e
      Robert Ricci authored
      again.
      
      One of the fixes changes the way in which we iterate through pclasses
      in find_pnode(). We used to treat the vector like a ring buffer, and
      start (randomly) someplace in the middle. This turns out to give some
      bad statistical properties when doing dynamic pclasses, since long
      chains of disabled pclasses will cause some pclasses to be selected
      more often. My old hack of just hopping around randomly in the
      disabled-pclass case was bad, because it's hard to tell when you've
      actually tried all the pclasses - so, we were getting false negatives
      where it was looking like there was no place available where we could
      map a vnode, which turned out to have worse effects than I had
      thought.
      
      So, now, we make a list of all the indices and randomize the order,
      then just iterate through that list.
      
      We also now count the number of pclasses that are enabled at every
      temperature step, and adjust the neighborhood size to remove them.
      This makes dynamic pclasses quite a bit faster - it cuts the time
      by 30% - 50% for my test case.
      
      Cleaned up find_pnode() by removing some #ifdef's that we don't use,
      and probably will never want to again - this makes the function almost
      readable!
      c7e2d29e
  8. 03 Jan, 2004 2 commits
  9. 23 Dec, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add new state PXEWAIT to be explained later ... · 22ee2594
      Leigh B. Stoller authored
      Change ptopgen to look at the eventstate of a node; a node is not considered
      free unless it is ISUP or PXEWAIT.
      
      Add TBAvailablePCs() to libdb, removing the corresponding code from
      assign_wrapper. This routine does the equiv of ptopgen, returning the
      number of PCs that are available for use (looking at eventstate).
      
      Change TBFreePCs in the web interface accordingly.
      
      The above changes correspond to an upcoming change in stated.
      22ee2594
  10. 15 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      Distributed NSE changes. In other words, simulation resources are · d266bd71
      Shashi Guruprasad authored
      now mapped to more than one PC if required. The simnode_capacity
      column in the node_types table determines how many sim nodes can
      be packed on one PC. The packing factor can also be controlled via
      tb-set-colocate-factor to be smaller than simnode_capacity.
      
      - No frontend code changes. To summarize:
        $ns make-simulated {
          ...
        }
        is still the easy way to put a whole bunch of Tcl code to be
        in simulation.
        One unrelated fix in the frontend code is to fix the
        xmlencode() function which prior to this would knock off
        newlines from columns in the XML output. This affected
        nseconfigs since it is one of the few columns with embedded
        newlines. Also changed the event type and event object type
        in traffic.tcl from TRAFGEN/MODIFY to NSE/NSEEVENT.
      
      - More Tcl code in a new directory tbsetup/nseparse
        -> Runs on ops similar to the main parser. This is invoked
           from assign_wrapper in the end if there are simnodes
        -> Partitions the Tcl code into multiple Tcl specifications
           and updates the nseconfigs table via xmlconvert
        -> Comes with a lot of caveats. Arbitrary Tcl code such as user
           specified objects or procedures will not be re-generated. For
           example, if a user wanted a procedure to be included in Tcl
           code for all partitions, there is no way for code in nseparse
           to do that. Besides that, it needs to be tested more thoroughly.
      
      - xmlconvert has a new option -s. When invoked with this option,
        the experiments table is not allowed to be modified. Also,
        virtual tables are just updated (as opposed to deleting
        all rows in the first invocation before inserting new rows)
      
      - nse.patch has all the IP address related changes committed in
        iversion 1.11 + 2 other changes. 1) MTU discovery support in
        the ICMP agent 2) "$ns rlink" mechanism for sim node to real
        node links
      
      - nseinput.tcl includes several client side changes to add IP
        routes in NSE and the kernel routing table for packets crossing
        pnodes. Also made the parsing of tmcc command output more robust
        to new changes. Other client side changes in libsetup.pm and other
        scripts to run nse, are also in this commit
      
      - Besides the expected changes in assign_wrapper for simulated nodes,
        the interfaces and veth_interfaces tables are updated with
        routing table identifiers (rtabid). The tmcd changes are already
        committed. This field is used only by sim hosts on the client side.
        Of course, they can be used by jails as well if desired.
      d266bd71
  11. 10 Dec, 2003 1 commit
  12. 07 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      Minimal NSE related changes on the mainbed so that I can work · 425b4e47
      Shashi Guruprasad authored
      in the dev tree. I'm tired of problems on the mini that wastes
      my time.
      
      Changes include 2 new tmcd commands: tmcc routelist returns
      the routes for all the vnodes hosted on a pnode. tmcc role
      returns the role of a reserved node, like 'virthost' or
      'simhost.  tmcc ifconfig now reports an RTABID field which
      is calculated in assign wrapper. All the new changes
      in assign wrapper will be checked in after I finish testing.
      
      All the DB changes are in: simnode_capacity in node_types, rtabid in
      interfaces and veth_interfaces. New NSE event_objtype and NSEEVENT
      event_eventtype. Changed the erole field in the reserved table
      to have 'simhost' instead of 'simnode'. Changed the correspoding
      libdb subroutines.
      425b4e47
  13. 03 Dec, 2003 1 commit
    • Robert Ricci's avatar
      Some support for gigabit interfaces. The biggest change is that we're · 851e2f1d
      Robert Ricci authored
      more liberal about which links don't need delays. Before, we were
      putting delays on anything that wasn't 100Mbps.
      
      The other change is that for delay nodes, we set virtual link
      bandwidths by rounding up to the next fastest speed of interface,
      instead of hardwiring 100Mbps. Similarly, we set the speed of
      physical delay interfaces to the card's 'native' speed instead of
      hardwiring 100Mbps.
      
      These changes work for our nodes and links, but are still not totally
      general. Really, what needs to happen is that assign needs to be the
      one to insert delay nodes, because it's the only place where we really
      know which interfaces of what speeds are available on which nodes.
      But, that is a very big project, since both assign wrapper and assign
      need to be changed.
      851e2f1d
  14. 01 Dec, 2003 1 commit
    • Robert Ricci's avatar
      New scripts: tarfiles_setup, fetchtar.proxy, and webtarfiles_setup . · c0c6547c
      Robert Ricci authored
      The idea is to give us hooks for grabbing experimenters' tarballs (and
      RPMs) from locations other than files on ops. Mainly, to remove
      another dependance on users having shells on ops.
      
      tarfiles_setup supports fetching files from http and ftp URLs right
      now, through wget. It places them into the experiment directory, so
      that they'll go away when the experiment is terminated, and the rest
      of the chain (ie. downloading to clients and os_setup's checks)
      remains unchaged.  It is now tarfiles_setup's job to copy tarballs and
      RPMs from the virt_nodes table to the nodes table for allocated nodes.
      This way, it can translate URLs into the local filenames it
      constructs. It get invoked from tbswap.
      
      Does the actual fetching over on ops, running as the user, with
      fetchtar.proxy.
      
      Should be idempotent, so we should be able to give the user a button
      to run webtarfiles_setup (none exists yet) yet to 'freshen' their
      tarballs. (We'd also have to somehow let the experiment's nodes know
      they need to re-fetch their tarballs.)
      
      One funny side effect of this is that the separator in
      virt_nodes.tarfiles is now ';' instead of ':' like nodes.tarballs,
      since we can now put URLs in the former. Making these consistent is a
      project for another day.
      c0c6547c
  15. 23 Nov, 2003 1 commit
  16. 17 Nov, 2003 1 commit
    • Leigh B. Stoller's avatar
      Merge the two state machines (batchstate and state) into a single · 2025e0bd
      Leigh B. Stoller authored
      state machine (state). All of the stuff that was previously handled by
      using batchstate is now embedded into the one state machine. Of
      course, these mostly overlapped, so its not that much of a change,
      except that we also redid the machine, adding more states (for
      example, modify phases are now explicit. To get a picture of the
      actual state machine, on boss:
      
      		stategraph -o newstates EXPTSTATE
      		gv newstates.ps
      
      Things to note:
      
      * The "batchstate" slot of the experiments table is now used solely to
        provide a lock for batch daemon. A secondary change will be to
        change the slot name to something more appropriate, but it can
        happen anytime after this new stuff is installed.
      
      * I have left expt_locked for now, but another later change will be to remove
        expt_locked, and change it to active_busy or some such new state name in
        the state machine. I have removed most uses of expt_locked, except those
        that were necessary until there is a ...
      2025e0bd
  17. 05 Nov, 2003 1 commit
  18. 31 Oct, 2003 1 commit
  19. 20 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Bring wanassign back from the bit rot abyss. Three changes. · fe9eba11
      Leigh B. Stoller authored
      * Remove all of the code that dealt with allocating unconnected nodes.
        It used to be assign_wrapper passed all widearea node allocation
        decisions to wanassign, those in links and those that were
        unconnected. assign_wrapper now handles all unconnected nodes since
        assign is much better with features/desires and node type stuff.
      
      * Do not modify any database state in wanassign; It used to do the
        actual nalloc calls, but now it just returns the mapping to
        assign_wrapper so that we can more easily track "recoverability" and
        because there is existing code in assign_wrapper to allocate vnodes
        on the selected pnodes. No point in duplication.
      
      * Switch from mapping to vnodes, to mapping to pnodes. We made this
        change for other virtual nodes; instead of "fixing" to a vnode on a
        pnode, fix to the pnode. The resulting mappings are also given as
        pnodes, and assign_wrapper does the allocation on those selected
        nodes.
      
      Now all we need is...
      fe9eba11
  20. 19 Oct, 2003 1 commit
  21. 15 Oct, 2003 2 commits
  22. 13 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Aside from another round of cleanup, there is a significant change. · a70aef53
      Leigh B. Stoller authored
      I have implemented the suggestion Jay made a couple of weeks ago
      about allowing partial allocation in assign_wrapper, and retrying with a
      modified set of "fixed" nodes.
      
      My basic approach was to change nalloc to optionally allow partial
      allocations, returning the number of nodes that could not be allocated as
      its return value. In assign_wrapper, I determine which nodes we were able
      to get (in each loop), set their allocstate to INIT_DIRTY, augment the
      fixed_node set, and recreate the top file. Then I try again, up to the
      current number of maxtries. If assign fails with an unretryable error, or
      if we could not nalloc a user directed fixed node, then I stop right away
      since the experiment is not going to map (in the near term) if the fixed
      node list cannot be allocated.
      
      I am confident that this works okay, although testing is a little
      difficult. The main problem is how this interacts with experiment modify.
      Chad's implementation is that a modify can be reverted (recovered from)
      only as long as the DB is not modified by assign_wrapper. Well, a partial
      allocation, followed by failure, obviously modifies the DB, and so is
      deemed not recoverable. I am still trying to figure out the effects of
      this, and whether I can relax this requirement, but in the meantime
      lets install it and see what happens (won't affect many people).
      a70aef53
  23. 30 Sep, 2003 3 commits
    • Leigh B. Stoller's avatar
      Init delaynodes and jail hosting nodes with startstatus=0 so that · 55db053e
      Leigh B. Stoller authored
      the batch system see's them as always done. There is no reason to do
      this from the node itself, since it would be really hard to have
      either a jail or delay node without other nodes in the topology!
      55db053e
    • Leigh B. Stoller's avatar
      Remove tiny bit of debugging code. · f3c381da
      Leigh B. Stoller authored
      f3c381da
    • Leigh B. Stoller's avatar
      Up to now we have had two state variables associated with an experiment, · 4269dad1
      Leigh B. Stoller authored
      plus a lock field. The lock field was a simple "experiment locked, go away"
      slot that is easy to use when you do not care about the actual state that
      an experiment is in, just that it is in "transition" and should not be
      messed with.
      
      The other two state variables are "state" and "batchstate". The former
      (state) is the original variable that Chris added, and was used by the tb*
      scripts to make sure that the experiment was in the state each particular
      script wanted them to be in. But over time (and with the addition of so
      much wrapper goo around them), "state" has leaked out all over the place to
      determine what operations on an experiment are allowed, and if/when it
      should be displayed in various web pages. There are a set of transition
      states in addition to the usual "active", "swapped", etc like "swapping"
      that make testing state a pain in the butt.
      
      I added the other state variable ("batchstate") when I did the batch
      system, obvious...
      4269dad1
  24. 26 Sep, 2003 1 commit
  25. 18 Sep, 2003 1 commit
  26. 17 Sep, 2003 2 commits
  27. 16 Sep, 2003 1 commit
  28. 13 Sep, 2003 1 commit
  29. 12 Sep, 2003 1 commit
  30. 11 Sep, 2003 1 commit
    • Leigh B. Stoller's avatar
      Widearea changes, mostly started for plab but affects all widearea · a5e1e2ee
      Leigh B. Stoller authored
      nodes. The intent is to better support last mile types, which right
      now are a total mess cause the types are associated with the virtual
      nodes, and a node can have just a single type. This info has now been
      moved in the node_types_auxtypes table and the node_types table.
      
      Anyway, we no longer use wanassign on unconnected widearea nodes, but
      use assign directly, much like we use assign to allocate virtual nodes
      on local physical nodes. ptopgen inserts the physical widearea nodes
      (even though they are allocated), and the proper counts for the types
      that are available on them. assign will pick the nodes, and
      assign_wrapper will allocate the necessary vnodes on the pnodes (but
      in the case of wideare nodes, the underlying physical node does not
      need to be allocated).
      
      Also added some fixes for dealing with vtypes when used in conjunction
      with widearea nodes. Rather then generating an error like it used to,
      you can create a vtype in your NS file:
      
      	tb-make-soft-vtype mytype {pcvroninet pcvwainet}
      	tb-set-hardware $v0 mytype
      
      and assign_wrapper now looks at the underlying types to figure out
      what it needs. Note: No consistency checking yet; mixing a remote/virt
      and a local/real type will break.
      a5e1e2ee
  31. 09 Sep, 2003 1 commit
  32. 02 Sep, 2003 1 commit
  33. 29 Aug, 2003 1 commit
    • Leigh B. Stoller's avatar
      Temporary patch to solve the non-connected veth interface problems, · 5f214b74
      Leigh B. Stoller authored
      which happens on lans of vnodes that are split between pnodes. assign
      spits out trivial links for the nodes collocated on the pnodes, but if
      there are two groups of vnodes on different pnodes, the connection is
      not explicit in the link statements that assign gives (techinically,
      they should not be trivial links, but Rob is still thinking that
      over). Fortunately, I have enough info from assign to extend the vlan
      and to patch the veth interfaces afterwards. Its god-awful stuff, and
      I hope I can strip it out soon.
      5f214b74
  34. 27 Aug, 2003 1 commit
    • Leigh B. Stoller's avatar
      Added "subnode" support, primarily for the IXPs. The main feature of · 4088258c
      Leigh B. Stoller authored
      subnodes is dealing with the hosting node, and the "fakelink" that
      we have to insert until assign is taught how to deal with hosting
      nodes. The fakelink causes assign to grab both nodes together.
      
      Also: * Minor cleanup.
            * Change control_net to control_iface.
            * Set the routertype to "manual" for jail hosts, and "none" for
              delay nodes. This is done in InitPnode.
            * Get rid of more shark code.
      4088258c