1. 08 Mar, 2004 1 commit
    • Robert Ricci's avatar
      Make the getbandwidth() function aware of node types - it takes as a · 5cb2e92f
      Robert Ricci authored
      parameter the virtual node it is being called for, and considers only
      bandwidths that can be supported by physical nodes that virtual node
      can be mapped to.
      
      This fixes some cases where we were mistakenly setting interfaces
      on slow links to 10Mbps, which does not work reliably.
      
      Not surpisingly, tracking down problems with vtypes was half the
      trouble with making this change.
      5cb2e92f
  2. 23 Feb, 2004 1 commit
  3. 16 Feb, 2004 1 commit
  4. 12 Feb, 2004 2 commits
    • Leigh B. Stoller's avatar
      Fix problem with trivial links not getting lindelays. In the case that · 0dc661b1
      Leigh B. Stoller authored
      a 100Mb link is mapped to a trivial link, a linkdelay has to be
      inserted since the actual loopback bandwidth is much higher. The
      problem is that unlike other delays where requires_delay() will return
      true, this decision has to wait until after assign does the mapping
      and we find out that a link was mapped to a trivial link. When this
      happens, I insert a linkdelay "on the fly" when processing the link
      statements from assign.
      0dc661b1
    • Robert Ricci's avatar
      Unbreak vtypes by putting entries for them into the %node_type_linkbw · fa4f5764
      Robert Ricci authored
      hash.
      
      This means we have to call LoadVirtTypes() from a slightly different
      place, so that we have the virt types before calling LoadPhysInfo(),
      where %node_type_linkbw is built.
      fa4f5764
  5. 10 Feb, 2004 1 commit
  6. 03 Feb, 2004 3 commits
  7. 30 Jan, 2004 2 commits
  8. 29 Jan, 2004 1 commit
    • Shashi Guruprasad's avatar
      Fixes the bug unearthed by Tim. When a link is created between a jail · 8e236ecc
      Shashi Guruprasad authored
      vnode and a PC node, under normal circumstances, a real physical link
      is used. However, a user can use tb-set-multiplexed on the link that
      allows assign to map multiple vlinks over the same physical link.
      This is fine when the PC is running FBSD-STD which can use veth
      interfaces. Not so if it is running Linux or a non-standard BSD.
      This fix forces EMULATED links and VETH devices when the PC is running
      FBSD-STD and turns those attributes off otherwise so that assign
      will not give a mapping that cannot be realized on the client side.
      For the gory details of this problem, look at testbed-ops email thread
      "pcvm route problem" dated 29th Jan 2004.
      
      A problem still exists in the non-BSD router + jail vnodes in a delayed
      LAN. VLANs table is not getting the correct entries. A fix will come
      soon. Follow the testbed-ops email thread "fix for the pcvm route problem"
      dated 29th Jan 2004 for the gory details.
      8e236ecc
  9. 21 Jan, 2004 1 commit
  10. 12 Jan, 2004 1 commit
  11. 09 Jan, 2004 1 commit
    • Shashi Guruprasad's avatar
      Changes to do auto re-swap of expts with simnodes when an nse on a simhost · 375f87c1
      Shashi Guruprasad authored
      (or more than one simhost) is unable to keep up with real-time. It includes
      changes to assign_wrapper to handle swap modify for simnodes, the simple
      algorithm in nseswap that bumps up the nodeweight of simnodes being hosted
      on a simhost that reports "can't keep up with real-time" (aka nse violation),
      ptopgen and sim.tcl to prefer nodes that already have the FBSD-NSE image.
      Also, changes to other files to send out NSESWAP event.
      
      One unrelated change: We now have per-swapin .top files and assign.log
      files along with .ptop files. This helps in debugging across multiple
      swapins since files remain in the form of
      <pid>-<eid>-<process_id>.{top,ptop} and assign-<pid>-<eid>-<process_id>.log
      Also useful for archiving.
      375f87c1
  12. 06 Jan, 2004 1 commit
    • Robert Ricci's avatar
      Fix dynamic pclasses, and make them the default for virtual nodes · c7e2d29e
      Robert Ricci authored
      again.
      
      One of the fixes changes the way in which we iterate through pclasses
      in find_pnode(). We used to treat the vector like a ring buffer, and
      start (randomly) someplace in the middle. This turns out to give some
      bad statistical properties when doing dynamic pclasses, since long
      chains of disabled pclasses will cause some pclasses to be selected
      more often. My old hack of just hopping around randomly in the
      disabled-pclass case was bad, because it's hard to tell when you've
      actually tried all the pclasses - so, we were getting false negatives
      where it was looking like there was no place available where we could
      map a vnode, which turned out to have worse effects than I had
      thought.
      
      So, now, we make a list of all the indices and randomize the order,
      then just iterate through that list.
      
      We also now count the number of pclasses that are enabled at every
      temperature step, and adjust the neighborhood size to remove them.
      This makes dynamic pclasses quite a bit faster - it cuts the time
      by 30% - 50% for my test case.
      
      Cleaned up find_pnode() by removing some #ifdef's that we don't use,
      and probably will never want to again - this makes the function almost
      readable!
      c7e2d29e
  13. 03 Jan, 2004 2 commits
  14. 23 Dec, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add new state PXEWAIT to be explained later ... · 22ee2594
      Leigh B. Stoller authored
      Change ptopgen to look at the eventstate of a node; a node is not considered
      free unless it is ISUP or PXEWAIT.
      
      Add TBAvailablePCs() to libdb, removing the corresponding code from
      assign_wrapper. This routine does the equiv of ptopgen, returning the
      number of PCs that are available for use (looking at eventstate).
      
      Change TBFreePCs in the web interface accordingly.
      
      The above changes correspond to an upcoming change in stated.
      22ee2594
  15. 15 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      Distributed NSE changes. In other words, simulation resources are · d266bd71
      Shashi Guruprasad authored
      now mapped to more than one PC if required. The simnode_capacity
      column in the node_types table determines how many sim nodes can
      be packed on one PC. The packing factor can also be controlled via
      tb-set-colocate-factor to be smaller than simnode_capacity.
      
      - No frontend code changes. To summarize:
        $ns make-simulated {
          ...
        }
        is still the easy way to put a whole bunch of Tcl code to be
        in simulation.
        One unrelated fix in the frontend code is to fix the
        xmlencode() function which prior to this would knock off
        newlines from columns in the XML output. This affected
        nseconfigs since it is one of the few columns with embedded
        newlines. Also changed the event type and event object type
        in traffic.tcl from TRAFGEN/MODIFY to NSE/NSEEVENT.
      
      - More Tcl code in a new directory tbsetup/nseparse
        -> Runs on ops similar to the main parser. This is invoked
           from assign_wrapper in the end if there are simnodes
        -> Partitions the Tcl code into multiple Tcl specifications
           and updates the nseconfigs table via xmlconvert
        -> Comes with a lot of caveats. Arbitrary Tcl code such as user
           specified objects or procedures will not be re-generated. For
           example, if a user wanted a procedure to be included in Tcl
           code for all partitions, there is no way for code in nseparse
           to do that. Besides that, it needs to be tested more thoroughly.
      
      - xmlconvert has a new option -s. When invoked with this option,
        the experiments table is not allowed to be modified. Also,
        virtual tables are just updated (as opposed to deleting
        all rows in the first invocation before inserting new rows)
      
      - nse.patch has all the IP address related changes committed in
        iversion 1.11 + 2 other changes. 1) MTU discovery support in
        the ICMP agent 2) "$ns rlink" mechanism for sim node to real
        node links
      
      - nseinput.tcl includes several client side changes to add IP
        routes in NSE and the kernel routing table for packets crossing
        pnodes. Also made the parsing of tmcc command output more robust
        to new changes. Other client side changes in libsetup.pm and other
        scripts to run nse, are also in this commit
      
      - Besides the expected changes in assign_wrapper for simulated nodes,
        the interfaces and veth_interfaces tables are updated with
        routing table identifiers (rtabid). The tmcd changes are already
        committed. This field is used only by sim hosts on the client side.
        Of course, they can be used by jails as well if desired.
      d266bd71
  16. 10 Dec, 2003 1 commit
  17. 07 Dec, 2003 1 commit
    • Shashi Guruprasad's avatar
      Minimal NSE related changes on the mainbed so that I can work · 425b4e47
      Shashi Guruprasad authored
      in the dev tree. I'm tired of problems on the mini that wastes
      my time.
      
      Changes include 2 new tmcd commands: tmcc routelist returns
      the routes for all the vnodes hosted on a pnode. tmcc role
      returns the role of a reserved node, like 'virthost' or
      'simhost.  tmcc ifconfig now reports an RTABID field which
      is calculated in assign wrapper. All the new changes
      in assign wrapper will be checked in after I finish testing.
      
      All the DB changes are in: simnode_capacity in node_types, rtabid in
      interfaces and veth_interfaces. New NSE event_objtype and NSEEVENT
      event_eventtype. Changed the erole field in the reserved table
      to have 'simhost' instead of 'simnode'. Changed the correspoding
      libdb subroutines.
      425b4e47
  18. 03 Dec, 2003 1 commit
    • Robert Ricci's avatar
      Some support for gigabit interfaces. The biggest change is that we're · 851e2f1d
      Robert Ricci authored
      more liberal about which links don't need delays. Before, we were
      putting delays on anything that wasn't 100Mbps.
      
      The other change is that for delay nodes, we set virtual link
      bandwidths by rounding up to the next fastest speed of interface,
      instead of hardwiring 100Mbps. Similarly, we set the speed of
      physical delay interfaces to the card's 'native' speed instead of
      hardwiring 100Mbps.
      
      These changes work for our nodes and links, but are still not totally
      general. Really, what needs to happen is that assign needs to be the
      one to insert delay nodes, because it's the only place where we really
      know which interfaces of what speeds are available on which nodes.
      But, that is a very big project, since both assign wrapper and assign
      need to be changed.
      851e2f1d
  19. 01 Dec, 2003 1 commit
    • Robert Ricci's avatar
      New scripts: tarfiles_setup, fetchtar.proxy, and webtarfiles_setup . · c0c6547c
      Robert Ricci authored
      The idea is to give us hooks for grabbing experimenters' tarballs (and
      RPMs) from locations other than files on ops. Mainly, to remove
      another dependance on users having shells on ops.
      
      tarfiles_setup supports fetching files from http and ftp URLs right
      now, through wget. It places them into the experiment directory, so
      that they'll go away when the experiment is terminated, and the rest
      of the chain (ie. downloading to clients and os_setup's checks)
      remains unchaged.  It is now tarfiles_setup's job to copy tarballs and
      RPMs from the virt_nodes table to the nodes table for allocated nodes.
      This way, it can translate URLs into the local filenames it
      constructs. It get invoked from tbswap.
      
      Does the actual fetching over on ops, running as the user, with
      fetchtar.proxy.
      
      Should be idempotent, so we should be able to give the user a button
      to run webtarfiles_setup (none exists yet) yet to 'freshen' their
      tarballs. (We'd also have to somehow let the experiment's nodes know
      they need to re-fetch their tarballs.)
      
      One funny side effect of this is that the separator in
      virt_nodes.tarfiles is now ';' instead of ':' like nodes.tarballs,
      since we can now put URLs in the former. Making these consistent is a
      project for another day.
      c0c6547c
  20. 23 Nov, 2003 1 commit
  21. 17 Nov, 2003 1 commit
    • Leigh B. Stoller's avatar
      Merge the two state machines (batchstate and state) into a single · 2025e0bd
      Leigh B. Stoller authored
      state machine (state). All of the stuff that was previously handled by
      using batchstate is now embedded into the one state machine. Of
      course, these mostly overlapped, so its not that much of a change,
      except that we also redid the machine, adding more states (for
      example, modify phases are now explicit. To get a picture of the
      actual state machine, on boss:
      
      		stategraph -o newstates EXPTSTATE
      		gv newstates.ps
      
      Things to note:
      
      * The "batchstate" slot of the experiments table is now used solely to
        provide a lock for batch daemon. A secondary change will be to
        change the slot name to something more appropriate, but it can
        happen anytime after this new stuff is installed.
      
      * I have left expt_locked for now, but another later change will be to remove
        expt_locked, and change it to active_busy or some such new state name in
        the state machine. I have removed most uses of expt_locked, except those
        that were necessary until there is a new state to replace it.
      
      * These new changes are an implementation of the new state machine,
        but I have not done anything fancy. Most of the code is the same as
        it was before.
      
      * I suspect that there are races with the batch daemon now, but they
        are going to be rare, and the end result is probably that a
        cancelation is delayed a little bit.
      2025e0bd
  22. 05 Nov, 2003 1 commit
  23. 31 Oct, 2003 1 commit
  24. 20 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Bring wanassign back from the bit rot abyss. Three changes. · fe9eba11
      Leigh B. Stoller authored
      * Remove all of the code that dealt with allocating unconnected nodes.
        It used to be assign_wrapper passed all widearea node allocation
        decisions to wanassign, those in links and those that were
        unconnected. assign_wrapper now handles all unconnected nodes since
        assign is much better with features/desires and node type stuff.
      
      * Do not modify any database state in wanassign; It used to do the
        actual nalloc calls, but now it just returns the mapping to
        assign_wrapper so that we can more easily track "recoverability" and
        because there is existing code in assign_wrapper to allocate vnodes
        on the selected pnodes. No point in duplication.
      
      * Switch from mapping to vnodes, to mapping to pnodes. We made this
        change for other virtual nodes; instead of "fixing" to a vnode on a
        pnode, fix to the pnode. The resulting mappings are also given as
        pnodes, and assign_wrapper does the allocation on those selected
        nodes.
      
      Now all we need is uptodate widearea data!
      fe9eba11
  25. 19 Oct, 2003 1 commit
  26. 15 Oct, 2003 2 commits
  27. 13 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Aside from another round of cleanup, there is a significant change. · a70aef53
      Leigh B. Stoller authored
      I have implemented the suggestion Jay made a couple of weeks ago
      about allowing partial allocation in assign_wrapper, and retrying with a
      modified set of "fixed" nodes.
      
      My basic approach was to change nalloc to optionally allow partial
      allocations, returning the number of nodes that could not be allocated as
      its return value. In assign_wrapper, I determine which nodes we were able
      to get (in each loop), set their allocstate to INIT_DIRTY, augment the
      fixed_node set, and recreate the top file. Then I try again, up to the
      current number of maxtries. If assign fails with an unretryable error, or
      if we could not nalloc a user directed fixed node, then I stop right away
      since the experiment is not going to map (in the near term) if the fixed
      node list cannot be allocated.
      
      I am confident that this works okay, although testing is a little
      difficult. The main problem is how this interacts with experiment modify.
      Chad's implementation is that a modify can be reverted (recovered from)
      only as long as the DB is not modified by assign_wrapper. Well, a partial
      allocation, followed by failure, obviously modifies the DB, and so is
      deemed not recoverable. I am still trying to figure out the effects of
      this, and whether I can relax this requirement, but in the meantime
      lets install it and see what happens (won't affect many people).
      a70aef53
  28. 30 Sep, 2003 3 commits
    • Leigh B. Stoller's avatar
      Init delaynodes and jail hosting nodes with startstatus=0 so that · 55db053e
      Leigh B. Stoller authored
      the batch system see's them as always done. There is no reason to do
      this from the node itself, since it would be really hard to have
      either a jail or delay node without other nodes in the topology!
      55db053e
    • Leigh B. Stoller's avatar
      Remove tiny bit of debugging code. · f3c381da
      Leigh B. Stoller authored
      f3c381da
    • Leigh B. Stoller's avatar
      Up to now we have had two state variables associated with an experiment, · 4269dad1
      Leigh B. Stoller authored
      plus a lock field. The lock field was a simple "experiment locked, go away"
      slot that is easy to use when you do not care about the actual state that
      an experiment is in, just that it is in "transition" and should not be
      messed with.
      
      The other two state variables are "state" and "batchstate". The former
      (state) is the original variable that Chris added, and was used by the tb*
      scripts to make sure that the experiment was in the state each particular
      script wanted them to be in. But over time (and with the addition of so
      much wrapper goo around them), "state" has leaked out all over the place to
      determine what operations on an experiment are allowed, and if/when it
      should be displayed in various web pages. There are a set of transition
      states in addition to the usual "active", "swapped", etc like "swapping"
      that make testing state a pain in the butt.
      
      I added the other state variable ("batchstate") when I did the batch
      system, obvious...
      4269dad1
  29. 26 Sep, 2003 1 commit
  30. 18 Sep, 2003 1 commit
  31. 17 Sep, 2003 2 commits