GitLab

inner plc on.

ElabInElab experiments, so as not to consume an interface.

I actually modified Keith's changes so that we can dynmaically choose
the single or dual model in the NS file (Keith's changes hardwired the
system to only single control network) since the single network model
has the disadvantage of not allowing new nodes to be added to an inner
elab, which would make it impossible to test some things (like the
newnode path!).

To choose the single control network model, place this in your NS file:

	tb-elabinelab-singlenet

other it defaults to the older dual network model.

* Add XMLRPC interface for template swapin,stoprun,startrun,swapout and
  add the appropriate wrappers to the script_wrapper on ops.

* Allow parameter descriptions in NS files. This is probably not in its
  final form since its a bit confusing as to what has priority; something
  in the NS file or a metadata item. Anyway, you can do this in your NS
  file:

	$ns define-template-parameter GUID "0/0" "The GUID to be analyzed"

  The rules are currently that the NS file description has priority and
  is copied to child templates, unless the user has modified a description
  via the web interface, in which case the NS file description is ignored.
  I know, sounds awful, but for the most part people are going to use the
  NS file anyway.

* Add "clear" option when starting a new experiment run; the per
  experiment DB at the logholes are cleared. Note that this is *not* the
  default behaviour; you have to either check the checkbox on the web form
  or use the -c option to the script wrapper, or clear=yes if talking
  directly to the XMLRPC server.

* Fix up how email is generated for template_swapin and template_create,
  so that Kevin can debug tblog/tbreport stuff, but also so that we maintain
  mail logs as before. I have made some improvements to libaudit so as to
  centralize the mail goo, and avoid duplicating all that stuff.

* Minor fixes to the program agent so that the new environment strings are
  sent before the program agent exits and reloads them!

* Other minor little things.

Used to debug NSE problem without changing standard FBSD image,
may come in handy again.

Actually, most of the changes here were just to generalize the "virtual
interface" state in the DB.  Other than the client-side scripts, there
is very little specific here specific to tagged VLANs.

In fact, you cannot specify "vlan" as a type yet as we haven't done the
snmpit support for setting up the switches.

For more info see bas:~mike/flux/doc/testbed-virtinterfaces.txt, which I
will integrate into the knowledge base and the Emulab doc at some point.

node.tcl: Fix the quoting of the -dir argument for the program agent
(yeah, I know what you are thinking; but I am just mimicing what is
already done in sim.tcl::make_event)

It shouldn't matter in this file (only tb_compat.tcl), but just in case...

the verify parse.

* setup some fields that parallel the
  elab-in-elab ones, for modifying the dhcpd.conf file.

* tb-set-node-plab-role to set a node as either 'plc' or 'node'
  used to set the DB role field above

you can turn it on in your devel tree by setting the $EXPOSETEMPLATES
variable in www/defs.php3.in to 1. BE SURE NOT TO CHECK THAT CHANGE IN!

security-level.

Some nsverify fixes:

	* tbsetup/ns2ir/nstb_compat.tcl: Change tb-set-node-startcmd to
	actually create the program agent.  Add an "unknown" instproc for
	LanNode so "mustdelay" and whatever else works.

	* tbsetup/nsverify/ns-2.27.patch, tbsetup/nsverify/ns-2.29.patch:
	Use real_set when setting last_host/last_whateva.

	* tbsetup/nsverify/nstbparse.in: Don't do a rename for
	non-globals.

	* tbsetup/nsverify/tb_compat.tcl: Match behavior in the NS parser
	where the user manually sets/clears red_/gred_ in a queue object.
	Also need to output "h" lines for node objects that weren't stored
	in a global variable and so only have an internal name.

Parse the NS file with the real NS parser so we can make sure linktest is
doing the "right" thing.

	* configure, configure.in: Add tbsetup/nsverify files.

	* tbsetup/GNUmakefile.in: Add nsverify subdir.

	* tbsetup/tbprerun.in: Run verify-ns on the experiments NS file.

	* tbsetup/ns2ir/nstb_compat.tcl: Bring up-to-date with the current
	world.

	* tbsetup/nsverify/GNUmakefile.in: Makefile.

	* tbsetup/nsverify/ns-2.27.patch: Patch file for NS version 2.27.

	* tbsetup/nsverify/nstbparse.in: Wrapper for the NS parser.

	* tbsetup/nsverify/tb_compat.tcl: Different version of
	tb_compat.tcl that is used to verify linktest parameters.

	* tbsetup/nsverify/verify-ns.in: Script that runs on boss and
	verifies that the testbed parser worked correctly.

	* tbsetup/ns2ir/parse-ns.in, tbsetup/ns2ir/parse.proxy.in: Tweaked
	a bit so parse.proxy can be used to run the regular NS parser in
	addition to the testbed one.

* ptopgen:

   1. Change to using interface_capabilities table to get the default
      speed for an interface type.

   2. The wrinkle is that an interface can have multiple speeds,
      depending on the protocol that is requested. In the wireless case the
      table looks like:

	| type             | capkey            | capval               |
	+------------------+-------------------+----------------------+
	| ath              | protocols         | 80211a,80211b,80211g |
	| ath              | 80211a_defspeed   | 54000                |
	| ath              | 80211b_defspeed   | 11000                |
	| ath              | 80211g_defspeed   | 54000                |

   3. We use the above info to tell us the default speed for links, based
      on the particular protocol.

   4. And lastly, create "air" switches for each of 80211a, 80211b, 80211g,
      and specify links from the switches to the nodes that have interfaces
      that support those protocols. At the moment, there is a hardwired
      list of 80211a, 80211b, 80211g in ptopgen, so create 3 new air
      switches and links from each of the nodes to each of the switches.
      Rob says that assign can handle this just fine (using the same
      interface on a node to connect to three different air switches, at
      different speeds).

* assign_wrapper:

   1. First off, I had to increase the bogousity factor of how we determine
      that a delay node is required (requires_delay() routine) on a link or
      lan. Using the interface_capabilities table, I determine the default
      speed for each protocol on each interface type.

   2. When calling requires_delay() I now the pass the lan so that we can
      get the "protocol" of the link/lan and use that to find the default
      speed. The main reason for all this crud is that if a user specifies
      a bandwidth of 54Mb on a wired ethernet, we want to make sure we
      insert a delay node, but not if the user is using 80211g. As it was,
      delay nodes would be skipped cause there are now nodes with
      interfaces that support those speeds. This change also allows to know
      that a delay node should be inserted on a wireless link, if we ever
      wanted to do something as silly as that!

   3. Okay, so now onto the actual wireless support!  Whenever the protocol
      for a lan is not "ethernet" spit out links to fake lan nodes, sorta
      like we used to do with ethernet lans before Rob made all those great
      changes to assign. So, a lan with protocol 80211a will get fake links
      to the 80211a air switch that ptopgen spits out (see above).  We
      currently bypass all delay node processing on wireless lans, and in
      fact we avoid most post-processing (after assign) of wireless lans
      (no vlans, veths, etc). At some point we can bring certain things
      back in (like emulated links) but that will be some time! See
      comments above about ptopgen.

   4. Added a section to copy over the virt_lan_settings and
      virt_lan_member_settings to the interface_settings table, for each
      node's interfaces in the mapped experiment. I also make up the ssid
      for the lan, and add the protocol to the settings so that the client
      side knows what it is supposed to do. I also set the acesspoint since
      that requires MAC addresses, and this is easiest place to get and set
      the MAC (all of the nodes in a lan need to know the MAC of the node
      acting as the access point). In the virtual topology, the accesspoint
      is specified as a node name.

* tbsetup/ns2ir:

   1. Add tb-set-lan-protocol(lan, protocol) to set the protocol to one of
      ethernet, 80211a, 80211b, 80211g. Defaults to ethernet.

   2. Add tb-set-lan-accesspoint(lan, accesspoint) to set the name of the
      access point for a lan. Obviously this makes sense for only wireless
      links; ignored for wired links.

   3. Add tb-set-lan-setting(lan, setting, value) to specify a setting that
      applies to the entire lan. At present, these are ignored for wired
      lans. Both setting and value are strings.

   4. Add tb-set-node-lan-setting(lan, node, setting, value) to specify a
      setting that applies to a particular node in a lan. At present, these
      are ignored for wired lans. Both setting and value are strings.

jail and delay node OSIDs. When set, will override anything in the
node_types table. Use with caution.

so that it issues exit status via startcmddone. User does not need to
do this, which Eric says is less error prone!

multiplexing of nodes.

in email.

	proc tb-use-endnodeshaping {onoff} {}
	proc tb-force-endnodeshaping {onoff} {}
	proc tb-set-multiplexed {link onoff} {}
	proc tb-set-endnodeshaping {link onoff} {}
	proc tb-set-noshaping {link onoff} {}

of emulated and linkdelays on a per-link/lan basis (although you
cannot make a lan emulated at this time).

	tb-set-link-uselinkdelay $link0 1
	tb-set-lan-uselinkdelay $lan0 1
	tb-set-link-emulated $link0 1

Added slots to the virt_lans table, used in assign_wrapper to control
these features on a per link/lan basis. Note, the global options are
still available to prefer/force linkdelays for an entire experiment.
When setting uselinkdelays on a per link/basis, if the link would not
otherwise be shaped, you are requesting that a linkdelay be inserted
anyway (in otherwords, its the equiv of tb-set-forcelinkdelays on an
individual basis).

link delays setup on the nodes.

Add tb-set-forcelinkdelays to force a linkdelay (when uselinkdelays is
on of course) to be inserted, even if the link was unshaped. This
allows it to be controlled later, say via the event system. This is
nice, cause in the current system an unshaped link cannot be later
shaped.

Note that these are currently require FBSD-47.

nodes. The second argument can now be an NS node instead of the name
of a real testbed node. For example:

	tb-set-hardware $node3  pc600
	tb-set-hardware $nodev1 pcvm600
	tb-fix-node $nodev1 $node3

So, "fix" $nodev1 to $node3. The intent is that once $node3 is
allocated by assign to a real testbed node, we can then allocate a
virtual node on pcXX to $nodev1. I did this primarily to allow for
easy testing of jails via my NS file, without having to hack assign
wrapper to deeply. Note there are still hacks in assign_wrapper to
support this, but they are not extensive.

Also my old usewatunnels stuff I never checked in:

	tb-set-usewatunnels 0/1

  - Fixes the routing problem
  - A new type "sim" has been created but not in the DB. node_types and
    nodes remain unchanged. This will change after we figure out how
    to represent in the DB the local multiple virtnodes in one phynode case
    The frontend tb_compat.tcl adds the sim type and ptopgen associates
    a huge number of sim nodes to all local PCs.
  - All simulated nodes go into one pc untill I finish coding the distributed
    nse case. Also, sim nodes go through assign but with an explicit
    "fix-node" directive in the top file. A random free pc is chosen using
    the avail script in assign_wrapper. If we don't fix node it, assign
    maps sim nodes to multiple phy nodes even when a valid all to one
    mapping is possible.
  - Syntax for nse:

   $ns make-simulated {

       set simnode1 [$ns node]
       set simnode2 [$ns node]

   }
   ...

   The old syntax

   set nsenode [$ns nsenode]
   $nsenode make-simulated {
   }

   is deprecated

  - All 38 frontend tests in the testsuite pass

  - A new table v2pmap has been added to handle multiple virt nodes to
    one phy node mapping. This is used in "tmcc hostnames" currently.

  - The phy node that is picked to run nse is loaded with a custom image
    FBSD45-NSE . This can use the default FBSD kernel if there is a mechanism
    to run at 1000HZ, have options IPFIREWALL_FORWARD and PERFMON turned on.
    The image is in the 'testbed' group. Do not delete this image.

  - Static routes now adds routes for the case dst == nexthop. The routing
    graph would be disconnected otherwise and I need to traverse this in
    assign_wrapper for nse. On the client side, such a route is filtered out in
    libsetup.pm

  - sim nodes are also correctly visualized

	tb-set-wasolver-weights delay bw plr

Where I am told proper defaults are 1, 10, 500. Setting any to zero
effectively drops that metric from consideration when the solver runs.
I've updated the tb_compat files and the documentation.

Also add required changes to wanlinkinfo and wanassign.