Emulab - Testbed NS Command Extensions

Contents


Introduction

In order to use the testbed specific commands you must include the following line near the top of your NS file (before any testbed commands are used):

source tb_compat.tcl

If you wish to use your file under NS you can use download this tb_compat.tcl. Place it in the same directory as your NS file. When run in this way under NS the testbed commands will have no effect, but NS will be able to parse your file.


TCL, NS, and node names

In your file you will be creating nodes with something like:
set node1 [$ns node]
What is really going on is that the simulator, represented by $ns is creating a new node, involving a bunch of internal data changes, and returning a reference to it which is stored in the variable node1. In almost all cases, when you need to refer to a node you will do it as $node1, the $ indicating that you want the value of the variable node1, i.e. the reference to the node. Thus you will be issuing commands like:
$ns duplex-link $node1 $node2 100Mb 150ms DropTail
tb-set-ip $node1 10.1.0.2

(Note the $'s)

You will notice that when your experiment is setup the node names and such will be node1. This happens because the parser detects what variable you are using to store the node reference and uses that as the node name. In the case that you do something like:

set node1 [$ns node2]
set A $node1

The node will still be called node1 as that was the first variable to contain the reference.

If you are dealing with many nodes you may store them in array, perhaps like this:

for {set i 0} {$i < 4} {incr i} {
   set nodes($i) [$ns node]
}

In this case the names of the node will be nodes-0, nodes-1, nodes-2, nodes-3. (In other words, the ( character is replaced with -, and ) is removed.) This slightly different syntax is used to avoid any problems that () may cause later in the process. For example, the () characters cannot appear in DNS entries.

As a final note, everything said above for nodes applies equally to lans. I.e.:

set lan0 [$ns make-lan "$node0 $node1" 100Mb 0ms]
tb-set-lan-loss $lan0 .02

(Again, note the $'s)

Links can also be named just like nodes and lans. The names can then be used to set loss rates or IP addresses. This technique is the only way to set such attributes when there are multiple links between two nodes.

set link1 [$ns duplex-link $node0 $node1 100Mb 0ms DropTail]
tb-set-link-loss $link1 0.05
tb-set-ip-link $node0 $link1 10.1.0.128

Captured NS file parameters

A common convention when writing NS files is to place any parameters in an array named "opt" at the beginning of the file. For example:

    set opt(CLIENT_COUNT) 5
    set opt(BW) 10mb;    Link bandwidth
    set opt(LAT) 10ms;   Link latency

   ...

    $ns duplex-link $server $router $opt(BW) $opt(LAT) DropTail

    for {set i 0} {$i < $opt(CLIENT_COUNT)} {incr i} {
        set nodes($i) [$ns node]
	...
    }

    set serverprog [$server program-agent -command "starter.sh"]
Normally, this convention is only used to help organize the parameters. In Emulab, however, the contents of the "opt" array are captured and made available to the emulated environment. For instance, the parameters are added as environment variables to any commands run by program-agents (only available on recent FBSD410-STD and RHL90-STD images). So, in the above example of NS code, the "starter.sh" script will be able to reference parameters by name, like so:
    #! /bin/sh

    echo "Testing with $CLIENT_COUNT clients."
    ...
Note that the contents of the "opt" array are not ordered, so you should not reference other parameters and expect the shell to expand them appropriately:
    set opt(prefix) "/foo/bar"
    set opt(BINDIR) '$prefix/bin'; # BAD

    set opt(prefix) "/foo/bar"
    set opt(BINDIR) "$opt(prefix)/bin"; # Good

Ordering Issues

tb- commands have the same status as all other Tcl and NS commands. Thus the order matters not only relative to each other but also relative to other commands. One common example of this is that IP commands must be issued after the links or lans are created.


Hardware Commands

tb-set-hardware

tb-set-hardware node type [args].

tb-set-hardware $node3 pc
tb-set-hardware $node4 shark
node - The name of the node.
type - The type of the node.

Notes:


IP Address Commands

Each node will be assigned an IP address for each interface that is in use. The following commands will allow you to explicitly set those IP addresses. IP addresses will be automatically generated for all nodes that you do not explicitly set IP addresses.

In the common case the IP addresses on either side of a link must be in the same subnet. Likewise, all IP addresses on a LAN should be in the same subnet. Generally the same subnet should not be used for more than one link or LAN in a given experiment, nor should one node have multiple interfaces in the same subnet. Automatically generated IP addresses will conform to these requirements. If part of a link or lan is explicitly specified with the commands below then the remainder will be automatically generated under the same subnet.

IP address assignment is deterministic and tries to fill lower IP's first, starting at 2. Except in the partial specification case (see above), all automatic IP addresses are in the network 10.

tb-set-ip

tb-set-ip node ip

tb-set-ip $node1 142.3.4.5
node - The node to assign the IP address to.
ip - The IP address.

Notes:

tb-set-ip-link

tb-set-ip-link node link ip

tb-set-ip-link $node0 $link0 142.3.4.6
node - The node to set the IP for.
link - The link to set the IP for.
ip - The IP address.

Notes:

tb-set-ip-lan

tb-set-ip-lan node lan ip

tb-set-ip-lan $node1 $lan0 142.3.4.6
node - The node to set the IP for.
lan - The lan the IP is on.
ip - The IP address.

Notes:

tb-set-ip-interface

tb-set-ip-interface node dst ip

tb-set-ip-interface $node2 $node1 142.3.4.6
node - The node to set the IP for.
dst - The destination of the link to set the IP for.
IP - The IP address.

Notes:

tb-set-netmask

tb-set-netmask lanlink netmask

tb-set-netmask $link0 "255.255.255.248"
lanlink - The lan or link to set the netmask for.
netmask - The netmask in dotted notation.

Notes:


OS Commands

tb-set-node-os

tb-set-node-os node os

tb-set-node-os $node1 FBSD-STD
tb-set-node-os $node1 MY_OS
node - The node to set the OS for.
os - The id of the OS for that node.

Notes:

tb-set-node-rpms

tb-set-node-rpms node rpms...

tb-set-node-rpms $node0 rpm1 rpm2 rpm3

Notes:

tb-set-node-startcmd

tb-set-node-startcmd node startupcmd

tb-set-node-startcmd $node0 "mystart.sh -a >& /tmp/node0.log"

Notes:

tb-set-node-cmdline

tb-set-node-cmdline node cmdline

tb-set-node-cmdline $node0 {???}

Notes:

tb-set-node-tarfiles

tb-set-node-tarfiles node install-dir1 tarfile1 ...
The tb-set-node-tarfiles command is used to install one or more tar files onto a node's local disk. This command is useful for installing files that are used frequently, but will change very little during the course of your experiments. For example, if your software depends on a third-party library not provided in the standard disk images, you can produce a tarball and have the library ready for use on all the experimental nodes. Another example would be the data sets for your software. The benefit of installing files using this method is that they will reside on the node's local disk, so your experimental runs will not be disrupted by NFS traffic. Finally, you will want to avoid using this command if the files are changing frequently because the tars are only (re)installed when the nodes boot.

Installing individual tar files or RPMs is a midpoint in the spectrum of getting software onto the experimental nodes. At one extreme, you can read everything over NFS, which works well if the files are changing constantly, but can generate a great deal of strain on the control network and disrupt your experiment. The tar files and RPMs are also read over NFS when the nodes initially boot, however, there won't be any extra NFS traffic while you are running your experiment. Finally, if you need a lot of software installed on a large number of nodes, say greater than 20, it might be best to create a custom disk image. Using a disk image is easier on the control network since it is transferred using multicast, thus greatly reducing the amount of NFS traffic when the experiment is swapped in.

Required Parameters:

Example usage:
  # Overwrite files in /bin and /sbin.
  tb-set-node-tarfiles $node0 /bin /proj/foo/mybinmods.tar /sbin /proj/foo/mysbinmods.tar

  # Programmatically generate the list of tarballs.
  set tb [list]
  # Add a tarball located on a web site.
  lappend tb / http://foo.bar/bazzer.tgz
  # Add a tarball located in the Emulab NFS space.
  lappend tb /usr/local /proj/foo/tarfiles/bar.tar.gz
  # Use 'eval' to expand the 'tb' list into individual
  # arguments to the tb-set-node-tarfiles command.
  eval tb-set-node-tarfiles $node1 $tb
See also: Notes:
  1. Because the files are installed as root, care must be taken to protect the tar file so it cannot be replaced with a trojan that allowed less privileged users to become root.
  2. Currently, you can only tell how/why an installation failed by examining the node's console log on bootup.

Link Loss Commands

This is the NS syntax for creating a link:

$ns duplex-link $node1 $node2 100Mb 150ms DropTail

Note that it does not allow for specifying link loss rates. Emulab does, however, support link loss. The following commands can be used to specify link loss rates.

tb-set-link-loss

tb-set-link-loss src dst loss
tb-set-link-loss link loss

tb-set-link-loss $node1 $node2 0.05
tb-set-link-loss $link1 0.02
src, dst - Two nodes to describe the link.
link - The link to set the rate for.
loss - The loss rate (between 0 and 1).

Notes:

tb-set-lan-loss

tb-set-lan-loss lan loss

tb-set-lan-loss $lan1 0.3
lan - The lan to set the loss rate for.
loss - The loss rate (between 0 and 1).

Notes:

tb-set-node-lan-delay

tb-set-node-lan-delay node lan delay

tb-set-node-lan-delay $node0 $lan0 40ms
node - The node we are modifying the delay for.
lan - Which LAN the node is in that we are affecting.
delay - The new node to switch delay (see below).

Notes:

tb-set-node-lan-bandwidth

tb-set-node-lan-bandwidth node lan bandwidth

tb-set-node-lan-bandwidth $node0 $lan0 20Mb
node - The node we are modifying the bandwidth for.
lan - Which LAN the node is in that we are affecting.
bandwidth - The new node to switch bandwidth (see below).

Notes:

tb-set-node-lan-loss

tb-set-node-lan-loss node lan loss

tb-set-node-lan-loss $node0 $lan0 0.05
node - The node we are modifying the loss for.
lan - Which LAN the node is in that we are affecting.
loss - The new node to switch loss (see below).

Notes:

tb-set-node-lan-params

tb-set-node-lan-params node lan delay bandwidth loss

tb-set-node-lan-params $node0 $lan0 40ms 20Mb 0.05
node - The node we are modifying the loss for.
lan - Which LAN the node is in that we are affecting.
delay - The new node to switch delay.
bandwidth - The new node to switch bandwidth.
loss - The new node to switch loss.

Notes:

tb-set-link-simplex-params

tb-set-link-simplex-params link src delay bw loss

tb-set-link-simplex-params $link1 $srcnode 100ms 50Mb 0.2
link - The link we are modifying.
src - The source, defining which direction we are modifying.
delay - The source to destination delay.
bw - The source to destination bandwidth.
loss - The source to destination loss.

Notes:

tb-set-lan-simplex-params

tb-set-lan-simplex-params lan node todelay tobw toloss fromdelay frombw fromloss

tb-set-lan-simplex-params $lan1 $node1 100ms 10Mb 0.1 5ms 100Mb 0
lan - The lan we are modifying.
node - The member of the lan we are modifying.
todelay - Node to lan delay.
tobw - Node to lan bandwidth.
toloss - Node to lan loss.
fromdelay - Lan to node delay.
frombw - Lan to node bandwidth.
fromloss - Lan to node loss.

Notes:

tb-set-endnodeshaping

tb-set-endnodeshaping link-or-lan enable

tb-set-endnodeshaping $link1 1
tb-set-endnodeshaping $lan1 1
link-or-lan - The link or LAN we are modifying.
enable - Set to 1 to enable, 0 to disable.

Notes:

tb-set-noshaping

tb-set-noshaping link-or-lan enable

tb-set-noshaping $link1 1
tb-set-noshaping $lan1 1
link-or-lan - The link or LAN we are modifying.
enable - Set to 1 to disable bandwidth shaping, 0 to enable.

Notes:

tb-use-endnodeshaping

tb-use-endnodeshaping enable

tb-use-endnodeshaping 1
enable - Set to 1 to enable end-node traffic shaping on all links and LANs.

Notes:

tb-force-endnodeshaping

tb-force-endnodeshaping enable

tb-force-endnodeshaping 1
enable - Set to 1 to force end-node traffic shaping on all links and LANs.

Notes:

tb-set-multiplexed

tb-set-multiplexed link allow

tb-set-multiplexed $link1 1
link - The link we are modifying.
allow - Set to 1 to allow multiplexing of the link, 0 to disallow.

Notes:


Virtual Type Commands

Virtual Types are a method of defining fuzzy types. I.e. types that can be fulfilled by multiple different physical types. The advantage of virtual types (vtypes) is that all nodes of the same vtype will usually be the same physical type of node. In this way, vtypes allows logical grouping of nodes.

As an example, imagine we have a network with internal routers connecting leaf nodes. We want the routers to all have the same hardware, and the leaf nodes to all have the same hardware, but the specifics do not matter. We have the following fragment in our NS file:

...
tb-make-soft-vtype router {pc600 pc850}
tb-make-soft-vtype leaf {pc600 pc850}

tb-set-hardware $router1 router
tb-set-hardware $router2 router
tb-set-hardware $leaf1 leaf
tb-set-hardware $leaf2 leaf

Here we have set up two soft (see below) vtypes, router and leaf. Our router nodes are then specified to be of type router, and the leaf nodes of type leaf. When the experiment is swapped in the testbed will attempt to make router1 and router2 be of the same type, and similarly, leaf1 and leaf2 of the same type. However, the routers/leafs may be pc600s or they may be pc850s, whichever is easier to fit in to the available resources.

As a basic use, vtypes can be used to request nodes that are all the same type, but can be of any available type:

...
tb-make-soft-vtype N {pc600 pc850}

tb-set-hardware $node1 N
tb-set-hardware $node2 N
...

Vtypes come in two varieties, hard and soft. With soft vtypes, the testbed will try to make all nodes of that vtype the same physical type, but may do otherwise if resources are tight. Hard vtypes behave just like soft vtypes except that the testbed will give higher priority to vtype consistency and swapping in will fail if the vtypes can not be satisfied. So, if you use soft vtypes you are more likely to swap in but there is a chance your node of a specific vtype will not all be the same. If you use hard vtypes all nodes of a given vtype will be the same, but swapping in may fail.

Finally, you can have weighted soft vtypes. Here you assign a weight from 0 to 1 exclusive to your vtype. The testbed will give higher priority to consistency in the higher weighted vtypes. The primary use of this is to rank multiple vtypes by importance of consistency. Soft vtypes have a weight of 0.5 by default.

As a final note, when specifying the types of a vtype, use the most specific type possible. For example: tb-make-soft-vtype router {pc pc600}, is not very useful, as pc600 is a sub type of pc. You may very well end up with two routers as type pc with different hardware, as pc covers multiple types of hardware.

tb-make-soft-vtype

tb-make-soft-vtype vtype {types}
tb-make-hard-vtype vtype {types}
tb-make-weighted-vtype vtype weight {types}

tb-make-soft-vtype router {pc600 pc850}
tb-make-hard-vtype leaf {pc600 pc850}
tb-make-weighted-vtype A 0.1 {pc600 pc850}
vtype - The name of the vtype to create.
types - One or more physical types.
weight - The weight of the vtype, 0 < weight < 1.

Notes:

Misc. Commands

tb-fix-node

tb-fix-node vnode pnode

tb-fix-node $node0 pc42
vnode - The node we are fixing.
pnode - The physical node we want used.

Notes:

tb-set-uselatestwadata

tb-set-uselatestwadata 0
tb-set-uselatestwadata 1

Notes:

tb-set-wasolver-weights

tb-set-wasolver-weights delay bw plr

tb-set-wasolver-weights 1 10 500
delay - The weight to give delay when solving.
bw - The weight to give bandwidth when solving.
plr - The weight to give lossrate when solving.

Notes:

tb-set-node-failure-action

tb-set-node-failure-action node action

tb-set-node-failure-action $nodeA "fatal"
tb-set-node-failure-action $nodeB "nonfatal"
node - The node name.
action - One of "fatal" or "nonfatal"

Notes: