GitLab

on fatal errors, and error codes >= 1 for assign errors. The batch
system depends on these error codes to determine if a batch needs
to be punted from the system.

and virt_lans to determine which links/lans need to be dropped out of
the list given to assign (.top file). Basically, if a simple link has
virtual node in it, get rid of the link (both sides). If a lan has a
virtual node in it, then get rid of just that member of the lan (the
other regular members need to be allocated as usual). There is a minor
problem of a lan with just one physical member left after dropping the
virtual members; not sure what to do about that so I punted on it for
now.

Note that I *do* send the virt_nodes into assign for allocation as
unconnected nodes. Its up to the user to create the routes properly,
although maybe we have to do that automatically?

provides a mapping from an object ($cbr0) to the node on which the
agent is running ($nodeA). It also includes the type of agent
(TRAFGEN, LINK, etc). There were a number of reasons for adding this
table:

* To avoid a series of specialized table lookups in the event
  scheduler to map from a name (link0) to the node on which the agent
  is running. Previously, it was looking the delays table and the
  virt_trafgens table. Well, now it just needs to look at this one
  table and store the mapping internally. When a dynamic event comes
  in, we can figure out where to send it easily.

* For NSE traffic generation. Unlike the simple TG based CBRs, Shashi
  wants to be able to send events to any of the objects in the config
  (udp0, telnet0, ftp0, etc). Well, the virt_trafgens table certainly
  does not store that info, and it would have been painful to work
  this into it. For every agent/application, just add an entry in the
  virt_agents table and the scheduler knows where to send the events.

* It nicely supports the new program object; just plug an entry in the
  virt_agents table.

Okay, there is one messy aspect; delays nodes! Delay nodes are not
computed until after assign wrapper runs, so in addition to munging
the static event list in assign_wrapper, we now munge the virt_agents
table as well. Not much to do about this; delay nodes are handled
outside the normal path everywhere.

getting other experiments node names!

this off, but here is what I did.

Parser: Allow for the following syntax

	set link0  [$ns duplex-link $nodeA $nodeB 100Mb 0ms RED]
	set queue0 [[$ns link $nodeA $nodeB] queue]
	$queue0 set gentle_ 1
	$queue0 set queue-in-bytes_ 0
	$queue0 set limit_ 50
	$queue0 set maxthresh_ 20
	$queue0 set thresh_ 7
	$queue0 set linterm_ 11
	$queue0 set q_weight_ 0.004

    NB: This differs from the NS syntax (and is the part that Chris
    needs to fix) in that there is just a single queue object per
    duplex link, thus the parameters cannot be set asymmetrically.
    Note, the delay node *does* use a RED/GRED queue in each
    direction, but its params are the same. These TCL hacks took a
    long time for me to get right!

    Also note that I have no idea how this stuff relates to LANS! I
    do not allow LANS to be created with RED queues (another item for
    Chris to work on perhaps?).

assign_wrapper: A horrible hack to pass the new fields added to
virt_lans onto the delays table setup. Also another minor hack ensure
that a delay node is added when a RED queue is used (for the case when
no other traffic shaping is done). At the moment, the virt_lans table
has a single set of fields, while the delays table has the double set;
one for each direction of the pipe. Here is a listing.

    alter table delays add q0_limit int default 0 after lossrate0;
    alter table delays add q0_maxthresh int default 0 after q0_limit;
    alter table delays add q0_minthresh int default 0 after q0_maxthresh;
    alter table delays add q0_weight float default 0.0 after q0_minthresh;
    alter table delays add q0_linterm int default 0 after q0_weight;
    alter table delays add q0_qinbytes tinyint default 0 after q0_linterm;
    alter table delays add q0_bytes tinyint default 0 after q0_qinbytes;
    alter table delays add q0_meanpsize int default 0 after q0_bytes;
    alter table delays add q0_wait int default 0 after q0_meanpsize;
    alter table delays add q0_setbit int default 0 after q0_wait;
    alter table delays add q0_droptail int default 0 after q0_setbit;
    alter table delays add q0_red tinyint default 0 after q0_droptail;
    alter table delays add q0_gentle tinyint default 0 after q0_red;

tmcd/tmcd.c: Change dodelays to pass back all of these fields (for
both pipes; there are equiv q1 fields in the delays table). Yikes! Its
done in a backwards compatable manner though, so existing delay nodes
will continue to work just fine.

tmcd/freebsd/liblocsetup.pm: Change the delays configuration script to
get all these fields and do something useful with. Of course, our
delay nodes cannot use a lot of these fields, but the information is
sent through for the eventuality that we have more sophisticated
delays nodes.

Test Suite: Add red test dir that has the above syntax as its test.

assign_wrapper.in: Hack in a change that ensures a delay node is
created for any link on which an event is posted (up,down,modify),
no matter what its initial parameters are. ie: If a link is created
with no delay, but there is an event that adds a delay later, then we
must drop in a delay node. Same for up/down on a link. We do this in
the delay node. I am reasonably confident that this change is fine for
duplex links, but I am less sure of the effect on lans!

eventsys_control.in: Checkpoint latest changes. Add "replay" option,
which right now just stops and starts the event scheduler so that it
reloads the entire event list. Add check for existing experiment, and
that the experiment is either active or swapping (do not want to start
a scheduler for a swapped out experiment!). Add check to see if there
are any events, and skip startup if there are not events in the DB.
Lastly, get very serious about preventing more than one scheduler from
being started, either by accident or intentionally. My protocol is to
lock the table, grab and set the pid to -pid, test the pid for a
positive value, and if positive, send the scheduler a kill(TERM) so
that it can cleanup, clear the pid to zero in the DB, and exit. This
approach ensures that we do not try to send a kill to a pid that is no
longer active or owned by the user (this last part is not really
necessary cause of how pids are reused, but it was easy to add so why
not).

exports_setup.in: Trivial change to make it easier to turn this on
temporarily in devel trees.
named_setup.in: Ditto.

node_reboot.in: Add call to TBdbfork() in child cause of apparent DB
connection problems across forks. In the child, set the eventstatus
for the node to REBOOT if successful (not this event status stuff is
temporary, will be recast in next set of revisions).

GNUmakefile:  Add new controlling program, eventsys_control.
power.in:     Ditto previous comment about REBOOT.
os_setup.in:  Non event system cleanups.
tbend.in:     Add DB cleanup of the new virt_trafgens and eventlist tables.
tbprerun.in:  Ditto.
tbreport.in:  Print out the event list in a pretty print format.
tbswapin.in:  Add call to start the event system. Also a big fix; move
              the named script up above the os_setup so that the named
              tables have been updated by the time the first node
              reboots. I noticed that nodes were failing on gethostbyname().
tbswapout.in: Add call to stop the event system.

supporting autocreating and autoloading images. The imageid form now
sports a field to specify a nodeid to create the image from; If set,
the backend create_image script is invoked. Thats the easy part.
Slightly harder is autoloading images based on the osid specified in
the NS file. To support this, I have added a new DB table called
osidtoimageid, which holds the mapping from osid/pctype to imageid.
When users create images, they must specify what node types that image
is good for. Obviously, the mappings have to be unique or it would be
impossible to figure it out! Anyway, once that image mapping is
in place and the image created, the user can specify that ID in the NS
file. I've changed os_setup to to look for IDs that are not loaded,
and to try and find one in the osidtoimageid. If found, it invokes
os_load. To keep things running in parallel as much as possible,
os_setup issues all the loads/reboots (could be more than a single set
of loads is multiple IDs are in the NS file) at once, and waits for
all the children to exit. I've hacked up os_load a bit to try and be
more robust in the face of PXE failures, which still happen and are
rather troublsesome. Need an event system!

Contained in this revision are unrelated changed to make the OS and
Image IDs per-project unique instead of globally unique, since thats a
pain for the users. This turns out to be very messy, since underneath
we do not want to pass around pid/ID in all the various places its
used. Rather, I create a globally unique name and extened the OS and
Image tables to include pid/name/ID. The user selects pid/name, and I
create the globally unique ID. For the most part this is invisible
throughout the system, except where we interface with the user, say in
the web pages; the user should see his chosen name where possible, and
the should invoke scripts (os_load, create_image, etc) using his/her
name not the internal ID. Also, in the front end the NS file should
use the user name not the ID. All in all, this accounted for a number
of annoying changes and some special cases that are unavoidable.

code had been added in a bunch of places to clear the data out.

The portmap table in the database can now be dropped.

	tb-set-node-routertype $node none|gated

Added slots to virt_nodes and nodes table. This stuff is carried
through so that tmcd can get at it and pass it back to client.

did the lanlink change. assign_wrapper does not really need any
changes at all, but I dropped a comment in. The only real change is
needed in the client side to convert a floating point from tmcd, into
an int to give to ipfw. Rather cheesy too, given that Perl does not
have a real conversion function (int() truncates).

the virtual topology and the physical topology. See equiv changes to
constants in ptopgen. The interface_types table is also changed to
reflect Kbps instead of Mbps.

strings.

is a control interface. This can occur if you swap an experiment out, and
swap it back in on a different hardware type.

problems with assign though, which uses Mbs. Do that in next step.

stuff so that the web page did not need to do anything except display
and form processing. Add tbsetup/node_control for backend so that it
can be called from the command line too. The virt_nodes table is also
updated (for those values that have virt_nodes equivalents), and this
mostly implies that changes can be applied only to swapped in
experiments since we use the reserved table to map pcXXX to its vname
so that the virt_nodes table can be updated. It is an easy extension
to allow changes based on the pid/eid/vname, but I do not see a reason
to support this ability yet. Note usage:

    Usage: node_control name=value [name=value ...] node [node ...]
           node_control -e pid,eid name=value [name=value ...]
           node_control -l
    For multiword values, use name='word0 ... wordN'
    Use -l to get a list of operational parameters you can change.
    Use -e to change parameters of all nodes in an experiment.

    {824} stolle...

hold separate values for each of the two pipes. This allows us to
treat the links asymmetrically with respect to the delay node
configuration. Changes to tmcd to return the expanded info, and rework
(completely) delay_config to allow user to change the duplex
characteristics (just like before), or just simplex characteristics by
specifying the pipe number. By convention (I invented) pipe 0 is
traffic coming in on iface0. Of course, none of the front end stuff
has been changed, so the changing a single pipe does not cause
anything in virt_lans to be changed, so those changes will be lost at
swapout. Also get rid of all the ssh'ed pipe config stuff in
delay_config. Instead, add a client side script called update_delays
to /etc/testbed on the experiment nodes. After updating the delays
table, just ssh over and run that script, which calls the existing
code. Less distribution of knowledge.

we can now handle cross-switch links.

to ptopgen to use just one switch, and pass that from assign. Only
until new snmpit is ready.

that assign does not handle well.

Also moved most subroutines to the end of the file, to match the style of
our other scripts.

Fixed case where virt_nodes has no osid because they didn't choose one, and asked for a class of nodes instead of a type. now if they don't have an os, we choose the default os for the type of node that was chosen.

Fixed some warnings in assign_wrapper that were subsequently caught.

exitonwarn is a library that, when used in a script, causes it to exit 255
when warnings are sent.  It still shows all warnings, just does an ugly
perl hack to error on script termination.

delay nodes for 10Mbit links.  This last feature is currently disabled
until we get the backend in place to set port speeds.

1. 'tb-set-hardware ... shark' and 'tb-set-hardware ... dnard' are now
functionally identical.  Previously only the former worked but both passed
the parser.

2. Assign will now exit very quickly in the case that, for a given virtual
nodes, there are no physical nodes that could match in type.  This should
never happen as the parser and assign_wrapper have checks that usually
prevent this.  However, in the case of problems in the code (such as #1)
this'll make it easier to debug.  In addition, as we add more types of
nodes and our estimates becoming increasingly inaccurate cases where this
might occur could slip in.  All calling code treats this identically to an
'insufficient resources' failure.

Many changes and updates for handling new types. The db now has types like 'pc600', 'pc850', and 'dnard', and each type has a class like 'pc' or 'shark'. This updates scripts that use types to use classes where appropriate, and to handle the new types where there were hardcoded things that couldn't be eliminated right now.

to tell it how to parse interswitch links from assign.

It still assumes a single VLAN domain.

a bootstatus field to the nodes table. os_setup sets this to one of
okay, failed, unknown. This is to be used with the still to be defined
method of specifying certain nodes that can fail reboot on experiment
creation. Right now sharks are wired to this, and this information is
presented in the web page. Its also essential for the batch system,
which needs to consider nodes that failed to reboot, or else batch
experiments would never end. Might still need a way for an experiment
to tell the batch system its done though.