ipassign_wrapper no longer does that. Revert back to always invoking
the static route calculator (which figures out what to do). Remove the
stiff about generating a linkmap, which was an interim measure for
ddijk routing; now done with gentopfile.

gentopofile.  Note that the call to "staticroutes -j" is still there
while we transition over; eventually the -j option (which created an
input file for ddijkstra) will be removed, since the client side
rc.routes will generate that file on the remote node using the topomap
generated by gentopofile.

currently I run this all the time regardless of routertype or
use_ipassign flags, since its really quick. Will improve later.

from commvirtsig-branch into main trunk

Two testsuite tests fail: tbcmd and vtypes . Need to fix.

of virt_tables so that it is saved and restored like the rest of
the virtual state.

state machine (state). All of the stuff that was previously handled by
using batchstate is now embedded into the one state machine. Of
course, these mostly overlapped, so its not that much of a change,
except that we also redid the machine, adding more states (for
example, modify phases are now explicit. To get a picture of the
actual state machine, on boss:

		stategraph -o newstates EXPTSTATE
		gv newstates.ps

Things to note:

* The "batchstate" slot of the experiments table is now used solely to
  provide a lock for batch daemon. A secondary change will be to
  change the slot name to something more appropriate, but it can
  happen anytime after this new stuff is installed.

* I have left expt_locked for now, but another later change will be to remove
  expt_locked, and change it to active_busy or some such new state name in
  the state machine. I have removed most uses of expt_locked, except those
  that were necessary until there is a new state to replace it.

* These new changes are an implementation of the new state machine,
  but I have not done anything fancy. Most of the code is the same as
  it was before.

* I suspect that there are races with the batch daemon now, but they
  are going to be rare, and the end result is probably that a
  cancelation is delayed a little bit.

swapped out (non-recoverable) by tbswap. swapexp was leaving the
experiment in the running state instead of paused. We need to check
this after tbswap since we do not get reasonable error codes back.
Also some cleanup with respect to how aborted modifies are handled.
I think I understand what Chad did ...

A general comment; we need to be better about returning meaningful
error codes!

not have to wait 3 minutes for it to finish before he can watch his
experiment swapin fail for some other reason.

I adopted the same pid mechanism as in eventsys_control.in, which uses
a slot in the experiments table.

Running "prerender" puts the render into the background and stores
the pid. Running "prerender -r" kills a running prerender and removes
the existing info from the DB.

Fixed the problem with swapmod not restoring the old vis; swapmod now
kills any running prerender, and restarts one if the swapmod fails
(the prerun of the new NS file starts up another prerender in the
background).

Add setpriority() call in prerender to nice it and children to 15.

files.

Various Other changes to get Expt Modify ready for prime time.

 - If assign fails on a modify, experiment will
   be restored to old state, *not* swapped out.

 - Reboot option has been improved to reboot all
   nodes as part of os_setup, not in separate
   step.

 - Different assign error codes result in different
   retry behavior for assign_wrapper
   (Follow's Rob's change to assign to make it
    pass back special code for non-retriable faults)

 - '64' bit in assign_wrapper exit code indicates to tbswap
   that db/phys state hadn't been mucked with before
   the exit occurred
   (ergo, '65' and '1' are the common return codes,
    though the old 4,8,16,32 are still there for assign failing.)

 - (tbswap still returns codes from assign wrapper)

 - Added 5 sec pause between assign attempts.

 - Cleaned up tbswap code.

 - Physical state backup/restore removed from tbprerun,
   put into swapexp.

 - Interfaces table now getting cleaned up correctly
   (Mike noticed problem)

 - Changed menu display in showexp to show
   the "modify" menu option for swapped out experiments
   (like it used to.)

 - A couple other changes.

Note:
 Still admin-only, but I plan to change that soon.

To do:
 - Erase expt backups in /tmp after using them.
 - Re-viz failed experiments.

get an updated copyrights message.

Added new feature 'Experiment Modify'.
Now available (to admins only for now) from the showexp page.

Warning! doing a modify which alters the topology will probably
require a "reboot all nodes" afterwards.
(There will be a checkbox soon in the modify experiment page.)

Adding/removing delay nodes seems to work fine without reboots, though.

Warning! If the new version of the experiment cannot be mapped
 (not enough nodes available, for instance) the experiment will be
 swapped out! This will get fixed later.

Prerun backs up the experiment topology, so using a bad NS
file doesn't result in experiment termination.

As part of this, added library functions to libdb to
delete, backup, and restore both virtual and physical experiment state.

Stored in $TB/www/thumbs/,
with filenames generated by hashing random data with experiment name.
(Stored in vis_experiments table in db.)

New visualizer; uses "vis_nodes" table in database to store precomputed solutions.
Added "prerender" call to tbprerun and tbend.
"prerender_all" can be used to precompute vis solutions for all experiments.
Also, removed legacy vis cruft.

parser runs. The staticroutes script is a wrapper for Chad's route
solver. The network optimization is currently turned off; use -t to
turn it on, until I know if its correct.

Note that Chad gets credit for routecalc.cc; I'm just committing the
file for him, with a couple of trivial changes that I made.

	# Turn on manual routing.
	$ns rtproto Manual

	# Set manual routes
	$nodeA add-route $nodeC $nodeB
	$nodeC add-route $nodeA $nodeB

results in this information being returned from the tmcd routing
command:

	ROUTERTYPE=manual
	ROUTE DEST=192.168.2.3 DESTTYPE=host DESTMASK=255.255.255.0 \
		NEXTHOP=192.168.3.2 COST=0

The reason for DESTTYPE and DESTMASK is so that we can also support
routing to links and lans, since doing it on a per host basis if not
only hugely tedious, but plain impossible if the destination node has
multiple links; the add-route syntax takes a node, but we need the IP
of the relevant link in order to run the route add commands on the
nodes. So, I've "extended" the syntax of add-route so that you can
give it a Link or a Lan as the dest:

	$nodeA add-route $link0 $nodeB
	$nodeA add-route [$ns link $nodeB $nodeC] $nodeB

In this case, the DESTTYPE=net, and the netmask is no longer ignored;
it is used in the route add command. Currently, the mask is hardwired
in the DB to 255.255.255.0, but by providing it in the tmcd command,
we change it later if needed.

I did not implement add-route-to-adj-node since that is not really
useful in our context, and we definitely do not want the user to
change the default routes on his nodes. But, its easy to add if we
need to.

The client side stuff is not done yet.

provides a mapping from an object ($cbr0) to the node on which the
agent is running ($nodeA). It also includes the type of agent
(TRAFGEN, LINK, etc). There were a number of reasons for adding this
table:

* To avoid a series of specialized table lookups in the event
  scheduler to map from a name (link0) to the node on which the agent
  is running. Previously, it was looking the delays table and the
  virt_trafgens table. Well, now it just needs to look at this one
  table and store the mapping internally. When a dynamic event comes
  in, we can figure out where to send it easily.

* For NSE traffic generation. Unlike the simple TG based CBRs, Shashi
  wants to be able to send events to any of the objects in the config
  (udp0, telnet0, ftp0, etc). Well, the virt_trafgens table certainly
  does not store that info, and it would have been painful to work
  this into it. For every agent/application, just add an entry in the
  virt_agents table and the scheduler knows where to send the events.

* It nicely supports the new program object; just plug an entry in the
  virt_agents table.

Okay, there is one messy aspect; delays nodes! Delay nodes are not
computed until after assign wrapper runs, so in addition to munging
the static event list in assign_wrapper, we now munge the virt_agents
table as well. Not much to do about this; delay nodes are handled
outside the normal path everywhere.

Telnet sources)

assign_wrapper.in: Hack in a change that ensures a delay node is
created for any link on which an event is posted (up,down,modify),
no matter what its initial parameters are. ie: If a link is created
with no delay, but there is an event that adds a delay later, then we
must drop in a delay node. Same for up/down on a link. We do this in
the delay node. I am reasonably confident that this change is fine for
duplex links, but I am less sure of the effect on lans!

eventsys_control.in: Checkpoint latest changes. Add "replay" option,
which right now just stops and starts the event scheduler so that it
reloads the entire event list. Add check for existing experiment, and
that the experiment is either active or swapping (do not want to start
a scheduler for a swapped out experiment!). Add check to see if there
are any events, and skip startup if there are not events in the DB.
Lastly, get very serious about preventing more than one scheduler from
being started, either by accident or intentionally. My protocol is to
lock the table, grab and set the pid to -pid, test the pid for a
positive value, and if positive, send the scheduler a kill(TERM) so
that it can cleanup, clear the pid to zero in the DB, and exit. This
approach ensures that we do not try to send a kill to a pid that is no
longer active or owned by the user (this last part is not really
necessary cause of how pids are reused, but it was easy to add so why
not).

exports_setup.in: Trivial change to make it easier to turn this on
temporarily in devel trees.
named_setup.in: Ditto.

node_reboot.in: Add call to TBdbfork() in child cause of apparent DB
connection problems across forks. In the child, set the eventstatus
for the node to REBOOT if successful (not this event status stuff is
temporary, will be recast in next set of revisions).

GNUmakefile:  Add new controlling program, eventsys_control.
power.in:     Ditto previous comment about REBOOT.
os_setup.in:  Non event system cleanups.
tbend.in:     Add DB cleanup of the new virt_trafgens and eventlist tables.
tbprerun.in:  Ditto.
tbreport.in:  Print out the event list in a pretty print format.
tbswapin.in:  Add call to start the event system. Also a big fix; move
              the named script up above the os_setup so that the named
              tables have been updated by the time the first node
              reboots. I noticed that nodes were failing on gethostbyname().
tbswapout.in: Add call to stop the event system.

an infinite loop rather easily via the NS file TCL hooks. Added a
perl wrapper around parse.tcl called parse-ns, which forks a child to
run the parser. The parser is invoked "nice +10" and the CPU limit for
the child is set to 60 seconds, which should be enough for any parse.
If the limit is exceeded, send email to tbops since this indicates a
big problem or a user being dumb/malicious.

to use the DB library access routines, which also changed in response
to what the tb scripts needed. Added some functions and mor constants.
Removed the -nologfile option from all the scripts (startexp and
endexp too), since there is no reason for these scripts to worry about
log files. Thats handled in the wrappers. Tested with the testsuite
and live in my own tree.

Lots of small changes for turning our 'require lib*' lines into 'use lib*' lines. Proper modules declare themselves as a package, and use Exporter to export the names of the subroutines that should be visible from the outside world. Many of ours didn't do that, it was just a file with a bunch of subs in it. So now I've fixed many of them to be proper, and removed the requires and 'push(@INC,...)' hacks and changed it to the proper 'use lib @prefix@/lib/;' and use lib*.

Fixed some warnings in assign_wrapper that were subsequently caught.

exitonwarn is a library that, when used in a script, causes it to exit 255
when warnings are sent.  It still shows all warnings, just does an ugly
perl hack to error on script termination.

with the testing stuff.

them on the first run to accurate predictions.

fixes we missed.

tbprerun - Fixed estimations to be slightly more accurate.  Hard to get
complete accuracy without duplicating much of assign_wrapper.  This'll
probably change in someway in the near future.

replaced by the "images" table. New os_info table is added. New web
pages to add and delete OSIDs to/from the os_info table, for use in
the NS file. tb-create-os is gone. handle_os no longer operates on the
tbcmds file, and no longer writes anything into the ir file. Moved the
setting up of os state (nodes table) from os_setup to handle_os, where
it should be. os_load and sched_reload now take a single argument, the
name of the imageid from the images table.

the batch daemon knows when configuraton failed cause there were not
enough nodes. The batch daemon uses this in an information manner
right now, but later we should kick out experiments that fail for any
reason other than lack of resources.

Include assign.log in email message when assign fails.

a safe place in /usr/testbeb/expinfo for archival purposes.