tbswap is doing a swapretry (this happens during a normal swapin, when
a node fails). In this case, we want to fix all the nodes (overriding
user setting) when replacing failed nodes!

a 100Mb link is mapped to a trivial link, a linkdelay has to be
inserted since the actual loopback bandwidth is much higher. The
problem is that unlike other delays where requires_delay() will return
true, this decision has to wait until after assign does the mapping
and we find out that a link was mapped to a trivial link. When this
happens, I insert a linkdelay "on the fly" when processing the link
statements from assign.

hash.

This means we have to call LoadVirtTypes() from a slightly different
place, so that we have the virt types before calling LoadPhysInfo(),
where %node_type_linkbw is built.

from commvirtsig-branch into main trunk

Two testsuite tests fail: tbcmd and vtypes . Need to fix.

node type, but allow it to be overridden with tb-set-jail-os().

other changes to the commvirtsig branch.

* Run assign with nice; when assign is cranking (or when two are
  cranking!) boss gets really sluggish (web, DB).

* Minor change to "impotent" option; run a little longer and print out
  what the vlan table would look like.

* Two trivial grammer changes.

parser with new functions tb-set-jail-os() and tb-set-delay-os()).
Prefer these over anything in the node_types table. Makes it easier to
test new images in the various configurations.

pnode. Used to no be allowed, but with multiple routing tables this is
now possible. My code to construct link names on the fly when delay
nodes are inserted, was generating non-unique names which was causing
vlans to be dropped.

vnode and a PC node, under normal circumstances, a real physical link
is used. However, a user can use tb-set-multiplexed on the link that
allows assign to map multiple vlinks over the same physical link.
This is fine when the PC is running FBSD-STD which can use veth
interfaces. Not so if it is running Linux or a non-standard BSD.
This fix forces EMULATED links and VETH devices when the PC is running
FBSD-STD and turns those attributes off otherwise so that assign
will not give a mapping that cannot be realized on the client side.
For the gory details of this problem, look at testbed-ops email thread
"pcvm route problem" dated 29th Jan 2004.

A problem still exists in the non-BSD router + jail vnodes in a delayed
LAN. VLANs table is not getting the correct entries. A fix will come
soon. Follow the testbed-ops email thread "fix for the pcvm route problem"
dated 29th Jan 2004 for the gory details.

(or more than one simhost) is unable to keep up with real-time. It includes
changes to assign_wrapper to handle swap modify for simnodes, the simple
algorithm in nseswap that bumps up the nodeweight of simnodes being hosted
on a simhost that reports "can't keep up with real-time" (aka nse violation),
ptopgen and sim.tcl to prefer nodes that already have the FBSD-NSE image.
Also, changes to other files to send out NSESWAP event.

One unrelated change: We now have per-swapin .top files and assign.log
files along with .ptop files. This helps in debugging across multiple
swapins since files remain in the form of
<pid>-<eid>-<process_id>.{top,ptop} and assign-<pid>-<eid>-<process_id>.log
Also useful for archiving.

again.

One of the fixes changes the way in which we iterate through pclasses
in find_pnode(). We used to treat the vector like a ring buffer, and
start (randomly) someplace in the middle. This turns out to give some
bad statistical properties when doing dynamic pclasses, since long
chains of disabled pclasses will cause some pclasses to be selected
more often. My old hack of just hopping around randomly in the
disabled-pclass case was bad, because it's hard to tell when you've
actually tried all the pclasses - so, we were getting false negatives
where it was looking like there was no place available where we could
map a vnode, which turned out to have worse effects than I had
thought.

So, now, we make a list of all the indices and randomize the order,
then just iterate through that list.

We also now count the number of pclasses that are enabled at every
temperature step, and adjust the neighborhood size to remove them.
This makes dynamic ...

not skipped in the case of 10Mb 0ms links.

Change ptopgen to look at the eventstate of a node; a node is not considered
free unless it is ISUP or PXEWAIT.

Add TBAvailablePCs() to libdb, removing the corresponding code from
assign_wrapper. This routine does the equiv of ptopgen, returning the
number of PCs that are available for use (looking at eventstate).

Change TBFreePCs in the web interface accordingly.

The above changes correspond to an upcoming change in stated.

now mapped to more than one PC if required. The simnode_capacity
column in the node_types table determines how many sim nodes can
be packed on one PC. The packing factor can also be controlled via
tb-set-colocate-factor to be smaller than simnode_capacity.

- No frontend code changes. To summarize:
  $ns make-simulated {
    ...
  }
  is still the easy way to put a whole bunch of Tcl code to be
  in simulation.
  One unrelated fix in the frontend code is to fix the
  xmlencode() function which prior to this would knock off
  newlines from columns in the XML output. This affected
  nseconfigs since it is one of the few columns with embedded
  newlines. Also changed the event type and event object type
  in traffic.tcl from TRAFGEN/MODIFY to NSE/NSEEVENT.

- More Tcl code in a new directory tbsetup/nseparse
  -> Runs on ops similar to the main parser. This is invoked
     from assign_wrapper in the end if there are simnodes
  -> Partitions the Tcl code into multiple Tcl specifications
     and updates the nseconfigs table via xmlconvert
  -> Comes with a lot of caveats. Arbitrary Tcl code such as user
     specified objects or procedures will not be re-generated. For
     example, if a user wanted a procedure to be included in Tcl
     code for all partitions, there is no way for code in nseparse
     to do that. Besides that, it needs to be tested more thoroughly.

- xmlconvert has a new option -s. When invoked with this option,
  the experiments table is not allowed to be modified. Also,
  virtual tables are just updated (as opposed to deleting
  all rows in the first invocation before inserting new rows)

- nse.patch has all the IP address related changes committed in
  iversion 1.11 + 2 other changes. 1) MTU discovery support in
  the ICMP agent 2) "$ns rlink" mechanism for sim node to real
  node links

- nseinput.tcl includes several client side changes to add IP
  routes in NSE and the kernel routing table for packets crossing
  pnodes. Also made the parsing of tmcc command output more robust
  to new changes. Other client side changes in libsetup.pm and other
  scripts to run nse, are also in this commit

- Besides the expected changes in assign_wrapper for simulated nodes,
  the interfaces and veth_interfaces tables are updated with
  routing table identifiers (rtabid). The tmcd changes are already
  committed. This field is used only by sim hosts on the client side.
  Of course, they can be used by jails as well if desired.

allows real PCs to run any OS instead of being constrained to FreeBSD. A
detailed discussion on this topic can be found in the testbed-dev mailing
list under the thread "links between vnodes and real PCs"

in the dev tree. I'm tired of problems on the mini that wastes
my time.

Changes include 2 new tmcd commands: tmcc routelist returns
the routes for all the vnodes hosted on a pnode. tmcc role
returns the role of a reserved node, like 'virthost' or
'simhost.  tmcc ifconfig now reports an RTABID field which
is calculated in assign wrapper. All the new changes
in assign wrapper will be checked in after I finish testing.

All the DB changes are in: simnode_capacity in node_types, rtabid in
interfaces and veth_interfaces. New NSE event_objtype and NSEEVENT
event_eventtype. Changed the erole field in the reserved table
to have 'simhost' instead of 'simnode'. Changed the correspoding
libdb subroutines.

more liberal about which links don't need delays. Before, we were
putting delays on anything that wasn't 100Mbps.

The other change is that for delay nodes, we set virtual link
bandwidths by rounding up to the next fastest speed of interface,
instead of hardwiring 100Mbps. Similarly, we set the speed of
physical delay interfaces to the card's 'native' speed instead of
hardwiring 100Mbps.

These changes work for our nodes and links, but are still not totally
general. Really, what needs to happen is that assign needs to be the
one to insert delay nodes, because it's the only place where we really
know which interfaces of what speeds are available on which nodes.
But, that is a very big project, since both assign wrapper and assign
need to be changed.

The idea is to give us hooks for grabbing experimenters' tarballs (and
RPMs) from locations other than files on ops. Mainly, to remove
another dependance on users having shells on ops.

tarfiles_setup supports fetching files from http and ftp URLs right
now, through wget. It places them into the experiment directory, so
that they'll go away when the experiment is terminated, and the rest
of the chain (ie. downloading to clients and os_setup's checks)
remains unchaged.  It is now tarfiles_setup's job to copy tarballs and
RPMs from the virt_nodes table to the nodes table for allocated nodes.
This way, it can translate URLs into the local filenames it
constructs. It get invoked from tbswap.

Does the actual fetching over on ops, running as the user, with
fetchtar.proxy.

Should be idempotent, so we should be able to give the user a button
to run webtarfiles_setup (none exists yet) yet to 'freshen' their
tarballs. (We'd also have to somehow let the experiment's nodes know
they need to re-fetch their tarballs.)

One funny side effect of this is that the separator in
virt_nodes.tarfiles is now ';' instead of ':' like nodes.tarballs,
since we can now put URLs in the former. Making these consistent is a
project for another day.

trivial simulated links. This is coz existing sim node support that
goes on 1 pnode can't handle this. Also added a 'sim' type in the
node_types table and a issimnode column in it.

Now, the old sim node support is working and demoable.

state machine (state). All of the stuff that was previously handled by
using batchstate is now embedded into the one state machine. Of
course, these mostly overlapped, so its not that much of a change,
except that we also redid the machine, adding more states (for
example, modify phases are now explicit. To get a picture of the
actual state machine, on boss:

		stategraph -o newstates EXPTSTATE
		gv newstates.ps

Things to note:

* The "batchstate" slot of the experiments table is now used solely to
  provide a lock for batch daemon. A secondary change will be to
  change the slot name to something more appropriate, but it can
  happen anytime after this new stuff is installed.

* I have left expt_locked for now, but another later change will be to remove
  expt_locked, and change it to active_busy or some such new state name in
  the state machine. I have removed most uses of expt_locked, except those
  that were necessary until there is a new state to replace it.

* These new changes are an implementation of the new state machine,
  but I have not done anything fancy. Most of the code is the same as
  it was before.

* I suspect that there are races with the batch daemon now, but they
  are going to be rare, and the end result is probably that a
  cancelation is delayed a little bit.

contain the latest image suitable for running nse as opposed to being
tied to an old FBSD 4.5 image.

does not reduce the number of nodes required by the number of already
allocated nodes. Even so, the test is fuzzy and assign might fail
anyway cause of node type requirements.

* Remove all of the code that dealt with allocating unconnected nodes.
  It used to be assign_wrapper passed all widearea node allocation
  decisions to wanassign, those in links and those that were
  unconnected. assign_wrapper now handles all unconnected nodes since
  assign is much better with features/desires and node type stuff.

* Do not modify any database state in wanassign; It used to do the
  actual nalloc calls, but now it just returns the mapping to
  assign_wrapper so that we can more easily track "recoverability" and
  because there is existing code in assign_wrapper to allocate vnodes
  on the selected pnodes. No point in duplication.

* Switch from mapping to vnodes, to mapping to pnodes. We made this
  change for other virtual nodes; instead of "fixing" to a vnode on a
  pnode, fix to the pnode. The resulting mappings are also given as
  pnodes, and assign_wrapper does the allocation on those selected
  nodes.

Now all we need is uptodate widearea data!

previous revision.

Bump maxrun for plab experiments from 1 to 2 (special case predated my
changes).

I have implemented the suggestion Jay made a couple of weeks ago
about allowing partial allocation in assign_wrapper, and retrying with a
modified set of "fixed" nodes.

My basic approach was to change nalloc to optionally allow partial
allocations, returning the number of nodes that could not be allocated as
its return value. In assign_wrapper, I determine which nodes we were able
to get (in each loop), set their allocstate to INIT_DIRTY, augment the
fixed_node set, and recreate the top file. Then I try again, up to the
current number of maxtries. If assign fails with an unretryable error, or
if we could not nalloc a user directed fixed node, then I stop right away
since the experiment is not going to map (in the near term) if the fixed
node list cannot be allocated.

I am confident that this works okay, although testing is a little
difficult. The main problem is how this interacts with experiment modify.
Chad's implementation is that a modify can be reverted (recovered from)
only as long as the DB is not modified by assign_wrapper. Well, a partial
allocation, followed by failure, obviously modifies the DB, and so is
deemed not recoverable. I am still trying to figure out the effects of
this, and whether I can relax this requirement, but in the meantime
lets install it and see what happens (won't affect many people).

the batch system see's them as always done. There is no reason to do
this from the node itself, since it would be really hard to have
either a jail or delay node without other nodes in the topology!

plus a lock field. The lock field was a simple "experiment locked, go away"
slot that is easy to use when you do not care about the actual state that
an experiment is in, just that it is in "transition" and should not be
messed with.

The other two state variables are "state" and "batchstate". The former
(state) is the original variable that Chris added, and was used by the tb*
scripts to make sure that the experiment was in the state each particular
script wanted them to be in. But over time (and with the addition of so
much wrapper goo around them), "state" has leaked out all over the place to
determine what operations on an experiment are allowed, and if/when it
should be displayed in various web pages. There are a set of transition
states in addition to the usual "active", "swapped", etc like "swapping"
that make testing state a pain in the butt.

I added the other state variable ("batchstate") when I did the batch
system, obviously! It was intended as a wrapper state to control access to
the batch queue, and to prevent batch experiments from being messed with
except when it was really okay (for example, its okay to terminate a
swapped out batch experiment, but not a swapped in batch experiment since
that would confuse the batch daemon). There are fewer of these states, plus
one additional state for "modifying" experiments.

So what I have done is change the system to use "batchstate" for all
experiments to control entry into the swap system, from the web interface,
from the command line, and from the batch daemon. The other state variable
still exists, and will be brutally pushed back under the surface until its
just a vague memory, used only by the original tb* scripts. This will
happen over time, and the "batchstate" variable will be renamed once I am
convinced that this was the right thing to do and that my changes actually
work as intended.

Only people who have bothered to read this far will know that I also added
the ability to cancel experiment swapin in progress. For that I am using
the "canceled" flag (ah, this one was named properly from the start!), and
I test that at various times in assign_wrapper and tbswap. A minor downside
right now is that a canceled swapin looks too much like a failed swapin,
and so tbops gets email about it. I'll fix that at some point (sometime
after the boss complains).

I also cleaned up various bits of code, replacing direct calls to exec
with calls to the recently improved SUEXEC interface. This removes
some cruft from each script that calls an external script.

Cleaned up modifyexp.ph3 quite a bit, reformatting and indenting.
Also fixed to not run the parser directly! This was very wrong; should
call nscheck instead. Changed to use "nobody" group instead of group
flux (made the same change in nscheck).

There is a script in the sql directory called newstates.pl. It needs
to be run to initialize the batchstate slot of the experiments table
for all existing experiments.

put:
$node add-desire desire weight

This will end up in the virt_node_desires table. assign_wrapper now
puts the desires from this table into the top file.

during a modify. Not supposed to!

run.

pclasses seem to be broken.

Note: this only affects experiments with vodes.