GitLab

multiple interfaces. The multiple interfaces issue is going to take
some thought and work to get right.

should not count against the global capacity limits.

attribute in the types table to cap the number of allowed virtual
nodes on each physnode (by type). We have to look in the reserved
table to see how many already allocated on each physnode, and adjust
the counts accordingly. Doing it here (via assign_wrapper) is not
atomic and so it could fail to catch an over subscription, but on a
lightly used testbed this will not be a problem in practice.

lan_member_attributes, and ifaces. Making the world safe for protogenis.

to it.

link type (assumed to be 'ethernet' for anything in the wires table),
also include types with the switch, type, and speed of the interface.

This allows us to ask for them more directly in the NS file.

users can specifically ask for nodes that don't (or do, if they're
crazy) have specific bandwidth caps.

table, into a new table called node_type_attributes, which is intended
to be a more extensible way of describing nodes.

The only things left in the node_types table will be type,class and the
various isXXX boolean flags, since we use those in numerous joins all over
the system (ie: when discriminating amongst nodes).

For the most part, all of that other stuff is rarely used, or used in
contexts where the information is needed, but not for type descrimination.
Still, it made for a lot of queries to change!

Along the way I added a NodeType library module that represents the type
info as a perl object. I also beefed up the existing Node module, and
started using it in more places. I also added an Interfaces module, but I
have not done much with that yet.

I have not yet removed all the slots from the node_types table; I plan to
run the new code for a few days and then remove the slots.

Example using the new NodeType object:

	use NodeType;

	my $typeinfo = NodeType->Lookup($type);

        if ($typeinfo->control_interface(\$control_iface) ||
            !$control_iface) {
  	    warn "No control interface for $type is defined in the DB!\n";
        }

or using the Node:

	use Node;

        my $nodeobject = Node->Lookup($node_id);
        my $imageable  = $nodeobject->NodeTypeInfo()->imageable();
or
        my $rebootable = $nodeobject->isrebootable();
or
        $nodeobject->NodeTypeAttribute("control_interface", \$control_iface);

Lots of way to accomplish the same thing, but the main point is that the
Node is able to override the NodeType (if it wants to), which I think is
necessary for flexibly describing one/two of a kind things like switches, etc.

nodes, and include the code necessary in the parser and assign_wrapper
to handle this.

The idea is to be able to talk about 'internet' on PlanetLab nodes - ie.
to request link tracing on it.

ones that are already on and/or won't need to be recharged soon.

First cut at a daemon that does regular checkups of the testbed
hardware/software.

	* configure, configure.in: Add tbsetup/checkup directory.

	* db/audit.in: Add a listing of stuck checkups.

	* install/boss-install.in: Add 'elabckup' user.

	* rc.d/3.testbed.sh.in: Startup the checkup_daemon.

	* sql/database-create.sql, sql/database-migrate.txt: Add the
	checkups tables.

	* tbsetup/GNUmakefile.in: Descend into the checkup directory.

	* tbsetup/checkup: The checkup daemon, man page, and
	  associated scripts.

	* tbsetup/ptopgen.in: Add a feature with a value of 0.9 to
	  prereserved nodes to keep them from being allocated unless
	  they're really wanted.

	* utils/firstuser.in: Add some other options so the script can be
	  used to create other pseudo users.

lost.  Since the exempt eid is always given when doing a swap in, the
prereserve wasn't doing anything.

allows takes you to new (admin only) page to select a project that
node will be reserved for.

* The node is not actually *reserved*, it is *pre* reserved! The node
  stays in the free pool, and is available only for use in the project
  to which it is reserved.

* The node can already be reserved to some other project when you
  pre-reserve it. It is not until the current owner releases the node
  that the pre-reservation takes effect.

* The node free counts (on the web pages) count a free a node with a
  pre-reservation, as allocated. This way people do not see a free
  count that includes a node they will never be able to get.

> 4Gbps trunks.

'connected-to-X' features for every other switch/node it's connected
to. This way, you can easily ask for nodes on cisco3, or whatever...

Firewalled experiments (see tbsetup/elabinelab.in for the other stuff).

* To support firewalled experiments, needed to add a new virt_firewalls
  table to split the existing firewalls table up, which included both
  virtual and physical stuff. There are the usual frontend changes and a
  few other things scattered around, including tmcd.c.

* The firewall code in tbswap got some beefing up to support adding and
  deleting nodes from the its special control net vlan. Note that I have
  not made any progress on containment of deleted nodes, just as we do not
  do anything now for teardown (unless its paniced, in which case the
  experiment cannot be modified anyway).

* ptopgen and assign_wrapper got some interesting modifications: Unlike
  regular swapmod, we cannot just tear down all the vlans since that would
  interrupt everything inside the inner elab. Instead, leave the vlans as
  is. The problem is that when assign runs, it can just as easily pick
  different interfaces on the same nodes, which would be a royal pain in
  the ass to deal with! So, ptopgen got a new option (-u) that assign
  wrapper uses to tell ptopgen that it should prune out unused interfaces
  from nodes that are already allocated to the experiment. This is, at
  best, as pathetically gross hack, but it makes sure that all the
  interfaces stay the same across swapmods.

* The unrelated revision of elabinelab has a bunch of new code for adding
  and deleting nodes from the inner elab. Mostly it deals with dhcpd (inner
  and outer, waiting for nodes to reboot, etc). It also deals with updating
  the vlans table in the DB, pruning out any nodes (ports) that are deleted
  but for which there are still interfaces in existing vlans. Said ports
  are them moved back to the default vlan with calls to snmpit. Also under
  another revision a a couple of weeks ago are the web interface changes to
  support the newnode MFS inside an inner Emulab.

* swapexp and endexp got some more checks for firewalled and paniced
  experiments, which were missing.

Use node_startloc table to figure out where robots are physically
located initially, instead of using location_info for current location
and initial.

	* tbsetup/ptopgen.in: When adding features for nodes attached to a
	building (e.g. robots), pull the building names from node_startloc
	instead of location_info.  This change lets the robots exist in
	two different areas.

	* tbsetup/ns2ir/parse-ns.in: Pull building names from
	node_startloc instead of location_info.

allocated.

	tb-set-delay-capacity 1

Will override the default delay capacity as specfied in the node_types
table for each node type, and set it for all types when generating the
ptop file.

This is a big stick, to be used with caution, as it will effectively
double the number of nodes used for delay nodes (withing an experiment).

More robot integration and some event system updates.

	* configure, configure.in: Detect rsync for loghole and add
	utils/loghole to the list of template files.

	* db/libdb.pm.in, db/xmlconvert.in: Add virt_node_startloc to the
	list of virtual tables.

	* event/lib/event.h, event/lib/event.c, event/lib/tbevent.py.tail:
	Add event_stop_main function to break out of the event_main()
	loop.  Add timeline to the address tuple.

	* event/sched/GNUmakefile.in, event/sched/error-record.h,
	event/sched/error-record.c, event/sched/event-sched.8,
	event/sched/event-sched.h, event/sched/event-sched.c,
	event/sched/group-agent.h, event/sched/group-agent.c,
	event/sched/listNode.h, event/sched/listNode.c,
	event/sched/local-agent.h, event/sched/local-agent.c,
	event/sched/node-agent.h, event/sched/node-agent.cc,
	event/sched/queue.c, event/sched/rpc.h, event/sched/rpc.cc,
	event/sched/simulator-agent.h, event/sched/simulator-agent.c,
	event/sched/timeline-agent.h, event/sched/timeline-agent.c:
	Updated event scheduler, not completely finished, but well enough
	along for the robots.

	* lib/libtb/GNUmakefile.in, lib/libtb/popenf.h,
	lib/libtb/popenf.c, lib/libtb/systemf.h, lib/libtb/systemf.c: Add
	some handy versions of system/popen that take format arguments.

	* lib/libtb/tbdefs.h, lib/libtb/tbdefs.c: Add some more event and
	object types.

	* tbsetup/assign_wrapper.in: Add the virt_node_startloc building
	to desires string for a node.

	* tbsetup/ptopgen.in: Add a node's location to the feature list.

	* tbsetup/tbreport.in: Display the timeline/sequence an event is a
	part of.

	* tbsetup/ns2ir/GNUmakefile.in: Add timeline, sequence, and
	topography files.

	* tbsetup/ns2ir/node.tcl: Add initial position for nodes and allow
	them to be attached to "topographys".

	* tbsetup/ns2ir/parse-ns.in: Make a hwtype_class array with a
	node_type's class.  Make an 'areas' array that holds the
	buildings where nodes are located.  Make an 'obstacles' table
	with any obstacles in the building.

	* tbsetup/ns2ir/parse.tcl.in: Move named-args function from
	tb_compat.tcl to here.  Add reltime-to-secs function that converts
	time given in a format like "10h2m1s" to a seconds value, used in
	"$ns at" so its easier to write time values.  Add "K", "Kb", and
	"Kbps" as possible units for bandwidth (only the lowercase
	versions were available before).

	* tbsetup/ns2ir/program.tcl: Add "dir" and "timeout" attributes,
	although they don't go anywhere at the moment.

	* tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/timeline.tcl,
	tbsetup/ns2ir/topography.tcl: Initial versions.

	* tbsetup/ns2ir/sim.tcl.in: Add support for timelines and
	sequences.  Add 'node-config' method to change the default
	configuration for nodes produced by the Simulator object.  Send an
	initial MODIFY event to any trafgen objects so their configuration
	gets through, even when there are no start/stop events.  Move
	event parsing to the 'make_event' method.

	* utils/loghole.1, utils/loghole.in: Loghole utility, used for
	retrieving logs from experimental nodes and creating archives of
	the logs.

	* xmlrpc/emulabclient.py.in: Escape any strange characters in the
	output field.

	* xmlrpc/emulabserver.py.in: Add virt_node_startloc to the list of
	virtual_tables.  Add emulab.vision_config and
	emulab.obstacle_config methods for getting information pertaining
	to the robots.  Change the OSID listing to include more fields.
	Add a "physical" aspect to experiment.info to get information
	about the physical nodes.  Add parent field to the events in the
	array returned by eventlist.  Add sshdescription to get extra
	information needed to log into a vnode.  Add node.statewait so you
	can wait for nodes to come up.

assume it always has its default OSID (from node_types) loaded and
ready to go (add this as the OS- feature). This so assign will agree
to the allocation (assign_wrapper adds a desire that says it has to be
running the OSID the user has selected, or the default OSID from the
node_types table).

Watch out for problems ...

delay:2, you get delay-pc600:2 (or delay-pc850:2, etc.). With this,
you can do this in your NS file:

	namespace eval TBCOMPAT {
	  set hwtypes(delay-pc600) 1
	}
	tb-make-hard-vtype delay delay-pc600

which will cause delays to be pc600. The TB_COMPAT is a temp thing,
until Rob comes back and tells us a better way to do this.

       nextosid mechinism of 1.114 making it possible to map a generic *-STD
       OSID based on the time in which an experiment is created.  This
       provides backward compatibility for old experiments when the standard
       images are changed.

       The osid_map table lookup is triggered when the value of the nextosid
       field is set to 'MAP:osid_map'.  The nextosid also continues to behave
       as before: if it contains a valid osid, that OSID value is used to map
       independent of the experiment creation time.  The two styles can also
       be mixed, for example FBSD-JAIL has a nextosid of FBSD-STD which in
       turn is looked up and redirects to the osid_map and selects one of
       FBSD47-STD or FBSD410-STD depending on the time.

	CREATE TABLE osid_map (
	  osid varchar(35) NOT NULL default '',
	  btime datetime NOT NULL default '1000-01-01 00:00:00',
	  etime datetime NOT NULL default '9999-12-31 23:59:59',
	  nextosid varchar(35) default NULL,
	  PRIMARY KEY  (osid,btime,etime)
	) TYPE=MyISAM;

       Yeah, yeah, I'm using another magic date as a sentinel value.
       Tell ya what, in 7995 years, find out where I'm buried, dig me up,
       and kick my ass for being so short-sighted...

       The following commands are not strictly needed, they just give
       an example, default population of the table.  They cause the standard
       images to be revectored through the table and then remapped, based on
       two time ranges, to the exact same image.  Obviously, the second set
       would normally be mapped to a different set of images (say RHL90 and
       FBSD410):

	INSERT INTO osid_map (osid,etime,nextosid) VALUES \
	  ('RHL-STD','2004-09-08 08:59:59','emulab-ops-RHL73-STD');
	INSERT INTO osid_map (osid,etime,nextosid) VALUES \
	  ('FBSD-STD','2004-09-08 08:59:59','emulab-ops-FBSD47-STD');

	INSERT INTO osid_map (osid,btime,nextosid) VALUES \
	  ('RHL-STD','2004-09-08 09:00:00','emulab-ops-RHL73-STD');
	INSERT INTO osid_map (osid,btime,nextosid) VALUES \
	  ('FBSD-STD','2004-09-08 09:00:00','emulab-ops-FBSD47-STD');

	UPDATE os_info SET nextosid='MAP:osid_map' \
	  WHERE osname IN ('RHL-STD','FBSD-STD');

In detail:

1. Added TBDB_NODESTATE_ALWAYSUP to libdb.pm for representing the ALWAYSUP
eventstate.

2. Modified free node calculation in ptopgen to include ALWAYSUP nodes.

3. Added code to ptopgen to correctly handle the case of a NULL iface
column, which happens when switches have interface (as they do in
Wisconsin), but assign_wrapper expects (null) for their iface rather than
"".

resolve it using a library function that will go through multiple
levels of indirction, so that we can point FBSD-JAIL at FBSD-STD.

supports the OS they are asking for. Puts features 'OS-<osid>' on each
pnode, listing the OSes that the pnode supports, and a desire on each
vnode with a weight of 1 for the OS the user wants to run. This way,
assign will not accidentally pick a node (such as a wireless PC) that
the user's image will not run on.

Note: This makes ptop files much, much larger, and makes assign take
somewhat longer to run.

webfeedback to talk to the newer version of canaryd.  Add feedback
"estimate" stuff so that if we have no data (because of an overloaded
node) to work with, we can make some sort of "reasonable" guesstimate
on every iteration.

* Fix for dynamic vnodes (previous revision); I forgot to make
  equivalent changea to the code for widearea virtual nodes.

* Add modelnet support; [-m c,v] option to use from assign_wrapper
  specifies the number of modelnet-core and modelnet-edge nodes to
  place into the ptop file for local nodes. Assign will end up doing
  modelnet assignment based on the number of physical core/edge nodes
  the user specifies in the NS file.

hardwiring it in this script.

Intended for debugging, and to use used with assign's pre-check
feature to catch experiments that will never be mappable.

In the ptop file, put the specific protocol (ie. 80211a) on the link
line.

In the ptop file, there is now only one fake airswitch. It can take
on all wireless types. We now only print one link line for each
physical wireless interface - it has the maximum bandwidth supported
by the interface, and is able to take on any protocol supported by the
interface.

and ptopgen.

Add link typing to assign. Each virtual link is given a single type.
Each physical link is given one or more types. A virtual link will
only be mapped to a physical link which can satisfy its type. In both
the top and ptop files, the link types are now mandatory, and they
fall at the end of the mandatory link arguments.

This differers from the 'regular' type system in two ways. First, a
plink is not constrained to filling only one type at a time. If we are
using emulated links, a plink could satisfy, say, an 'ethernet' link
and an 'fxp' link at the same time. This seems to more naturally match
the way we'll use link types.  Second, there are no counts assoicated
with link types, as there are for node types. ie. a link is not an
'ethernet:1' link, it's an 'ethernet' link. Presumably, when
multiplexing virtual links onto a physical one, it's bandwidth that's
the factor that limits the multiplexing.

The link type is now taken into account when constructing pclasses,
and in the mapping precheck.

As a side-effect of these changes, the silly 'count' argument on the
end of link lines in the ptop file, which was used for the fake LAN
nodes, is no longer supported.

The implementation could be a bit more efficient, but that would mean
tossing more of the stuff we do with boost's graph library. I think
this should happen, but today is not the day.

Modify assign_wrapper and ptopgen to spit out top and ptop files in
the new format.

Changed the constant LINK_UKNOWN to LINK_UNMAPPED - the new name more
accurately reflects the way this constant is used.

Add a new '-n' flag that tells assign not to do annealing, just exit
after the type precheck.

Clarify the usage message for the -c flag.

Removed some dead code for dealing with LAN nodes.

* ptopgen:

   1. Change to using interface_capabilities table to get the default
      speed for an interface type.

   2. The wrinkle is that an interface can have multiple speeds,
      depending on the protocol that is requested. In the wireless case the
      table looks like:

	| type             | capkey            | capval               |
	+------------------+-------------------+----------------------+
	| ath              | protocols         | 80211a,80211b,80211g |
	| ath              | 80211a_defspeed   | 54000                |
	| ath              | 80211b_defspeed   | 11000                |
	| ath              | 80211g_defspeed   | 54000                |

   3. We use the above info to tell us the default speed for links, based
      on the particular protocol.

   4. And lastly, create "air" switches for each of 80211a, 80211b, 80211g,
      and specify links from the switches to the nodes that have interfaces
      that support those protocols. At the moment, there is a hardwired
      list of 80211a, 80211b, 80211g in ptopgen, so create 3 new air
      switches and links from each of the nodes to each of the switches.
      Rob says that assign can handle this just fine (using the same
      interface on a node to connect to three different air switches, at
      different speeds).

* assign_wrapper:

   1. First off, I had to increase the bogousity factor of how we determine
      that a delay node is required (requires_delay() routine) on a link or
      lan. Using the interface_capabilities table, I determine the default
      speed for each protocol on each interface type.

   2. When calling requires_delay() I now the pass the lan so that we can
      get the "protocol" of the link/lan and use that to find the default
      speed. The main reason for all this crud is that if a user specifies
      a bandwidth of 54Mb on a wired ethernet, we want to make sure we
      insert a delay node, but not if the user is using 80211g. As it was,
      delay nodes would be skipped cause there are now nodes with
      interfaces that support those speeds. This change also allows to know
      that a delay node should be inserted on a wireless link, if we ever
      wanted to do something as silly as that!

   3. Okay, so now onto the actual wireless support!  Whenever the protocol
      for a lan is not "ethernet" spit out links to fake lan nodes, sorta
      like we used to do with ethernet lans before Rob made all those great
      changes to assign. So, a lan with protocol 80211a will get fake links
      to the 80211a air switch that ptopgen spits out (see above).  We
      currently bypass all delay node processing on wireless lans, and in
      fact we avoid most post-processing (after assign) of wireless lans
      (no vlans, veths, etc). At some point we can bring certain things
      back in (like emulated links) but that will be some time! See
      comments above about ptopgen.

   4. Added a section to copy over the virt_lan_settings and
      virt_lan_member_settings to the interface_settings table, for each
      node's interfaces in the mapped experiment. I also make up the ssid
      for the lan, and add the protocol to the settings so that the client
      side knows what it is supposed to do. I also set the acesspoint since
      that requires MAC addresses, and this is easiest place to get and set
      the MAC (all of the nodes in a lan need to know the MAC of the node
      acting as the access point). In the virtual topology, the accesspoint
      is specified as a node name.

* tbsetup/ns2ir:

   1. Add tb-set-lan-protocol(lan, protocol) to set the protocol to one of
      ethernet, 80211a, 80211b, 80211g. Defaults to ethernet.

   2. Add tb-set-lan-accesspoint(lan, accesspoint) to set the name of the
      access point for a lan. Obviously this makes sense for only wireless
      links; ignored for wired links.

   3. Add tb-set-lan-setting(lan, setting, value) to specify a setting that
      applies to the entire lan. At present, these are ignored for wired
      lans. Both setting and value are strings.

   4. Add tb-set-node-lan-setting(lan, node, setting, value) to specify a
      setting that applies to a particular node in a lan. At present, these
      are ignored for wired lans. Both setting and value are strings.

was getting added twice with my previous commit.