GitLab

might not be able to write the output file. Better to us syslog anyway
since its a low usage command.

logging to the power command (to power.log).

Contributed by IBBT/UGent

Everything else works the same; stick the required entries in the
interfaces tables and the outlets_remoteauth table.

Oh, I ripped out the existing parallelizzation code and replaced it
with the library code emutil::ParRun(), which is slowly making its way
across the system, replacing all the one-off cases of using fork to
parallelize things. Very handy.

Seems to be needed since I changed the 'use' to a 'require' on power_rmcp.

Make sure power doesn't blow up if rmcp code is not installed.

last commit.

lumping the VLAN number in with the OID, leaving the field we expect
to contain the VLAN number blank. I'm not sure why this happens, but
I've seen it before, and I think that simply dealing with it is going
to be more successful than finding the root cause.

the db, to slogin to the node's ilo interface (which has to be specified
in the interfaces table), then uses a simple expect-like sequence over a
pty to control power.

power cycle. For those we need to send wakeonlan packets to kick them.
This is complicated by the fact that the laptops take an incredibly
long time to go from poweron to a point where it will listen to a
wakeonlan packet. No idea why. I use StateWait() to find out when the
node hits bootinfo (which sends a PXEBOOTING event). Keep looping
sending more wakeonlan packets until we hear from the node. Time out
the operation after two minutes (probably need something to control
this).

To find the outgoing interface required by tools/whol program, I look
in the interfces table for boss' control network.

The mac address of the target node comes from the interfaces table.

The flag that controls whether a node needs wakeonlan is in the
node_attributes table:

	mysql> insert into node_attributes set
	          attrkey='wakeonlan_afterpower',
		  attrvalue=1, node_id='pcxxx';

That is, they are not in the nodes table but they are in the outlets table.
Useful during initial bring up.

describing the number of outlets possible on a power controller.

have to re-run the swig-wrappers target in tools/rmanage/GNUmakefile to
generate the wrapper and perl module; this must of course be done when
changes are made to the rmcp libs.

  * GNUmakefile.in, configure, configure.in: add tools/rmanage
  * tbsetup/GNUmakefile.in, tbsetup/power*.in: add rmcp to power command
  * tools/GNUmakefile.in: add rmanage
  * tools/rmanage/*.c,*.h: bugfixes, swig helper methods, etc.
  * tools/rmanage/rmcp.i: swig import control file
  * tools/rmanage/rmcp.pm,rmcp_wrap.c: rmcp wrapper/module generated by swig

table, into a new table called node_type_attributes, which is intended
to be a more extensible way of describing nodes.

The only things left in the node_types table will be type,class and the
various isXXX boolean flags, since we use those in numerous joins all over
the system (ie: when discriminating amongst nodes).

For the most part, all of that other stuff is rarely used, or used in
contexts where the information is needed, but not for type descrimination.
Still, it made for a lot of queries to change!

Along the way I added a NodeType library module that represents the type
info as a perl object. I also beefed up the existing Node module, and
started using it in more places. I also added an Interfaces module, but I
have not done much with that yet.

I have not yet removed all the slots from the node_types table; I plan to
run the new code for a few days and then remove the slots.

Example using the new NodeType object:

	use NodeType;

	my $typeinfo = NodeType->Lookup($type);

        if ($typeinfo->control_interface(\$control_iface) ||
            !$control_iface) {
  	    warn "No control interface for $type is defined in the DB!\n";
        }

or using the Node:

	use Node;

        my $nodeobject = Node->Lookup($node_id);
        my $imageable  = $nodeobject->NodeTypeInfo()->imageable();
or
        my $rebootable = $nodeobject->isrebootable();
or
        $nodeobject->NodeTypeAttribute("control_interface", \$control_iface);

Lots of way to accomplish the same thing, but the main point is that the
Node is able to override the NodeType (if it wants to), which I think is
necessary for flexibly describing one/two of a kind things like switches, etc.

 * allow requesting status by naming a node on a controller rather than
   needing to know the controller name
 * make output of status command more grep-able

(eg. "power status rpc14-1" or "power status all").

In rpc module, add a "status" function that is called to collect,
well..status.  This is used by the generic power code, but more importantly,
by the new "powermon" script (which could have just called "power status",
but no point in parsing output twice).

to do that on the Cisco 2950s currently.

Some power-by-mail hacking:

  - Bump the timeout for waiting for the operators to flip the switch
    to 20 minutes.

  - Fail fast if the node is in hwdown.  This case is intended to make
    an os_load fail for a robot-mounted mote whose robot is in hwdown.

  - Fail if the robotlab is not open since noone is around to do
    anything about it anyways.

  - Assume success if the event state for a node was updated
    "recently."  This is a fall back in case the powertime web page
    isn't used to notify the system that the node was powered
    on/cycled.  Also, do not send the SHUTDOWN event in this case.

  - Add a TBNodeEventStateUpdated() function to libdb.pm that returns
    true if the eventstate for a node was updated within N seconds
    from the current time.

To enable WhOL, you need to add outlets table entries for nodes which are
whol-enabled.  The power_id encodes the interface on boss to use:

+---------+-----------+--------+----------------+
| node_id | power_id  | outlet | last_power     |
+---------+-----------+--------+----------------+
| pcwf6   | whol-fxp0 |      0 | 20050318152119 |
+---------+-----------+--------+----------------+

You then need interfaces and wires table entries for that interface on
boss, so that snmpit works with syntax like "boss:1".  This is probably not
really needed once the VLAN has been setup.

You need a magic VLAN called "WhOL", I used VLAN 999.  Add it to all the
switches and trunks.  Put boss's port in it.  It will remain there, enabled,
forever.

In the interfaces table entry for every interface that supports WhOL,
you need to set the 'whol' field to 1.

Checkin some changes related to experiment automation and vnode feedback:

	* configure, configure.in: Add sensors/canaryd/feedbacklogs
	template.

	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
	table that holds environment variable names and values.

	* event/lib/event.c: Allocate memory of the right size for
	event_notifications.

	* event/program-agent/GNUmakefile.in: Add version.c file and
	add install targets for the man page.

	* event/program-agent/program-agent.8: Man page describing the
	program-agent daemon.

	* event/program-agent/program-agent.c: Add a bunch of convenience
	features: let the user specify the working directory for commands;
	save output to separate files on every invocation of an agent; let
	the user specify a timeout for a command; make the set of
	environment variables sane and add vars given in the NS file in
	the opt array; a "status" file containing process information is
	written out when children are collected.  Internal changes: child
	processes are collected immediately, instead of waiting for the
	next START event, so we can send back COMPLETE events; the daemon
	now runs with a real-time priority, to increase the chances of
	receiving events.

	* event/proxy/evproxy.c: Made it bidirectional so the
	program-agent's COMPLETE events make it back to the scheduler.

	* event/sched/error-record.c: Change the default log directory.

	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
	environment similar to a program-agent to run the user's log
	digester.

	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
	that runs create_image for the node.

	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
	Let the user specify a "DIGESTER" script that digests the log
	files into a summary of the results.  Add event handler for
	remapping a vnode experiment.

	* event/sched/timeline-agent.c: Accept the RUN event as well as
	the START event.

	* os/GNUmakefile.in: Install the install-tarfile.1 man page.

	* os/install-tarfile: Automatically chown/chgrp any files that do
	not have valid user or group IDs, the new owner will be the user
	that swapped in the experiment.  Include the install directory in
	the DB file.  Add a "list" mode that just dumps what files have
	been installed and where.  Add a "force" option so the user can
	forcefully install the file, even though the DB says its already
	there.

	* os/install-tarfile.1: Man page describing the install-tarfile
	tool.

	* os/syncd/GNUmakefile.in: Install man pages on ops.

	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
	install "feedbacklogs" tool.

	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.

	* sensors/canaryd/canarydEvents.c: Log errors.

	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
	data from canaryd log files.

	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.

	* sensors/slothd/digest-slothd: Fix some bugs and write out an
	"alert" file with all the nodes/links that were overloaded.

	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
	argument that lets you specify that you want to wait for the disk
	to finish loading and/or wait for the node to come back up in the
	new OS.

	* tbsetup/power.in: Remove debugging printf.

	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
	quoting problems with event-sequences.  Add -expected-exit-code
	and -tag options to the "$program run" event.  Add -digester to
	the "$ns report" event that lets the user specify a program to run
	to digest the log files.

	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
	factor for feedback nodes to 1%, instead of 100%.

	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
	returns the values in "virt_user_environment".  Return new program
	agent fields: dir, timeout, and expected_exit_code.

	* tmcd/common/GNUmakefile.in: Install rc.canaryd.

	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
	a real-time priority, they can't do it from inside the jail.

	* tmcd/common/rc.canaryd: Rc script for canaryd.

	* tmcd/common/watchdog: Don't fail outright if there is a bad line
	in the battery.log

	* tmcd/common/rc.progagent: Append "userenv" data to the
	program-agent config file.

	* utils/GNUmakefile.in: Install loghole and its man page on ops.

	* utils/loghole.1: Document "clean" command and the change in
	loghole directories.

	* utils/loghole.in: Add "clean" command and parallelization.

	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
	Order the eventlist by "idx" and time, needed for sequences.  And
	removed unnecessary nologin checks.

Robot related stuff: power via e-mail, client-install fixups, checking
coords against camera boundaries.

	* configure, configure.in: Add tbsetup/power_mail.pm to the list
	of template files.

	* doc/cross-compiling.txt: More stargate notes.

	* event/sched/rpc.cc: Updates for the addition of the cameras
	table.

	* robots/GNUmakefile.in, robots/emc/GNUmakefile.in,
	robots/mtp/GNUmakefile.in, robots/rmcd/GNUmakefile.in,
	robots/tbsetdest/GNUmakefile.in, robots/vmcd/GNUmakefile.in:
	client-install fixups.

	* tbsetup/GNUmakefile.in: Add power_mail.pm.

	* tbsetup/os_setup.in: Don't skip reboot of robots anymore.

	* tbsetup/power.in: Add special case for a power_id of "mail",
	which calls into the power_mail.pm backend.

	* tbsetup/power_mail.pm.in: E-mail backend for power, it sends an
	e-mail to tbops and waits for the outlets.last_power value to be
	updated from the power.php3 web page.

	* tbsetup/ns2ir/parse-ns.in: Add the contents of the cameras table
	to the TBCOMPAT namespace.

	* tbsetup/ns2ir/sim.tcl.in: More checking of "setdest" inputs.

	* tbsetup/ns2ir/topography.tcl: Update the checkdest method to
	check destination points against the camera list.

	* www/powertime.php3: Webpage used to update the last power time
	for nodes.

	* www/shownode.php3: Add "Update Power Time" menu button.

'power' command.

library to call out to the outer emulab to do the actual power cycle.

have a power outlet! At least we will know ...

get an updated copyrights message.

then remove special case for sending REBOOTING event in node_reboot/power
when using NORMAL mode. Now SHUTDOWN is always sent. (Important side note:
SHUTDOWN needs to be a valid state in every machine now.)

Changes to watch out for:

- db calls that change boot info in nodes table are now calls to os_select

- whenever you want to change a node's pxe boot info, or def or next boot
osids or paths, use os_select.

- when you need to wait for a node to reach some point in the boot process
(like ISUP), check the state in the database using the lib calls

- Proxydhcp now sends a BOOTING state for each node that it talks to.

- OSs that don't send ISUP will have one generated for them by stated
either when they ping (if they support ping) or immediately after they get
to BOOTING.

- States now have timeouts. Actions aren't currently carried out, but they
will be soon. If you notice problems here, let me know... we're still
tuning it. (Before all timeouts were set to "none" in the db)

One temporary change:

- While I make our new free node manager daemon (freed), all nodes are
forced into reloading when they're nfreed and the calls to reset the os
are disabled (that will move into freed).

two benefits: (1) More general (2) Regains ability to run without the
event system. Previously, since programs that watned to set node state
had to 'use event', this broke our ability to run without the event
system. Now, we can do a check in libdb for the event system, and not
use it if EVENTSYS is not set. If not, we update state in the database
directly rather than sending an event.

Also added equivalent calls for node operational mode, as well as new
constants for both state and mode.

Converted power and node_reboot to use this new scheme.

TBNODESTATE/REBOOTING event. While I was at it, converted power to use
Getopt::Std for getting command-line arguments.

node_reboot uses this argument to prevent the event from getting sent twice
(sice node_reboot has already sent it.)

Changed scripts that used the 'eventstatus' column to use the more
descriptively-named 'eventstate' column.

The FreeBSD and Linux starup scripts report a 'REBOOTED' state to tmcd
when they start, and 'ISUP' when the starup script is done.

node_reboot and power now send TBNODESTATE/REBOOTING events.

other!

assign_wrapper.in: Hack in a change that ensures a delay node is
created for any link on which an event is posted (up,down,modify),
no matter what its initial parameters are. ie: If a link is created
with no delay, but there is an event that adds a delay later, then we
must drop in a delay node. Same for up/down on a link. We do this in
the delay node. I am reasonably confident that this change is fine for
duplex links, but I am less sure of the effect on lans!

eventsys_control.in: Checkpoint latest changes. Add "replay" option,
which right now just stops and starts the event scheduler so that it
reloads the entire event list. Add check for existing experiment, and
that the experiment is either active or swapping (do not want to start
a scheduler for a swapped out experiment!). Add check to see if there
are any events, and skip startup if there are not events in the DB.
Lastly, get very serious about preventing more than one scheduler from
being started, either by accident or intentionally. My protocol is to
lock the table, grab and set the pid to -pid, test the pid for a
positive value, and if positive, send the scheduler a kill(TERM) so
that it can cleanup, clear the pid to zero in the DB, and exit. This
approach ensures that we do not try to send a kill to a pid that is no
longer active or owned by the user (this last part is not really
necessary cause of how pids are reused, but it was easy to add so why
not).

exports_setup.in: Trivial change to make it easier to turn this on
temporarily in devel trees.
named_setup.in: Ditto.

node_reboot.in: Add call to TBdbfork() in child cause of apparent DB
connection problems across forks. In the child, set the eventstatus
for the node to REBOOT if successful (not this event status stuff is
temporary, will be recast in next set of revisions).

GNUmakefile:  Add new controlling program, eventsys_control.
power.in:     Ditto previous comment about REBOOT.
os_setup.in:  Non event system cleanups.
tbend.in:     Add DB cleanup of the new virt_trafgens and eventlist tables.
tbprerun.in:  Ditto.
tbreport.in:  Print out the event list in a pretty print format.
tbswapin.in:  Add call to start the event system. Also a big fix; move
              the named script up above the os_setup so that the named
              tables have been updated by the time the first node
              reboots. I noticed that nodes were failing on gethostbyname().
tbswapout.in: Add call to stop the event system.

standardize on using /usr/bin/perl instead of /usr/local/bin/perl (most already
did.)