GitLab

Mote and robot related stuff.  The main thing is the addition of relay
capabilities to capture and related things.

	* GNUmakefile.in: Add the capture and tip subdirectories to the
	client and client-install targets.

	* configure, configure.in, config.h.in: Detect srandomdev() for
	capture and add "mote/newmote" script.

	* capture/GNUmakefile.in, capture/capture.c: Add "relay"
	capabilities to capture.

	* capture/capquery.c: Query the capserver for the relay receiver's
	port number.

	* capture/capserver.c: Small hack to return the port number
        for a node.

	* db/libdb.pm.in, db/xmlconvert.in: Add virt_tiptunnels table.

	* event/program-agent/program-agent.c: Change log file names to
	something a little more user-friendly.  Add a "MODIFY" event
	handler that lets the user set agent attributes (command, tag,
	timeout) without having to run a program.

	* event/sched/GNUmakefile.in, event/sched/console-agent.cc,
	event/sched/console-agent.h, event/sched/event-sched.c: Add
	console agents that can be used to snapshot a section of the
	capture log file.

	* event/sched/node-agent.cc: Some minor cleanup.

	* event/sched/simulator-agent.cc, event/sched/simulator-agent.h:
	Add the config data to the report mail.  Add a "RESET" event
	handler that runs "loghole clean".  Save the report mail in a file
	so it gets archived with the rest of the logs.

	* lib/libtb/tbdefs.h: Add CONSOLE object type.

	* mote/GNUmakefile.in, mote/newmote: Add newmote script, just a
	quick hack to add motes to the DB.

	* mote/tbuisp.in: Add another backend for loading motes through
	their relay capture server.

	* robots/mtp/mtp_dump.c: Dump the min/max values for x and y,
	handy for figuring out the bounds of the camera.

	* sql/database-fill.sql: Change the RELOAD-MOTE/SHUTDOWN ->
	ALWAYSUP/SHUTDOWN mode transition to ALWAYSUP/ISUP since stated
	doesn't seem to run triggers after a state change by a mode
	transition.

	* tbsetup/tbreport.in: Change the ordering of the eventlist so it
	displays event-sequences appropriately.

	* tbsetup/ns2ir/GNUmakefile.in, tbsetup/ns2ir/console.tcl,
	tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/parse.tcl.in,
	tbsetup/ns2ir/sim.tcl.in: Add a "console" agent that represents
	the serial console for a node.

	* tbsetup/ns2ir/sequence.tcl: Add an "append" method so it is
	easier to build sequences dynamically.

	* tbsetup/ns2ir/topography.tcl: Make checkdest available to
	regular users.

	* tip/GNUmakefile.in, tip/tiptunnel.c: Add support for uploading a
	file to a relay version of capture and exporting the end
	connection as a pty.

	* tmcd/decls.h, tmcd/common/libsetup.pm: Bump version number since
	the dosubnodelist change is not backwards compatible.

	* tmcd/tmcd.c: Make dosubnodelist and dosubconfig callable even
	when a node isn't allocated.  Add dotiptunnels command that
	returns which serial consoles are to be mounted on a node.  Add
	mote version of subconfig that returns information needed to
	startup the relay version of capture.

	* tmcd/common/bootsubnodes: For motes, startup the relay version
	of capture (XXX stargate specific).

	* tmcd/common/libsetup.pm, tmcd/common/libtmcc.pm,
	tmcd/common/config/rc.config, tmcd/common/config/rc.tiptunnels:
	Client side changes for mounting another nodes serial line.

	* tmcd/common/rc.bootsetup: Always boot the subnodes, even when
	free.  This is used for motes since their capture needs to be up
	for reloading at the time.

	* tmcd/linux/ixpboot: Shuffle some code around so the script
	doesn't fail if the ixp isn't allocated.

	* utils/loghole.in: Add "digest.out" and "report.mail" as global
	logs to be saved in archives and display the "report.mail" file
	when showing a loghole archive.

	* xmlrpc/emulabserver.py.in: Scrub more of the return values to
	get rid of "None"s.

(to tftpboot-elabinelab.tar.gz) so we can evolve it
separately without affecting the advertised tarball.

install.  This had been fixed for ops-install, but not boss.

I did the "back half" support.  If the 'mustwipe' field is non-zero
in the reserved table entry for a node then its disk must be zeroed.
How the zeroing is done, depends on the value of the mustwipe field.
Right now, '1' means pass the '-z' option to frisbee to have it zero
all non-allocated blocks.  The value '2' is reserved for enabling a
"full wipe" pass of the disk before running frisbee, which Keith Sklower
(DETER) wanted to be able to do.  Note that 1 and 2 are effectively the
same, if we are loading a full-disk image; i.e. all non-allocated blocks
from the new image are zeroed.  But if the disk were being loaded with
a single-partition image, then "frisbee -z" would only wipe unused
blocks in that partition.

The reload_daemon has been modified to extract the mustwipe info and
invoke os_load accordingly.   os_load now takes a "-z <type>" option
to enable the zeroing by setting a value in the current_reloads table.
tmcd will read and return that info to its caller in the "loadinfo" command.
Finally, the rc.frisbee script that runs in the frisbee MFS extracts the
loadinfo info and crafts the frisbee startup command.

What still needs to be done is the "front end," how the user specifies
the value and how it winds up in the DB reserved table.  This will probably
involve addition of state to the experiments table as this will likely be
a per-experiment setting.

vs. not on the control net.

vs. not on the control net.  The "media" command wasn't implemented in the
Linux dhclient-script anyway.

Checkin some changes related to experiment automation and vnode feedback:

	* configure, configure.in: Add sensors/canaryd/feedbacklogs
	template.

	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
	table that holds environment variable names and values.

	* event/lib/event.c: Allocate memory of the right size for
	event_notifications.

	* event/program-agent/GNUmakefile.in: Add version.c file and
	add install targets for the man page.

	* event/program-agent/program-agent.8: Man page describing the
	program-agent daemon.

	* event/program-agent/program-agent.c: Add a bunch of convenience
	features: let the user specify the working directory for commands;
	save output to separate files on every invocation of an agent; let
	the user specify a timeout for a command; make the set of
	environment variables sane and add vars given in the NS file in
	the opt array; a "status" file containing process information is
	written out when children are collected.  Internal changes: child
	processes are collected immediately, instead of waiting for the
	next START event, so we can send back COMPLETE events; the daemon
	now runs with a real-time priority, to increase the chances of
	receiving events.

	* event/proxy/evproxy.c: Made it bidirectional so the
	program-agent's COMPLETE events make it back to the scheduler.

	* event/sched/error-record.c: Change the default log directory.

	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
	environment similar to a program-agent to run the user's log
	digester.

	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
	that runs create_image for the node.

	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
	Let the user specify a "DIGESTER" script that digests the log
	files into a summary of the results.  Add event handler for
	remapping a vnode experiment.

	* event/sched/timeline-agent.c: Accept the RUN event as well as
	the START event.

	* os/GNUmakefile.in: Install the install-tarfile.1 man page.

	* os/install-tarfile: Automatically chown/chgrp any files that do
	not have valid user or group IDs, the new owner will be the user
	that swapped in the experiment.  Include the install directory in
	the DB file.  Add a "list" mode that just dumps what files have
	been installed and where.  Add a "force" option so the user can
	forcefully install the file, even though the DB says its already
	there.

	* os/install-tarfile.1: Man page describing the install-tarfile
	tool.

	* os/syncd/GNUmakefile.in: Install man pages on ops.

	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
	install "feedbacklogs" tool.

	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.

	* sensors/canaryd/canarydEvents.c: Log errors.

	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
	data from canaryd log files.

	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.

	* sensors/slothd/digest-slothd: Fix some bugs and write out an
	"alert" file with all the nodes/links that were overloaded.

	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
	argument that lets you specify that you want to wait for the disk
	to finish loading and/or wait for the node to come back up in the
	new OS.

	* tbsetup/power.in: Remove debugging printf.

	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
	quoting problems with event-sequences.  Add -expected-exit-code
	and -tag options to the "$program run" event.  Add -digester to
	the "$ns report" event that lets the user specify a program to run
	to digest the log files.

	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
	factor for feedback nodes to 1%, instead of 100%.

	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
	returns the values in "virt_user_environment".  Return new program
	agent fields: dir, timeout, and expected_exit_code.

	* tmcd/common/GNUmakefile.in: Install rc.canaryd.

	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
	a real-time priority, they can't do it from inside the jail.

	* tmcd/common/rc.canaryd: Rc script for canaryd.

	* tmcd/common/watchdog: Don't fail outright if there is a bad line
	in the battery.log

	* tmcd/common/rc.progagent: Append "userenv" data to the
	program-agent config file.

	* utils/GNUmakefile.in: Install loghole and its man page on ops.

	* utils/loghole.1: Document "clean" command and the change in
	loghole directories.

	* utils/loghole.in: Add "clean" command and parallelization.

	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
	Order the eventlist by "idx" and time, needed for sequences.  And
	removed unnecessary nologin checks.

"programs" command.

To be used as a backup for NFS.

Report battery stats using the watchdog:

	* robots/emc/emcd.c: Don't send events with battery levels
	anymore.

	* tmcd/tmcd.c: Add 'battery' command that updates the
	battery_percentage, battery_voltage, and battery_timestamp values
	in the nodes table.  It's currently only called by the watchdog
	running on the garcias.

	* tmcd/common/libtmcc.pm: Add TMCCCMD_BATTERY function.

	* tmcd/common/watchdog: Updated to send back battery information
	obtained from garcia-pilot.

Zap gated setup.

Enable windows support by default now.  This ensures exports_setup creates
samba shares.

* rc.mkelab - modify logic s.t. --disable-windows is passed to configure if
  WINSUPPORT is not set.

or not, otherwise the proper regmon module isn't installed and mote
loading won't work.

a widearea node.

Robot-related improvements based on feedback from Sid:

	* event/sched/event-sched.h, event/sched/event-sched.c,
	event/sched/rpc.h, event/sched/rpc.cc: Send a notification of the
	start of event time and timelines.  Add a '-r' option that forces
	it to use the default rpc path (so dan can use a custom
	scheduler/rmcd/etc on ops).

	* robots/GNUmakefile.in, robots/emc/GNUmakefile.in,
	robots/mtp/GNUmakefile.in, robots/primotion/GNUmakefile.in,
	robots/rmcd/GNUmakefile.in, robots/vmcd/GNUmakefile.in,
	robots/tbsetdest/tbsetdest.cc: Add a control-install target.

	* robots/primotion/garcia-pilot.cc: Take out some experimental
	code.

	* robots/primotion/dashboard.hh, robots/primotion/dashboard.cc,
	robots/primotion/pilotClient.cc, robots/primotion/wheelManager.hh,
	robots/primotion/wheelManager.cc: When doing a COMMAND_STOP, make
	sure we only send one update packet, an idle or an abort.  Also
	send back the actual distance travelled/pivoted.

	* robots/tbsetdest/tbsetdest.cc: Don't output setdest event's with
	a speed of zero.

	* tbsetup/eventsys_control.in: When replaying/stopping, clear the
	start of event time for the experiment.

	* tbsetup/power_mail.pm.in: Decrease the threshold used to tell if
	a node has been power cycled "recently".

	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/sim.tcl.in: Bah, backout
	last change, orientation is supposed to be in degrees at these
	points.  Also, when automatically picking the sync server, don't
	use nodes that are subnodes.

	* tmcd/common/bootsubnodes: Send an ISUP for motes.

	* www/moteleds.php3: Show the mote node names instead of the
	robots.

	* www/powertime.php3: Add option to mark nodes as powered off.

	* www/robotmap.php3: Display the elapsed time for the overall
	event time and for each active timeline.

	* www/showexp.php3: Only display the blinky lights menu item when
	there are motes.

	* www/shownode.php3: Change "Update Power Time" -> "Update Power
	State".

	* www/tutorial/mobilewireless.php3: Fix some nits.

	* xmlrpc/emulabserver.py.in: Add event_time_start method for
	storing/getting/clearing the start of event time for an
	experiment/timeline.

constructed. This is avoid the emulab source code leaking out too
easily.

We leave it on boss, since at the moment mere users do not get shells
on inner boss, so getting the source code that way is a bit harder
(although not that hard of course). Besides, for development its much
nicer to have that source left on boss.

Minor, but important cleanup operation.

Some more robot integration.

	* event/lib/event.h, event/lib/event.c: Add some
	event_notification creation functions that get used in
	event-sched.

	* event/sched/event-sched.c, event/sched/rpc.h,
	event/sched/rpc.cc: Sync the start of event time with the
	robots reaching their initial positions.  This is done by
	creating a master event-sequence that takes care of sending
	the SETDEST events and then starting the Simulator timeline.

	* mote/tbuisp.in: Use node_reboot instead of ssh'ing in.

	* robots/GNUmakefile.in: Don't build tbsetdest if ulsshxmlrpcpp is
	not available.

	* robots/emc/loclistener.in: Clear the destination values for a
	node when it reaches its destination.

	* robots/primotion/garcia-pilot.cc,
	robots/primotion/pilotClient.cc: Some cleanup and debugging.

	* robots/primotion/wheelManager.cc: Check the rear sensors for
	obstructions and then decide which way to pivot.

	* robots/rmcd/pilotConnection.h, robots/rmcd/pilotConnection.c,
	robots/rmcd/rclip.h: Even more tweaking.

	* robots/tbsetdest/tbsetdest.cc: Don't generate points that are
	outside the camera bounds or inside an obstacle.

	* robots/vmcd/visionTrack.c, robots/vmcd/vmcd.h,
	robots/vmcd/vmcd.c:  Add more debugging output.

	* tbsetup/os_setup.in: Removed the robot hack used when deciding
	which nodes to reconfig/reboot.  Added a robot hack to avoid
	rebooting a robot whose mote is being os_load'd, since it would
	interrupt tbuisp which does the reboot anyways.  Also fixed a
	small typo.

	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/sim.tcl.in: Oops, forgot
	to convert degrees to radians.

	* tbsetup/ns2ir/topography.tcl: When checking for node destination
	points in obstacles, include the implicit exclusion zone.

	* tmcd/common/bootsubnodes: Add empty "mote" case.

	* tmcd/linux-sg/GNUmakefile.in: Make some of the /etc subdirs when
	doing the install.

	* tmcd/linux-sg/rc.stargate: Start garcia-pilot.

	* vis/floormap.in: Add options for showing the camera bounds,
	obstacle exclusion zones, and displaying vnames instead of pnames.

	* www/ledpipe.php3: Finally figured out how to use a socket
	instead of popening a perl script.

	* www/robotmap.php3: Add checkboxes for displaying/not displaying
	the camera bounds and obstacle exclusion zones.  Add a legend
	showing what actual vs. destination points are.  Pass pid/eid
	through to vis/floormap if it is given.

	* www/showexp.php3: Add a "Robot Map" link to experiments that
	have allocated garcias.

	* www/floormap/map_legend_node.gif,
	www/floormap/map_legend_node_dst.gif: Icons used by the robot map
	legend.

	* www/floormap/robots-4.jpg: Added an obstacle around the entryway
	so people are slightly less likely to trip over the robots.  Added
	a coordinate system legend to the top left corner.

	* www/tutorial/mobilewireless.php3: Add links to David's movie of
	the robot making its way around the pillar.

	* www/tutorial/robot_anim.gif: A nifty gifanim clip of the robot
	movie.

	* xmlrpc/emulabserver.py.in: Pull the camera data from the DB,
	instead of returning hardcoded stuff.

Was using $BOOTDIR which is different in the frisbee MFS than anywhere else.

oops - hook in rc.stargate.

interface to IP address assignment, and is trivially parsable.
This is less onerous then telling people to parse the output of
ifconfig with perl.

Stargate client-side boot support.

Similar to linux, but the sg linux distribution is based on debian, so some
things are a little different.

WARNING: if you do a client-install in a cross-compile environment
(with DESTDIR set), watch out for bogus symlinks in <root>/etc/rc*.d!
Have to fix these up by hand in this case.

Also note that the stargate rc script doesn't yet start the robot pilot
daemon - Tim said he needed to do finalize some stuff w.r.t. this, so
he would take care of modifying the rc script.

The ntp settings are still not ideal.  Even though the root fs is on a
journalling jffs filesys, its still not good to write to it regularly.  Need
to point the ntp drift file off into /var (ramfs)