GitLab

created before trying to install.

plain "make".  There seemed to be a number of missing dependencies that
prevented this from working (how did this ever work?)

Cross compilation fixes for the stargates, 'gmake client' should now
build, link, and install properly.  Haven't really tried to run stuff though.

	* GNUmakerules: Add target for stripping executables, used instead
	of "install -s" since that doesn't work for cross-compiling.

	* Makeconf.in: Add ELVIN_CONFIG variable that refers to
	'elvin-config'.

	* configure, configure.in: Detect and save the elvin-config path
	since we need a different one for cross-compiling.

	* event/lib/GNUmakefile.in, event/link-agent/GNUmakefile.in,
	event/linktest/GNUmakefile.in, event/program-agent/GNUmakefile.in,
	event/proxy/GNUmakefile.in, event/tbgen/GNUmakefile.in,
	event/trafgen/GNUmakefile.in, os/dijkstra/GNUmakefile.in,
	os/syncd/GNUmakefile.in, sensors/slothd/GNUmakefile.in,
	tmcd/GNUmakefile.in, tmcd/linux/GNUmakefile.in: Cross compilation
	fixes, don't statically link on arm, create "foo-debug"
	executables with debugging info and install separately stripped
	ones instead of passing "-s" to install.

during ops-install.

ops as the user, but that breaks for admin people who have real shells
on boss and no unencrypted key in the .ssh dir. So, switched it to a
root ssh, and a simple proxy on the other side that flips to the user
and invokes the run_linktest stuff.

directly to ops via opsdir.

* Force tty allocation in the control script. Also force protocol 1 and
  ignore the users .ssh/config file.

* Do not start up linktest on delay nodes!

* The per-experiment event scheduler now runs on ops instead of boss.
  Boss still runs elvind and uses events internally, but the user part
  of the event system has moved.

* Part of the guts of eventsys_control moved to new script, eventsys.proxy,
  which runs on ops and fires off the event scheduler. The only tricky part
  of this is that the scheduler runs as the user, but killing it has to be
  done as root since a different person might swap out the experiment. So,
  the proxy is a perl wrapper invoked from a root ssh from boss, which
  forks, writes the pid file into /var/run/emulab/evsched/$pid_$eid.pid,
  then flips to the user and execs the event scheduler (which is careful
  not to fork). Obviously, if the kill is done as root, the pid file has to
  be stored someplace the user is not allowed to write.

* The event scheduler has been rewritten to use Tim's C++ interface to the
  sshxmlrpc server on boss. Actually, I reorg'ed the scheduler so that it
  can be built either as a mysql client, or as RPC client. Note that it can
  also be built to use the SSL version of the XMLRPC server, but that will
  not go live until I finish the server stuff up. Also some goo for dealing
  with building the scheduler with C++.

* Changes to several makefiles to install the ops binaries over NFS to
  /usr/testbed/opsdir. Makes life easier, but only if boss and ops are
  running the same OS. For now, using static linking on the event scheduler
  until ops upgraded to same rev as boss.

* All of the event clients got little tweaks for dealing with the new CNAME
  for the event system server (event-sever). Will need to build new images
  at some point. Old images and clients will continue to work cause of an
  inetd hack on boss that uses netcat to transparently redirect elvind
  connections to ops.

* Note that eventdebug needs some explaining. In order to make the inetd
  redirect work, elvind cannot be listening on the standard port. So, the
  boss event system uses an alternate port since there are just a few
  subsystems on boss that use the server, and its easy to propogate changes
  on boss. Anyway, the default for eventdebug is to connect to the standard
  port on localhost, which means it will work as expected on ops, but will
  require -b argument on boss.

* Linktest changes were slightly more involved. No longer run linktest on
  boss when called from the experiment swapin path, but ssh over to ops to
  fire it off. This is done as the user of course, and there are some
  tricks to make it possible to kill a running linktest and its ssh when
  experiment swapin is canceled (or from the command line) by forcing
  allocation of a tty. I will probably revisit this at some point, but I
  did not want to spend a bunch of time on linktest.

* The upgrade path detailed in doc/UPDATING is necessarily complicated and
  bound to cause consternation at remote sites doing an upgrade.

* The linktest daemon (the one that runs on the nodes) no longer talks
  to boss directly, but instead contacts the local elvind; rc.linktest
  is changed to reflect that.

* A bunch of signal handler changes to run_linktest.pl; do not rely on
  events to stop linktest when it is running on boss; when the user
  kills a running linktest make sure all the processes are killed
  explicitly.

* New wrapper script (linktest_control) for use on boss, specifically
  when being called from the web interface. This script handles the DB
  part (getting linktest_level and linktest_pid), making sure that
  only one linktest is running at a time (on boss) and reseting the
  pid in the DB as needed. The -k option kills a running linktest, and
  is invoked from the web interface when the user wants to kill one in
  progress. This gets the pid from the DB and sends it a TERM signal,
  which sends a TERM to the run_linktest.pl script, which sends a TERM
  to the ltevent helper app.

  Note that this wrapper is also suitable for the XMLRPC interface,
  although I have not added it there yet.

* Do not have linktest daemon connect to boss; have it connect to
  local node elvind like all other local agents. Remove the event
  generation code (linktest was sending a KILL event to all other
  linktest programs), and replace with a system() call to tevc, which
  sends the event through the scheduler and exits; this will avoid a
  zillion tcp connctions to boss from the linktest daemon.

* A couple of process group changes to linktest daemon; the daemon
  appeared to be killing itself off.

* Fix to run_linktest.pl; It was just hanging after it completed,
  cause its child ltevent process was still running. Changed to record
  child pid, and kill/close ltevent child before exiting.

possible to:

	gmake client
	sudo gmake client-install

on a FBSD4, FBSD5, RHL7.3, and RHL9.0 client node.

There are still some dependencies that are not explicit and which would
prevent a build/install from working on a "clean" OS.  Two that I know of are:
you must install our version of the elvin libraries and you must install boost.

1. "make clean" will just remove stuff built in the process of a regular build
2. "make distclean" will also clean out configure generated files.

This is how it was always supposed to be, there was just some bitrot.

Now it installs from your source tree (rather than from /usr/testbed/lib,
which might not exist) into /usr/local/etc/emulab

the top level.  This will build all the necessary binaries and then install
them.  This works on FBSD4 and RHL7.3.  It still doesn't work on FBSD5
(newer compiler that no longer supports a style of use of _FUNCTION_ in the
event lib) or RHL9 (event lib needs SSL lib which has a bad dependency
on Kerberos).  Notes:

- requires that elvin libraries be installed on nodes (they are) to build
  event agents, requires linuxthreads be installed on FBSD (it is now) to
  build imagezip (which is installed, but is not strictly necessary)

- installed event-agents and other binaries are stripped

- added a few missing files to the source tree for bsd (healthd.conf)
  and linux (healthd.conf, rc.local)

- the only thing that doesn't get rebuilt in /usr/local/etc/emulab is
  healthd, I couldn't quickly find how it gets built

- uses a scaled down version of libtb with no DB functions (since mysql
  isn't installed on nodes).  N.B. DO NOT DO A CLIENT INSTALL FROM YOUR
  REGULAR OBJ TREE OR ELSE YOU MAY WIND UP WITH A NEUTERED VERSION OF
  libtb.a!

The build-as-well-as-install semantics are counter to the regular install
targets, but this is what we gotta do for now.  Once the TB source builds
under Linux and newer BSDs, we could undo this and just require that people
do a regular "make" followed by "make client-install"  OTOH, there should
be no reason to require installation of mysql and other server-side packages
just to build clients (or make them sit through the compilation of assign),
so maybe we will keep the client build special.

Note: more changes are pending for the makefile related to moving to standard makefile style/suppressing errors.

makefile needs to be fixed (remove fbsd/linux subdir cruft).

I am not using paperbag to run it on ops, but just installing the two
programs that are needed. Eventually should paperbag it.

This same script runs on both boss/ops and on experimental nodes to
fire off the linktest daemon on the nodes and wait. This is icky as
some crufty stuff has to be done so that it will run in both
environments. ltevent is of course not needed on boss, but do not want
to bother it now since it *is* needed on the nodes.

To run it on boss script needed severe cleanup and taint checking.
Also added the use of event keyfile, which is currently optional, but
needs to be mandatory once the images are updated.

The current problem is that run_linktest can hang waiting for the
clients to finish?. We can use quick mode in the experiment setup
path, but maybe we need a cancel operation?

Removed hardcoded "boss" from linktest.pl and run_linktest.pl
Changed historical log location.

and wait for linktest events, and integrated it into the perl scripts.

At this point I'm working on small potatoes such as the rc.setup script
to invoke linktest daemon and nailing down where to put platform-specific
stuff such as the ns-linktest build and the customized tb-compat.

Also am testing linktest in cases where I've deliberately inserted an error
to make sure it catches it.

This includes an updated client-install.

However, this is not yet ready for builds due to certain libraries not
present on the nodes and ops. For those I will be writing a static
linked c program to handle the linktest "done" event in both linktest.pl
and run_linktest.pl.

and addded parameter passing the daemon.

Also fixed up the makefile for compilation and installation (client).

described in my final project paper.

is to add HMACs to events to ensure they that events cannot be
injected into an experiment by an unauthorized client.

* The frontend now generates a secret key for each experiment and
  stores that into a file and in the DB.

* Each of the event clients, as well as the event producers
  (scheduler, tevc) have a new -k option to specify the name of the
  file. Two new event library functions were added for clients to give
  the key:

    event_handle_t
    event_register_withkeyfile(char *name, int threaded, char *keyfile);

    event_handle_t
    event_register_withkeydata(char *name, int threaded,
	   		       unsigned char *keydata, int keylen);

* When the library is in possesion of a key, it will generate an HMAC
  and attach it to outgoing notifications. A client receiving a
  notification will compute an HMAC and compare it against the HMAC in
  the notification. If they do not compare, the notification is
  dropped with a warning message printed (the client callback never
  gets the notification). If the client has not provided a key, then
  the HMAC in the incoming notification is ignored.

* The scheduler also takes a -k option, and will compute HMACs for all
  of the static events ahead of time. That keeps it off the critical
  path.

* The tevc client also takes a -k option. However, tevc will always
  try to find the keyfile (default path) so that it can attach the
  HMAC to dynamic events before sending them to the scheduler (which
  will check to make sure it matches). The scheduler will not accept
  dynamic events without unless the HMAC is present and matches.

* I have rebuilt the elvin librarys, removing all of the X goop and
  the SSL goop. Smaller binaries. So, I had to add -lcrypto to all of
  the client makefiles to that programs link.

* The program-agent got a few more changes. The command string is no
  longer passed inside the event; it comes in when the program agent
  is started, via a config file generated from tmcd data. This gets
  rid of our mostly insecure remote execution facility.

of stuff in the new image. Also added client side install targets
every place I could think of.

and reponds to PROGRAM events. Currently, just start and stop. Start
takes a COMMAND= argument, and allows arbitrary command lines since I
pass the whole thing off to the shell. Caveat; the agent runs as root
and starts the program as root. You can has as many program objects in
your NS file as you like, but each one can be started once; you have
to either stop or wait for the old one to finish before trying to
start again.

system programs. Build two versions of the event library, one for non
threaded programs and another (libevent_r.a) for threaded versions.
Remove all that -pthread goo from all of the makefiles, except for the
scheduler (which is threaded); it gets linked as before, but with
-levent_r instead. If you try and use the threaded API without the
proper link, the library will print an error message and quit.

tbsend.c and tbrecv.c, which are trivial examples of how to send
and receive a sample event.

Also several changes for building on FreeBSD. I've left the older
Makefiles in place so Ian can continue to build in his environmant
(Linux?).