1. 03 Sep, 2008 1 commit
  2. 28 Nov, 2007 1 commit
    • Leigh B. Stoller's avatar
      Add some small stuff to support the current implementation of the plab · b30e4f95
      Leigh B. Stoller authored
      evproxy, which uses set_failover and set_connection_retries. The event
      library had these as noops, so that was an easy change. Also, add back
      in the async add/remove subscription stuff, which was already
      implemented in pubsub but not hooked up from the event library.
      
      In the tmcd/plab directory I purged all mention of elvin and changed it
      to "event server". I also renamed the runelvin script to runevents.
      
      In events/proxy I cleaned up the makefile and added evproxyplab to the
      targets list since it should now build okay (no longer needs to link
      against elvin stuff). Renamed elvindtest program to eventping, and
      otherwise purged lots of "elvin" tokens.
      b30e4f95
  3. 16 Aug, 2007 1 commit
  4. 25 May, 2007 1 commit
  5. 24 Apr, 2007 1 commit
  6. 23 Apr, 2007 1 commit
  7. 23 Mar, 2007 1 commit
  8. 02 Feb, 2007 1 commit
    • Mike Hibler's avatar
      The paint wasn't even dry on the last plab tarball before I have moved on... · c2570cfc
      Mike Hibler authored
      Rev 20 of the tarball has no new function, it is just slimmed down.
      I replaced our large statically linked binaries with dynamically linked ones.
      Also found and stripped a couple of other binaries.
      
      The resulting tarball goes from 5.7MB to 0.9MB.
      
      The Makefile changes are to incorporate the NOSTATIC=1 environment hack
      to enable building the dynamic binaries above.
      c2570cfc
  9. 25 Aug, 2006 1 commit
    • Kirk Webb's avatar
      · e3c90cb9
      Kirk Webb authored
      Fix experiment event stream unsubscribe bug.
      e3c90cb9
  10. 11 Aug, 2006 1 commit
    • Kirk Webb's avatar
      · 169d5076
      Kirk Webb authored
      Fix up another place where the hostname lookup can fail, and thus cause
      the proxy (or anything else) to exit.  Both evproxyplab and the event lib
      now first try to lookup the hostname to get the IP, and then fall
      back to grabbing the IP from /var/emulab/myip.
      169d5076
  11. 10 Aug, 2006 1 commit
    • Kirk Webb's avatar
      · 7901885c
      Kirk Webb authored
      The other half of the changes that cause the plab event proxy to now try
      to get the routable IP of the node from tmcd rather than relying on
      the success of a hostname lookup.  It will still fall back to trying a
      hostname lookup if it can't get the IP from tmcd.
      7901885c
  12. 19 May, 2006 1 commit
    • Kirk Webb's avatar
      · e0c33c75
      Kirk Webb authored
      More reliability fixes/enhancements.  Hopefully this will catch the remaining
      problems I've seen happen.
      e0c33c75
  13. 19 Apr, 2006 1 commit
    • Kirk Webb's avatar
      · a5ea34cb
      Kirk Webb authored
      Update the plab proxy to manage restoring the connection to the remote
      elvind on ops itself (instead of leaving it up to elvin).
      
      This whole program needs cleanup/refactoring attention.  Later.
      a5ea34cb
  14. 28 Mar, 2006 1 commit
    • Kirk Webb's avatar
      · 30ff680b
      Kirk Webb authored
      Let's increase the idle check period a bit...
      30ff680b
  15. 27 Mar, 2006 1 commit
    • Kirk Webb's avatar
      · 74fc532b
      Kirk Webb authored
      Some event system fixes for the plab proxy and testing enhancements to tbsend.
      
      * event.c / event.h
      
      Fix assertion failure that appears to be caused by notifications coming in
      on a subscription that is pending deletion (via async API call).
      
      Also add idle polling to the ops endpoint connection to help maintain
      connectivity (will do failover).
      
      Add function to set failover flag on connection handle.
      
      * tbsend.c
      
      Add ability to set connection retry, failover, idle polling, and perform
      looped and triggered testing.
      74fc532b
  16. 09 Mar, 2006 1 commit
    • Sachin Goyal's avatar
      · 2ca5a9e4
      Sachin Goyal authored
      Modifications done for plab scheduler:
      * pass thru the UPLOAD/CLEAR events to local elvind so that
        plab-scheduler can get them.
      * filter events with <plabsched, 1>
      * listen to scheduler==2 evennts also from ops elvind
      2ca5a9e4
  17. 02 Feb, 2006 1 commit
    • Kirk Webb's avatar
      · 16686f26
      Kirk Webb authored
      Added "global passthru" event subscription to the plab event proxy.
      16686f26
  18. 01 Feb, 2006 1 commit
    • Kirk Webb's avatar
      · 08a74ca6
      Kirk Webb authored
      Fix up the plav event proxy a bit.  It was sending RELOAD notifications
      when it should have been scheduling UPDATEs.  The RELOAD event handler was
      also updated to schedule and update.  Replaced some sprintfs with snprintfs,
      and added a log file and some helpful log output.
      
      pa-wrapper: tell libtmcc we don't want a cached "plabconfig" result.
      08a74ca6
  19. 29 Jan, 2006 1 commit
    • Sachin Goyal's avatar
      · 92a687b9
      Sachin Goyal authored
      Added 2 new programs:
      
      1- elvindtest: a simple program to verify if a particular
          elvindserver is up or not
      
      2- evproxyplab.cc: evproxy for planetlab nodes. It subscribes
         to the following events:
         host = node's ip addr
         objtype = EVPROXY
         objname = __$nodeid_proxy
         eventtype = UPDATE | CLEAR | RELOAD
      
         On UPDATE, it adds a new subscription to listen to an
         experiment conveyed by EXPT field of UPDATE message.
      
         On CLEAR, it removes the subscription for a experiment
         experiment conveyed by EXPT field of CLEAR message.
      
         On RELOAD, it removes all active experiment subscriptions.
      
         Also, while it comes up, it sends a RELAOD event to emulab's
         event-server to reload it with all existing experiments:
      
         host = node's ip addr
         objtype = EVPROXY
         objname = __$nodeid_proxy
         eventtype = RELOAD
      92a687b9
  20. 07 Mar, 2005 1 commit
    • Timothy Stack's avatar
      · 898cf9a2
      Timothy Stack authored
      Checkin some changes related to experiment automation and vnode feedback:
      
      	* configure, configure.in: Add sensors/canaryd/feedbacklogs
      	template.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
      	table that holds environment variable names and values.
      
      	* event/lib/event.c: Allocate memory of the right size for
      	event_notifications.
      
      	* event/program-agent/GNUmakefile.in: Add version.c file and
      	add install targets for the man page.
      
      	* event/program-agent/program-agent.8: Man page describing the
      	program-agent daemon.
      
      	* event/program-agent/program-agent.c: Add a bunch of convenience
      	features: let the user specify the working directory for commands;
      	save output to separate files on every invocation of an agent; let
      	the user specify a timeout for a command; make the set of
      	environment variables sane and add vars given in the NS file in
      	the opt array; a "status" file containing process information is
      	written out when children are collected.  Internal changes: child
      	processes are collected immediately, instead of waiting for the
      	next START event, so we can send back COMPLETE events; the daemon
      	now runs with a real-time priority, to increase the chances of
      	receiving events.
      
      	* event/proxy/evproxy.c: Made it bidirectional so the
      	program-agent's COMPLETE events make it back to the scheduler.
      
      	* event/sched/error-record.c: Change the default log directory.
      
      	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
      	environment similar to a program-agent to run the user's log
      	digester.
      
      	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
      	that runs create_image for the node.
      
      	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
      	Let the user specify a "DIGESTER" script that digests the log
      	files into a summary of the results.  Add event handler for
      	remapping a vnode experiment.
      
      	* event/sched/timeline-agent.c: Accept the RUN event as well as
      	the START event.
      
      	* os/GNUmakefile.in: Install the install-tarfile.1 man page.
      
      	* os/install-tarfile: Automatically chown/chgrp any files that do
      	not have valid user or group IDs, the new owner will be the user
      	that swapped in the experiment.  Include the install directory in
      	the DB file.  Add a "list" mode that just dumps what files have
      	been installed and where.  Add a "force" option so the user can
      	forcefully install the file, even though the DB says its already
      	there.
      
      	* os/install-tarfile.1: Man page describing the install-tarfile
      	tool.
      
      	* os/syncd/GNUmakefile.in: Install man pages on ops.
      
      	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
      	install "feedbacklogs" tool.
      
      	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.
      
      	* sensors/canaryd/canarydEvents.c: Log errors.
      
      	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
      	data from canaryd log files.
      
      	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.
      
      	* sensors/slothd/digest-slothd: Fix some bugs and write out an
      	"alert" file with all the nodes/links that were overloaded.
      
      	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
      	argument that lets you specify that you want to wait for the disk
      	to finish loading and/or wait for the node to come back up in the
      	new OS.
      
      	* tbsetup/power.in: Remove debugging printf.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
      	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
      	quoting problems with event-sequences.  Add -expected-exit-code
      	and -tag options to the "$program run" event.  Add -digester to
      	the "$ns report" event that lets the user specify a program to run
      	to digest the log files.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
      	factor for feedback nodes to 1%, instead of 100%.
      
      	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
      	returns the values in "virt_user_environment".  Return new program
      	agent fields: dir, timeout, and expected_exit_code.
      
      	* tmcd/common/GNUmakefile.in: Install rc.canaryd.
      
      	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
      	a real-time priority, they can't do it from inside the jail.
      
      	* tmcd/common/rc.canaryd: Rc script for canaryd.
      
      	* tmcd/common/watchdog: Don't fail outright if there is a bad line
      	in the battery.log
      
      	* tmcd/common/rc.progagent: Append "userenv" data to the
      	program-agent config file.
      
      	* utils/GNUmakefile.in: Install loghole and its man page on ops.
      
      	* utils/loghole.1: Document "clean" command and the change in
      	loghole directories.
      
      	* utils/loghole.in: Add "clean" command and parallelization.
      
      	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
      	Order the eventlist by "idx" and time, needed for sequences.  And
      	removed unnecessary nologin checks.
      898cf9a2
  21. 14 Jan, 2005 1 commit
    • Timothy Stack's avatar
      · dee46d59
      Timothy Stack authored
      Cross compilation fixes for the stargates, 'gmake client' should now
      build, link, and install properly.  Haven't really tried to run stuff though.
      
      	* GNUmakerules: Add target for stripping executables, used instead
      	of "install -s" since that doesn't work for cross-compiling.
      
      	* Makeconf.in: Add ELVIN_CONFIG variable that refers to
      	'elvin-config'.
      
      	* configure, configure.in: Detect and save the elvin-config path
      	since we need a different one for cross-compiling.
      
      	* event/lib/GNUmakefile.in, event/link-agent/GNUmakefile.in,
      	event/linktest/GNUmakefile.in, event/program-agent/GNUmakefile.in,
      	event/proxy/GNUmakefile.in, event/tbgen/GNUmakefile.in,
      	event/trafgen/GNUmakefile.in, os/dijkstra/GNUmakefile.in,
      	os/syncd/GNUmakefile.in, sensors/slothd/GNUmakefile.in,
      	tmcd/GNUmakefile.in, tmcd/linux/GNUmakefile.in: Cross compilation
      	fixes, don't statically link on arm, create "foo-debug"
      	executables with debugging info and install separately stripped
      	ones instead of passing "-s" to install.
      dee46d59
  22. 03 Jan, 2005 1 commit
  23. 30 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      The bulk of the event system changes. · 9aa6b5ca
      Leigh B. Stoller authored
      * The per-experiment event scheduler now runs on ops instead of boss.
        Boss still runs elvind and uses events internally, but the user part
        of the event system has moved.
      
      * Part of the guts of eventsys_control moved to new script, eventsys.proxy,
        which runs on ops and fires off the event scheduler. The only tricky part
        of this is that the scheduler runs as the user, but killing it has to be
        done as root since a different person might swap out the experiment. So,
        the proxy is a perl wrapper invoked from a root ssh from boss, which
        forks, writes the pid file into /var/run/emulab/evsched/$pid_$eid.pid,
        then flips to the user and execs the event scheduler (which is careful
        not to fork). Obviously, if the kill is done as root, the pid file has to
        be stored someplace the user is not allowed to write.
      
      * The event scheduler has been rewritten to use Tim's C++ interface to the
        sshxmlrpc server on boss. Actually, I reorg'ed the scheduler so that it
        can be built either as a mysql client, or as RPC client. Note that it can
        also be built to use the SSL version of the XMLRPC server, but that will
        not go live until I finish the server stuff up. Also some goo for dealing
        with building the scheduler with C++.
      
      * Changes to several makefiles to install the ops binaries over NFS to
        /usr/testbed/opsdir. Makes life easier, but only if boss and ops are
        running the same OS. For now, using static linking on the event scheduler
        until ops upgraded to same rev as boss.
      
      * All of the event clients got little tweaks for dealing with the new CNAME
        for the event system server (event-sever). Will need to build new images
        at some point. Old images and clients will continue to work cause of an
        inetd hack on boss that uses netcat to transparently redirect elvind
        connections to ops.
      
      * Note that eventdebug needs some explaining. In order to make the inetd
        redirect work, elvind cannot be listening on the standard port. So, the
        boss event system uses an alternate port since there are just a few
        subsystems on boss that use the server, and its easy to propogate changes
        on boss. Anyway, the default for eventdebug is to connect to the standard
        port on localhost, which means it will work as expected on ops, but will
        require -b argument on boss.
      
      * Linktest changes were slightly more involved. No longer run linktest on
        boss when called from the experiment swapin path, but ssh over to ops to
        fire it off. This is done as the user of course, and there are some
        tricks to make it possible to kill a running linktest and its ssh when
        experiment swapin is canceled (or from the command line) by forcing
        allocation of a tty. I will probably revisit this at some point, but I
        did not want to spend a bunch of time on linktest.
      
      * The upgrade path detailed in doc/UPDATING is necessarily complicated and
        bound to cause consternation at remote sites doing an upgrade.
      9aa6b5ca
  24. 24 Jun, 2004 2 commits
    • Mike Hibler's avatar
      Improve the client-side install. With these changes, it should now be · 976133e4
      Mike Hibler authored
      possible to:
      
      	gmake client
      	sudo gmake client-install
      
      on a FBSD4, FBSD5, RHL7.3, and RHL9.0 client node.
      
      There are still some dependencies that are not explicit and which would
      prevent a build/install from working on a "clean" OS.  Two that I know of are:
      you must install our version of the elvin libraries and you must install boost.
      976133e4
    • Mike Hibler's avatar
      Minor lint for GCC 3.3 · a36ccc7b
      Mike Hibler authored
      (early stages of getting Emulab software to build under FreeBSD 5)
      a36ccc7b
  25. 11 May, 2004 1 commit
  26. 20 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Improve the client-install. You can now do a "make client-install" from · 361ee691
      Mike Hibler authored
      the top level.  This will build all the necessary binaries and then install
      them.  This works on FBSD4 and RHL7.3.  It still doesn't work on FBSD5
      (newer compiler that no longer supports a style of use of _FUNCTION_ in the
      event lib) or RHL9 (event lib needs SSL lib which has a bad dependency
      on Kerberos).  Notes:
      
      - requires that elvin libraries be installed on nodes (they are) to build
        event agents, requires linuxthreads be installed on FBSD (it is now) to
        build imagezip (which is installed, but is not strictly necessary)
      
      - installed event-agents and other binaries are stripped
      
      - added a few missing files to the source tree for bsd (healthd.conf)
        and linux (healthd.conf, rc.local)
      
      - the only thing that doesn't get rebuilt in /usr/local/etc/emulab is
        healthd, I couldn't quickly find how it gets built
      
      - uses a scaled down version of libtb with no DB functions (since mysql
        isn't installed on nodes).  N.B. DO NOT DO A CLIENT INSTALL FROM YOUR
        REGULAR OBJ TREE OR ELSE YOU MAY WIND UP WITH A NEUTERED VERSION OF
        libtb.a!
      
      The build-as-well-as-install semantics are counter to the regular install
      targets, but this is what we gotta do for now.  Once the TB source builds
      under Linux and newer BSDs, we could undo this and just require that people
      do a regular "make" followed by "make client-install"  OTOH, there should
      be no reason to require installation of mysql and other server-side packages
      just to build clients (or make them sit through the compilation of assign),
      so maybe we will keep the client build special.
      361ee691
  27. 08 Mar, 2004 1 commit
  28. 06 Nov, 2003 1 commit
  29. 05 Nov, 2003 1 commit
    • Leigh B. Stoller's avatar
      Middle part of the event system changes. The main part of this change · 54bc15c4
      Leigh B. Stoller authored
      is to add HMACs to events to ensure they that events cannot be
      injected into an experiment by an unauthorized client.
      
      * The frontend now generates a secret key for each experiment and
        stores that into a file and in the DB.
      
      * Each of the event clients, as well as the event producers
        (scheduler, tevc) have a new -k option to specify the name of the
        file. Two new event library functions were added for clients to give
        the key:
      
          event_handle_t
          event_register_withkeyfile(char *name, int threaded, char *keyfile);
      
          event_handle_t
          event_register_withkeydata(char *name, int threaded,
      	   		       unsigned char *keydata, int keylen);
      
      * When the library is in possesion of a key, it will generate an HMAC
        and attach it to outgoing notifications. A client receiving a
        notification will compute an HMAC and compare it against the HMAC in
        the notification. If they do not compare, the notification is
        dropped with a warning message printed (the client callback never
        gets the notification). If the client has not provided a key, then
        the HMAC in the incoming notification is ignored.
      
      * The scheduler also takes a -k option, and will compute HMACs for all
        of the static events ahead of time. That keeps it off the critical
        path.
      
      * The tevc client also takes a -k option. However, tevc will always
        try to find the keyfile (default path) so that it can attach the
        HMAC to dynamic events before sending them to the scheduler (which
        will check to make sure it matches). The scheduler will not accept
        dynamic events without unless the HMAC is present and matches.
      
      * I have rebuilt the elvin librarys, removing all of the X goop and
        the SSL goop. Smaller binaries. So, I had to add -lcrypto to all of
        the client makefiles to that programs link.
      
      * The program-agent got a few more changes. The command string is no
        longer passed inside the event; it comes in when the program agent
        is started, via a config file generated from tmcd data. This gets
        rid of our mostly insecure remote execution facility.
      54bc15c4
  30. 05 Jun, 2003 1 commit
    • Leigh B. Stoller's avatar
      New event proxy. This proxy is used in lieu of Elvin clustering or · b5d82850
      Leigh B. Stoller authored
      federation, which is not supported in the version we have source to.
      Basically, we run an elvind on each node. The proxy on each node
      subscribes to all events for that node from the boss elvind, and hands
      them to the local elvind, Each client on the node subscribes to the
      local elvind, and gets its events via the proxy. This should reduce
      the number of connections to boss, and makes it possible to run agents
      inside each virtual node without an FD explosion on boss.
      b5d82850