1. 09 Mar, 2005 1 commit
  2. 07 Mar, 2005 1 commit
    • Timothy Stack's avatar
      · 898cf9a2
      Timothy Stack authored
      Checkin some changes related to experiment automation and vnode feedback:
      
      	* configure, configure.in: Add sensors/canaryd/feedbacklogs
      	template.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
      	table that holds environment variable names and values.
      
      	* event/lib/event.c: Allocate memory of the right size for
      	event_notifications.
      
      	* event/program-agent/GNUmakefile.in: Add version.c file and
      	add install targets for the man page.
      
      	* event/program-agent/program-agent.8: Man page describing the
      	program-agent daemon.
      
      	* event/program-agent/program-agent.c: Add a bunch of convenience
      	features: let the user specify the working directory for commands;
      	save output to separate files on every invocation of an agent; let
      	the user specify a timeout for a command; make the set of
      	environment variables sane and add vars given in the NS file in
      	the opt array; a "status" file containing process information is
      	written out when children are collected.  Internal changes: child
      	processes are collected immediately, instead of waiting for the
      	next START event, so we can send back COMPLETE events; the daemon
      	now runs with a real-time priority, to increase the chances of
      	receiving events.
      
      	* event/proxy/evproxy.c: Made it bidirectional so the
      	program-agent's COMPLETE events make it back to the scheduler.
      
      	* event/sched/error-record.c: Change the default log directory.
      
      	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
      	environment similar to a program-agent to run the user's log
      	digester.
      
      	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
      	that runs create_image for the node.
      
      	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
      	Let the user specify a "DIGESTER" script that digests the log
      	files into a summary of the results.  Add event handler for
      	remapping a vnode experiment.
      
      	* event/sched/timeline-agent.c: Accept the RUN event as well as
      	the START event.
      
      	* os/GNUmakefile.in: Install the install-tarfile.1 man page.
      
      	* os/install-tarfile: Automatically chown/chgrp any files that do
      	not have valid user or group IDs, the new owner will be the user
      	that swapped in the experiment.  Include the install directory in
      	the DB file.  Add a "list" mode that just dumps what files have
      	been installed and where.  Add a "force" option so the user can
      	forcefully install the file, even though the DB says its already
      	there.
      
      	* os/install-tarfile.1: Man page describing the install-tarfile
      	tool.
      
      	* os/syncd/GNUmakefile.in: Install man pages on ops.
      
      	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
      	install "feedbacklogs" tool.
      
      	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.
      
      	* sensors/canaryd/canarydEvents.c: Log errors.
      
      	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
      	data from canaryd log files.
      
      	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.
      
      	* sensors/slothd/digest-slothd: Fix some bugs and write out an
      	"alert" file with all the nodes/links that were overloaded.
      
      	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
      	argument that lets you specify that you want to wait for the disk
      	to finish loading and/or wait for the node to come back up in the
      	new OS.
      
      	* tbsetup/power.in: Remove debugging printf.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
      	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
      	quoting problems with event-sequences.  Add -expected-exit-code
      	and -tag options to the "$program run" event.  Add -digester to
      	the "$ns report" event that lets the user specify a program to run
      	to digest the log files.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
      	factor for feedback nodes to 1%, instead of 100%.
      
      	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
      	returns the values in "virt_user_environment".  Return new program
      	agent fields: dir, timeout, and expected_exit_code.
      
      	* tmcd/common/GNUmakefile.in: Install rc.canaryd.
      
      	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
      	a real-time priority, they can't do it from inside the jail.
      
      	* tmcd/common/rc.canaryd: Rc script for canaryd.
      
      	* tmcd/common/watchdog: Don't fail outright if there is a bad line
      	in the battery.log
      
      	* tmcd/common/rc.progagent: Append "userenv" data to the
      	program-agent config file.
      
      	* utils/GNUmakefile.in: Install loghole and its man page on ops.
      
      	* utils/loghole.1: Document "clean" command and the change in
      	loghole directories.
      
      	* utils/loghole.in: Add "clean" command and parallelization.
      
      	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
      	Order the eventlist by "idx" and time, needed for sequences.  And
      	removed unnecessary nologin checks.
      898cf9a2
  3. 14 Jan, 2005 3 commits
    • Kirk Webb's avatar
      Get rid of some debugging stuff. · 231845af
      Kirk Webb authored
      231845af
    • Kirk Webb's avatar
      0e27c632
    • Timothy Stack's avatar
      · dee46d59
      Timothy Stack authored
      Cross compilation fixes for the stargates, 'gmake client' should now
      build, link, and install properly.  Haven't really tried to run stuff though.
      
      	* GNUmakerules: Add target for stripping executables, used instead
      	of "install -s" since that doesn't work for cross-compiling.
      
      	* Makeconf.in: Add ELVIN_CONFIG variable that refers to
      	'elvin-config'.
      
      	* configure, configure.in: Detect and save the elvin-config path
      	since we need a different one for cross-compiling.
      
      	* event/lib/GNUmakefile.in, event/link-agent/GNUmakefile.in,
      	event/linktest/GNUmakefile.in, event/program-agent/GNUmakefile.in,
      	event/proxy/GNUmakefile.in, event/tbgen/GNUmakefile.in,
      	event/trafgen/GNUmakefile.in, os/dijkstra/GNUmakefile.in,
      	os/syncd/GNUmakefile.in, sensors/slothd/GNUmakefile.in,
      	tmcd/GNUmakefile.in, tmcd/linux/GNUmakefile.in: Cross compilation
      	fixes, don't statically link on arm, create "foo-debug"
      	executables with debugging info and install separately stripped
      	ones instead of passing "-s" to install.
      dee46d59
  4. 03 Jan, 2005 1 commit
  5. 05 Nov, 2004 1 commit
  6. 05 Oct, 2004 1 commit
  7. 27 Jul, 2004 1 commit
  8. 12 Jul, 2004 1 commit
  9. 01 Jul, 2004 1 commit
  10. 24 Jun, 2004 2 commits
    • Mike Hibler's avatar
      Improve the client-side install. With these changes, it should now be · 976133e4
      Mike Hibler authored
      possible to:
      
      	gmake client
      	sudo gmake client-install
      
      on a FBSD4, FBSD5, RHL7.3, and RHL9.0 client node.
      
      There are still some dependencies that are not explicit and which would
      prevent a build/install from working on a "clean" OS.  Two that I know of are:
      you must install our version of the elvin libraries and you must install boost.
      976133e4
    • Mike Hibler's avatar
      Minor lint for GCC 3.3 · a36ccc7b
      Mike Hibler authored
      (early stages of getting Emulab software to build under FreeBSD 5)
      a36ccc7b
  11. 10 May, 2004 1 commit
  12. 26 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Cleanup Makefiles: · 297019fb
      Mike Hibler authored
      1. "make clean" will just remove stuff built in the process of a regular build
      2. "make distclean" will also clean out configure generated files.
      
      This is how it was always supposed to be, there was just some bitrot.
      297019fb
  13. 20 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Improve the client-install. You can now do a "make client-install" from · 361ee691
      Mike Hibler authored
      the top level.  This will build all the necessary binaries and then install
      them.  This works on FBSD4 and RHL7.3.  It still doesn't work on FBSD5
      (newer compiler that no longer supports a style of use of _FUNCTION_ in the
      event lib) or RHL9 (event lib needs SSL lib which has a bad dependency
      on Kerberos).  Notes:
      
      - requires that elvin libraries be installed on nodes (they are) to build
        event agents, requires linuxthreads be installed on FBSD (it is now) to
        build imagezip (which is installed, but is not strictly necessary)
      
      - installed event-agents and other binaries are stripped
      
      - added a few missing files to the source tree for bsd (healthd.conf)
        and linux (healthd.conf, rc.local)
      
      - the only thing that doesn't get rebuilt in /usr/local/etc/emulab is
        healthd, I couldn't quickly find how it gets built
      
      - uses a scaled down version of libtb with no DB functions (since mysql
        isn't installed on nodes).  N.B. DO NOT DO A CLIENT INSTALL FROM YOUR
        REGULAR OBJ TREE OR ELSE YOU MAY WIND UP WITH A NEUTERED VERSION OF
        libtb.a!
      
      The build-as-well-as-install semantics are counter to the regular install
      targets, but this is what we gotta do for now.  Once the TB source builds
      under Linux and newer BSDs, we could undo this and just require that people
      do a regular "make" followed by "make client-install"  OTOH, there should
      be no reason to require installation of mysql and other server-side packages
      just to build clients (or make them sit through the compilation of assign),
      so maybe we will keep the client build special.
      361ee691
  14. 23 Dec, 2003 1 commit
  15. 15 Oct, 2003 1 commit
    • Mike Hibler's avatar
      Uniform syslog'ing. Change everything I could find to use a syslog facility · cc6d6fa7
      Mike Hibler authored
      as defined in the defs-* file (e.g. "TBLOGFACIL=local2").  The default is
      "local5" which is what we are setup to use so you shouldn't need to mess
      with your defs- file!
      
      perl scripts just get this value configured in when configure is run.
      C programs get the value in two ways.  For programs that are intimate with
      the testbed infrastructure, and include "config.h", they just get it from
      that file.  For programs that we sometimes use outside the Emulab build
      environment (e.g., frisbee, capture) and that don't include config.h,
      the value is set via a "-DLOG_TESTBED=..." in the GNUmakefile build line.
      If the value isn't set, it defaults to what it used to be (usually LOG_USER).
      
      Still to do: healthd, hmcd (whose build doesn't seem to be completely
      integrated) and plabdaemon.in (since its icky python :-)
      cc6d6fa7
  16. 22 Aug, 2003 1 commit
  17. 31 Jul, 2003 1 commit
    • Kirk Webb's avatar
      Minor changes: · 073c9473
      Kirk Webb authored
      1) Use libtb's reporting funcs rather than my own (the former are better)
      2) Produce more informational "malformed interface record" errors.
      073c9473
  18. 28 Apr, 2003 1 commit
  19. 21 Mar, 2003 1 commit
    • Kirk Webb's avatar
      Well, here it is: The new slothd and sdcollectd stuff. · a337f7b9
      Kirk Webb authored
      Several things have changed; here is an outline:
      
       - "Thresholds" added to slothd
      	These are checked at each iteration against current idle data
              to determine whether or not to set bits in the "active" vector.
      	- LAST_TTY_ACT (now) > LAST_TTY_ACT (previous)
              - LOAD > <provided value>
              - EXPER_PKT_COUNT(now) - EXPER_PKT_COUNT(prev) > <provided value>
      	- CNTRL_PKT_COUNT(now) - CNTRL_PKT_COUNT(prev) > <provided value>
      	- Values passed in via command line: do ./slothd -h for help
       - "Aggressive" mode added to slothd
      	When a regular interval's worth of time has passed, and no activity
              has been noted, slothd goes into "aggressive" mode.  In this mode,
      	slothd changes its sleep interval, checking for activity much more
      	frequently (still reporting every <reg_interval>) until activity
      	is noted.  This is based on the threshold logic above.  When activity
      	is seen, slothd sends a report, and goes back to "regular" reporting.
      	- params passed via command line for regular, and aggressive intervals.
       - sdcollectd updates data in new "node_activity" table
      	- Updates timestamps based on activity bits in received slothd packets
      	- No new rows are added.
       - Option added to sdcollectd to insert into "old" node_idlestats and
      	iface_counters tables
         	- This is not the default (-o)
       - New sdcollectd still accepts old client reports
      	- but does not update the node_activity table with them.
       - Misc.
      	- Code cleanup
      	- Scan "active terminals only" option & code removed from slothd
      	- Always send report immediately on startup (no more option)
      	- Randomize time of second report to mitigate synchronization
      	- Send report on shutdown (receipt of INT, TERM, or QUIT)
      a337f7b9
  20. 18 Dec, 2002 1 commit
  21. 07 Oct, 2002 1 commit
  22. 12 Jul, 2002 1 commit
    • Mac Newbold's avatar
      Fix flags passed to slothd. Before, -f meant send an immediate update (which... · 3a643160
      Mac Newbold authored
      Fix flags passed to slothd. Before, -f meant send an immediate update (which wasn't the default behavior). The latest slothd updates (several weeks ago at least) changed the polarity of the flag. It still does the non-default thing, but now that makes it wait an hour before sending an update. So we remove -f so it does the right thing.
      3a643160
  23. 10 Jul, 2002 1 commit
  24. 09 Jul, 2002 1 commit
  25. 24 Jun, 2002 1 commit
  26. 19 Jun, 2002 1 commit
  27. 18 Jun, 2002 1 commit
  28. 11 Jun, 2002 1 commit
  29. 05 Jun, 2002 1 commit
    • Leigh B. Stoller's avatar
      Changes to sshtb. Remove sshremote, and convert sshtb into a perl · 231fc2b1
      Leigh B. Stoller authored
      script that checks the database to see if local or remote. The problem
      with this is that the ssh syntax makes it hard to determine the host
      name by inspection. Would need to parse all the ssh args (bad idea),
      ot work backwards and try to figure out the difference between the
      command (which is not a string but a sequence of args) and the host
      and the preceeding ssh args. Hell with that! Changed sshtb to require
      a specific -host argument. Read the args and look for it. Error out of
      not found, to catch improper usage.
      
      The moral of this update: "sshtb [ssh args] -host <host> [more args ...]
      231fc2b1
  30. 31 May, 2002 1 commit
  31. 24 May, 2002 4 commits
    • Mac Newbold's avatar
      Big changes. Add idlecheck, sdisrunning and sddeploy to configure and the make... · 5aece56b
      Mac Newbold authored
      Big changes. Add idlecheck, sdisrunning and sddeploy to configure and the make files. Install sd* above into sbin dir. Rename sddeploy.pl and isrunning.pl.
      5aece56b
    • Mac Newbold's avatar
      Big changes. · 2c03632f
      Mac Newbold authored
      Added default behavior so that when no pid/eid is given, it looks up
      all the nodes that haven't reported in the last 2 hours and runs it
      on those nodes.
      Also added magic around ssh calls. Fork and set an alarm, if it doesn't come
      back, declare the node unreachable and kill the child.
      (This checkin is for checkpointing, I'm now going to add this to configure
      so that it properly gets devel libdb.pm instead of hardcoded to the real one.)
      2c03632f
    • Kirk Webb's avatar
      Some important changes to slothd · d500b090
      Kirk Webb authored
      1) Removed the ten iterations limit from main daemon loop I had set during
      testing a while back (duh).  This was why slothd was mysteriously disappearing
      after half a day.
      
      2) Updated the makefile to produce static slothd binaries
      
      3) Added a locking mechanism to avoid accidentally starting two slothd's
      simulatneously.
      d500b090
    • Mac Newbold's avatar
      30452e80
  32. 22 May, 2002 1 commit
    • Kirk Webb's avatar
      This commit includes a script to clean old idle data from the emulab · 3bfcfe9f
      Kirk Webb authored
      database (iddata_clean), and a script to generate fake idle entries for
      testing scripts on the minibed (idloadup.pl).
      
      iddata_clean, by default, removes all idle entries older than 36 hours.
      This can be changed by providing -m <maxage> (in seconds) to it.
      
      idloadup.pl adds random idle data at time offsets from now, to 5 days
      prior (1 sec granularity).  It adds anywhere from 1 to 10 entries per
      PC to the node_idlestats page, and from 1 - 10 x # interfaces to
      the iface_counters table.
      3bfcfe9f
  33. 14 May, 2002 1 commit
  34. 13 May, 2002 1 commit
    • Kirk Webb's avatar
      Forgot to check this stuff in last week: · 286e2019
      Kirk Webb authored
      Slothd changes to force an initial wait of <interval> (def 1hr).  Selectable
      with -w.  Also changed to scan ALL ttys by default (can use UTMP with -a).
      
      Finally, deploy script changed a bit to be more robust wrt down nodes.
      286e2019