1. 07 Sep, 2005 1 commit
  2. 02 Sep, 2005 1 commit
  3. 30 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      Some console support changes: · 43b978df
      Leigh Stoller authored
      * utils/spewconlog.in: New script to ssh to the host tipserver and cat
        the console log for a node. In admin mode cats the .log file, while
        in mere user mode, cats the .run file.
      
      * www/shownode.php3: Added a "Show Console Log" function.
      
      * www/showconlog.php3: New script to invoke spewconlog backend script
        and dump the console log to the user.
      43b978df
  4. 12 Jul, 2005 1 commit
  5. 07 Jul, 2005 1 commit
    • Leigh Stoller's avatar
      Oh, such a silly little project ... Added CVS support to Emulab. When · 9b17b075
      Leigh Stoller authored
      enabled in the defs file:
      
      	CVSSUPPORT=1
      
      each project gets a stub CVS tree created (using 'cvs init') in
      /proj/$pid/CVS. It is up to users obviously to do something with
      that tree, and of course they have to either set their CVSROOT
      env variable, or use the -d option to cvs.
      
      The showproject page gets a link to the per-project CVS tree, using
      the cvsweb interface, which I hacked up a bit to allow restricted
      access to specific project trees, via a ?pid=$pid argument to the URL.
      Without the ?pid argument, it falls back to normal behaviour, which is
      check the cvsallowed bit in the users table, and provide access to the
      Emulab source repo.
      
      If you are curious, go here:
      
      	https://www.emulab.net/cvsweb/cvsweb.php3/?pid=testbed
      9b17b075
  6. 22 Jun, 2005 1 commit
    • Leigh Stoller's avatar
      Added my simplistic link tracing and monitoring. Example usage and · 7942119e
      Leigh Stoller authored
      some details can be found in the advanced tutorial that I wrote up.
      See this link:
      
      http://www.emulab.net/tutorial/docwrapper.php3?docname=advanced.html#Tracing
      
      The basic idea is that each virt_lan entry gets a couple of new slots
      describing the type of tracing that is desired.
      
        traced tinyint(1) default '0',
        trace_type enum('header','packet','monitor') NOT NULL default 'header',
        trace_expr tinytext,
        trace_snaplen int(11) NOT NULL default '0',
        trace_endnode tinyint(1) NOT NULL default '0',
      
      There is a new physical table called "traces" that is a little bit
      like the current delays table. A new tmcd command returns the trace
      configuration to the client nodes (tmcd/common/config/rc.trace).
      
      The delays table got a new boolean called "noshaping" that tells the
      delay node to bridge, but not set up any pipes. This allows us to
      capture traffic at the delay node, but without much less overhead on
      the packets.
      
      The pcapper got bloated up to do packet capture and more event stuff.
      I also had to add some mutex locking around calls into the pcap
      library and around malloc, since the current setup used linuxthreads,
      which is not compatable with the standard libc_r library. I was
      getting all kinds of memory corruption, and I am sure that if someone
      breathes on the pcapper again, it will break in some new way.
      7942119e
  7. 16 May, 2005 1 commit
    • Leigh Stoller's avatar
      Add ability to check out a specific source tag from the mirrored CVS · 603b59f9
      Leigh Stoller authored
      tree we keep on boss (which is update each night). Still need to
      arrange for tmcd to pass back CVSSRCTAG=xyz in the emulabconfig
      request, but if it did, that tag is passed along to the web interface,
      which calls new backend script to do the checkout, tar it up, gzip
      it, and spit to stdout.
      603b59f9
  8. 22 Apr, 2005 1 commit
    • Leigh Stoller's avatar
      Pump up grabwebcam script. Actually, I threw out the existing code · 162036f5
      Leigh Stoller authored
      that picked up single images with scp, and replaced it with stuff that
      can be used to capture movies.
      
      	Usage: grabwebcams [-d] [-v] [-m] [-t timeout]
      	       grabwebcams [-d] [-v] [-t timeout] [-k] pid eid
      	switches and arguments:
      	-d      - Debug mode, use to prevent daemonization
      	-v      - Verbose mode (causes vlc to spit lots of goo)
      	-m      - Movie option; create a 10fps movie from each camera
      	-t <N>  - Terminate automatically and N seconds
      	-k      - Kill a daemonized grabwebcams (only use with pid/eid)
      	pid eid - Project and Experiment (for use with swapin)
      
      The first form allows you to capture a low frame rate movie from the
      webcams. Low means 2fps. The files are written in the current
      directory. Without the -d option, the script goes into the background
      and runs forever, unless you give it a -t option, in which case the
      movies will be terminated after that many seconds.
      
      Add the -m option, and you get 15fps movies, suitable for showing off
      to people. This option IS VERY CPU INTENSIVE. Use it sparingly. It
      might get better once I hear back from the axis people, telling me if
      there is a way to turn up the mpeg fps via a URL (like most everything
      else can be on these cameras). Until then, I am transcoding mjpeg into
      mpeg, and that sucks.
      
      The second form is for use from the swapexp path, to start capturing
      movies when an experiment is swapped in, and then to then to kill it
      off (-k) when the experiment is swapped out. At 2fps, it generates
      about 1MB per camera per minute. Thats a lot of data, so I am not
      hooking this in quite yet; want to give it some more thought.
      
      We may want to export an interface to the web (and commandline) to
      that people can take movies of their experiments at particular times.
      Not sure yet.
      162036f5
  9. 04 Apr, 2005 1 commit
    • Timothy Stack's avatar
      · 463ee6b1
      Timothy Stack authored
      Mote and robot related stuff.  The main thing is the addition of relay
      capabilities to capture and related things.
      
      	* GNUmakefile.in: Add the capture and tip subdirectories to the
      	client and client-install targets.
      
      	* configure, configure.in, config.h.in: Detect srandomdev() for
      	capture and add "mote/newmote" script.
      
      	* capture/GNUmakefile.in, capture/capture.c: Add "relay"
      	capabilities to capture.
      
      	* capture/capquery.c: Query the capserver for the relay receiver's
      	port number.
      
      	* capture/capserver.c: Small hack to return the port number
              for a node.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add virt_tiptunnels table.
      
      	* event/program-agent/program-agent.c: Change log file names to
      	something a little more user-friendly.  Add a "MODIFY" event
      	handler that lets the user set agent attributes (command, tag,
      	timeout) without having to run a program.
      
      	* event/sched/GNUmakefile.in, event/sched/console-agent.cc,
      	event/sched/console-agent.h, event/sched/event-sched.c: Add
      	console agents that can be used to snapshot a section of the
      	capture log file.
      
      	* event/sched/node-agent.cc: Some minor cleanup.
      
      	* event/sched/simulator-agent.cc, event/sched/simulator-agent.h:
      	Add the config data to the report mail.  Add a "RESET" event
      	handler that runs "loghole clean".  Save the report mail in a file
      	so it gets archived with the rest of the logs.
      
      	* lib/libtb/tbdefs.h: Add CONSOLE object type.
      
      	* mote/GNUmakefile.in, mote/newmote: Add newmote script, just a
      	quick hack to add motes to the DB.
      
      	* mote/tbuisp.in: Add another backend for loading motes through
      	their relay capture server.
      
      	* robots/mtp/mtp_dump.c: Dump the min/max values for x and y,
      	handy for figuring out the bounds of the camera.
      
      	* sql/database-fill.sql: Change the RELOAD-MOTE/SHUTDOWN ->
      	ALWAYSUP/SHUTDOWN mode transition to ALWAYSUP/ISUP since stated
      	doesn't seem to run triggers after a state change by a mode
      	transition.
      
      	* tbsetup/tbreport.in: Change the ordering of the eventlist so it
      	displays event-sequences appropriately.
      
      	* tbsetup/ns2ir/GNUmakefile.in, tbsetup/ns2ir/console.tcl,
      	tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/parse.tcl.in,
      	tbsetup/ns2ir/sim.tcl.in: Add a "console" agent that represents
      	the serial console for a node.
      
      	* tbsetup/ns2ir/sequence.tcl: Add an "append" method so it is
      	easier to build sequences dynamically.
      
      	* tbsetup/ns2ir/topography.tcl: Make checkdest available to
      	regular users.
      
      	* tip/GNUmakefile.in, tip/tiptunnel.c: Add support for uploading a
      	file to a relay version of capture and exporting the end
      	connection as a pty.
      
      	* tmcd/decls.h, tmcd/common/libsetup.pm: Bump version number since
      	the dosubnodelist change is not backwards compatible.
      
      	* tmcd/tmcd.c: Make dosubnodelist and dosubconfig callable even
      	when a node isn't allocated.  Add dotiptunnels command that
      	returns which serial consoles are to be mounted on a node.  Add
      	mote version of subconfig that returns information needed to
      	startup the relay version of capture.
      
      	* tmcd/common/bootsubnodes: For motes, startup the relay version
      	of capture (XXX stargate specific).
      
      	* tmcd/common/libsetup.pm, tmcd/common/libtmcc.pm,
      	tmcd/common/config/rc.config, tmcd/common/config/rc.tiptunnels:
      	Client side changes for mounting another nodes serial line.
      
      	* tmcd/common/rc.bootsetup: Always boot the subnodes, even when
      	free.  This is used for motes since their capture needs to be up
      	for reloading at the time.
      
      	* tmcd/linux/ixpboot: Shuffle some code around so the script
      	doesn't fail if the ixp isn't allocated.
      
      	* utils/loghole.in: Add "digest.out" and "report.mail" as global
      	logs to be saved in archives and display the "report.mail" file
      	when showing a loghole archive.
      
      	* xmlrpc/emulabserver.py.in: Scrub more of the return values to
      	get rid of "None"s.
      463ee6b1
  10. 28 Mar, 2005 2 commits
  11. 07 Mar, 2005 1 commit
    • Timothy Stack's avatar
      · 898cf9a2
      Timothy Stack authored
      Checkin some changes related to experiment automation and vnode feedback:
      
      	* configure, configure.in: Add sensors/canaryd/feedbacklogs
      	template.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
      	table that holds environment variable names and values.
      
      	* event/lib/event.c: Allocate memory of the right size for
      	event_notifications.
      
      	* event/program-agent/GNUmakefile.in: Add version.c file and
      	add install targets for the man page.
      
      	* event/program-agent/program-agent.8: Man page describing the
      	program-agent daemon.
      
      	* event/program-agent/program-agent.c: Add a bunch of convenience
      	features: let the user specify the working directory for commands;
      	save output to separate files on every invocation of an agent; let
      	the user specify a timeout for a command; make the set of
      	environment variables sane and add vars given in the NS file in
      	the opt array; a "status" file containing process information is
      	written out when children are collected.  Internal changes: child
      	processes are collected immediately, instead of waiting for the
      	next START event, so we can send back COMPLETE events; the daemon
      	now runs with a real-time priority, to increase the chances of
      	receiving events.
      
      	* event/proxy/evproxy.c: Made it bidirectional so the
      	program-agent's COMPLETE events make it back to the scheduler.
      
      	* event/sched/error-record.c: Change the default log directory.
      
      	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
      	environment similar to a program-agent to run the user's log
      	digester.
      
      	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
      	that runs create_image for the node.
      
      	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
      	Let the user specify a "DIGESTER" script that digests the log
      	files into a summary of the results.  Add event handler for
      	remapping a vnode experiment.
      
      	* event/sched/timeline-agent.c: Accept the RUN event as well as
      	the START event.
      
      	* os/GNUmakefile.in: Install the install-tarfile.1 man page.
      
      	* os/install-tarfile: Automatically chown/chgrp any files that do
      	not have valid user or group IDs, the new owner will be the user
      	that swapped in the experiment.  Include the install directory in
      	the DB file.  Add a "list" mode that just dumps what files have
      	been installed and where.  Add a "force" option so the user can
      	forcefully install the file, even though the DB says its already
      	there.
      
      	* os/install-tarfile.1: Man page describing the install-tarfile
      	tool.
      
      	* os/syncd/GNUmakefile.in: Install man pages on ops.
      
      	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
      	install "feedbacklogs" tool.
      
      	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.
      
      	* sensors/canaryd/canarydEvents.c: Log errors.
      
      	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
      	data from canaryd log files.
      
      	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.
      
      	* sensors/slothd/digest-slothd: Fix some bugs and write out an
      	"alert" file with all the nodes/links that were overloaded.
      
      	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
      	argument that lets you specify that you want to wait for the disk
      	to finish loading and/or wait for the node to come back up in the
      	new OS.
      
      	* tbsetup/power.in: Remove debugging printf.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
      	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
      	quoting problems with event-sequences.  Add -expected-exit-code
      	and -tag options to the "$program run" event.  Add -digester to
      	the "$ns report" event that lets the user specify a program to run
      	to digest the log files.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
      	factor for feedback nodes to 1%, instead of 100%.
      
      	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
      	returns the values in "virt_user_environment".  Return new program
      	agent fields: dir, timeout, and expected_exit_code.
      
      	* tmcd/common/GNUmakefile.in: Install rc.canaryd.
      
      	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
      	a real-time priority, they can't do it from inside the jail.
      
      	* tmcd/common/rc.canaryd: Rc script for canaryd.
      
      	* tmcd/common/watchdog: Don't fail outright if there is a bad line
      	in the battery.log
      
      	* tmcd/common/rc.progagent: Append "userenv" data to the
      	program-agent config file.
      
      	* utils/GNUmakefile.in: Install loghole and its man page on ops.
      
      	* utils/loghole.1: Document "clean" command and the change in
      	loghole directories.
      
      	* utils/loghole.in: Add "clean" command and parallelization.
      
      	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
      	Order the eventlist by "idx" and time, needed for sequences.  And
      	removed unnecessary nologin checks.
      898cf9a2
  12. 28 Feb, 2005 1 commit
  13. 14 Feb, 2005 1 commit
  14. 10 Jan, 2005 1 commit
    • Leigh Stoller's avatar
      A quick hack job to get the webcams onto the web interface. · d46902e1
      Leigh Stoller authored
      * Add new DB table "webcams" which hold the id of the webcam, the
        server it is attached to, and the last update time.
      
      * Add new sitevars webcam/anyone_can_view and webcam/admins_can_view.
        Should be obvious what they mean.
      
      * Add trivial script grabwebcams (invoked from cron) to grab the images
        from the servers and stash in /usr/testbed/webcams. The images are
        grabbed with scp, protected by a 5 second timeout. Fine for a couple
        of cameras.
      
      * Add web page stuff to display webcams, linked from the robot mape page.
      
      Permission to view the webcams is currently admin, or in a project that is
      allowed to use a robot. We can tighten this up later as needed.
      d46902e1
  15. 22 Dec, 2004 1 commit
  16. 15 Dec, 2004 1 commit
    • Robert Ricci's avatar
      Blinky light support for motes on stargates. · 9323263d
      Robert Ricci authored
      Display a new 'Blinky Lights' button on the showexp page. In order to
      do this, I have to get a list of which classes/types are in use in
      the experiment.
      
      This leads to moteleds.php3, which displays the blink lights using
      Tim's cool Java applet.
      9323263d
  17. 06 Dec, 2004 1 commit
  18. 28 Oct, 2004 1 commit
  19. 06 Oct, 2004 1 commit
  20. 18 Aug, 2004 3 commits
  21. 09 Aug, 2004 1 commit
    • Leigh Stoller's avatar
      Major rework of the script interface to Emulab. Up to now we have been · 5ef8f70a
      Leigh Stoller authored
      supporting both a shell script driven interface, plus the newer XMLRPC
      interface. This change removes the script driven interface from boss,
      replacing it with just the XMLRPC interface. Since we like to maintain
      backwards compatability with interfaces we have advertised to users (and
      which we know are being used), I have implemented a script wrapper that
      exports the same interface, but which converts the operations into XMLRPC
      requests to the server. This wrapper is written in python and uses our
      locally grown xmlrpc-over-ssh library. Like the current "demonstation"
      client, you can take this wrapper to your machine that has python and ssh
      installed, and use it there; you do not need to use these services from
      just users.emulab.net. Other things to note:
      
      * The wrapper is a single python script that has a "class" for each wrapped
        script. Running the wrapper without any arguments will list all of the
        operations it supports. You can invoke the wrapper with the operation as
        its argument:
      
          {987} stoller$ script_wrapper.py swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
          where:
               -w   - Wait for experiment to finish swapping
               -e   - Project and Experiment ID
               in   - Swap experiment in  (must currently be swapped out)
              out   - Swap experiment out (must currently be swapped in)
      
          Wrapper Options:
              --help      Display this help message
              --server    Set the server hostname
              --login     Set the login id (defaults to $USER)
              --debug     Turn on semi-useful debugging
      
         But more convenient is to create a set of symlinks so that you can just
         invoke the operation by its familiar scriptname. This is what I have
         done on users.emulab.net.
      
          {987} stoller$ /usr/tesbed/bin/swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
      
      
      * For those of you talking directly to the RPC server from python, I have
        added a wrapper class so that you can issue requests to any of the
        modules from a single connection. Instead using /xmlrpc/modulename, you
        can use just /xmlrpc, and use method names of the form experiment.swapexp,
        node.reboot, etc.
      
        Tim this should be useful for the netlab client which I think opens up
        multiple ssh connections?
      
      * I have replaced the paperbag shell with a stripped down xmlrpcbag shell
        that is quite a bit simpler since we no longer allow access to anything
        but the RPC server. No interactive mode, no argument processing, no
        directory changing, etc. My main reason for reworking the bag is to make
        it easier to understand, maintain, and verify that it is secure. The new
        bag also logs all connections to syslog (something we should have done in
        the orginal). I also added some setrlimit calls (core, maxcpu). I also
        thought about niceing the server down, but that would put RPC users at a
        disadvantage relative to web interface users. When we switch the web
        interface to use the XMLRPC backend, we can add this (reniceing from the
        web server would be a pain cause of its scattered implementation).
      5ef8f70a
  22. 08 Jul, 2004 1 commit
  23. 11 May, 2004 1 commit
    • Leigh Stoller's avatar
      New event agent to control wireless links. At present, this agent is · 6cf05acb
      Leigh Stoller authored
      very specific to wireless links in general, and to iwconfig on Redhat
      9.0. It allows you to control the entire lan or an individual member
      of a wireless lan via the event system. For example to change the
      accesspoint of a wireless lan, you could do this:
      
      	tevc -e foo/bar now lan0 modify accesspoint=00:09:5B:93:0B:A4
      
      The agent deciphers the event arguments and calls iwconfig with the
      appropriate as needed. Note that there are many ways to make the lan
      unusable doing this, so you want to be careful. You can get the MAC
      addresses from the experiment info page (tbreport).
      
      New script called link_config, which might be badly named since it
      implies generality) to front end tevc. Operates mostly like
      delay_config in that it will change the physical table settings, and
      optionally (-m) the virtual table entries. So,
      
      	link_config testbed two-wireless lan0 accesspoint=00:09:5B:93:0B:A4
      
      You can change individual members of a lan too:
      
      	link_config -s nodew1 testbed two-wireless lan0 txpower=50
      
      Currently no web interface; too much work. I will add an xmlrpc
      interface though since that is easy to do.
      6cf05acb
  24. 10 May, 2004 1 commit
    • Leigh Stoller's avatar
      Silly script to take the testbed commit log that is rsync'ed over each · 39ff000a
      Leigh Stoller authored
      night, and process it into a file to stick into the testbed www
      directory. This replaces the current approach of wishfully thinking
      that people will remember to add their commit log messages to another
      file and them install it. I'm one of the worst offenders these days ...
      
      So, take the commit log, skip up to the first entry in 2004/01, then
      take the rest of the file and reverse the entries. Copy the resulting
      file over to the www directory.
      
      This script is run from cron on our site only!
      
      Note to Opera users; opera does not respect <XMP> properly, and so the
      page is somewhat scrogged by HTML entities that are supposed to be
      ignored when inside an XMP. Works okay on other browsers.
      39ff000a
  25. 26 Apr, 2004 1 commit
    • Mike Hibler's avatar
      Cleanup Makefiles: · 297019fb
      Mike Hibler authored
      1. "make clean" will just remove stuff built in the process of a regular build
      2. "make distclean" will also clean out configure generated files.
      
      This is how it was always supposed to be, there was just some bitrot.
      297019fb
  26. 04 Dec, 2003 1 commit
    • Leigh Stoller's avatar
      New script. · f06d9a59
      Leigh Stoller authored
      Grant permission to use types/class of nodes. If given a specific
      "type", then grant permission to use just that type. If its a class, then
      permission to use all of the types in that class, the class itself, and any
      aux nodetypes for the type/class (node_types_auxtypes table).
      
      For example:
      
      	wap grantnodetype -p testbed pc2000
      	wap grantnodetype -p testbed pcvm
      	wap grantnodetype -p testbed pcplab
      f06d9a59
  27. 26 Nov, 2003 1 commit
  28. 14 Jul, 2003 1 commit
  29. 07 Mar, 2003 1 commit
    • Mac Newbold's avatar
      A few changes to stated: · 92fa4ae2
      Mac Newbold authored
       - fix bad indenting to a uniform 4 spaces (before was 2, 4 and 8 mixed)
       - Move ping-for-isup functionality into a separate script
       - Make sure every transition triggered by stated (directly or indirectly)
         sends an event, instead of taking shortcuts.
      
      This called for a new script, eventping, which just pings until the node
      is pingable, then sends an ISUP event. Stated runs this in the background
      where necessary, and nothing else should run it.
      
      Adding eventping meant modifying configure and the utils makefile, too.
      92fa4ae2
  30. 13 Feb, 2003 1 commit
  31. 26 Aug, 2002 2 commits
    • Leigh Stoller's avatar
      Rework all of the ssh key handling. Moved the parsing and verification · ae77bdb6
      Leigh Stoller authored
      to an external perl script, and use ssh-keygen to attempt conversion
      off SSH2/SECSH key formats. This is actually a simplification of the
      php code, which is not generally very good at this kind of thing (or
      maybe I mean perl is just better at it). The parsing and error
      handling it also much improved.
      ae77bdb6
    • Leigh Stoller's avatar
      Minor reorg of cvsupd startup. It now gets started by perl script that · 541a3586
      Leigh Stoller authored
      flips the uid/gid to nobody/nobody. It would be good to run this in a
      chroot shell, but that would be difficult given that we cannot easily
      rebuild cvsupd (modula-3). The right solution is to either run it in a
      jail or to move it to ops.
      
      Note, files in the sup tree obviously have to be world readable for
      cvsupd to send them off.
      541a3586
  32. 07 Jul, 2002 1 commit
  33. 01 Jul, 2002 1 commit
  34. 24 Jun, 2002 1 commit
  35. 13 Jun, 2002 1 commit
  36. 05 Jun, 2002 1 commit
    • Leigh Stoller's avatar
      Changes to sshtb. Remove sshremote, and convert sshtb into a perl · 231fc2b1
      Leigh Stoller authored
      script that checks the database to see if local or remote. The problem
      with this is that the ssh syntax makes it hard to determine the host
      name by inspection. Would need to parse all the ssh args (bad idea),
      ot work backwards and try to figure out the difference between the
      command (which is not a string but a sequence of args) and the host
      and the preceeding ssh args. Hell with that! Changed sshtb to require
      a specific -host argument. Read the args and look for it. Error out of
      not found, to catch improper usage.
      
      The moral of this update: "sshtb [ssh args] -host <host> [more args ...]
      231fc2b1