1. 16 Jun, 2006 1 commit
  2. 14 Jun, 2006 1 commit
    • Leigh B. Stoller's avatar
      The template "datastore" ... · fe9aa6a4
      Leigh B. Stoller authored
      Each template has a datastore, which is really just a subdirectory that can
      be populated with files, and committed to the subversion archive.  Note,
      the datastore os specific to the template itself. The Template Archive link
      on the Show Template page takes you to the subdirectory, which by
      convention I am calling "datastore".
      
      The directory actually lives in /proj/pid/exp/eid/TGUID-VERS ... but that
      path is printed out for you on the archive page.
      
      Anyway, put stuff in the datastore directory, and then commit the template
      archive so there is a tag associated with it.
      
      When an instance is created, a checkout of the datastore is placed in the
      experiment directory (/proj/pid/eid/exp/template_datastore). The current
      tag (from above) is stored with the instance so that we can later recreate
      the enviroment for the instance, say for rerun.
      
      Tarfiles and rpms in the datastore can be referenced as xxx://foo.rpm (in
      your NS file).  tarfiles_setup transforms those when the instance is
      swapped in, sorta like it does other URLs, only it does not actually fetch
      them, just need to rewrite the paths so they reference datastore.
      
      The program agent gets another environment variable so you can refer to the
      datastore without hardwiring paths ($DATASTORE). Eventually I want to move
      the checkout someplace else, but it was easy to drop it into the experiment
      directory for now.
      fe9aa6a4
  3. 12 Jun, 2006 1 commit
  4. 08 Jun, 2006 1 commit
  5. 11 May, 2006 1 commit
    • Mike Hibler's avatar
      Initial support for plab-in-elab: · 088c8e74
      Mike Hibler authored
      * setup some fields that parallel the
        elab-in-elab ones, for modifying the dhcpd.conf file.
      
      * tb-set-node-plab-role to set a node as either 'plc' or 'node'
        used to set the DB role field above
      088c8e74
  6. 31 Mar, 2006 1 commit
    • Leigh B. Stoller's avatar
      More experiment run stuff. Also the NS file parsing support that was · 74bf8d1d
      Leigh B. Stoller authored
      needed (reparsing of NS file with current bindings).
      
      Mostly I decided that I made a huge mistake in using the exptidx as the
      main cross index. I started to change that, which took a bunch of time.
      Still more tables to fix up, but I think all of this code is gonna need
      a major reorg once I am no longer under the gun to get it done.
      74bf8d1d
  7. 27 Dec, 2005 1 commit
    • Mike Hibler's avatar
      More tightly connect the notion of a firewall and the security level. · f1206314
      Mike Hibler authored
      If you specify an explicit firewall, you are implicitly assigned security
      level 2 and you cannot explicitly specify the security level.  Likewise,
      if you specify a security level, you cannot also specify a firewall.
      
      The reason for this is that security level 1 (aka "Blue") now has a slightly
      different meaning.  It is intended for protecting the inside from the outside
      rather than visa-versa.  The only practical implication of this is that for
      level 1, we don't do all the fancy power-off-boot-into-MFS-zapbootblock stuff
      that we do for higher levels.
      
      Anyway, I wanted to make sure that if you specify your own firewall, you
      DO have to go through the full cleansing swapout since we can't trust a
      firewall that the Average Joe sets up.
      f1206314
  8. 30 Sep, 2005 1 commit
  9. 07 Sep, 2005 1 commit
    • Mike Hibler's avatar
      Inner-elab role changes: · 88926d1c
      Mike Hibler authored
      	'boss' -> 'boss+router'
      	'ops'  -> 'ops+fs'
      	'fs'      (new role)
      	'router'  (new role)
      
      These are in preparation for allowing configurations with split ops and fs
      nodes (and sometime down the road, split boss and router nodes).
      
      This checkin is just the DB state changes along with the scripts that look
      at that state.  The Big One, which actually sets up separate nodes
      automatically, is undergoing more testing but will be Coming Soon.
      88926d1c
  10. 04 Aug, 2005 1 commit
  11. 25 May, 2005 1 commit
  12. 16 May, 2005 1 commit
    • Leigh B. Stoller's avatar
      Add support for specifying the CVS tag to use when getting the source code · f1863cfd
      Leigh B. Stoller authored
      for the inner elab.
      
      	tb-set-elabinelab-cvstag dist-foo
      
      Will result in this branch getting checked out from the mirrored repository
      (updated nightly) on boss and sent back to the node, instead of the usual
      source tarball that we keep in /usr/testbed/src (still the default
      behaviour if no tag is specified. You can also do this if you like:
      
      	tb-set-elabinelab-cvstag HEAD
      
      which of course is a special tag to CVS.
      f1863cfd
  13. 27 Apr, 2005 1 commit
  14. 06 Apr, 2005 1 commit
  15. 30 Mar, 2005 1 commit
  16. 07 Mar, 2005 1 commit
    • Timothy Stack's avatar
      · 898cf9a2
      Timothy Stack authored
      Checkin some changes related to experiment automation and vnode feedback:
      
      	* configure, configure.in: Add sensors/canaryd/feedbacklogs
      	template.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
      	table that holds environment variable names and values.
      
      	* event/lib/event.c: Allocate memory of the right size for
      	event_notifications.
      
      	* event/program-agent/GNUmakefile.in: Add version.c file and
      	add install targets for the man page.
      
      	* event/program-agent/program-agent.8: Man page describing the
      	program-agent daemon.
      
      	* event/program-agent/program-agent.c: Add a bunch of convenience
      	features: let the user specify the working directory for commands;
      	save output to separate files on every invocation of an agent; let
      	the user specify a timeout for a command; make the set of
      	environment variables sane and add vars given in the NS file in
      	the opt array; a "status" file containing process information is
      	written out when children are collected.  Internal changes: child
      	processes are collected immediately, instead of waiting for the
      	next START event, so we can send back COMPLETE events; the daemon
      	now runs with a real-time priority, to increase the chances of
      	receiving events.
      
      	* event/proxy/evproxy.c: Made it bidirectional so the
      	program-agent's COMPLETE events make it back to the scheduler.
      
      	* event/sched/error-record.c: Change the default log directory.
      
      	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
      	environment similar to a program-agent to run the user's log
      	digester.
      
      	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
      	that runs create_image for the node.
      
      	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
      	Let the user specify a "DIGESTER" script that digests the log
      	files into a summary of the results.  Add event handler for
      	remapping a vnode experiment.
      
      	* event/sched/timeline-agent.c: Accept the RUN event as well as
      	the START event.
      
      	* os/GNUmakefile.in: Install the install-tarfile.1 man page.
      
      	* os/install-tarfile: Automatically chown/chgrp any files that do
      	not have valid user or group IDs, the new owner will be the user
      	that swapped in the experiment.  Include the install directory in
      	the DB file.  Add a "list" mode that just dumps what files have
      	been installed and where.  Add a "force" option so the user can
      	forcefully install the file, even though the DB says its already
      	there.
      
      	* os/install-tarfile.1: Man page describing the install-tarfile
      	tool.
      
      	* os/syncd/GNUmakefile.in: Install man pages on ops.
      
      	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
      	install "feedbacklogs" tool.
      
      	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.
      
      	* sensors/canaryd/canarydEvents.c: Log errors.
      
      	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
      	data from canaryd log files.
      
      	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.
      
      	* sensors/slothd/digest-slothd: Fix some bugs and write out an
      	"alert" file with all the nodes/links that were overloaded.
      
      	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
      	argument that lets you specify that you want to wait for the disk
      	to finish loading and/or wait for the node to come back up in the
      	new OS.
      
      	* tbsetup/power.in: Remove debugging printf.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
      	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
      	quoting problems with event-sequences.  Add -expected-exit-code
      	and -tag options to the "$program run" event.  Add -digester to
      	the "$ns report" event that lets the user specify a program to run
      	to digest the log files.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
      	factor for feedback nodes to 1%, instead of 100%.
      
      	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
      	returns the values in "virt_user_environment".  Return new program
      	agent fields: dir, timeout, and expected_exit_code.
      
      	* tmcd/common/GNUmakefile.in: Install rc.canaryd.
      
      	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
      	a real-time priority, they can't do it from inside the jail.
      
      	* tmcd/common/rc.canaryd: Rc script for canaryd.
      
      	* tmcd/common/watchdog: Don't fail outright if there is a bad line
      	in the battery.log
      
      	* tmcd/common/rc.progagent: Append "userenv" data to the
      	program-agent config file.
      
      	* utils/GNUmakefile.in: Install loghole and its man page on ops.
      
      	* utils/loghole.1: Document "clean" command and the change in
      	loghole directories.
      
      	* utils/loghole.in: Add "clean" command and parallelization.
      
      	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
      	Order the eventlist by "idx" and time, needed for sequences.  And
      	removed unnecessary nologin checks.
      898cf9a2
  17. 17 Feb, 2005 1 commit
  18. 02 Feb, 2005 1 commit
    • Leigh B. Stoller's avatar
      Add tb-set-delay-capacity NS directive: · 21044084
      Leigh B. Stoller authored
      	tb-set-delay-capacity 1
      
      Will override the default delay capacity as specfied in the node_types
      table for each node type, and set it for all types when generating the
      ptop file.
      
      This is a big stick, to be used with caution, as it will effectively
      double the number of nodes used for delay nodes (withing an experiment).
      21044084
  19. 28 Jan, 2005 1 commit
    • Leigh B. Stoller's avatar
      Changes to elabinelab setup. The source code for inner boss/ops no · 03cf92cb
      Leigh B. Stoller authored
      longer comes from the project directory (really, my source directory
      in the various projects). Instead, the source code comes from one of
      two places:
      
      * Using fetch on the inner ops and boss, make a special request to the
        outer webserver to return a tarball, currently stored in
        /usr/testbed/src, and created by toplevel makefile target "elabinelab".
        So, in your object tree (preferably one that is pure, without any of your
        own hacks) run "make elabinelab" and it will create the tarfile for you.
        The tarball is returned only to elabinelab experiments providing the
        experiment secret key (a variant of the existing spewrpmtar file stuff).
      
      * Using the standard tarfile install command, albiet indrectly cause of how
        we hide all of the elabinelab NS goo. So, in your NS file you can do:
      
      	tb-elab-in-elab 1
      	set ::TBCOMPAT::elabinelab_source_tarfile \
      			"/proj/testbed/tarfiles/emulab-src.tar.gz"
      
        Which will arrange for this tarfile to be installed in /usr/src, and then
        copied to the approriate place later when inner boss and inner ops are
        setup. The tarfile should be created in the top of your testbed source
        tree. This is given preference over method 1 above.
      
      XXX Currently the tarfile is unpacked in /usr/src cause install-tarfile
      does not create the target directory. As soon as we have new images, I'll
      move this unpacking to a more suitable place.
      03cf92cb
  20. 17 Jan, 2005 1 commit
    • Timothy Stack's avatar
      · bf489797
      Timothy Stack authored
      More robot integration and some event system updates.
      
      	* configure, configure.in: Detect rsync for loghole and add
      	utils/loghole to the list of template files.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add virt_node_startloc to the
      	list of virtual tables.
      
      	* event/lib/event.h, event/lib/event.c, event/lib/tbevent.py.tail:
      	Add event_stop_main function to break out of the event_main()
      	loop.  Add timeline to the address tuple.
      
      	* event/sched/GNUmakefile.in, event/sched/error-record.h,
      	event/sched/error-record.c, event/sched/event-sched.8,
      	event/sched/event-sched.h, event/sched/event-sched.c,
      	event/sched/group-agent.h, event/sched/group-agent.c,
      	event/sched/listNode.h, event/sched/listNode.c,
      	event/sched/local-agent.h, event/sched/local-agent.c,
      	event/sched/node-agent.h, event/sched/node-agent.cc,
      	event/sched/queue.c, event/sched/rpc.h, event/sched/rpc.cc,
      	event/sched/simulator-agent.h, event/sched/simulator-agent.c,
      	event/sched/timeline-agent.h, event/sched/timeline-agent.c:
      	Updated event scheduler, not completely finished, but well enough
      	along for the robots.
      
      	* lib/libtb/GNUmakefile.in, lib/libtb/popenf.h,
      	lib/libtb/popenf.c, lib/libtb/systemf.h, lib/libtb/systemf.c: Add
      	some handy versions of system/popen that take format arguments.
      
      	* lib/libtb/tbdefs.h, lib/libtb/tbdefs.c: Add some more event and
      	object types.
      
      	* tbsetup/assign_wrapper.in: Add the virt_node_startloc building
      	to desires string for a node.
      
      	* tbsetup/ptopgen.in: Add a node's location to the feature list.
      
      	* tbsetup/tbreport.in: Display the timeline/sequence an event is a
      	part of.
      
      	* tbsetup/ns2ir/GNUmakefile.in: Add timeline, sequence, and
      	topography files.
      
      	* tbsetup/ns2ir/node.tcl: Add initial position for nodes and allow
      	them to be attached to "topographys".
      
      	* tbsetup/ns2ir/parse-ns.in: Make a hwtype_class array with a
      	node_type's class.  Make an 'areas' array that holds the
      	buildings where nodes are located.  Make an 'obstacles' table
      	with any obstacles in the building.
      
      	* tbsetup/ns2ir/parse.tcl.in: Move named-args function from
      	tb_compat.tcl to here.  Add reltime-to-secs function that converts
      	time given in a format like "10h2m1s" to a seconds value, used in
      	"$ns at" so its easier to write time values.  Add "K", "Kb", and
      	"Kbps" as possible units for bandwidth (only the lowercase
      	versions were available before).
      
      	* tbsetup/ns2ir/program.tcl: Add "dir" and "timeout" attributes,
      	although they don't go anywhere at the moment.
      
      	* tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/timeline.tcl,
      	tbsetup/ns2ir/topography.tcl: Initial versions.
      
      	* tbsetup/ns2ir/sim.tcl.in: Add support for timelines and
      	sequences.  Add 'node-config' method to change the default
      	configuration for nodes produced by the Simulator object.  Send an
      	initial MODIFY event to any trafgen objects so their configuration
      	gets through, even when there are no start/stop events.  Move
      	event parsing to the 'make_event' method.
      
      	* utils/loghole.1, utils/loghole.in: Loghole utility, used for
      	retrieving logs from experimental nodes and creating archives of
      	the logs.
      
      	* xmlrpc/emulabclient.py.in: Escape any strange characters in the
      	output field.
      
      	* xmlrpc/emulabserver.py.in: Add virt_node_startloc to the list of
      	virtual_tables.  Add emulab.vision_config and
      	emulab.obstacle_config methods for getting information pertaining
      	to the robots.  Change the OSID listing to include more fields.
      	Add a "physical" aspect to experiment.info to get information
      	about the physical nodes.  Add parent field to the events in the
      	array returned by eventlist.  Add sshdescription to get extra
      	information needed to log into a vnode.  Add node.statewait so you
      	can wait for nodes to come up.
      bf489797
  21. 10 Jan, 2005 1 commit
  22. 09 Dec, 2004 1 commit
    • Leigh B. Stoller's avatar
      Okay, here is the current development approach for dealing with · 5a4e9df8
      Leigh B. Stoller authored
      ElabinElab experiments that wrap another experiment, either firewalled
      or not. This instead of my security level stuff, that I decided was
      too much of a pain the user, at least for now. New NS syntax:
      
      	tb-set-inner-elab-eid two-simple
      
      In the ElabinElab file, sets the name of an existing experiment in the
      same project. Experiment is parsed, and after the parse we notice in
      tbprerun that we have an inner eid, so we reparse the NS file, only
      this time we pass in the maximum number of nodes needed by the inner
      eid (tbprerun now computes min/max nodes at prerun time, instead of
      later as first part of swapin). This number is used to allocatethe
      appropriate number of inner experimental nodes. Why do it this way?
      Cause the NS parser is the only tool we have for generating the virt
      topology, and I do not want go down the path of inventing a new
      frontend.
      
      Anyway, after the reparse, we now have the proper number of nodes in
      the wrapper experiment. Now its simply a matter of copying over the
      type and fixnode info from the inner experiment to the outer
      experiment.  Why? So that when the outer experiment is swapped in, it
      gets the nodes (of the right type/fixnode) that the inner experiment
      is going to want later, when it is swapped in by the inner emulab!
      
      Another approach would be to make elabinelab and elabinelab_eid
      options to batchexp (and thus the web form and XMLRPC interface) so
      that we can avoid the double parse. I suspect people do not want more
      crap on the web form, so I did not do it this way.
      5a4e9df8
  23. 08 Dec, 2004 1 commit
  24. 06 Dec, 2004 1 commit
  25. 08 Oct, 2004 1 commit
    • Timothy Stack's avatar
      · ed8262c1
      Timothy Stack authored
      Made a pass over tar file related things:
      
      	* os/install-tarfile: Added support for bzip2 compressed files and
      	fixed the '.tgz' regex (it detected '.tar.tgz' before).  Also
      	binmode'd some file handles to squelch a warning on Redhat 9.
      
      	* tbsetup/fetchtar.proxy.in: Added 'https' as a valid protocol for
      	URLs and 'tar.bz2' as a valid extension.  Changed the wget call to
      	do 'timestamping', which means it should only download files that
      	have changed since the last download.
      
      	* tbsetup/tarfiles_setup.in: More 'https' and '.tar.bz2' stuff.
      	When downloading tarballs, copy the file name extension from the
      	URL onto the local file name so 'install-tarfile' can figure out
      	how to decompress it properly.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Added a bunch of checks to the
      	tb-set-node-tarfiles function.
      
      	* www/faq.html, www/tutorial/nscommands.html,
      	www/tutorial/tutorial.html: Flesh out and clean up tarfile related
      	documentation.  Also added anchors for the individual functions so
      	you can link to them from other pages.
      ed8262c1
  26. 17 Sep, 2004 1 commit
  27. 10 Aug, 2004 1 commit
  28. 27 Jul, 2004 1 commit
  29. 19 Jul, 2004 1 commit
  30. 12 Jul, 2004 1 commit
  31. 07 Jul, 2004 1 commit
  32. 10 May, 2004 1 commit
    • Robert Ricci's avatar
      Allow the user to give an estimated bandwidth for links and/or LANs. · 923063cc
      Robert Ricci authored
      This is in addition to the normal bandwidth, and the difference is
      that the estimated bandwidth is not enforced. It's just fed to assign
      for mapping decisions. The idea is to allow for more efficient
      packings by doing non-conservative bandwidth allocation.
      
      Not documented yet, because it might change a little after I made it
      work properly with asymmetric links.
      923063cc
  33. 30 Apr, 2004 1 commit
    • Leigh B. Stoller's avatar
      Small hack requested by Jay to aid in creating wireless nodes; added · 45e5e3c7
      Leigh B. Stoller authored
      new directive for your NS file: tb-use-physnaming(1) which means that
      if you have a line like this:
      
      	set pc6 [$ns node]
      
      and pc6 is an actual name of a physical node, then do an implicit
      "fixnode" to pc6. In other words, pc6 is the name of the node in your
      NS file (virtual topo) *and* will be fixed to the real pc6. Needless
      to say, if pc6 is allocated when you go to swapin, you lose.
      45e5e3c7
  34. 12 Apr, 2004 1 commit
    • Leigh B. Stoller's avatar
      The core of the wireless support code. · 86155d03
      Leigh B. Stoller authored
      * ptopgen:
      
         1. Change to using interface_capabilities table to get the default
            speed for an interface type.
      
         2. The wrinkle is that an interface can have multiple speeds,
            depending on the protocol that is requested. In the wireless case the
            table looks like:
      
      	| type             | capkey            | capval               |
      	+------------------+-------------------+----------------------+
      	| ath              | protocols         | 80211a,80211b,80211g |
      	| ath              | 80211a_defspeed   | 54000                |
      	| ath              | 80211b_defspeed   | 11000                |
      	| ath              | 80211g_defspeed   | 54000                |
      
         3. We use the above info to tell us the default speed for links, based
            on the particular protocol.
      
         4. And lastly, create "air" switches for each of 80211a, 80211b, 80211g,
            and specify links from the switches to the nodes that have interfaces
            that support those protocols. At the moment, there is a hardwired
            list of 80211a, 80211b, 80211g in ptopgen, so create 3 new air
            switches and links from each of the nodes to each of the switches.
            Rob says that assign can handle this just fine (using the same
            interface on a node to connect to three different air switches, at
            different speeds).
      
      * assign_wrapper:
      
         1. First off, I had to increase the bogousity factor of how we determine
            that a delay node is required (requires_delay() routine) on a link or
            lan. Using the interface_capabilities table, I determine the default
            speed for each protocol on each interface type.
      
         2. When calling requires_delay() I now the pass the lan so that we can
            get the "protocol" of the link/lan and use that to find the default
            speed. The main reason for all this crud is that if a user specifies
            a bandwidth of 54Mb on a wired ethernet, we want to make sure we
            insert a delay node, but not if the user is using 80211g. As it was,
            delay nodes would be skipped cause there are now nodes with
            interfaces that support those speeds. This change also allows to know
            that a delay node should be inserted on a wireless link, if we ever
            wanted to do something as silly as that!
      
         3. Okay, so now onto the actual wireless support!  Whenever the protocol
            for a lan is not "ethernet" spit out links to fake lan nodes, sorta
            like we used to do with ethernet lans before Rob made all those great
            changes to assign. So, a lan with protocol 80211a will get fake links
            to the 80211a air switch that ptopgen spits out (see above).  We
            currently bypass all delay node processing on wireless lans, and in
            fact we avoid most post-processing (after assign) of wireless lans
            (no vlans, veths, etc). At some point we can bring certain things
            back in (like emulated links) but that will be some time! See
            comments above about ptopgen.
      
         4. Added a section to copy over the virt_lan_settings and
            virt_lan_member_settings to the interface_settings table, for each
            node's interfaces in the mapped experiment. I also make up the ssid
            for the lan, and add the protocol to the settings so that the client
            side knows what it is supposed to do. I also set the acesspoint since
            that requires MAC addresses, and this is easiest place to get and set
            the MAC (all of the nodes in a lan need to know the MAC of the node
            acting as the access point). In the virtual topology, the accesspoint
            is specified as a node name.
      
      * tbsetup/ns2ir:
      
         1. Add tb-set-lan-protocol(lan, protocol) to set the protocol to one of
            ethernet, 80211a, 80211b, 80211g. Defaults to ethernet.
      
         2. Add tb-set-lan-accesspoint(lan, accesspoint) to set the name of the
            access point for a lan. Obviously this makes sense for only wireless
            links; ignored for wired links.
      
         3. Add tb-set-lan-setting(lan, setting, value) to specify a setting that
            applies to the entire lan. At present, these are ignored for wired
            lans. Both setting and value are strings.
      
         4. Add tb-set-node-lan-setting(lan, node, setting, value) to specify a
            setting that applies to a particular node in a lan. At present, these
            are ignored for wired lans. Both setting and value are strings.
      86155d03
  35. 07 Apr, 2004 1 commit
  36. 10 Feb, 2004 1 commit
  37. 03 Feb, 2004 1 commit
  38. 29 Jan, 2004 1 commit
  39. 28 Jan, 2004 1 commit
  40. 20 Jan, 2004 1 commit