Commit d40ec74c authored by Mac Newbold's avatar Mac Newbold
Browse files

Updated before th sw release. Finished up arch a little bit, and now it is

a "source tree map" of sorts. Did some reformatting to overview, and added
a small para about idle detection to the expt life cycle para.
parent 9e26c63a
Testbed Architecture
Overview of parts and organization
(Started April 2, 2003)
Emulab Source Tree Map
Parts, in no particular order yet:
This file documents roughly the contents of our source tree as of
April, 2003. Some of the entries in here are per-script, others are
for a group of scripts, in which case the documentation inside the
individual scripts should be sufficient explanation. The end of the
file also has some overview-ish stuff about abstractions and things
like that.
[This file maintained by]
- unix accounts
......@@ -52,7 +59,6 @@ Capture/console (node consoles - "'zero-penalty' remote research")
- changes quickly when node changes "ownership"
- simple, secure remote access
- ACLs, authenticated ssl tunnel program + standard telnet client
- [do we still want to talk about tip?]
CD-ROM (remote node mgmt/robustness, adding nodes to the system)
- simple to add a node
......@@ -73,16 +79,13 @@ Database (centralized store for persistent shared system state)
- sw engineering issues
- db schema must match sw build
IXP (special hw resources?)
IXP (special hw resources)
- use as testbed infrastructure
- traffic shaping
- use for experimentation
- shared facil. gives more people access, increases usage
- emulab is good environment w/many tools
discvr? (topology discovery tool(s))
- verification tool?
Event system (distributed event coordination/communication)
- publish/subscribe system (written by someone else)
- used in several directions
......@@ -121,14 +124,12 @@ install (emulab cluster site configuration tools)
- one for configuring an "ops" node (public server)
ipod/apod (node control without power control hardware)
[this should fit under something]
- "ICMP Ping-Of-Death" and big brother, "Authenticated Ping-Of-Death"
- reboot pingable but hung node without external intervention
- adds robustness and greater control
- especially important where only other alternative is a human
Libaries (Software engineering?)
[this should fit somewhere else probably]
- shared constants
- common interfaces
- database routines and abstractions
......@@ -173,9 +174,9 @@ Security
- plasticwrap/paperbag - transparently run commands on secure server
- suexec during web execution adds extra layer of security and
permission checks
- lastlogs [should be somewhere else?]
- lastlogs
- track logins on servers and nodes, report into main db
- giving away root on the nodes causes issues [discuss elsewhere?]
- giving away root on the nodes causes issues
- passwords
- we enforce good ones via checkpass/cracklib
- have expirations
......@@ -191,13 +192,7 @@ Sensors
- higher latency okay for detecting beginning of inactivity
- when its active, stay out of the way...
SQL (database schema and prefilled data, and how to upgrade schemas)
[discuss elsewhere, under database]
SSL (secure wide-area communications)
[discuss elsewhere, if at all]
TBSetup [break into several groups? Organize some other way?]
- core of testbed software
- primary focus: expt config tasks
- and auxiliary functions necessary for expt config stuff
......@@ -210,8 +205,8 @@ TBSetup [break into several groups? Organize some other way?]
- allows for more automation
- submit expt even when no resources are avail., runs later
- checkports - ?
- console reset/setup [move somewhere?]
- control console access [see also capture section]
- console reset/setup
- control console access (see also capture section)
- db2ns - dump our db data rep back into an ns file
- eventsys start/control
- start up event schedulers for each expt - see event section
......@@ -335,7 +330,7 @@ TMCD - Testbed Master Control Daemon
- currently supported on FreeBSD and Linux, and ported to OpenBSD
- tool for nodes<->emulab communication
- part of a set of node initialization scripts
- Node self-configuration process [move elsewhere? up a level?]
- Node self-configuration process
- report "I'm alive"
- update config scripts (currently via sup)
- run the config, which sets up:
......@@ -373,7 +368,6 @@ Web Interface
- disk images, custom OS's, etc.
- for admins etc, also provides web db access and cvs web access
Stated ("state-dee") - node state management daemon
- listens for node state events
- performs triggered actions
......@@ -390,13 +384,11 @@ Netbed Wide-area nodes
- same methods/abstractions/tools used in LAN or WAN environment
- easy to switch from a wide-area run to an emulated run (or simulated)
- Boot process a little different
- [need some details here... Leigh? can you tell me how it works?]
- Many parallels to local area case
- SFS instead of NFS for shared homedirs
- Can set up links as tunnels with 192.168.* addresses
- Accounts same (except for rootness)
- Traffic generation
- [more?]
Simulated Nodes
- many nodes simulated inside NSE on a single phys. node
......@@ -416,21 +408,43 @@ Multiplexed Nodes
Cross-cutting Abstractions
- Four different environments
- Emulab (dedicated phys.) nodes, wide-area nodes, simulated nodes,
and multiplexed ("virtual") nodes
- Emulab/emulation (dedicated phys.) nodes, wide-area nodes,
simulated nodes, and multiplexed ("virtual") nodes
- can mix and match in same expt
- in many cases, same expt can run in any (or several) of the
environments with few or no changes
- Nodes
- E: (emulab) dedicated physical nodes
- completely controllable network characteristics
- Emulated/emulab: dedicated physical nodes in a cluster
- get root, can reboot, serial console, total control of node
- including OS, disk imaging, etc.
- W: (widearea) shared nodes, geographically distributed
- get an account (non-root)
HW config (switch/router configs specific to emulab)?
rc.d (daemons on boss/ops/tipservers for running emulab)?
sysadmin (apachelogroll)?
- Widearea: shared nodes, geographically distributed
- get an account (non-root, typically)
- sometimes get a jail / "virtual server"
- less control (of OS, rebooting, etc.)
- Simulated: nodes inside of an NS simulator
- nodes are simulated, don't run an OS, etc.
- functionality programmed via NS models
- Multiplexed: jails / virtual servers on cluster nodes
- Almost as real as emulation nodes
- allows bigger scale, risks potential for side-effects
- same level of control as emulation nodes
- Links
- Emulated/emulab:
- completely controllable network characteristics
- including LAN speeds or shaped links
- isolated control network
- very realistic, predictable, repeatable
- Widearea:
- network is the real/raw internet
- tunnels are optionally configured
- no separate control network
- completely realistic, but unpredictable
- Simulated:
- links inside NSE (NS Emulator)
- NSE does shaping
- real and sim worlds can talk to each other
- Multiplexed:
- Same capabilities as normal emulated/emulab links
- some tricks involved to get everything to work right
This diff is collapsed.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment