Commit d40ec74c authored by Mac Newbold's avatar Mac Newbold

Updated before th sw release. Finished up arch a little bit, and now it is

a "source tree map" of sorts. Did some reformatting to overview, and added
a small para about idle detection to the expt life cycle para.
parent 9e26c63a
Testbed Architecture
Overview of parts and organization
(Started April 2, 2003)
======================
Emulab Source Tree Map
======================
Parts, in no particular order yet:
This file documents roughly the contents of our source tree as of
April, 2003. Some of the entries in here are per-script, others are
for a group of scripts, in which case the documentation inside the
individual scripts should be sufficient explanation. The end of the
file also has some overview-ish stuff about abstractions and things
like that.
[This file maintained by testbed-ops@emulab.net]
Accounts
- unix accounts
......@@ -52,7 +59,6 @@ Capture/console (node consoles - "'zero-penalty' remote research")
- changes quickly when node changes "ownership"
- simple, secure remote access
- ACLs, authenticated ssl tunnel program + standard telnet client
- [do we still want to talk about tip?]
CD-ROM (remote node mgmt/robustness, adding nodes to the system)
- simple to add a node
......@@ -73,16 +79,13 @@ Database (centralized store for persistent shared system state)
- sw engineering issues
- db schema must match sw build
IXP (special hw resources?)
IXP (special hw resources)
- use as testbed infrastructure
- traffic shaping
- use for experimentation
- shared facil. gives more people access, increases usage
- emulab is good environment w/many tools
discvr? (topology discovery tool(s))
- verification tool?
Event system (distributed event coordination/communication)
- publish/subscribe system (written by someone else)
- used in several directions
......@@ -121,14 +124,12 @@ install (emulab cluster site configuration tools)
- one for configuring an "ops" node (public server)
ipod/apod (node control without power control hardware)
[this should fit under something]
- "ICMP Ping-Of-Death" and big brother, "Authenticated Ping-Of-Death"
- reboot pingable but hung node without external intervention
- adds robustness and greater control
- especially important where only other alternative is a human
Libaries (Software engineering?)
[this should fit somewhere else probably]
- shared constants
- common interfaces
- database routines and abstractions
......@@ -173,9 +174,9 @@ Security
- plasticwrap/paperbag - transparently run commands on secure server
- suexec during web execution adds extra layer of security and
permission checks
- lastlogs [should be somewhere else?]
- lastlogs
- track logins on servers and nodes, report into main db
- giving away root on the nodes causes issues [discuss elsewhere?]
- giving away root on the nodes causes issues
- passwords
- we enforce good ones via checkpass/cracklib
- have expirations
......@@ -191,13 +192,7 @@ Sensors
- higher latency okay for detecting beginning of inactivity
- when its active, stay out of the way...
SQL (database schema and prefilled data, and how to upgrade schemas)
[discuss elsewhere, under database]
SSL (secure wide-area communications)
[discuss elsewhere, if at all]
TBSetup [break into several groups? Organize some other way?]
TBSetup
- core of testbed software
- primary focus: expt config tasks
- and auxiliary functions necessary for expt config stuff
......@@ -210,8 +205,8 @@ TBSetup [break into several groups? Organize some other way?]
- allows for more automation
- submit expt even when no resources are avail., runs later
- checkports - ?
- console reset/setup [move somewhere?]
- control console access [see also capture section]
- console reset/setup
- control console access (see also capture section)
- db2ns - dump our db data rep back into an ns file
- eventsys start/control
- start up event schedulers for each expt - see event section
......@@ -335,7 +330,7 @@ TMCD - Testbed Master Control Daemon
- currently supported on FreeBSD and Linux, and ported to OpenBSD
- tool for nodes<->emulab communication
- part of a set of node initialization scripts
- Node self-configuration process [move elsewhere? up a level?]
- Node self-configuration process
- report "I'm alive"
- update config scripts (currently via sup)
- run the config, which sets up:
......@@ -373,7 +368,6 @@ Web Interface
- disk images, custom OS's, etc.
- for admins etc, also provides web db access and cvs web access
Stated ("state-dee") - node state management daemon
- listens for node state events
- performs triggered actions
......@@ -390,13 +384,11 @@ Netbed Wide-area nodes
- same methods/abstractions/tools used in LAN or WAN environment
- easy to switch from a wide-area run to an emulated run (or simulated)
- Boot process a little different
- [need some details here... Leigh? can you tell me how it works?]
- Many parallels to local area case
- SFS instead of NFS for shared homedirs
- Can set up links as tunnels with 192.168.* addresses
- Accounts same (except for rootness)
- Traffic generation
- [more?]
Simulated Nodes
- many nodes simulated inside NSE on a single phys. node
......@@ -416,21 +408,43 @@ Multiplexed Nodes
Cross-cutting Abstractions
- Four different environments
- Emulab (dedicated phys.) nodes, wide-area nodes, simulated nodes,
and multiplexed ("virtual") nodes
- Emulab/emulation (dedicated phys.) nodes, wide-area nodes,
simulated nodes, and multiplexed ("virtual") nodes
- can mix and match in same expt
- in many cases, same expt can run in any (or several) of the
environments with few or no changes
- Nodes
- E: (emulab) dedicated physical nodes
- completely controllable network characteristics
- Emulated/emulab: dedicated physical nodes in a cluster
- get root, can reboot, serial console, total control of node
- including OS, disk imaging, etc.
- W: (widearea) shared nodes, geographically distributed
- get an account (non-root)
HW config (switch/router configs specific to emulab)?
rc.d (daemons on boss/ops/tipservers for running emulab)?
sysadmin (apachelogroll)?
- Widearea: shared nodes, geographically distributed
- get an account (non-root, typically)
- sometimes get a jail / "virtual server"
- less control (of OS, rebooting, etc.)
- Simulated: nodes inside of an NS simulator
- nodes are simulated, don't run an OS, etc.
- functionality programmed via NS models
- Multiplexed: jails / virtual servers on cluster nodes
- Almost as real as emulation nodes
- allows bigger scale, risks potential for side-effects
- same level of control as emulation nodes
- Links
- Emulated/emulab:
- completely controllable network characteristics
- including LAN speeds or shaped links
- isolated control network
- very realistic, predictable, repeatable
- Widearea:
- network is the real/raw internet
- tunnels are optionally configured
- no separate control network
- completely realistic, but unpredictable
- Simulated:
- links inside NSE (NS Emulator)
- NSE does shaping
- real and sim worlds can talk to each other
- Multiplexed:
- Same capabilities as normal emulated/emulab links
- some tricks involved to get everything to work right
---EOF---
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment