Commit 6bd26211 authored by Kevin Atkinson's avatar Kevin Atkinson

Removed web pages that are now in the WIKI. See MOVED-TO-WIKI.

parent 6bdf04b5
Copyright (c) 2000-2005 University of Utah and the Flux Group.
All rights reserved.
<h1>Linktest Tutorial</h1>
<em>Linktest is an online validation test of the network characteristics
in an Emulab experiment. It's a check that Emulab set up the network
as you requested. If you care about the fidelity of your network, you
should set linktest to run on all or some swapins, and you should read
this entire tutorial.</em>
<li> <a href="#QuickStart">Quick Start</a>
<li> <a href="#Limits">Limitations</a>
<li> <a href="#Understand">Understanding Linktest</a>
<li> <a href="#Level0">Level 0 - Do not run Linktest</a>
<li> <a href="#Level1">Level 1 - Connectivity and Latency</a>
<li> <a href="#Level2">Level 2 - Static Routing</a>
<li> <a href="#Level3">Level 3 - Loss</a>
<li> <a href="#Level4">Level 4 - Bandwidth</a>
<li> <a href="#Advanced">Advanced Topics</a>
<li> <a href="#Web">Running Linktest from the Web Interface</a>
<li> <a href="#Ops">Running Linktest on Ops</a>
<li> <a href="#Log">Linktest Log Directory</a>
<a name="QuickStart"></a>
<h2>Quick Start</h2>
Temporarily, while we are validating and tuning linktest,
on every swapin or modify:<br>
(1) We ignore users' requests to set the linktest level below 3;
i.e., we always run linktest, at level 3 or greater.<br>
(2) Although the activity log still reports all linktest results, including
failures, we email failures only to testbed-ops, not the user.<br>
(If you <a href = "#Advanced">explictly invoke linktest</a>
from the "Run Linktest" page or on ops, you get whatever level you request.)
To run linktest, select a Linktest test level from the dropdown on the
<a href="beginexp_html.php3">Begin an Experiment</a> page.
<li> <a href="#Level1"><b>Level 1</b> - Connectivity and Latency</a>:
For fastest response, select level 1. Through ping, this will check
link-level connectivity and link latencies.
<li> <a href="#Level2"><b>Level 2</b> - Plus Static Routing</a>:
In addition to the above, tests static routing (if applicable).
<li> <a href="#Level3"><b>Level 3</b> - Plus Link Loss</a>: Check link loss
characteristics. Test levels are cumulative, so this option also
includes connectivity, latency, and static routing (if applicable).
<li> <a href="#Level4"><b>Level 4</b> - Plus Bandwidth</a>: If bandwidth is
important in your experiment, select level 4. Be warned that the
bandwidth test takes up to 20 seconds per link. <b>Also note that
not all bandwidths can be accurately measured</b>. If that
happens, a warning will be placed in the log file for each link
that is out of range for bandwidth testing.
If you select a test level other than zero, Linktest will run
after the experiment completes its swapin. If a problem is found,
testbed-ops is automatically notified and a message will appear in the
activation log. You will also receive an email message to ensure that
you are aware of the problem. Otherwise, no notification will
appear. A failure in linktest will <em><b>not</b></em> cause the
swapin to fail!</em> If traffic shaping parameters are of critical
importance to your experiments, make sure you take a closer look if
linktest reports failures!
<a name="Limits"></a>
<h2>Limitations (Important, Please Read)</h2>
<li> Not all bandwidths can be accurately measured, and linktest <a href="#Level4">will
skip links</a> that it knows will give false results (e.g., slow or lossy links).
Please check the output, and be sure to test those links yourself if your
results depend on total accuracy.
<li> As with any automated testing procedure, we have to balance the desire
for accuracy with the possibility of false positives. To reduce
the number of false positives, we allow for a small amount of
fudge on any link. <b>If your results are dependent on total
accuracy, then you should test your links yourself!</b>
<li> When using "endnodeshaping" (so-called linkdelays), latency is less accurate
because of the 1ms clock resolution at which the kernel runs. At
worst, latency can be off by up to 3ms, say for a roundtrip ping
<li> Linktest can take a <b>long time</b> on large experiments. Even
on very small experiments (5-10 nodes), doing the full bandwidth test
can add 2-3 minutes. You should probably <em>not</em> do bandwidth
tests at swapin on any experiment over 20 nodes unless you are
prepared to wait a <em>long</em> time for the experiment to swap
in. If you decide you have waited long enough, you can use the
<b>Stop Linktest</b> menu option on the Show Experiment page. This
will cancel linktest and allow the swapin to complete normally.
<a name="Understand"></a>
<h2>Understanding Linktest</h2>
Linktest is an end-to-end validation test for Emulab experiments. It
verifies that experiment nodes are up, that they are reachable by
static routes (when applicable), and that traffic shaping on delay
nodes matches the experiment NS script.
Linktest works by reading a data file of all of the links and their
attributes, and then invoking external measurement tools -- namely
ping, <a href="">Rude and Crude</a> and <a
href=>iperf</a>. Linktest
compares the results against margins of error calculated in advance to
identify major errors in configuration.
Linktest runs on each experiment node. The Linktest daemon waits for a
custom event instructing it to begin testing. When it receives the
event, it invokes the Linktest script to conduct the actual tests. The
script invokes external processes to validate links and log any errors
found. If a node detects an error, it writes an explanatory message
to the experiment tbdata/linktest directory. Otherwise, no messages
appear in the directory after Linktest completes its run.
Linktest uses test levels to select which tests to perform. Test
levels are cumulative, so that selecting a higher test level ensures
lower-level tests are also run. Test levels are ordered in length of
time to complete, so that <a href="#Level4">Level 4 - Bandwidth</a>
takes the most time and
<a href="#Level1">Level 1 - Connectivity and Latency</a> takes the least.
Read more about each test level in the following sections:
<a NAME="Level0"></a>
<li> <h3>Level 0 - Do not run Linktest</h3>
The default test level is Level 0 - Do not run Linktest. Use this
level to leave Linktest turned off, performing no validation of
experiment links after swapin.
<a NAME="Level1"></a>
<li> <h3>Level 1 - Connectivity and Latency</h3>
Each Linktest node on a lan or direct link pings the node on the other
side of the link. From the responses, the node detects whether the
link is up and the latency of the link. Linktest compares the measured
latency with the expected latency of the link, adjusting for known
delay crossing the testbed backplane. If the measured latency
is outside the 99% confidence interval for latencies at that setting,
Linktest reports an error.
<b>Note:</b> In an effort not to get bogged down by reporting too many
false positive errors, Linktest will look at the actual delta and
report an error only if the measured latency is more then 0.5
millseconds <em>below</em> the desired value, or more than 3.5
millseconds <em>above</em> the desired value.
<a NAME="Level2"></a>
<li> <h3>Level 2 - Static Routing</h3>
If the routing mode of the experiment is static, each Linktest node
pings the remainder of nodes in the experiment. If any node cannot be
reached, the Linktest node reports an error.
<a NAME="Level3"></a>
<li> <h3>Level 3 - Loss</h3>
Each Linktest node on a lan or direct link with loss > 0 sends a burst
of packets to the node on the other side of the link using Rude and
Crude, a real-time packet emitter and collector. If the percentage of
packets lost is outside the 99% confidence interval for
normally-distributed loss at that setting, Linktest reports an
<a NAME="Level4"></a>
<li> <h3>Level 4 - Bandwidth</h3>
Each Linktest node on a lan or direct link uses iperf to measure the
bandwidth of the link, provided that the link is >= 1 Mbps and does
not have a link loss value. If the measured bandwidth is outside the
margin of error, Linktest reports an error.
The Bandwidth test adds up to 20 seconds per distinct link in the
experiment; 10 seconds in each direction. Linktest attempts to run
tests in parallel whenever
possible, but topologies such as a star will lead to longer runtimes
since Linktest allows only one sender or receiver to run on a node at a
<b>Note:</b> In an effort not to get bogged down by reporting too many
false positive errors, Linktest will look at the actual delta and
report an error only if the measured bandwidth is more then 5%
<em>below</em> the desired value, or more than 1% <em>above</em> the
desired value.
<a name="Advanced"></a>
<h2>Advanced Topics</h2>
To run Linktest after experiment swapin, you may use Emulab's Web
Interface, or you may manually invoke the script <tt></tt> on
ops. You may also examine Linktest output in its log directory. Read
about these options in the following sections:
<li> <a href="#Web">Running Linktest from the Web Interface</a>
<li> <a href="#Ops">Running Linktest on Ops</a>
<li> <a href="#Log">Linktest Log Directory</a>
<a NAME="Web"></a>
<li> <h3>Running Linktest from the Web Interface</h3>
If you go to the "Show Experiment" page for your experiment, you will
see an option called "Run Linktest" in the auxiliary menu for the
experiment. Clicking on that link will take you to the run linktest
page, where you can select a level, and then start linktest running by
clicking on the Start button. Once linktest starts running, you can
stop it by clicking on the Stop button. Please be patient; linktest
can take a long time to run. Eventually, you will be notified of its
results in the window below the Start/Stop button.
No email is sent.
<a NAME="Ops"></a>
<li> <h3>Running Linktest on Ops</h3>
Use to run Linktest on ops. The option "-e" is
mandatory for specifying the
project and experiment id. Running with "-q" will run all tests except
the bandwidth test. Running without "-q" runs all tests. Running with "-o" allows you to specify an output directory
for log messages. Invoke with no options for a help
<code><pre> -e utahstud/simple -q -o /tmp/linktest.log</code></pre>
<p>You may also specify the test level using the "-l" option. Example:
<code><pre> -e utahstud/simple -l 1</code></pre>
After Linktest completes, prints out errors found
during the run, if any. For scripting, returns 0 if no
errors were found, or !0 if at least one error was found.
No email is sent.
<a NAME="Log"></a>
<li> <h3>Linktest Log Directory</h3>
Linktest logs the results and any error reports in the tbdata/linktest
directory for each experiment. By default this directory's path is
<tt>/proj/&lt;myproj&gt;/exp/&lt;myexpt&gt;/tbdata/linktest/</tt> .
Copyright (c) 2000-2003, 2005 University of Utah and the Flux Group.
All rights reserved.
Overview of the Authorization Scheme, Policy, <br> and "How To Get Started"
We use a hierarchical structure: we authorize a project under a
principal investigator (e.g. a faculty member) and delegate authority
to that person to authorize the project's members-- and accountability
for their behavior.
<h3>Who can start a project?</h3>
In general, a faculty member or well-known senior staff member must
start each project. Students cannot lead projects unless they
have had prior approval by the
<a href="">
Emulab Approval Committee</a>. See the next section for the
reasoning behind this rule.
<h3>How do I get started?</h3>
Briefly, you use the links at your left to create and join
<em>projects</em>. Typically, someone who will be the <em>project
leader</em> requests permission from Testbed Ops/Admin, via the web
interface, to <em>create</em> a project. In academic parlance, a
project leader is a "principal investigator." That person is expected
to be someone who is responsible, whose position is more or less
verifiable by us, and is therefore accountable. Specifically, the
project leader is held responsible for the actions of members of
his/her project.
For example, if you are a grad student who "owns" a project and no
faculty member is really involved, normally you should still get your
advisor or other professor to be the project leader. Exceptions could
include your being a senior student well-known in the research
community. If you are not a student, but a senior/core member of an
open source project, either you or someone more official in
the project should be leader, as appropriate.
If you are in a research lab and are not brand new there, you would
probably be the project leader.
Typically, after an hour to a day later, or up to week (rarely),
you will receive email from the testbed admin folks,
either approving or denying your project. You will then be able to
really use the testbed: you will be able to perform various functions
through the Web interface and through a Unix login account.
People working on the project
(students, staff, etc.) will request permission from the project
leader, also via the web interface, to <em>join</em> the project.
These requests can precede project approval; they will be queued.
Once project members have been authorized by the leader, they can use
the Web interface and their Unix login to start and run experiments,
reserve and configure nodes, etc.
More detailed information on this
process can be found in the
<a href="faq.php3">Emulab FAQ</a>.
<h3>Another way of saying the same thing</h3>
If you didn't understand that, then how about this. Use this set of
Web pages:
<li> to gain authorization to use the testbed, either as
<li> a project leader ("principal investigator") who is
starting a new project ("start project"), or
<li> as a worker bee in a particular project ("join project");
<li> as a project leader, to approve or deny pending project members;
<li> to authenticate ("login") to the Web-based testbed services.
When your project or membership request is approved or denied you will
receive email.
<h3>Seems awfully complicated</h3>
Experience shows that it's far easier in practice than it sounds.
We need accountability. However, we want to avoid slowing things
down by checking every user-- thus we delegate that authority
to the PI's. Since the PI (project leader) has so much authority,
we need more info from them, such as their postal address.
If you think this sounds bad, try getting access to a telescope
or supercomputer.
We are certainly open to suggestions, however.
<h3>I've been approved. How do I use my account?</h3>
The first step would be to come back here and log in to the Web
interface. That will update the list of options in the side bar.
You might be authorized to start projects or experiments, or
maybe just to use the nodes in an experiment. Either way, your
options will show up in the side bar.
Those will normally include starting a new "experiment" which leads to
reserving a set of nodes, which leads to automatic creation of Unix
accounts on those nodes for all members in your project. You will be
able to use ssh to log into those machines.
You will also receive an account on the users' master host
"", and from there will be able to access the test
nodes' serial line consoles via 'console' as well as access console log
# Copyright (c) 2000-2008 University of Utah and the Flux Group.
# All rights reserved.
# Standard Testbed Header
<h3>Getting Started on the Testbed</h3>
<p>These documents are highly recommended reading for all Emulab users.</p>
<dt><b><a href="docwrapper.php3?docname=auth.html">
How to get an account on the Testbed</a></b></dt>
<dd>Starting or joining projects, authorization, and policies.</dd>
<dt><b><a href="tutorial/docwrapper.php3?docname=tutorial.html">
Emulab "Getting Started" Tutorial</a></b></dt>
<dd>The basics of creating experiments, and a quick intro to frequently
used features, avoiding common pitfalls, and quick solutions to the
most common problems people encounter.</dd>
<dt><b><a href="faq.php3">
Frequently Asked Questions (FAQ)</a></b></dt>
<dd>Probably our most important reference document. Check here before
asking for help. Includes sections on "Getting Started" on Emulab,
using the testbed, hardware setup, software setup, security issues,
and troubleshooting. Over 50 questions and answers.</dd>
<dt><b><a href="doc/docwrapper.php3?docname=plab.html">
Emulab PlanetLab Interface</a></b></dt>
<dd>Documentation for our interface to PlanetLab.</dd>
<h3>Reference Material</h3>
<li><b><a href="tutorial/docwrapper.php3?docname=tutorial.html#Advanced">
Emulab Advanced Tutorial</a></b>
<li><b><a href="tutorial/docwrapper.php3?docname=advanced.html">
Emulab Advanced Example</a></b>
# Link to kb-search through $TBBASE (https:), not $TBDOCBASE (http:).
# On https:, the browser sends HashCookie, so we get CHECKLOGIN_LOGGEDIN
# status. Going via http:, we get CHECKLOGIN_MAYBEVALID, and can't know
# whether to show admin KB entries.
echo "<li><b><a href=\"$TBBASE/kb-search.php3\">\n
Search or Browse the Emulab Knowledge Base</a></b>\n";
<li><b><a href="tutorial/docwrapper.php3?docname=nscommands.html">
Emulab-specific NS Extensions Reference Manual</a></b>
<li><b><a href = "xmlrpcapi.php3">Emulab's XML-RPC interface reference</a></b>
<li><b><a href = "docwrapper.php3?docname=swapping.html">
Node Usage Policies and "Swapping" Experiments</a></b>
<li><b><a href = "doc/docwrapper.php3?docname=netbuilddoc.html">
NetBuild Reference Manual</a></b>
<li><b><a href = "docwrapper.php3?docname=security.html">
Security Issues</a></b>
<li><b><a href = "docwrapper.php3?docname=groups.html">
About Project Groups</a></b>
<li><b><a href = "doc/docwrapper.php3?docname=internals.html">
Emulab Internals</a></b>
<li><b><a href = "doc/docwrapper.php3?docname=hw-recommend.html">
Want to build your own Testbed?</a></b>
<li><b><a href = "hardware.php">
Current Hardware Overview</a></b>
<li><b><a href = "docwrapper.php3?docname=software.html">
Current Software Overview</a></b>
<li><b><a href = "docwrapper.php3?docname=policies.html">
Administrative Policies and Disclaimer</a></b>
<li><b><a href = "docwrapper.php3?docname=sponsors.html">
Emulab Sponsors</a></b>
<li><b><a href = "news.php3">
Current and Past Emulab News Items</a></b>
<li><b><a href = "doc/docwrapper.php3?docname=ChangeLog.txt">
Changelog/Technical Details</a></b>
<li><b><a href = "pubs.php3">
Papers and Talks</a></b>
<a name="tutorial">
<h3>SIGCOMM Tutorial</h3>
You might also find it helpful to look through the tutorial given at
SIGCOMM '02 by Jay Lepreau, Robert Ricci, and Mac
Newbold, entitled "How To Use the Netbed (Emulab++) Network Testbeds."
<li><b>Slides</b> from the tutorial are available in
<a href = "">
PowerPoint Format</a>,
and as PDFs, with
<a href = "">
2 slides per page (full color)</a>, and
<a href = "">
6 slides per page (greyscale)</a>.
<li><b>Tutorial files</b> such as NS files for the experiments, are available
as a
<a href = "">
tarball</a>, or
<a href = "">
<font size=-2>
# Standard Testbed Footer
Copyright (c) 2005 University of Utah and the Flux Group.
All rights reserved.
<h1>Motes on E-Motes</h1>
<li> <a href="#INTRO">Introduction</a>
<li> <a href="#HARDWARE">Hardware</a>
<li> <a href="#SPECIFICATION">Specification in NS Files</a>
<li> <a href="#PROGRAMMING">Programming Motes</a>
<li> <a href="#SERIAL">Serial Consoles</a>
<li> <a NAME="INTRO"></a>
The <a href="/tutorial/mobilewireless.php3">mobile wireless testbed</a>
includes two static nodes hosted on
<a href="">E-Mote</a>
Ethernet gateways. This document describes how these devices fit into the
larger Emulab system.
<li> <a NAME="HARDWARE"></a>
The static motes in Utah's emulab are hosted on E-Motes. E-Motes (also known
as the MIB600CA) from <a href="">Crossbow</a> provide three
key features:
<li>They have a power supply that comes from the power grid, so there is no
need to worry about battery life on the attached mote.</li>
<li>They provide the ability to reprogram the attached mote over Ethernet -
thus you don't need to worry about wedging your motes, we can simply
reprogram them to recover them to a known state.</li>
<li>They export the mote's serial console to a TCP port. This opens up
several possibilities. First, it means that you can use any of them as base
stations to gateway between the Internet and your mote network. Second, it
provides a back channel over which to send commands to the motes or extract
data from them.</li>
The overall goal of the second two features for us is to provide something
analogous to our PCs'
<a href="/tutorial/docwrapper.php3?docname=tutorial.html#ControlNet">control
<h3>Specification in NS Files</h3>
Since motes in Emulab are hosted on other nodes such as E-Motes and Stargates,
you must specify two nodes in your NS file: the mote itself and its host. A
mote specification in your NS file may look something like this:
<blockquote style="border-style:solid; border-color:#bbbbbb; border-width: thin">
set inky [$ns node]
set inkyhost [$ns node]
tb-set-hardware $inky mica2
tb-set-hardware $inkyhost motehost
tb-fix-node $inky $inkyhost