Commit 8a268c48 authored by Mike Hibler's avatar Mike Hibler

Add a section on the control net

parent 9e11a70d
......@@ -25,6 +25,8 @@
<li> <a href="#Advanced">Advanced Topics</a>
<ul>
<li> <a href="#ADVEX">A more advanced example</a>
<li> <a href="#ControlNet">Understanding the Control Network</a>
<img src="../new.gif" alt="&lt;NEW&gt;">
<li> <a href="#RPMS">Installing RPMS automatically</a>
<li> <a href="#TARBALLS">Installing Tar files automatically</a>
<li> <a href="#Startupcmd">
......@@ -35,6 +37,8 @@
Setting up IP routing between nodes</a>
<li> <a href="#Simem">
Hybrid Experiments with Simulation and Emulation</a>
<li> <a href="docwrapper.php3?docname=vnodes.html">
Multiplexed Virtual Nodes</a>
<img src="../new.gif" alt="&lt;NEW&gt;">
</ul>
<li> <a href="#BatchMode">Batch Mode Experiments</a>
......@@ -550,12 +554,15 @@ Again, please feel free to contact us.
<ul>
<li> <a href="#ADVEX">A more advanced example</a>
<li> <a href="#ControlNet">Understanding the Control Network</a>
<img src="../new.gif" alt="&lt;NEW&gt;">
<li> <a href="#RPMS">Installing RPMS automatically</a>
<li> <a href="#TARBALLS">Installing Tar files automatically</a>
<li> <a href="#Startupcmd">Starting your application automatically</a>
<li> <a href="#SyncServer">How do I know when all my nodes are ready?</a>
<li> <a href="#Routing">Setting up IP routing between nodes</a>
<li> <a href="#Simem">Hybrid Experiments with Simulation and Emulation</a>
<li> <a href="docwrapper.php3?docname=vnodes.html">Multiplexed Virtual Nodes</a>
<img src="../new.gif" alt="&lt;NEW&gt;">
</ul>
<p>
......@@ -573,6 +580,123 @@ integration of network simulation (NS/NSE).
</p>
<li> <a NAME="ControlNet"></a>
<h3>The Emulab "Control Network"</h3>
<p>
The Emulab <i>control network</i> is the source of a number of problems related
to the behavior of network links in an experiment. So in this section we
attempt to explain exactly what the control network is, why problems occur,
and how to avoid them.
</p><p>
Every physical node in the
testbed has one interface connected to a common 100Mb LAN. This interface is
used by the testbed infrastructure to configure experiments (e.g. distribute
account info, load disks, etc.). It is also used by experimenter to communicate
with the nodes from outside Emulab or from users.emulab.net (e.g., ssh).
Finally, it may be used by an experimenter to monitor activity during an
experiment. Control net links have a fixed address and not configured
by the user.
</p><p>
The control network is differentiated from the <i>experimental network</i>
which is the set of links specified in the experiment topology over which
experiment applications should communicate. Experiment links are configured
by the user and may be shaped and otherwise controlled by that user.
</p><p>
To illustrate the difference, consider the simple two-nodes-and-a-router
configuration described by the NS snippet:
<code><pre>
source tb_compat.tcl
set ns [new Simulator]
set node1 [$ns node]
set router [$ns node]
set node2 [$ns node]
set linkA [$ns duplex-link $node1 $router 100Mb 0ms DropTail]
set linkB [$ns duplex-link $router $node2 1Mb 10ms DropTail]
$ns rtproto Static
$ns run
</code></pre>
which is instantiated as shown in red in Figure 1.
<br>
<center>
<img src="control-net.png"><br><br>
<b>Figure 1.</b> Control network interfaces (blue) to an
experiment topology (red).
</center>
<br>
From your vantage point out in the Internet, or on the machine
"users.emulab.net," you can access a node in your experiment via its
canonical Emulab name
(e.g., "pc12.emulab.net")
or by the DNS alias that is assigned when your experiment is created
(e.g., "node2.foo.testbed.emulab.net").
Both types of names use the fixed 100Mb control network link
(IP: 155.101.132.12) to access the node. These control net links are shown
in blue in Figure 1.
</p><p>
The view from inside an experiment node is different. In an ideal world,
applications running inside the experiment would not even know about the
control network, since it is not part of the topology. However, the control
net must be visible to nodes for several practical reasons, most notably
allowing login from remote sites and the ability of the node to access
the shared NFS /proj and /users filesystems.
<p></p>
As the control net is visible to applications on a node, it can lead to
its inadvertent use by applications. In the above example, consider a ping
from node1 to node2. The expectation is that traffic will pass through the
included router and over the shaped link to node2, resulting in round-trip
times of 20ms:
<code><pre>
1 node1.foo.testbed.emulab.net> ping node2
PING node2-linkB (10.1.1.2) from 10.1.2.2 : 56(84) bytes of data.
64 bytes from node2-linkB (10.1.1.2): icmp_seq=1 ttl=63 time=20.3 ms
64 bytes from node2-linkB (10.1.1.2): icmp_seq=2 ttl=63 time=20.3 ms
...
</code></pre>
But if the ping travels over the control network rather than the experimental
network, there will be no delay:
<code><pre>
2 node1.foo.testbed.emulab.net> ping node2.foo.testbed.emulab.net
PING pc12.emulab.net (155.101.132.12) from 155.101.132.10 : 56(84) bytes of data.
64 bytes from pc12.emulab.net (155.101.132.12): icmp_seq=1 ttl=64 time=0.291 ms
64 bytes from pc12.emulab.net (155.101.132.12): icmp_seq=2 ttl=64 time=0.124 ms
...
</code></pre>
Notice the subtle difference between the pings, the correct one uses an
unqualified name, the incorrect one uses the fully qualified name. Unqualified
names are resolved from a local <code>/etc/hosts</code> file that is created
on each node. Fully qualified names are resolved by the Emulab nameserver and
return the routable, control net address. Figure 1 shows the various names
each node can be named by, and which interface they will resolve to.
Accidental use of the control net interface in an experiment commonly occurs
due to one of three reasons:
<ul>
<li> The experiment user configures an application incorrectly.
By specifying either a node's fully qualified name or its control net address
when configuring an application, that application will use the wrong interface.
<li> An application itself decides which interface to use based on a
node's hostname. Since the hostname is set to the fully qualified name,
that name will resolve to the control net address.
<li> An application uses all available interfaces. By default, many server
applications will listen on all interfaces they discover via ioctls or other
kernel mechanisms. This will include the control interface.
</ul>
<p>
In the first case, you just need to be careful to use the correct name or
address. For the latter two cases, you might need to modify the application.
Fortunately, most applications include options enabling you to explicitly
specify which interfaces to use (or not use).
We recognize that this is not ideal, and will be trying to find better ways
to "hide" the control net in the future.
</p>
<li> <a NAME="RPMS"></a>
<h3>Installing RPMS automatically</h3>
<p>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment