Commit 805a7b58 authored by Jay Lepreau's avatar Jay Lepreau
Browse files

All paper stuff moved to the 'papers' cvs tree: papers/testbed/ipathemu

parent 6d9b210e
We present a new Internet path emulation model for use in the Emulab testbed.
This gives the experimenters acccess to load-free and completely controllable
host conditions and relistic network conditions. We use a
lightweight monitoring system to take periodic snapshots of the network
conditions on Planetlab and detect shared bottlenecks. Our system reliably
detects bottlenecks due to congestion and measures how the bandwidth available
to TCP transfers on Planetlab scales with the number of streams. We use a
modified FreeBSD Dummynet network configurator to faithfully reproduce these
network conditions in Emulab.Unlike the dynamic Internet model provided by
Flexlab, the static model scales reliably and the size of the experiments
possible is only limited by the number of available nodes in Emulab.
Our evaluation suggests that this new static Internet emulation model provides
an improved method for setting realistic network characteristics in Emulab
while preserving its strengths of repeatibility and host control.
Possible additions/changes to the abstract:
- Intro about Emulab & Planetlab:
- Network conditions are artificial in Emulab.
- Planetlab is overloaded and execution environment(Eg: Kernel)
can't be modified.
- Something about Node-caps(&/HTB) and how they might affect experiments?
- Reliable experiment setup: For large experiments, some Planetlab nodes
always fail to come up. We can eliminate this problem with our static
model and Emulab.
- More emphasis on repeatability and being able to run experiments
against particular snapshots of Internet conditions, useful for debugging.
- Mention the changes to Dummynet to enable Backfill, realistic emulation
of asymmetric paths and quantifying their prescence/extent on Planetlab.
Paper Outline:
- High level goal: A better static model in Emulab to emulate the available bandwidth seen
on Planetlab paths.
- Contributions ?
- A new(simpler/less complex/?..) method for emulating Internet conditions
inside Emulab, while providing control over the host environment and non-overloaded
host conditions.
- The experiments will be repeatable using any snapshot of the static
conditions stored in our database - ability to debug the applications while
running them under realistic Internet conditions.
- Our Dummynet mods ?? - realistic emulation of available bandwidth in case
of asymmetric paths.
- Quantification of the extent and effect of Node-caps on Planetlab.
- Problems with the existing static model(s):
- The cloud model creates independant pipes between each pair
of nodes. It fails to take into account potential bottlenecks in
the internet.
- The LAN model assumes that each node -> all other nodes is a bottleneck.
Again, this is an over-simplication.
- Hybrid model assumes that only Inet -> everywhere else paths are
- Glean whatever else is possible from the Flexlab paper/Bittorrent results.
- What is the new model ?
- Detects bottlenecks using Wavelet based Correlation detection. Light-weight
UDP probes sent out on all paths of interest and their delays are correlated.
- The Wavelet method is based on a SIGCOMM '04 paper(which is an increment over
Rubenstein's SIGMETRICS '02 paper) and it has been
verified to correctly detect congestion in an Emulab experiment.
- Setup the Dummynet pipes with the detected bottlenecks
- The bandwidth for these bottlenecks is currently being set as
the maximum bandwidth (as reported by Flexmon) of the paths in
- Insert about Jon's current work: Instead of doing the above for
bottleneck bandwidth, we measure the bandwidth obtained for a given
number of streams on Planetlab on these paths. While running
the static model in Emulab, we could detect the number of streams
going through the bottleneck and vary the amount of available bandwidth
to match what we saw earlier in Planetlab according to our snapshot.
We don't have the numbers on how well this performs yet.
- Issues in emulating the available bandwidth:
- ~10% of Planetlab paths have asymmetric links of > 3:1 asymmetry.
- Simply setting up dummynet pipes with the available bandwidth values
causes problems in the case of asymmetric links( with asymmetry > 3:1 ).
- We have experimental proof of this phenomenon(Jon's IPerf mesh runs in Emulab).
Its main cause is the large queuing delay caused on the reverse path and
the ACKs of the forward path transfer getting stuck there while waiting
to be transmitted on the slow link.
- This problem can be alleviated if we setup the links as high-capacity, but
providing the appropriate amount of available bandwidth. On the Internet,
this is accomplished by the cross-traffic from other sources.
- We do this by way of our modifications to Dummynet. We add a new parameter
called "Backfill" ( background fill ) to restrict the amount of available
bandwidth. This provides better performance in the case of asymmetric links
which have TCP transfers going on in both directions. ( we have the numbers for
this too).
- Our Dummynet modifications do not increase the load on the delay nodes.
( Top numbers available ).
- Evaluation(We don't have any of these numbers yet):
- IPerf mesh run in parallel on Planetlab and on Emulab.
- We need to show a contrast between how the Cloud/Hybrid models
perform against our new static model.
- We could run BitTorrent between the same nodes that we put in the
Flexlab NSDI paper and check how the new model does on that set of nodes.
Cloud model:
1) Each node has a seperate bandwidth pipe to every possible destination.
- If the node has a node cap being applied to some or all of its destinations,
then simple cloud model fails when the sum of bandwidths of all flows being used by the
application exceeds node cap.
- Cloud model does not take into account multiple flows between the same pair
of hosts - it probably results in lower throughput
a) if the real world flows are limited by bandwidth-delay product and not
by congestion. Or
b) The bottleneck link has more ethan a modest level of statistical
multiplexing. In this case, creating two flows gives 2x throughput - whereas the
cloud only gives 'x' total throughput for any number of flows between a pair of hosts.
- It also does not work when the bottleneck is the last mile link on the down stream. Two streams from two different nodes might get bottlenecked at the destination node
- we do not handle this case at the moment.
2) Shared config:
- This is a simplification which is applicable mostly to slow DSL and commodity
internet nodes.
- Heavily used Inet2 nodes on Planetlab might also exhibit this behaviour -
if their last mile access is congested.
a) However DSL/Inet nodes have relatively low statistical multiplexing. The
only flows congesting the link are our own experiment flows.
-- Usually RTT variation is considerable even without active IPerf
measurement by us. So we can easily identify this type of bottlenecks.
b) Congested Inet2 nodes have high statistical multiplexing. But RTT remains
relatively stable unless measurement flows are introduced -- tricky to identify which
destinations share the bottleneck - Heisenberg effects.
- The trouble here is that we have no idea how far out the bottleneck
link is and how many destination routes pass through that bottleneck.
- Assuming we identify and group destinations sharing a bottleneck, how do we
set the bandwidth for each group ?
- We assume high multiplexing, so each new flow should get the same amount
of throughput.
- What about flows going to different destinations with different RTT ?
Potential problems for evaluation:
1) We will have to compare a run of (IPerf mesh and/or Bittorrent) on Planetlab to a run with our static model, which uses Flexmon measurements. If the available bandwidth conditions hange between Flexmon measurement and our Planetlab run, then we'll have a hard time
figuring out whether the difference is because of our model or transient traffic conditions
on the Planetlab paths.
2) Flexmon measurements are taken one path at a time. There is a chance that these
throughput values are going to be lower when all the streams are run at once on
Planetlab - of course this could point out the bottleneck links we missed using
our UDP probes when there wasn't any significant cross-traffic.
Asymmetric paths and bandwidth emulation:
When the paths being emulated have relatively low bandwidths ( < 30 Mbps I think - not
competely sure ) and the asymmetry is > 3:1, then simply setting up Dummynet pipes with
the bandwidth and delay values is going to cause anomalies (upto 70% throughput loss) in
the forward path TCP when there is/are TCP flows on the reverse path.
Remedy to this i twofold. The key observation is that most of the Inet links have
high capacities. The available bandwidth varies because of the amount of
cross-traffic. Even links with very low available bandwidth transmit packets at a high
link rate. In contrast, if a dummynet pipe is setup with the low available bandwidth
as its capacity, then it is going to cause large queuing delays for the ACK packets of
the forward path TCP flows. This inflates the RTT of the forward TCP flow and results
in a significant and drastic throughput drop compared to the expected
forward throughput.
In order to correct this problem, we introduce a new parameter "BACKFILL" during
the dummynet pipe configuration . All the experiment links are setup as 50 Mbps(OC-1)
capacity. The amount of backfill on a link is link capacity - available bandwidth. We
introduce a CBR packet flow into the Dummynet pipe to act as cross-traffic at this
"BACKFILL" rate.
We modified Dummynet to introduce this "BACKFILL" directly into the bandwidth
queues. So, the cross-traffic is inserted at a fine granularity and is removed
from the queues and is simply ignored ie. it is not forwarded to any of the
experiment nodes. Therefore, it does not interfere with any other experiment
traffic other than serving to shape traffic on a particular Dummynet pipe. Our
modifications do not result in any noticeable increase in load on the delay nodes.
Another aspect of accurate bandwidth emulation is the setting up of queue
sizes. To emulate real world routers, the queues are set in terms of bytes, rather
than slots. The size of a particular queue depends on the available bandwidth and RTT
of the path - it is set to restrict the maximum window size of the TCP flow(s) on
the path.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment