NSE scaling, accuracy and capacity

The data reported here are from our OSDI publication

Contents


Scaling

An instance of nse simulated 2Mb constant bit rate UDP flows between pairs of nodes on 2Mb links with 50ms latencies. To measure nse's ability to keep pace with real time, and thus with live traffic, a similar link was instantiated inside the same nse simulation, to forward live TCP traffic between two physical Emulab nodes, again at a rate of 2Mb. On an 850MHz PC, we were able to scale the number of simulated flows up to 150 simulated links and 300 simulated nodes, while maintaining the full throughput of the live TCP connection. With additional simulated links, the throughput dropped precipitously. We also measured nse's TCP model on the simulated links: the performance dropped after 80 simulated links due to a higher event rate from the acknowledgment traffic in the return path.

Accuracy and Capacity

As a capacity test, we generated streams of UDP round-trip traffic between two nodes, with an interposed 850 Mhz PC running nse on a FreeBSD 4.5 1000HZ kernel. A maximum stable packet rate of 4000 packets per second was determined over a range of packet rates and link delays using 64-byte and 1518-byte packets. Since these are round trip measurements, the packet rates are actually twice the numbers reported. With this capacity, we performed experiments to measure the delay, bandwidth and loss rates for representative values. The results are summarized in Tables 1, 2 and 3

Emulab's integration of nse is much less mature than its support for dummynet based emulation. This is reflected in the large percentage error values in bandwidth and loss rates. Integrating nse has already uncovered a number of problems that have since been solved; as we continue to gain experience with nse, we expect the situation to improve.


Table 1: Accuracy of nse delay at maximum packet rate (4000 PPS) as a function of packet size and link delay. The 0ms measurement represents the base overhead of the link. Adjusted RTT is the observed value minus the base overhead.
delay
(ms)
packet size
(bytes)
observed adjusted
RTT (ms) stddev %err RTT (ms) %err
0 64 0.238 0.004 N/A N/A N/A
1518 1.544 0.025 N/A N/A N/A
5 64 10.251 0.295 2.51 10.013 0.13
1518 11.586 0.067 15.86 10.032 0.32
10 64 20.255 0.014 1.28 20.017 0.09
1518 21.675 0.093 8.38 20.121 0.61
50 64 100.474 0.029 0.47 100.236 0.24
1518 102.394 3.440 2.39 100.840 0.84
300 64 601.690 0.546 0.28 601.452 0.24
1518 602.999 0.093 0.49 601.445 0.24



Table 2: Accuracy of nse bandwidth as a function of link bandwidth and packet size.
bandwidth
(Kbps)
packet size
(bytes)
observed
bw (Kbps) %err
56 64 55.60 0.71
1518 56.63 1.12
384 64 376.3 2.00
1518 382.1 0.49
1544 64 1444.5 6.44
1518 1531.0 0.84
10000 64 N/A N/A
1518 9659.6 3.40
45000 1518 39857 11.43

Table 3: Accuracy of nse packet loss rate as a function of link loss rate and packet size.
packet loss
rate (%)
packet size
(bytes)
observed
loss rate (%) %err
0.8 64 0.819 2.37
1518 0.820 2.50
2.5 64 2.477 0.92
1518 2.477 0.92
12 64 11.88 1.00
1518 11.89 0.91