faq.html 35.3 KB
Newer Older
1
<center>
2
<h2>Frequently Asked Questions</h2>
3
4
5
</center>

<h2>Contents</h2>
6
<ul>
7
8
<li> <a href="#GS">Getting Started</a>
     <ul>
9
     <li> <a href="#GS-Eligible">Who is Eligible to use Emulab.Net?</a>
10
11
     <li> <a href="#GS-1">How do I start a project?</a>
     <li> <a href="#GS-2">How do I join a project?</a>
12
     <li> <a href="#GS-3">I have an Emulab account. Now what?</a>
13
     <li> <a href="#GS-PATH">Do I need to change my PATH variable?</a>
14
15
     <li> <a href="#GS-4">Can I be in more than one project?</a>
     <li> <a href="#GS-5">Can I change my Emulab password?</a>
16
17
     <li> <a href="#GS-6">I'm a project leader. Can I designate TAs?</a>
     <li> <a href="#GS-7">Where do I get help?</a>
18
19
20
21
22
     </ul>

<li> <a href="#UTT">Using the Testbed</a>
     <ul>
     <li> <a href="#UTT-1">Is there a tutorial?</a>
23
24
     <li> <a href="#UTT-NETBUILD">Do you have a <b>GUI</b> to help me
                                   create experiments?</a>
25
     <li> <a href="#UTT-NODES">How many nodes can I ask for?</a>
26
     <li> <a href="#UTT-HOWLONG">How long can I keep using my nodes?</a>
27
     <li> <a href="#UTT-TOOFEW">What if I need more nodes than are free?</a>
28
     <li> <a href="#UTT-2">Do I get root access on my nodes?</a>
29
     <li> <a href="#UTT-3">Do my nodes have consoles I can look at?</a>
30
     <li> <a href="#UTT-4">Can I reboot (power cycle) my nodes?</a>
31
     <li> <a href="#UTT-SCROGGED">I've scrogged my disk! Now what?</a>
32
33
     <li> <a href="#UTT-5">Where do I store files needed by my experiment?</a>
     <li> <a href="#UTT-6">Are my files on <b>users.emulab.net</b>
34
                           backed up (filesaved)?</a>
35
     <li> <a href="#UTT-7">Are the nodes in my experiment backed up
36
                           (filesaved)?</a> 
37
     <li> <a href="#UTT-Swapping">What is Swapping?</a> 
Robert Ricci's avatar
Robert Ricci committed
38
39
     <li> <a href="#UTT-8">How can I get switch statistics (such as packet
                           counts) for my experiment?</a> 
40
41
42
43
44
     </ul>

<li> <a href="#HDS">Hardware setup</a>
     <ul>
     <li> <a href="#HDS-1">How many nodes are there?</a>
45
46
     <li> <a href="#HDS-2">How many nodes are currently available?</a>
     <li> <a href="#HDS-3">How many ethernet cards are on each node?</a>
47
48
     <li> <a href="#HDS-4">How many nodes are currently available (free)?</a>
     <li> <a href="#HDS-5">Can I do traffic shaping on my links?</a>
Jay Lepreau's avatar
Jay Lepreau committed
49
     <li> <a href="#HDS-6">Can I modify the traffic shaping
50
                           parameters on my links?</a>
Jay Lepreau's avatar
Jay Lepreau committed
51
     <li> <a href="#HDS-7">Are there other traffic shaping parameters
52
                           besides latency, bandwidth, and PLR?</a>
53
54
55
56
57
     </ul>

<li> <a href="#SWS">Software setup</a>
     <ul>
     <li> <a href="#SWS-1">What OS do the nodes run?</a>
58
59
60
     <li> <a href="#SWS-2">How do I select which OS to run on each node?</a>
     <li> <a href="#SWS-3">Can I load my own software (RPMs) on my nodes?</a>
     <li> <a href="#SWS-4">Can I schedule programs to run
61
62
63
64
                           automatically when a node boots?</a>
     <li> <a href="#SWS-5">How can I turn on routing or set up routes
			   automatically in my nodes?</a>
     <li> <a href="#SWS-6">How does my software determine when other
65
                           nodes in my experiment are ready?</a>
66
     <li> <a href="#SWS-7">Can I run my own Operating System?</a>
67
     <li> <a href="#SWS-8">What if I need more disk space on my nodes?</a>
68
69
     <li> <a href="#SWS-9">Are there testbed-specific daemons that could
                           interfere with my experiment?</a>
70
     </ul>
71
72
73
74
75

<li> <a href="#SEC">Security Issues</a>
     <ul>
     <li> <a href="#SEC-1">Is Emulab Firewalled?</a>
     </ul>
76
77
78
79
80
81

<li> <a href="#TR">Troubleshooting</a>
     <ul>
     <li> <a href="#TR-1">My experiment is setup, but I cannot
     			  send packets between some of the nodes, why?</a>
     </ul>
82
83
84
</ul>

<hr>
85

86
87
88
<a NAME="GS"></a>
<h3>Getting Started</h3>
<ul>
89
90
91
92
93
94
95
96
97
98
99
100
101
<li><a NAME="GS-Eligible"></a>
    <h3>Who is Eligible to use Emulab.Net?</h3>
    <p>
    Emulab.Net is an NSF/DARPA sponsored project, with additional
    support from these <a href =
    "docwrapper.php3?docname=sponsors.html"> sponsors</a>. As such,
    eligibility to use Emulab is primarily granted to other NSF/DARPA
    sponsored projects, as well as current university research
    projects. There are exceptions of course. If you are unsure about
    your eligibility to use Emulab, please feel free to send us an email
    inquiry.
    </p>

102
<li><a NAME="GS-1"></a>
103
    <h3>How do I start a project?</h3>
104
105
    <p>
    If you are new to the Testbed, simply click on the "Start Project"
106
    link on the Emulab <a href="http://www.emulab.net">Home
107
108
109
110
    Page</a>. You will need to fill in the forms with your personal
    information and information about the project. Then click on the
    "Submit" button. Within a few days you will be contacted via email
    with an approval message. More information about starting projects
111
112
    can be found in <a href="docwrapper.php3?docname=auth.html">
    Authorization Page</a>.
113
    </p>
114
115
116
117
118
119
120
    <p>
    If you already have an Emulab account, and wish to start a second
    project, first log into the Web Interface. Then select the "Start
    Project" link; all of the personal information will already be
    filled in. You will need to complete just the project information
    section. 
    </p>
121

122
<li><a NAME="GS-2"></a>
123
    <h3>How do I join a project?</h3>
124
125
    <p>
    If you are new to the Testbed, simply click on the "Join Project"
126
    link on the Emulab <a href="http://www.emulab.net">Home
127
128
129
    Page</a>. You will need to fill in the form with your personal
    information, and provide the name of the project you are trying to
    join (typically, the <i>Project Leader</i> will have told you the
130
    name of the project). Then click on the "Submit" button, and wait
131
132
133
    for the project leader to approve you.  When approved you will
    receive an email message saying so, and you can then log into the
    Testbed.
134
135
136
137
138
139
140
141
142
143
    </p>

<li><a NAME="GS-3"></a>
    <h3>I have an Emulab account. Now what?</h3>
    <p>
    Once you have been approved to start (or join) your first project,
    you will be able to log into Emulab's user machine,
    <b>users.emulab.net</b>. We require that all Emulab users use ssh. For
    example, if your Emulab account name is "joe", then you would do:
    <pre>
144
	ssh users.emulab.net -l joe		</pre>
145
146
147
    </p>
    <p>
    Your password starts out the same as the password you initially
148
149
150
151
152
153
154
155
    supplied to the Start (or Join) web page.

<li><a NAME="GS-PATH"></a>
    <h3>Do I need any special directories in my PATH variable?</h3>
    <p>
    There are several useful (although not required) programs installed on
    <tt>users.emulab.net</tt> in <tt>/usr/testbed/bin</tt>. You should edit
    your dot files to include that directory in your search path. 
156
157
158
159
160
161
    </p>

<li><a NAME="GS-4"></a>
    <h3>Can I be in more than one project?</h3>
    <p>
    Yes. You may join (and/or start) as many projects as you like,
162
163
    subject to Emulab <a href="docwrapper.php3?docname=policies.html">
    administrative policies</a>.
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
    </p>

<li><a NAME="GS-5"></a>
    <h3>Can I change my Emulab password?</h3>
    <p>
    Yes. You can change your Emulab Web password and your Emulab login
    password (the password you use to log into <b>users.emulab.net</b>, as
    well as nodes in your experiments). To change your Web password,
    simply click on the "Update User Information" in the menu to your
    left, and then enter your new password in the location provided. 
    To change your login password, use the unix <tt>passwd</tt>
    utility when logged into <b>users.emulab.net</b>. 
    </p>

<li><a NAME="GS-6"></a>
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
    <h3>I'm a project leader. Can I designate TAs?</h3>
    <p>
    Yes. To designate a TA, you must first create a project
    <em>group</em>. A project group is a lot like a unix group, and
    in fact unix groups is the mechanism used to protect members of
    one group from members of another group. When you create a group,
    you designate a <em>group leader</em> who is responsible for
    approving users who apply to join the group. Group leaders may
    also terminate experiments that have been created by members of
    the group. As Project Leader, you may also shift members of your
    project in and out of your project's groups as you like, and you
    are automatically a member of all groups within your project. As a
    convenience, all new projects are created with one new group,
    termed the <em>default group</em>. As its name implies, whenever
    the group is left unspecified in a form, it defaults to the
    project group (this allows you to create a project without any sub
    groups at all; new members join the default group, new experiments
    are created in the default group, etc.).
    </p>

    <p>
    Project groups are created via the Project Information link at
    your left. Simply go to the project page in which you want to
    create a group, and look for the "Create New Group" link. More
    information on project groups is available via the <a
    href="doc.php3">Emulab Documentation</a> page in the
    <a href="docwrapper.php3?docname=groups.html">Groups Tutorial</a>.
    </p>

<li><a NAME="GS-7"></a>
209
210
211
    <h3>Where do I get help?</h3>
    <p>
    If you cannot find an answer to your question in the
212
    <a href="doc.php3">Emulab Documentation</a>, then you can
213
    send us an <a href="emailus.php3">email message</a>. We will try
214
215
    to answer your question as quickly as we can.
    </p>
216
</ul>
217

Leigh B. Stoller's avatar
Leigh B. Stoller committed
218
<hr>
219
220
221
222
223

<a NAME="UTT"></a>
<h3>Using the Testbed</h3>
<ul>
<li><a NAME="UTT-1"></a>
224
    <h3>Is there a tutorial?</h3>
225
    <p>
226
    Yes, we have an extensive <a href="tutorial/tutorial.php3">tutorial</a>
227
228
229
    on using the Testbed.
    </p>

Chad Barb's avatar
Chad Barb committed
230
<li><a NAME="UTT-NETBUILD"></a>
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
    <h3>Do you have a GUI to help me create experiments?</h3>
    <p>
    Yes, we provide a GUI that gives you an easy to use drawing
    palette on which you can place nodes, lans, and links. Testbed
    specific attributes such as operating system, hardware type, and
    link/lan characteristics, may be attached to each object. With a
    single click, you can instantiate your new topology on the Testbed
    as an experiment in one of your projects. Alternatively, you can
    save the auto-generated NS file on your machine, edit as required,
    and then submit it later when creating an experiment.
    </p>
    
    <p>
    To access the GUI, please log in and go to the Begin Experiment page.
    <em>Note: you need a Java compliant browser.</em>
Chad Barb's avatar
Chad Barb committed
246
247
    </p>

248
<li><a NAME="UTT-NODES"></a>
249
250
251
252
253
254
    <h3>How many nodes can I ask for?</h3>
    <p>
    You can ask for as many nodes as are currently available! You can
    click on the "Node Reservation Status" link at your left to see
    how many nodes are currently free. If you ask for more than are
    currently available, your experiment will be rejected (you
Jay Lepreau's avatar
Jay Lepreau committed
255
    will receive email notification shortly after you submit your NS
256
257
258
259
260
261
262
263
264
    file to the web interface).
    </p>

    <p>
    <em>We urge all new Emulab users to begin with a small 3-4 node
    experiment so that you will become familiar with NS syntax and the
    practical aspects of Emulab operation.</em>
    </p>

265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
<li><a NAME="UTT-HOWLONG"></a>
    <h3>How long can I keep using my nodes?</h3>
    <p>
    You can keep them as long as you need them, subject to our <a
    href="docwrapper.php3?docname=swapping.html">Node Usage
    Policies</a>.  In general, you should do your work, and then
    terminate your experiment as soon as you're done with it. If
    you're not done with it, but are through for a while, you should
    probably "swap out" your experiment (See the question <a
    href="#UTT-Swapping">What is Swapping</a> in this FAQ). It is
    especially important to swap out your experiment if you're through
    with it for the weekend. Emulab usually gets heavy use on the
    weekends by users who need to make very large experiments, so it
    is important to leave as many nodes available as possible.
    </p>

281
282
283
284
285
<li><a NAME="UTT-TOOFEW"></a>
    <h3>What if I need more nodes than are free?</h3>
    <p>
    For example, say you need 50 nodes but there are only 40 free. In
    general, getting this many nodes is going to require intervention
Jay Lepreau's avatar
spello    
Jay Lepreau committed
286
    from Testbed Operations, if only so we can ask other experimenters
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
    to free up nodes, if possible. Please send us email if you are not
    able to able to get the number of nodes you need for your experiment.
    </p>

    <p>
    Another alternative is to use the
    <a href="tutorial/tutorial.php3#BatchMode">Batch System.</a> 
    If your experiment is amenable to being batched (does not require
    human intervention to start and stop), then you can submit a batch
    request, which will be serviced when enough nodes become
    available.  Typically, you would start out with a few nodes,
    getting used to the batch system and creating whatever scripts are
    needed to make the experiment batchable. Then scale up to larger
    numbers of nodes. Thats the easiest way of getting a lot of nodes!
    </p>    
    
303
<li><a NAME="UTT-2"></a>
304
    <h3>Do I get root access on my nodes?</h3>
305
306
307
308
309
310
    <p>
    Yes. Project leaders get root access to all of the nodes in all of
    the experiments that are running in their project. Project members
    get root if their project leader grants them root access, when the
    leader approves the group <a href="#GS-1">membership request</a>.
    Root privileges are granted via the <code>sudo</code> command. The
311
    <a href="tutorial/tutorial.php3#RootAccess">tutorial</a> describes
312
313
    this in more detail.
    </p>
314
315
316
317
318
319

<li><a NAME="UTT-3"></a>
    <h3>Do my nodes have consoles I can look at?</h3>
    <p>
    Yes. Each of the PCs has its own serial console line that you can
    interact with using the unix <tt>tip</tt> utility. To "tip" to
Mike Hibler's avatar
Mike Hibler committed
320
321
    "pc1" in your experiment, ssh into <b>users.emulab.net</b>, and
    then type <tt>tip pc1</tt> at the unix prompt. You may then
322
323
324
325
    interact with the serial console. The console output is also saved
    for each node so that you may look at it it later. For each node,
    the console log is stored as <tt>/var/log/tiplogs/pcXXX.run</tt>.
    This <em>run</em> file is created when nodes are first allocated
Leigh B. Stoller's avatar
Leigh B. Stoller committed
326
    to an experiment, and the unix permissions of the run files permit
327
328
329
    just members of the project to view them. When the nodes are
    deallocated, the run files are cleared, so if you want to save
    them, you must do so before terminating the experiment. 
330
331
332
333
334
335
336
337
338
339
340
341
342
    </p>
    <p>
    The Sharks also have serial console lines, but because of the
    limited number of serial ports available on <b>users.emulab.net</b>, only
    one Shark, the last or "eighth", on each shelf has a console line
    attached. To tip to that shark, you would type <tt>tip shXX</tt>
    at the unix prompt, where "XX" is the shark shelf number. The
    shark shelf number is the first digit in the name. Using shark
    sh16-8 as an example, the shelf number is sixteen, and the number
    of the node on the shelf is eight.
    </p>

<li><a NAME="UTT-4"></a>
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
    <h3>Can I reboot (power cycle) my nodes?</h3>
    <p>
    Yes. Each of the PCs is independently power controlled. If your
    node becomes wedged, or otherwise unresponsive, you can use the
    <tt>node_reboot</tt> command, as discussed in the
    <a href="tutorial/tutorial.php3#Wedged">Emulab Tutorial.</a>
    </p>
    <p>
    The Sharks are also power controlled, but because of the limited
    number of power ports available, the entire shelf of 8 sharks is
    on a single controller. The <tt>node_reboot</tt> does its best to
    cleanly reboot individual sharks, but if a single shark is
    unresponsive, the entire shelf will be power cycled.
    </p>

358
359
360
361
362
363
364
365
366
367
368
369
<li><a NAME="UTT-SCROGGED"></a>
    <h3>I've scrogged my disk! Now what?</h3>
    <p>
    If you manage to corrupt a disk (or slice), no worries. You can easily
    repair the damage yourself by reloading a fresh copy of the default
    disk image. You will of course lose anything you have stored on that
    disk; it is a good idea to store only data that can be easily recreated, 
    or else store it in your project directory in <tt>/proj</tt>. Disk
    reloading is covered in more detail in the
    <a href="tutorial/tutorial.php3#Scrogged">Emulab Tutorial</a>.
    </p>

370
<li><a NAME="UTT-5"></a>
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
    <h3>Where do I store files needed by my experiment?</h3>
    <p>
    Each project has its own directory, rooted at <tt>/proj</tt>,
    which is available via NFS to all of the nodes in experiments
    running in that project. For example, when the "RON" project was
    created, a directory called /proj/RON was also created. This
    directory is owned by the project creator, and is in the unix
    group "RON." Its permission (mode) is 770; read/write/execute
    permitted by the project creator and by all of the members of the
    project RON, but protected against all access by people outside
    the RON project.
    </p>
    <p>
    Project members are encouraged to store any files needed by their
    experiments in the corresponding /proj project directory. 
    </p>

388
<li><a NAME="UTT-6"></a>
389
390
391
392
393
394
395
396
397
    <h3>Are my files on <b>users.emulab.net</b> backed up (filesaved)?</h3>
    <p>
    Yes. All of the files in your home directory on /users, and all of
    the files in your project directory in /proj are filesaved. While
    we can restore lost files in an emergency, we encourage you to
    back up critical data on your own to avoid (possibly long) delays
    in conducting your experiments.
    </p>
    
398
<li><a NAME="UTT-7"></a>
399
400
401
402
403
404
405
406
407
408
409
410
411
412
    <h3>Are the nodes in my experiment backed up (filesaved)?</h3>
    <p>
    No! The nodes in your experiment are not filesaved. Any changes
    you make to the local filesystems will be lost if the event of a
    disk failure. We plan to provide a mechanism for experimenters to
    create snapshots of their node state, but that is not done yet. In
    the meantime, any files that must not be lost should be stored in
    the project directory (/proj/<project_name>), which is available
    via NFS to all of the nodes in your experiment. You may also store
    files in your home directory (/users/<login>), also available via
    NFS to all of your nodes, but that is not the preferred location
    since quotas on /users are relatively small.
    </p>
    
413
<li><a NAME="UTT-Swapping"></a>
414
    <h3>What is Swapping?</h3>
415
    <p>
Jay Lepreau's avatar
Jay Lepreau committed
416
    Swapping is when you (or we) temporarily swap your experiment out,
417
418
419
420
421
    releasing all of the nodes in the experiment. Your experiment is
    still resident in the Emulab database, and you can see its status
    in the web interface, but no nodes are allocated. Once an
    experiment is swapped out, you can swap it back in via the web
    interface by going to the Experiment Information page for your
422
    experiment, and clicking on the swapin option. 
423
424
    </p>

425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
    <p>
    The <tt>swappable</tt> checkbox in the Begin Experiment web page
    is used to determine what experiments can be <em>automatically</em>
    swapped by the testbed scheduling system. Note that all experiments are
    capable of being swapped; even if you do not check the swappable box,
    you are free to swap your own experiments as you like. The only
    difference is that the testbed scheduling system will not consider your
    experiment when looking for experiments to swap out. You will sometimes
    notice that the Experiment Information page does not contain the swap
    link. That is because experiments cannot be swapped when they are in
    transition. For example, when the experiment is being swapped in (say,
    after first being created) the link will disappear until the experiment
    is fully swapped in, and it is capable of being swapped out. You will
    need to occasionally reload the page so that the updated state is
    recognized and the swap link appears. 
    </p>
    
442
443
444
445
446
447
448
    <p>
    Be aware that we do not currently save any files that you may have
    placed on your nodes. When your experiment is swapped back in, you
    will likely get different nodes, and with fresh copies of the disk
    images. For that reason, you should not swap your experiment out
    unless you make arrangements to save and restore any state you need.
    </p>
Robert Ricci's avatar
Robert Ricci committed
449

450
451
452
453
454
    <p>
    Make sure to take a look at our 
    <a href="docwrapper.php3?docname=swapping.html"> Node Usage Policies</a> 
    as well.
    </p>
Robert Ricci's avatar
Robert Ricci committed
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472

    <li><a NAME="UTT-8"></a>
    <h3>How can I get switch statistics (such as packet counts) for my
    experiment?</h3>
    <p>
    We have a command called <code>portstats</code> that allows you access
    to some of the port counters on our switches. To use it, you'll need
    to ssh to <b>users.emulab.net</b>. '<code>portstats &lt;pid&gt; 
    &lt;eid&gt;</code>' will get you stats for all experimental interfaces in
    your experiment. Run '<code>portstats -h</code>' to get a list of other
    options, such as different sets of stats.
    </p>

    <p>
    Note that the numbers returned by <code>portstats</code> do not get
    reset between experiments.
    </p>
  
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
</ul>

<hr>


<a NAME="HDS"></a>
<h3>Hardware Setup</h3>
<ul>
<li><a NAME="HDS-1"></a>
    <h3>What kind of computers are used for my nodes?
<li><a NAME="HDS-2"></a>
    How many nodes are there?
<li><a NAME="HDS-3"></a>
    How many ethernet cards are on each node?</h3>
    <p>
488
489
490
    Please see the <a href="docwrapper.php3?docname=hardware.html">
    Hardware Overview</a> page for a description and count of the
    computers that comprise the Testbed.
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
    </p>
    
<li><a NAME="HDS-4"></a>
    <h3>How many nodes are currently available (free)?</h3>
    <p>
    If you click on the "Node Reservation Status" link in the menu to
    your left, you will see a summary of the number of nodes (by type)
    that are currently available, followed by a listing of the
    reservation status of each individual node.
    </p>
    
<li><a NAME="HDS-5"></a>
    <h3>Can I do traffic shaping on my links?</h3>
    <p>
    Yes! You can specify the delay, bandwidth, and packet loss rate
    between any two nodes in your topology. Bandwidth and delay are
    specified in the NS <tt>duplex-link</tt> statement, while packet
    loss rate is specified with the Emulab <tt>tb-set-link-loss</tt>
    extension to NS. You may also specify delay, bandwidth, and packet
    loss rate between nodes in a regular LAN.
    </p>
    <p>
513
    Please see the
514
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
515
    page for a summary of all Emulab NS extensions, and the
516
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
517
518
    example. 
    </p>
519
520
521
522
523
524
525
526
527
528
529
530
531

<li><a NAME="HDS-6"></a>
    <h3>Can I modify the traffic shapping parameters on my links?</h3>
    <p>
    Yes! If your NS file specified traffic shaping on a link, then you
    can subsequently modify those parameters after the experiment has
    been swapped in. Note that you cannot <em>convert</em> a non shaped
    link into a shaped link; you can only modify the traffic shaping
    parameters of a link that is already being shaped.  To modify the
    parameters, log into <b>users.emulab.net</b> and use the
    <tt>delay_config</tt> program. This program requires that you know
    the symbolic names of the individual links. This information is
    available via the web interface on the Experiment Information
532
533
    page. The command line syntax for <tt>delay_config</tt> will be
    displayed when the <tt>-h</tt> option is given.
534
    </p>
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561

<li><a NAME="HDS-7"></a>
    <h3>Are there other traffic shapping parameters besides latency,
    bandwidth, and packet loss rate?</h3>
    <p>
    Yes! However, access to those other parameters is slightly more
    difficult since you cannot specify them in your NS file. First
    off, you should log into <tt>users.emulab.net</tt> and read the
    man page for
    <a href='http://www.freebsd.org/cgi/man.cgi?manpath=FreeBSD+4.3-RELEASE'>
    <tt>ipfw</tt></a>. Refer to the section on <tt>dummynet</tt>; ipfw
    is the user interface for the <tt>Dummynet</tt> traffic shaper.
    As noted in previous section above, you can alter the traffic
    shapping parameters of any delayed link (one in which you have
    specified a bandwidth, delay, or PLR that causes a delay node to
    be inserted). However, you will need to log into the delay node
    for the link you wish to modify and interact with <tt>ipfw</tt>
    directly. The easiest approach would be to make a copy of
    </tt>/etc/testbed/rc.delay</tt> and edit the pipe commands as
    desired (or replace the pipe commands with "queue" commands).  The
    pipe commands are indexed by number; the mapping from pipe number
    to virtual link is available via the web interface on the
    Experiment Information page for your experiment.  Be sure to leave
    the rest of the contents of the file as is. Once you have your
    changes made, simply execute the file using the <a href="#UTT-2">
    sudo</a> command.
    </p>    
562
563
564
565
566
567
568
569
570
571
</ul>

<hr>

<a NAME="SWS"></a>
<h3>Software Setup</h3>
<ul>
<li><a NAME="SWS-1"></a>
    <h3>What OS do the nodes run?</h3>
    <p>
572
573
574
    Please see the <a href="docwrapper.php3?docname=software.html">
    Software Overview</a> page for a description of the Operating
    Systems that can be run on each of the Testbed nodes.
575
576
577
578
579
580
581
582
583
584
585
586
    </p>

<li><a NAME="SWS-2"></a>
    <h3>How do I select which OS to run on each node?</h3>
    <p>
    When a choice of OS is available, you may specify which one you
    prefer for each node in the NS file using the Emulab
    <tt>tb-set-node-os</tt> extension to NS. When your experiment is
    configured, the appropriate disk image will be loaded on your
    nodes, and the selected operating system will boot up on each.
    </p>
    <p>
587
    Please see the
588
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
589
    page for a summary of all Emulab NS extensions, and the
590
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
    example. 
    </p>    
    
<li><a NAME="SWS-3"></a>
    <h3>Can I load my own software (RPMs) on my nodes?</h3>
    <p>
    Yes! If have an RPM (or more than one) that is appropriate for
    loading on the OS you have selected, you can arrange to have them
    loaded automatically when your experiment is configured. The
    Emulab NS extension <tt>tb-set-node-rpms</tt> is used in the NS
    file to specify a list of RPMS to install. You may specify a
    different list for each node in the experiment. When the node
    first boots after the experiment is configured, each of the RPMs
    will be installed (but only RPMs that have not already been
    installed). 
    </p>
    <p>
608
    Please see the
609
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
610
    page for a summary of all Emulab NS extensions, and the
611
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
612
613
    example. 
    </p>    
614
    
615
616
617
618
619
620
621
622
623
624
625
626
627
628
<li><a NAME="SWS-4"></a>
    <h3>Can I schedule programs to run automatically when a node boots?</h3>
    <p>
    Yes! You can arrange to run a single program or script when your
    node boots. The script is run as the UID of the experiment
    creator, and is run after all other node configuration (including
    RPM installation) has completed. The exit status of the script (or
    program) is reported back and is made available for you to view in
    Experiment Information link in the menu at your left. The Emulab
    NS extension <tt>tb-set-node-startup</tt> is used in the NS file
    to specify the path of the script (or program) to run. You may
    specify a different program for each node in the experiment.
    </p>
    <p>
629
    Please see the
630
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
631
    page for a summary of all Emulab NS extensions, and the
632
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
633
634
    example.
    </p>    
635

636
<li><a NAME="SWS-5"></a>
637
638
639
    <h3>How can I turn on routing or set up routes automatically 
    in my nodes?</h3>
    <p>
640
    By default, we do not setup any static routes or run any routing daemon
641
    on nodes in an experiment.  However, we do provide several options for
Jay Lepreau's avatar
spello    
Jay Lepreau committed
642
    experimenters, which are described in the
643
644
645
    <a href="tutorial/tutorial.php3#Routing">"Setting up IP routing
    between nodes"</a> section of the
    <a href="tutorial/tutorial.php3">Emulab Tutorial.</a>
646
647
648
    </p>
    
<li><a NAME="SWS-6"></a>
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
    <h3>How does my software determine when other nodes in my
    experiment are ready?</h3>
    <p>
    If your application requires synchronization to determine when all
    of the nodes in your experiment have started up and are ready to
    proceed, then you can use the Testbed's <i>ready bits</i>
    mechanism. The ready bits are really just a way of determining how
    many nodes have issued the <b>ready</b> command, and is returned
    to the application as a simple N of M string, where N is the
    number that have reported in, and M is the total number of nodes
    in the experiment. Applications can use this as a very simplistic
    form of barrier synchronization, albeit one that can be used just
    once and one that does not actually block!
    </p>
    <p>
    Use of the ready bits is described in more detail in the <a href =
665
    "tutorial/tutorial.php3">Emulab Tutorial</a> and in the <a href =
666
    "doc/docwrapper.php3?docname=tmcd.html"> Testbed Master Control
667
    Daemon</a> documentation.
668
669
    </p>
    
670
<li><a NAME="SWS-7"></a>
671
672
    <h3>Can I run my own Operating System?</h3>
    <p>
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
    Yes! You can run your own OS (or a customized version of an Emulab
    supported OS) on any of the PCs. You can also run <a href =
    "http://www.cs.utah.edu/flux/oskit/">OSKit</a> kernels on the
    PCs. Each of the PCs is partitioned with two DOS partitions large
    enough to hold the typical OS installation. The 1st and 2nd
    partitions are each 3GB. The 3rd partition is 500MB, and is
    labeled as Linux Swap. The 4th partition is the remainder of the
    disk, and varies in size depending on the pc type. We recommend
    that you use the 1st or 2nd partition; using the 4th partition
    will restrict the number of machines that you can run your OS on
    since it varies in size. Note that you must leave the MBR (Master
    Boot Record) in sector 0 alone, and that your custom partition
    must contain a proper DOS boot record in the first sector.
    </p>

    <p>
    Please note that while users are free to customize their disks and
    install their own operating systems, Emulab staff will not be able
    to offer more than encouragement and advice! We cannot install the
    OS for you, and we cannot load CDROMS, floppy disks, or tape drives!
    We <em>do</em> provide an easy way for you to boot FreeBSD from a
    memory based filesystem (MFS) so that you can more easily work
    with the disk (in case it is not possible to install your OS on a
    live disk). Beyond that, you are pretty much on your own!
    </p>

    <p>
    Many users had great success with customizing an Emulab supported
    OS (FreeBSD or Linux), and then creating a disk image that is
    autoloaded when the experiment is swapped in. We strongly
    encourage people to use this approach whenever possible! There is
    more information available in the <a
    href="tutorial/tutorial.php3#CustomOS">Custom OS</a> section of
    the <a href = "tutorial/tutorial.php3">Emulab Tutorial.</a>
707
    </p>
708

709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
<li><a NAME="SWS-8"></a>
    <h3>What if I need more disk space on my nodes?</h3>
    <p>
    Each node has a partition at the end of the disk that you can use if
    you wish. In Linux, the partition is <code>/dev/hda4</code> ; in FreeBSD,
    it's </code>/dev/ad0s4</code> . There is no filesystem on this partition,
    so you'll need to create it yourself. For example, in Linux:
    <blockquote>
	<code>
	    mkfs /dev/hda4;<br>
	    mount /dev/hda4 /mnt;
	</code>
    </blockquote>
    This partition is only 6 Gigs, the size of the leftover space on our
    smallest drives. If you need more space than this, it would be possible
    to enlarge this partition on some machines (for example, our pc850s
    have 40 GB disks,) but that is outside the scope of this FAQ.
    </p>

728
729
730
731
<li><a NAME="SWS-9"></a>
    <h3>Are there testbed-specific daemons that could interfere with my
	experiment?</h3>
    <p>
732
    By default, the testbed startup scripts currently start two daemons in 
733
734
735
736
737
738
    addition to the OS's standard set.  Other daemons may be started depending
    on the network services you ask for in your ns file (see below).
    </p>

    <p>
    Unconditionally started daemons:
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
    </p>

    <p>
    <blockquote>
    <li><code>healthd</code> - A low overhead hardware health monitor.
    </blockquote>
    </p>

    <p> 
    This deamon periodically polls the machine's health monitoring
    hardware and sends this information back to our <code>boss</code>
    node for analysis.  The hardware is polled once per second, and a
    status datagram is sent out once every five minutes.
    <code>Healthd</code>'s overhead is quite low, but it can be safely
    killed and disabled from startup if you're worried about possible
754
755
    side effects.  It is started by
    <code>/etc/testbed/rc.healthd</code>.
756
    </p>
757
758
759
760
761
762
763
764
765
766

    <p>
    <blockquote>
    <li><code>slothd</code> - A low overhead usage analysis tool.
    </blockquote>
    </p>

    <p>
    <code>Slothd</code> is important to efficient testbed utilization
    and should run on every node whenever possible.  Its overhead is
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
    almost negligible (essentially less than running <code>'ls -l
    /dev'</code> once per hour), and should not interfere with your
    work.  However if your experiment is exceptionally sensitive, then
    you may arrange with us to disable <code>slothd</code>.  Please
    note that we will restart this daemon if it is not running unless
    prior arrangements have been made.
    </p>

    <p>
    Conditionally started daemons:
    </p>

    <p>
    <blockquote>
    <li><code>gated</code> - A network routing daemon.
    </blockquote>
783
784
    </p>

785
786
787
788
789
    <p>
    If you have requested automatic routing on your nodes with the 
    <code>tb-set-ip-routing</code> command in your NS file, this will
    start <code>gated</code> on all of your nodes.
    </p>
790

791
792
793
794
795
796
797
798
    <p>
    We have left all daemons started by the operating systems' default
    configurations (such as <code>cron</code>) enabled, so you should also
    look at them if you are concered about running processes affecting
    your experiment.
    </p>
</ul>

799
800
801
802
803
804
805
806
<hr>

<a NAME="SEC"></a>
<h3>Security Issues</h3>
<ul>
<li><a NAME="SEC-1"></a>
    <h3>Is Emulab Firewalled?</h3>
    <p>
807
808
    Yes. Emulab blocks all of the <i>low numbered</i> ports (ports below 1024),
    with the exception of ports 20 and 21 (FTP), 22 (Secure Shell), and 80
Jay Lepreau's avatar
Jay Lepreau committed
809
    (HTTP). This is for the protection of experimenters, as well as to ensure
810
811
812
813
    that an errant application cannot become the source of a Denial of Service
    attack to sites outside of Emulab. If your application requires external
    access to other low numbered ports, please contact us to make special
    arrangements.
814
815
    </p>
</ul>
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837

<hr>

<a NAME="TR"></a>
<h3>Troubleshooting</h3>
<ul>
<li><a NAME="TR-1"></a>
    <h3>My experiment is setup, but I cannot
	send packets between some of the nodes, why?</h3>
    <p>
    The most common reason is that your topology
    includes nodes which are not directly connected, and you have
    not setup any routing.  Refer to
    "<a href="#SWS-5">How can I turn on routing or set up routes
    automatically in my nodes?</a>" for details.  If you cannot
    send packets between two machines which are directly connected
    (via a link or a lan), then there are two possibilities:
    either the nodes did not properly negotiate their speed and
    duplex with the Cisco switch, or the physical wire is loose
    or bad.  In these cases, you should contact us for help.
    </p>
</ul>