faq.html 60.5 KB
Newer Older
Leigh B. Stoller's avatar
Leigh B. Stoller committed
1
2
<!--
   EMULAB-COPYRIGHT
3
   Copyright (c) 2000-2003 University of Utah and the Flux Group.
Leigh B. Stoller's avatar
Leigh B. Stoller committed
4
5
   All rights reserved.
  -->
6
<center>
7
<h2>Frequently Asked Questions</h2>
8
9
10
</center>

<h2>Contents</h2>
11
<ul>
12
13
<li> <a href="#GS">Getting Started</a>
     <ul>
14
     <li> <a href="#GS-Eligible">Who is Eligible to use Emulab.Net?</a>
15
16
     <li> <a href="#GS-1">How do I start a project?</a>
     <li> <a href="#GS-2">How do I join a project?</a>
17
18
19
20
     <li> <a href="#GS-2b">I'm a project leader, and someone applied
	                   to join my project, but they're not on the 
	                   list to be approved.</a>
     <li> <a href="#GS-2c">Will Emulab send me email messages?</a>
21
     <li> <a href="#GS-3">I have an Emulab account. Now what?</a>
22
     <li> <a href="#GS-PATH">Do I need to change my PATH variable?</a>
23
24
     <li> <a href="#GS-4">Can I be in more than one project?</a>
     <li> <a href="#GS-5">Can I change my Emulab password?</a>
25
     <li> <a href="#GS-6">I'm a project leader. Can I designate TAs?</a>
26
27
     <li> <a href="#GS-7a">How do I report a specific problem?</a>
     <li> <a href="#GS-7">Where do I get more help?</a>
28
29
30
31
32
     </ul>

<li> <a href="#UTT">Using the Testbed</a>
     <ul>
     <li> <a href="#UTT-1">Is there a tutorial?</a>
33
34
     <li> <a href="#UTT-NETBUILD">Do you have a <b>GUI</b> to help me
                                   create experiments?</a>
35
     <li> <a href="#UTT-TOPO">Are there any constraints on my topology?</a>
36
     <li> <a href="#UTT-NODES">How many nodes can I ask for?</a>
37
     <li> <a href="#UTT-HOWLONG">How long can I keep using my nodes?</a>
38
     <li> <a href="#UTT-TOOFEW">What if I need more nodes than are free?</a>
39
     <li> <a href="#UTT-2">Do I get root access on my nodes?</a>
40
     <li> <a href="#UTT-3">Do my nodes have consoles I can look at?</a>
Chad Barb's avatar
Chad Barb committed
41
42
     <li> <a href="#UTT-TUNNEL">How do I connect directly to node consoles, 
                                without going through <b>users</b>?</a>
43
     <li> <a href="#UTT-4">Can I reboot (power cycle) my nodes?</a>
44
     <li> <a href="#UTT-SCROGGED">I've scrogged my disk! Now what?</a>
45
46
     <li> <a href="#UTT-5">Where do I store files needed by my experiment?</a>
     <li> <a href="#UTT-6">Are my files on <b>users.emulab.net</b>
47
                           backed up (filesaved)?</a>
48
     <li> <a href="#UTT-7">Are the nodes in my experiment backed up
49
                           (filesaved)?</a> 
50
     <li> <a href="#UTT-Swapping">What is Swapping?</a> 
51
     <li> <a href="#UTT-Restart">What is Experiment Restart?</a> 
Robert Ricci's avatar
Robert Ricci committed
52
53
     <li> <a href="#UTT-8">How can I get switch statistics (such as packet
                           counts) for my experiment?</a> 
54
55
     <li> <a href="#UTT-Naming">What names should I use to refer to the
			   nodes in my experiment?</a>
Chad Barb's avatar
   
Chad Barb committed
56
57
     <li> <a href="#UTT-Modify">Can I modify my experiment 
                           after creating it?</a> 
58
59
     <li> <a href="#UTT-SWPaths">Are there FreeBSD and/or Linux sources and 
                           packages available locally?</a>
60
61
62
63
64
     </ul>

<li> <a href="#HDS">Hardware setup</a>
     <ul>
     <li> <a href="#HDS-1">How many nodes are there?</a>
65
66
     <li> <a href="#HDS-2">How many nodes are currently available?</a>
     <li> <a href="#HDS-3">How many ethernet cards are on each node?</a>
67
68
     <li> <a href="#HDS-4">How many nodes are currently available (free)?</a>
     <li> <a href="#HDS-5">Can I do traffic shaping on my links?</a>
Jay Lepreau's avatar
Jay Lepreau committed
69
     <li> <a href="#HDS-6">Can I modify the traffic shaping
70
                           parameters on my links?</a>
Jay Lepreau's avatar
Jay Lepreau committed
71
     <li> <a href="#HDS-7">Are there other traffic shaping parameters
72
                           besides latency, bandwidth, and PLR?</a>
73
74
75
76
77
     </ul>

<li> <a href="#SWS">Software setup</a>
     <ul>
     <li> <a href="#SWS-1">What OS do the nodes run?</a>
78
79
80
     <li> <a href="#SWS-2">How do I select which OS to run on each node?</a>
     <li> <a href="#SWS-3">Can I load my own software (RPMs) on my nodes?</a>
     <li> <a href="#SWS-4">Can I schedule programs to run
81
82
83
84
                           automatically when a node boots?</a>
     <li> <a href="#SWS-5">How can I turn on routing or set up routes
			   automatically in my nodes?</a>
     <li> <a href="#SWS-6">How does my software determine when other
85
                           nodes in my experiment are ready?</a>
86
     <li> <a href="#SWS-7">Can I run my own Operating System?</a>
87
88
     <li> <a href="#MovingImage">Can I share a disk image between two
                           projects?</a>
89
     <li> <a href="#SWS-8">What if I need more disk space on my nodes?</a>
90
91
     <li> <a href="#SWS-9">Are there testbed-specific daemons that could
                           interfere with my experiment?</a>
Mac Newbold's avatar
Mac Newbold committed
92
     <li> <a href="#SWS-10">Does Emulab support IP Multicast?</a>
93
     </ul>
94
95
96
97
98

<li> <a href="#SEC">Security Issues</a>
     <ul>
     <li> <a href="#SEC-1">Is Emulab Firewalled?</a>
     </ul>
99
100
101

<li> <a href="#TR">Troubleshooting</a>
     <ul>
102
     <li> <a href="#TR-1">My experiment is set up, but I cannot
103
104
105
106
107
108
109
110
     			  send packets between some of the nodes. Why?</a>
     <li> <a href="#TR-2">I asked for traffic shaping, but everything
			  seems to be going at full LAN speeds. 
			  What's wrong?</a>
     <li> <a href="#TR-3">I set a non-zero packet-loss (or delay) but 'ping'
                          shows no packet-loss (or delay). Why?</a>
     <li> <a href="#TR-4">I set a non-zero packet-loss (or delay) but 'ping'
                          shows a different packet-loss (or delay). Why?</a>
111
     </ul>
112
113
114
</ul>

<hr>
115

116
<a NAME="GS"></a>
Chad Barb's avatar
   
Chad Barb committed
117
<font size='+1'><b>Getting Started</b></font>
118
<ul>
119
<li><a NAME="GS-Eligible"></a>
Chad Barb's avatar
   
Chad Barb committed
120
    <font size='+1'><b>Who is Eligible to use Emulab.Net?</b></font>
121
    <p>
122
123
124
125
126
127
128
129
    In principle, almost any research or educational use
    by those that have a need for it is appropriate and encouraged.
    This includes use by universities, industrial research labs, and both
    US and non-US institutions.   With some provisos, use for development
    and evaluation is also acceptable, even by companies.
    See our <a href ="docwrapper.php3?docname=policies.html">posted policies</a>
    for more detail.  If you are unsure about your eligibility to use
    Netbed/Emulab, please just send us an email inquiry.
130
131
    </p>

132
<li><a NAME="GS-1"></a>
Chad Barb's avatar
   
Chad Barb committed
133
    <font size='+1'><b>How do I start a project?</b></font>
134
135
    <p>
    If you are new to the Testbed, simply click on the "Start Project"
136
    link on the Emulab <a href="http://www.emulab.net">Home
137
138
139
140
    Page</a>. You will need to fill in the forms with your personal
    information and information about the project. Then click on the
    "Submit" button. Within a few days you will be contacted via email
    with an approval message. More information about starting projects
141
142
    can be found in <a href="docwrapper.php3?docname=auth.html">
    Authorization Page</a>.
143
    </p>
144
145
146
147
148
149
150
    <p>
    If you already have an Emulab account, and wish to start a second
    project, first log into the Web Interface. Then select the "Start
    Project" link; all of the personal information will already be
    filled in. You will need to complete just the project information
    section. 
    </p>
151

152
<li><a NAME="GS-2"></a>
Chad Barb's avatar
   
Chad Barb committed
153
    <font size='+1'><b>How do I join a project?</b></font>
154
155
    <p>
    If you are new to the Testbed, simply click on the "Join Project"
156
    link on the Emulab <a href="http://www.emulab.net">Home
157
158
159
    Page</a>. You will need to fill in the form with your personal
    information, and provide the name of the project you are trying to
    join (typically, the <i>Project Leader</i> will have told you the
160
    name of the project). Then click on the "Submit" button, and wait
161
162
163
    for an email with your new user key. When that email arrives, use
    the link in it (or the key itself), and use it with your password 
    to log into the web site and verify your account. Then just wait
164
165
166
    for the project leader to approve you.  When approved you will
    receive an email message saying so, and you can then log into the
    Testbed.
167
168
    </p>

169
<li><a NAME="GS-2b"></a>
Chad Barb's avatar
   
Chad Barb committed
170
171
172
    <font size='+1'><b>I'm a project leader, 
    and someone applied to join my project,
    but they're not on the list to be approved.</b></font>    
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
    <p>
      Joining a project has 3 stages. The first two are done by
      the person trying to join, and they both must be completed
      before you can approve their application. The first two are
      outlined <a href="#GS-2">in the previous question</a>, where the
      user fills out the "Join Project" form, and performs account
      verification. After these two steps are both complete, the
      project leader and any group leaders in the group 
      (<a href="#GS-6">More info here</a>) will get an email saying
      the account is ready to be approved, and it will appear on the
      list of new users waiting to be approved.
    <p>
      If someone says they've applied, but you haven't received an
      email from Emulab about it, and they don't show up on your
      list, the most likely cause is that they haven't finished the
      verification step.
    </p>

<li><a NAME="GS-2c"></a>
Chad Barb's avatar
   
Chad Barb committed
192
    <font size='+1'><b>Will Emulab send me email messages?</b></font>
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
    <p>
      Yes! Emulab uses email notifications to you in several different
      ways. Often it will send you a copy of information regarding
      experiments you set up, applications to projects, and other
      things you do at Emulab. Sometimes (like with account
      verification) the email is a critical part of being able to use
      Emulab.
    <p>
      For those reasons it is <b>critical</b> that any spam filtering
      software you have accept email from Emulab itself (anything
      coming from the emulab.net domain) and from Emulab staff (from
      the cs.utah.edu or flux.utah.edu domains). In many cases, it may
      also say that is from a specific machine in those domains. Our
      messages usually do not get flagged as spam by most filters, but
      in certain cases it can be a problem. It is also important that
      it not require manual intervention or confirmation to get emails
      through to you, so programs like SpamKiller can cause problems.
    <p>
      It is also important to read your email often while you are
      using Emulab, especially while you have machines reserved in 
      an experiment. A few emails may be the only notification you'll
      get before we swap out an experiment that appears to be idle,
      and if you don't respond, you may lose important work. Email is
      also our method for informing you about problems we may be
      experiencing, downtimes, or other important announcements. Your
      experience with Emulab (and ours with you) will be much more
      pleasant if everyone is responsive to email.
    </p>

222
<li><a NAME="GS-3"></a>
Chad Barb's avatar
   
Chad Barb committed
223
    <font size='+1'><b>I have an Emulab account. Now what?</b></font>
224
225
226
227
228
229
    <p>
    Once you have been approved to start (or join) your first project,
    you will be able to log into Emulab's user machine,
    <b>users.emulab.net</b>. We require that all Emulab users use ssh. For
    example, if your Emulab account name is "joe", then you would do:
    <pre>
230
	ssh users.emulab.net -l joe		</pre>
231
232
    </p>
    <p>
233
234
    Your password is the same as the password you supplied to the
    Start (or Join) Project web page.
235
236

<li><a NAME="GS-PATH"></a>
Chad Barb's avatar
   
Chad Barb committed
237
238
    <font size='+1'><b>Do I need any 
    special directories in my PATH variable?</b></font>
239
240
241
242
    <p>
    There are several useful (although not required) programs installed on
    <tt>users.emulab.net</tt> in <tt>/usr/testbed/bin</tt>. You should edit
    your dot files to include that directory in your search path. 
243
244
245
    </p>

<li><a NAME="GS-4"></a>
Chad Barb's avatar
   
Chad Barb committed
246
    <font size='+1'><b>Can I be in more than one project?</b></font>
247
248
    <p>
    Yes. You may join (and/or start) as many projects as you like,
249
250
    subject to Emulab <a href="docwrapper.php3?docname=policies.html">
    administrative policies</a>.
251
252
253
    </p>

<li><a NAME="GS-5"></a>
Chad Barb's avatar
   
Chad Barb committed
254
    <font size='+1'><b>Can I change my Emulab password?</b></font>
255
256
257
    <p>
    Yes. You can change your Emulab Web password and your Emulab login
    password (the password you use to log into <b>users.emulab.net</b>, as
258
    well as nodes in your experiments). To change your password,
259
    simply click on the "Update User Information" in the menu to your
260
261
262
263
    left, and then enter your new password in the location provided.
    Your new password will be installed on <b>users.emulab.net</b>
    immediately. Your experimental nodes will get the new password
    when they reboot.
264
265
266
    </p>

<li><a NAME="GS-6"></a>
Chad Barb's avatar
   
Chad Barb committed
267
    <font size='+1'><b>I'm a project leader. Can I designate TAs?</b></font>
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
    <p>
    Yes. To designate a TA, you must first create a project
    <em>group</em>. A project group is a lot like a unix group, and
    in fact unix groups is the mechanism used to protect members of
    one group from members of another group. When you create a group,
    you designate a <em>group leader</em> who is responsible for
    approving users who apply to join the group. Group leaders may
    also terminate experiments that have been created by members of
    the group. As Project Leader, you may also shift members of your
    project in and out of your project's groups as you like, and you
    are automatically a member of all groups within your project. As a
    convenience, all new projects are created with one new group,
    termed the <em>default group</em>. As its name implies, whenever
    the group is left unspecified in a form, it defaults to the
    project group (this allows you to create a project without any sub
    groups at all; new members join the default group, new experiments
    are created in the default group, etc.).
    </p>

    <p>
    Project groups are created via the Project Information link at
    your left. Simply go to the project page in which you want to
    create a group, and look for the "Create New Group" link. More
    information on project groups is available via the <a
    href="doc.php3">Emulab Documentation</a> page in the
    <a href="docwrapper.php3?docname=groups.html">Groups Tutorial</a>.
    </p>

296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
<li><a NAME="GS-7a"></a>
    <font size='+1'><b>How do I report a specific problem?</b></font>
    <p>
    Don't hesitate to send us <a href="emailus.php3">email</a>!
    <p>
    Ok, hesitate just a little and read the rest of this entry first.
    Before sending email, be sure to check out the
    <a href="#TR">Troubleshooting</a>
    entry which describes several common problems and possible causes.
    If you do send email, there are several pieces of information you
    should include to make our job easier.
    <ul>
    <li> The affected <b>project and experiment name</b> is pretty obvious,
    but some people forget.
    <li> The particular <b>nodes, OSes, and programs</b>
    involved will help us zero in on the problem more quickly.
    <li> The <b>time</b> at which the problem occurred.
    We need this to correlate with our various log files.
    <li> Any <b>relevant observations and actions</b> you have attempted.
    Did the problem only happen once?  Is it reproducible?  Did restarting
    a program or rebooting a node help?
    </ul>
    After sending email,
    <em>please do not swap or terminate the experiment</em>, if possible.
    It can be a lot harder to track down a problem after the experiment
    is gone.  Sometime you may need to terminate it, for example it is
    in the middle of the night our time and you really need the nodes
    that are tied up in the experiment.  But at least give us 15 minutes
    to respond to your message before acting, and then let us know that
    you did swap/terminate it.
    </p>

328
<li><a NAME="GS-7"></a>
329
    <font size='+1'><b>Where do I get more help?</b></font>
330
331
    <p>
    If you cannot find an answer to your question in the
332
    <a href="doc.php3">Emulab Documentation</a>, then you can
333
    send us an <a href="emailus.php3">email message</a>. We will try
334
335
    to answer your question as quickly as we can.
    </p>
336
</ul>
337

Leigh B. Stoller's avatar
Leigh B. Stoller committed
338
<hr>
339
340

<a NAME="UTT"></a>
Chad Barb's avatar
   
Chad Barb committed
341
<font size='+1'><b>Using the Testbed</b></font>
342
343
<ul>
<li><a NAME="UTT-1"></a>
Chad Barb's avatar
   
Chad Barb committed
344
    <font size='+1'><b>Is there a tutorial?</b></font>
345
    <p>
346
    Yes, we have an extensive <a href="tutorial/tutorial.php3">tutorial</a>
347
348
349
    on using the Testbed.
    </p>

Chad Barb's avatar
Chad Barb committed
350
<li><a NAME="UTT-NETBUILD"></a>
Chad Barb's avatar
   
Chad Barb committed
351
352
    <font size='+1'><b>Do you have a GUI to 
    help me create experiments?</b></font>
353
354
355
356
357
358
359
360
361
362
363
364
365
366
    <p>
    Yes, we provide a GUI that gives you an easy to use drawing
    palette on which you can place nodes, lans, and links. Testbed
    specific attributes such as operating system, hardware type, and
    link/lan characteristics, may be attached to each object. With a
    single click, you can instantiate your new topology on the Testbed
    as an experiment in one of your projects. Alternatively, you can
    save the auto-generated NS file on your machine, edit as required,
    and then submit it later when creating an experiment.
    </p>
    
    <p>
    To access the GUI, please log in and go to the Begin Experiment page.
    <em>Note: you need a Java compliant browser.</em>
Chad Barb's avatar
Chad Barb committed
367
368
    </p>

369
<li><a NAME="UTT-TOPO"></a>
Chad Barb's avatar
   
Chad Barb committed
370
    <font size='+1'><b>Are there any constraints on my topology?</b></font>
371
372
373
374
375
376
377
378
379
380
381
382
383
384
    <p>
    Yes, but only those imposed by the physical hardware that is
    currently available in our testbed. The constraints that people
    most commonly run into are the maximum speed of our links
    (100Mbps) and the maximum number of network interface cards (NICs)
    in our machines. You can't get any links faster than 100Mbps,
    since we don't yet have gigabit links for experimenters to
    use. Our nodes each have 4 experimental network interfaces, so
    each node can be a member of up to 4 links or LANs. A good
    strategy for making your topology fit within those limits is to
    replace multiple links to a node with a LAN or with a router
    node. Ask Testbed Operations if you need further assistance.
    </p>

385
<li><a NAME="UTT-NODES"></a>
Chad Barb's avatar
   
Chad Barb committed
386
    <font size='+1'><b>How many nodes can I ask for?</b></font>
387
388
389
390
391
    <p>
    You can ask for as many nodes as are currently available! You can
    click on the "Node Reservation Status" link at your left to see
    how many nodes are currently free. If you ask for more than are
    currently available, your experiment will be rejected (you
Jay Lepreau's avatar
Jay Lepreau committed
392
    will receive email notification shortly after you submit your NS
393
394
395
396
397
398
399
400
401
    file to the web interface).
    </p>

    <p>
    <em>We urge all new Emulab users to begin with a small 3-4 node
    experiment so that you will become familiar with NS syntax and the
    practical aspects of Emulab operation.</em>
    </p>

402
<li><a NAME="UTT-HOWLONG"></a>
Chad Barb's avatar
   
Chad Barb committed
403
    <font size='+1'><b>How long can I keep using my nodes?</b></font>
404
405
406
407
408
409
410
411
412
413
414
415
416
417
    <p>
    You can keep them as long as you need them, subject to our <a
    href="docwrapper.php3?docname=swapping.html">Node Usage
    Policies</a>.  In general, you should do your work, and then
    terminate your experiment as soon as you're done with it. If
    you're not done with it, but are through for a while, you should
    probably "swap out" your experiment (See the question <a
    href="#UTT-Swapping">What is Swapping</a> in this FAQ). It is
    especially important to swap out your experiment if you're through
    with it for the weekend. Emulab usually gets heavy use on the
    weekends by users who need to make very large experiments, so it
    is important to leave as many nodes available as possible.
    </p>

418
<li><a NAME="UTT-TOOFEW"></a>
Chad Barb's avatar
   
Chad Barb committed
419
    <font size='+1'><b>What if I need more nodes than are free?</b></font>
420
421
422
    <p>
    For example, say you need 50 nodes but there are only 40 free. In
    general, getting this many nodes is going to require intervention
Jay Lepreau's avatar
spello    
Jay Lepreau committed
423
    from Testbed Operations, if only so we can ask other experimenters
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
    to free up nodes, if possible. Please send us email if you are not
    able to able to get the number of nodes you need for your experiment.
    </p>

    <p>
    Another alternative is to use the
    <a href="tutorial/tutorial.php3#BatchMode">Batch System.</a> 
    If your experiment is amenable to being batched (does not require
    human intervention to start and stop), then you can submit a batch
    request, which will be serviced when enough nodes become
    available.  Typically, you would start out with a few nodes,
    getting used to the batch system and creating whatever scripts are
    needed to make the experiment batchable. Then scale up to larger
    numbers of nodes. Thats the easiest way of getting a lot of nodes!
    </p>    
    
440
<li><a NAME="UTT-2"></a>
Chad Barb's avatar
   
Chad Barb committed
441
    <font size='+1'><b>Do I get root access on my nodes?</b></font>
442
443
444
445
446
447
    <p>
    Yes. Project leaders get root access to all of the nodes in all of
    the experiments that are running in their project. Project members
    get root if their project leader grants them root access, when the
    leader approves the group <a href="#GS-1">membership request</a>.
    Root privileges are granted via the <code>sudo</code> command. The
448
    <a href="tutorial/tutorial.php3#RootAccess">tutorial</a> describes
449
450
    this in more detail.
    </p>
451
452

<li><a NAME="UTT-3"></a>
Chad Barb's avatar
   
Chad Barb committed
453
    <font size='+1'><b>Do my nodes have consoles I can look at?</b></font>
454
    <p>
455
456
457
458
    Yes. Each of the PCs has its own serial console line with which you can
    interact, either directly from your desktop (see next FAQ entry),
    or by hopping through the "users" machine,
    using our <tt>console</tt> program. To connect over serial line to
Mike Hibler's avatar
Mike Hibler committed
459
    "pc1" in your experiment, ssh into <b>users.emulab.net</b>, and
460
461
462
463
464
465
    then type <tt>console pc1</tt> at the Unix prompt. You may then
    interact with the serial console (hit "enter" to elicit output from
    the target machine).
    </p><p>
    In any case, all console output from each node is saved
    so that you may look at it it later. For each node,
466
467
    the console log is stored as <tt>/var/log/tiplogs/pcXXX.run</tt>.
    This <em>run</em> file is created when nodes are first allocated
468
469
    to an experiment, and the Unix permissions of the run files permit
    only members of the project to view them. When the nodes are
470
471
    deallocated, the run files are cleared, so if you want to save
    them, you must do so before terminating the experiment. 
472
473
474
475
476
    </p>
    <p>
    The Sharks also have serial console lines, but because of the
    limited number of serial ports available on <b>users.emulab.net</b>, only
    one Shark, the last or "eighth", on each shelf has a console line
477
478
    attached. To connect to that shark, you would type <tt>console shXX</tt>
    at the Unix prompt, where "XX" is the shark shelf number. The
479
480
481
482
483
    shark shelf number is the first digit in the name. Using shark
    sh16-8 as an example, the shelf number is sixteen, and the number
    of the node on the shelf is eight.
    </p>

Chad Barb's avatar
Chad Barb committed
484
<li><a NAME="UTT-TUNNEL"></a>
Chad Barb's avatar
   
Chad Barb committed
485
486
    <font size='+1'><b>How do I connect directly to node consoles, 
        without going through <b>users</b>?</b></font>
Chad Barb's avatar
Chad Barb committed
487
488
489
    <p>
    Clicking "Connect to Serial Line"
    in the Node Options page will send your browser a "text/x-testbed-acl"
Chad Barb's avatar
   
Chad Barb committed
490
491
492
493
494
495
496
    ".tbacl" file. 
    In windows, if you have installed <code>tiptunnel</code>, available
    below, you can save this file in a folder and double-click it
    to launch a tunneled connection to your node.
    In FreeBSD or Linux, you can save the file and pass it as an argument
    to <code>tiptunnel</code>, or associate it with
    <code>tiptunnel</code> in your web browser.
497
498
    Upon connection you typically first have to hit "enter" to
    elicit output from the target machine.
Chad Barb's avatar
   
Chad Barb committed
499
500
501

<!--
    If you have downloaded <code>tiptunnel</code> and set it as the 
Chad Barb's avatar
Chad Barb committed
502
503
504
505
506
    handler for that MIME type, <code>tiptunnel</code> will launch a new 
    telnet running in a new xterm (this may take a few seconds.) 
    That telnet will be connected to a local port, 
    which is tunneled through SSL to your node's console. 
    Closing the xterm, exiting telnet, or killing <code>tiptunnel</code>
Chad Barb's avatar
   
Chad Barb committed
507
508
    itself will end the connection. -->
    
Chad Barb's avatar
Chad Barb committed
509
    </p>
Chad Barb's avatar
   
Chad Barb committed
510
511
512
    <ul>
    <li>
    You can download the <code>tiptunnel</code>
513
    <a href="downloads/tiptunnel-win32.exe">installer for Windows here</a>.
Chad Barb's avatar
   
Chad Barb committed
514
515
    </li>
    <li>
Chad Barb's avatar
   
Chad Barb committed
516
517
    You can download the <code>tiptunnel</code> statically-linked x86
    <a href="downloads/tiptunnel-freebsd.tar.gz">binary for FreeBSD here</a>.
Chad Barb's avatar
   
Chad Barb committed
518
519
    </li>
    <li>
Chad Barb's avatar
   
Chad Barb committed
520
521
    You can download the <code>tiptunnel</code> statically-linked x86
    <a href="downloads/tiptunnel-linux.tar.gz">binary for Linux here</a>.
Chad Barb's avatar
   
Chad Barb committed
522
523
524
525
526
    </li>   
    <li>
    A source distribution will be available soon.
    </li>
    </ul>
527

Chad Barb's avatar
   
Chad Barb committed
528
    <font size='+1'><b>Instructions for Windows:</b></font>
Chad Barb's avatar
   
Chad Barb committed
529
530
531
532
533
534
535
536
537
538
    <ul>
      <li>Run the installer executable, and successfully complete the installation.</li>
      <li>In the Web Interface Node view, 
          click on the "Connect to serial line" link.</li>
      <li><b>Save</b> the resulting .tbacl file in an appropriate place.
          (for instance a folder off the desktop.)</li>
      <li>For the lifetime of your experiment, you can simply double-click
          these .tbacl files to connect.</li>
    </ul> 

Chad Barb's avatar
   
Chad Barb committed
539
    <font size='+1'><b>Instructions for Linux/FreeBSD:</b></font>
540
    <ul>
Chad Barb's avatar
   
Chad Barb committed
541
542
    <li>Use <code>gunzip</code>, 
    then <code>tar xvf</code> on the downloaded file.</li>
543
    <li>Move the resulting <code>tiptunnel</code> binary into 
Chad Barb's avatar
   
Chad Barb committed
544
    a directory of your choice (<code>/usr/local/bin</code>, 
545
546
547
    or <code>~/bin</code> are two good places.)</li>
    <li>Set up your browser to handle MIME type "text/x-testbed-acl"
    as outlined below.</li>
Chad Barb's avatar
   
Chad Barb committed
548
549
550
551
552
553
    <li>In the Web Interface Node view, 
        click on the "Connect to serial line" link.</li>
    <li>If your browser is properly configured to use <code>tiptunnel</code>,
        a new xterm window with a telnet session open to your node
        should emerge.</li>
    <li>(Alternately, you can tell your browser to save "text/x-testbed-acl"
Chad Barb's avatar
   
Chad Barb committed
554
555
556
557
        files in a directory and you can run them with 
	<code>tiptunnel</code> directly;
        this may be more convenient than using the 
	web interface every time you wish
Chad Barb's avatar
   
Chad Barb committed
558
        to connect to a node in your experiment.
Chad Barb's avatar
   
Chad Barb committed
559
560
        Note that these files are valid for the 
	lifetime of your experiment.)</li>
561
    </ul>
562

Chad Barb's avatar
   
Chad Barb committed
563
    <font size='+1'><b>Linux/FreeBSD and Netscape 4.7:</b></font>
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
    <ul>
    <li>Choose <code>preferences</code> from the <code>edit</code> menu.</li>
    <li>Select <code>Navigator</code>, then <code>Applications</code> under
        it.</li>
    <li>Click the <code>New...</code> button.</li>
    <li>In the <code>MIMEType</code> box, type <code>text/x-testbed-acl</code>
    </li>
    <li>In the <code>Suffixes</code> box, type <code>tbacl</code></li>
    <li>Choose <code>Application</code> in the <code>Handled by</code>
        box</li>
    <li>Next to <code>Application</code>, either type the path to the
        <code>tiptunnel</code> binary, or use <code>Choose...</code> to find
        it.</li>
    <li>Now, <b>be sure to</b> put a space, then <code>%s</code> after the
        path to the application in the box. This tells netscape to actually
        pass the aclfile into tiptunnel (Mozilla does not require this;
        see below.)</li>
    <li>Click <code>OK</code>, then <code>OK</code> again.</li>
    <li>Clicking a "connect to serial line" link should now
        bring up a connection in an xterm window.</li>
    </ul> 
585

Chad Barb's avatar
   
Chad Barb committed
586
    <font size='+1'><b>Linux/FreeBSD and Mozilla:</b></font>
587
588
589
590
591
    <ul>
    <li>Choose <code>preferences</code> from the <code>edit</code> menu.</li>
    <li>Select <code>Navigator</code>, then <code>Helper Applications</code> 
        under it.</li>
    <li>Click the <code>New Type...</code> button.</li>
Chad Barb's avatar
   
Chad Barb committed
592
593
    <li>In the <code>MIMEType</code> box, 
    type <code>text/x-testbed-acl</code></li>
594
595
596
    <li>In the <code>File extension</code> box, type <code>tbacl</code></li>
    <li>For <code>Application to use</code>, either type the path to the
        <code>tiptunnel</code> binary, or use <code>Choose...</code> to find
597
	it.</li>
598
599
600
601
602
    <li>In Mozilla do <b>not</b> add a <code>%s</code>.</li>
    <li>Click <code>OK</code>, then <code>OK</code> again.</li>
    <li>Clicking a "connect to serial line" link should now
        bring up a connection in an xterm window.</li>
    </ul> 
603
    </p>
604

605
<li><a NAME="UTT-4"></a>
Chad Barb's avatar
   
Chad Barb committed
606
    <font size='+1'><b>Can I reboot (power cycle) my nodes?</b></font>
607
608
609
610
611
612
613
614
615
616
617
618
619
620
    <p>
    Yes. Each of the PCs is independently power controlled. If your
    node becomes wedged, or otherwise unresponsive, you can use the
    <tt>node_reboot</tt> command, as discussed in the
    <a href="tutorial/tutorial.php3#Wedged">Emulab Tutorial.</a>
    </p>
    <p>
    The Sharks are also power controlled, but because of the limited
    number of power ports available, the entire shelf of 8 sharks is
    on a single controller. The <tt>node_reboot</tt> does its best to
    cleanly reboot individual sharks, but if a single shark is
    unresponsive, the entire shelf will be power cycled.
    </p>

621
<li><a NAME="UTT-SCROGGED"></a>
Chad Barb's avatar
   
Chad Barb committed
622
    <font size='+1'><b>I've scrogged my disk! Now what?</b></font>
623
624
625
626
627
628
629
630
631
632
    <p>
    If you manage to corrupt a disk (or slice), no worries. You can easily
    repair the damage yourself by reloading a fresh copy of the default
    disk image. You will of course lose anything you have stored on that
    disk; it is a good idea to store only data that can be easily recreated, 
    or else store it in your project directory in <tt>/proj</tt>. Disk
    reloading is covered in more detail in the
    <a href="tutorial/tutorial.php3#Scrogged">Emulab Tutorial</a>.
    </p>

633
<li><a NAME="UTT-5"></a>
Chad Barb's avatar
   
Chad Barb committed
634
635
    <font size='+1'><b>Where do I 
    store files needed by my experiment?</b></font>
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
    <p>
    Each project has its own directory, rooted at <tt>/proj</tt>,
    which is available via NFS to all of the nodes in experiments
    running in that project. For example, when the "RON" project was
    created, a directory called /proj/RON was also created. This
    directory is owned by the project creator, and is in the unix
    group "RON." Its permission (mode) is 770; read/write/execute
    permitted by the project creator and by all of the members of the
    project RON, but protected against all access by people outside
    the RON project.
    </p>
    <p>
    Project members are encouraged to store any files needed by their
    experiments in the corresponding /proj project directory. 
    </p>

652
<li><a NAME="UTT-6"></a>
Chad Barb's avatar
   
Chad Barb committed
653
654
    <font size='+1'><b>Are my files on 
    <b>users.emulab.net</b> backed up (filesaved)?</b></font>
655
656
657
658
659
660
661
662
    <p>
    Yes. All of the files in your home directory on /users, and all of
    the files in your project directory in /proj are filesaved. While
    we can restore lost files in an emergency, we encourage you to
    back up critical data on your own to avoid (possibly long) delays
    in conducting your experiments.
    </p>
    
663
<li><a NAME="UTT-7"></a>
Chad Barb's avatar
   
Chad Barb committed
664
665
    <font size='+1'><b>Are the nodes 
    in my experiment backed up (filesaved)?</b></font>
666
667
668
669
670
671
672
673
674
675
676
677
678
    <p>
    No! The nodes in your experiment are not filesaved. Any changes
    you make to the local filesystems will be lost if the event of a
    disk failure. We plan to provide a mechanism for experimenters to
    create snapshots of their node state, but that is not done yet. In
    the meantime, any files that must not be lost should be stored in
    the project directory (/proj/<project_name>), which is available
    via NFS to all of the nodes in your experiment. You may also store
    files in your home directory (/users/<login>), also available via
    NFS to all of your nodes, but that is not the preferred location
    since quotas on /users are relatively small.
    </p>
    
679
<li><a NAME="UTT-Swapping"></a>
Chad Barb's avatar
   
Chad Barb committed
680
    <font size='+1'><b>What is Swapping?</b></font>
681
    <p>
682
683
    Swapping is when you (or we, or the Emulab system) temporarily swaps
    out your experiment,
684
685
686
687
688
    releasing all of the nodes in the experiment. Your experiment is
    still resident in the Emulab database, and you can see its status
    in the web interface, but no nodes are allocated. Once an
    experiment is swapped out, you can swap it back in via the web
    interface by going to the Experiment Information page for your
689
    experiment, and clicking on the swapin option. 
690
    You can also <a href="UTT-Modify">modify it</a>.
691
692
    </p>

693
    <p>
694
    The <tt>idle-swap</tt> checkbox in the Begin Experiment web page
695
696
    is used to determine what experiments can be <em>automatically</em>
    swapped by the testbed scheduling system. Note that all experiments are
697
    capable of being swapped; even if you do not check the idle-swap box,
698
699
700
701
702
703
704
705
706
707
708
709
    you are free to swap your own experiments as you like. The only
    difference is that the testbed scheduling system will not consider your
    experiment when looking for experiments to swap out. You will sometimes
    notice that the Experiment Information page does not contain the swap
    link. That is because experiments cannot be swapped when they are in
    transition. For example, when the experiment is being swapped in (say,
    after first being created) the link will disappear until the experiment
    is fully swapped in, and it is capable of being swapped out. You will
    need to occasionally reload the page so that the updated state is
    recognized and the swap link appears. 
    </p>
    
710
711
712
    <p>
    Be aware that we do not currently save any files that you may have
    placed on your nodes. When your experiment is swapped back in, you
713
    will likely get different nodes, with fresh copies of the disk
714
715
716
    images. For that reason, you should not swap your experiment out
    unless you make arrangements to save and restore any state you need.
    </p>
Robert Ricci's avatar
Robert Ricci committed
717

718
    <p>
719
720
721
    <em>Please be sure to read our 
    <a href="docwrapper.php3?docname=swapping.html"> Node Usage Policies</a></em>,
    which contain detailed information on swapping.
722
    </p>
Robert Ricci's avatar
Robert Ricci committed
723

724
<li><a NAME="UTT-Restart"></a>
Chad Barb's avatar
   
Chad Barb committed
725
    <font size='+1'><b>What is Experiment Restart?</b></font>
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
    <p>
    Experiment restart (or perhaps more aptly, replay) allows you to
    rerun your experiment from scratch, but without the added expense
    of a swapin and swapout. In other words, the nodes that are
    currently allocated to your experiment are all rebooted, and the
    experiment startup state is cleared. This includes the
    <a href="#SWS-6">ready bits</a>, the boot status in the web page,
    and the <a href="#SWS-4">startup command status</a>. In addition,
    the event scheduler for the experiment is restarted, and your event
    sequence is replayed again. Note that your rpms and tarfiles are
    <b>not</b> installed again. Replay is obviously faster than
    swapout/swapin, and has the added benefit that you will not run
    the risk of not being able to swapin for lack of available nodes.
    </p>

741
<li><a NAME="UTT-8"></a>
Chad Barb's avatar
   
Chad Barb committed
742
743
    <font size='+1'><b>How can I 
    get switch statistics (such as packet counts) for my experiment?</b></font>
Robert Ricci's avatar
Robert Ricci committed
744
745
746
747
748
749
750
751
752
753
754
755
756
    <p>
    We have a command called <code>portstats</code> that allows you access
    to some of the port counters on our switches. To use it, you'll need
    to ssh to <b>users.emulab.net</b>. '<code>portstats &lt;pid&gt; 
    &lt;eid&gt;</code>' will get you stats for all experimental interfaces in
    your experiment. Run '<code>portstats -h</code>' to get a list of other
    options, such as different sets of stats.
    </p>

    <p>
    Note that the numbers returned by <code>portstats</code> do not get
    reset between experiments.
    </p>
757
758

<li><a NAME="UTT-Naming"></a>
Chad Barb's avatar
   
Chad Barb committed
759
760
    <font size='+1'><b>What names should I use to refer to the nodes in my
    experiment?</b></font>
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816

    <p>We set up names for your nodes in DNS, for use from outside,
    and <code>/etc/hosts</code> files for use on the nodes in the experiment.
    Since our nodes have multiple interfaces (the control network, and,
    depending on the experiment, possibly several experimental interfaces,)
    determining which name refers to which interface can be somewhat
    confusing. The rules below should help you figure this out.
    </p>

    <ul>
	<li><b>From the outside world</b> - 
	We set up names in the form
	<code><i>node</i>.<i>expt</i>.<i>proj</i>.emulab.net</code> in DNS,
	so that they visible anywhere on the Internet. This name always refers
	to the node's control network interface, which is the only one
	reachable from the outside world.
	</li>

	<li><b>On the nodes themselves</b> -
	There are three basic ways to refer to the interfaces of a node. The
	first is stored in DNS, and the second two are stored on the node in
	the <code>/etc/hosts</code> file.
	<ol>
	    <li><i>Fully-qualified hostnames</i> - These names the same ones
	    visible from the outside world, and referred to by attaching the
	    full domain name: ie.
	    <code><i>node</i>.<i>expt</i>.<i>proj</i>.emulab.net</code>. (note
	    that, since we put <code>.emulab.net</code> in nodes' domain
	    search paths, you can use
	    <code><i>node</i>.<i>expt</i>.<i>proj</i></code> as a shorthand.)
	    This name always refers to the control network
	    </li>

	    <li><i><code>node-link</code> form</i> - You can refer to an
	    individual experimental interface by suffixing it with the name of
	    the link or LAN (as defined in your NS file) that it is a member
	    of. For example, <code>nodeA-link0</code> or
	    <code>server-serverLAN</code>. This is the preferred way to refer
	    to experimental interfaces, since it uniquely and unambiguously
	    identifies an interface.
	    </li>

	    <li><i>Short form</i> - If a node is directly connected to the
	    node you're on, you can refer to that node simply with its name
	    (eg. <code>nodeA</code>.) Note that this differs from the fully-
	    qualified name in that no domain is given. If you have enabled
	    <a href="#SWS-5">routing</a>, we also create short names for 
	    any node that you have a route to. However, if two nodes are
	    connected with more than one interface, or there is more than
	    one route between them, there is no guarantee that the short name
	    has been associated with the one is on the 'best' (ie. shortest or
	    highest bandwidth) path - so, if there is ambiguity, we strongly
	    suggest that you use the <code><i>node-link</i></code> form.
	    </li>
	</li>
    </ul>
Chad Barb's avatar
   
Chad Barb committed
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
<br>
<li><a NAME="UTT-Modify"></a>
    <font size='+1'><b>Can I modify my experiment after creating it?
    </b></font>

    <p>Yes. On the experiment view page, choose "Modify this Experiment".
       This will allow you to modify an experiment, either swapped-out or in,
       by editing its NS file.</p>
    <p>If the experiment is swapped-out, 
       Experiment Modify will simply replace its topology with the
       newly specified one;
       this new topology will be mapped when the experiment is swapped in.</p>
    <p>If the experiment is already swapped-in, Modify will change the topology
       and map in the portions which have been changed. This allows dynamic
       addition, subtraction, and replacement of an experiment's nodes and links.
       However, when modifying swapped-in experiments, there are a couple
       things to keep in mind:
       <ul>
         <li>Any node with the same name in the old and new topology will remain
	 on the same physical machine, unaffected-- its disk will not be reloaded.
	 If you want to, for example, change the hardware on a machine, you will
	 have to call the machine something different in the new topology.
	 </li>
	 <br>
	 <li>
	 It is highly recommended that you leave the
	 "Reboot nodes in experiment" box checked in the Experiment Modify form.
	 This is especially important if changing your experiment topology 
         (adding or removing nodes, links, and LANs).
	 If adding/removing a delay to/from an existing link, or replacing 
	 a lost node <i>without modifying the experiment topology</i>,
	 this won't be necessary.
	 </li>
850
851
852
853
854
855
856
857
858
859
860
861
862
863
	 <li>
	 The event system is not automatically restarted for your
	 experiment, so you will not be able to modify the traffic
	 shaping for new links. In addition, if you add program
	 agents or traffic generators, these will not activate unless
	 you restart the event system by hand. Unfortunately, all
	 events will be replayed, so be careful: On
	 <tt>users.emulab.net</tt>, you may do the following:
	     <blockquote>
	     	<code>
	        eventsys_control [-f] replay pid eid
		</code>
	     </blockquote>
	 </li>
Chad Barb's avatar
   
Chad Barb committed
864
865
       </ul>
    </p>
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916

<li><a NAME="UTT-SWPaths"></a>
    <font size='+1'><b> 
    Are there Linux and/or FreeBSD sources and packages available locally?
    </b></font>
    <p>
    Yes.  We provide sources and packages for a few versions of
    FreeBSD and RedHat Linux.  The place to look for available
    software is under <b>/share</b> on either <b>users.emulab.net</b>
    or your experimental nodes.  This path is readonly (and NFS
    mounted on the nodes), so you'll need to make a copy of the
    contents found there if you need to do more than reference them
    for information or installation.
    </p>

    <ul>
        <li><b>FreeBSD Paths:</b>
        The FreeBSD kernel, and userland sources are available under
        <b>/share/freebsd</b>.  Look there to see if the version you are
        seeking is available.  You can also find Emulab additions and
        modifications to FreeBSD here.  Emulab kernel configurations are
        called TESTBED and located in <b>sys/i386/conf</b> relative to the
        FreeBSD source trees.  The README file in this directory has more
        information on the contents.
        </li>

        <br>
        <li><b>Linux Paths:</b>
        Linux kernel sources and RPMs for various versions of Redhat can
        be found under <b>/share/redhat</b>.  Look there to see if the
        version you are seeking is available.  You can also find Emulab
        additions and modifications to Linux here.  Emulab kernel
        configurations are called <b>config-emulab</b> and exist in the
        root of the kernel source trees.  The README file in this
        directory has more information on the contents.
        </li>
        
        <br>
        <li><b>Other Software:</b>
        We provide a few other generally useful software packages and
        sources under <b>/share</b> as well.  Have a look around.
        </li>

        <br>
        <li><b>Something Missing?</b> 
        If you think something should be added to <b>/share</b>, feel
        free to send your suggestion(s) to us via 
        <a href="emailus.php3">email</a>.  Note that we may retire some
        offerings if we determine them to be of little value.
        </li>
    </ul>
917
918
919
920
921
922
</ul>

<hr>


<a NAME="HDS"></a>
Chad Barb's avatar
   
Chad Barb committed
923
<font size='+1'><b>Hardware Setup</b></font>
924
925
<ul>
<li><a NAME="HDS-1"></a>
Chad Barb's avatar
   
Chad Barb committed
926
    <font size='+1'><b>What kind of computers are used for my nodes?
927
928
929
<li><a NAME="HDS-2"></a>
    How many nodes are there?
<li><a NAME="HDS-3"></a>
Chad Barb's avatar
   
Chad Barb committed
930
    How many ethernet cards are on each node?</b></font>
931
    <p>
932
933
934
    Please see the <a href="docwrapper.php3?docname=hardware.html">
    Hardware Overview</a> page for a description and count of the
    computers that comprise the Testbed.
935
936
937
    </p>
    
<li><a NAME="HDS-4"></a>
Chad Barb's avatar
   
Chad Barb committed
938
    <font size='+1'><b>How many nodes are currently available (free)?</b></font>
939
940
941
942
943
944
945
946
    <p>
    If you click on the "Node Reservation Status" link in the menu to
    your left, you will see a summary of the number of nodes (by type)
    that are currently available, followed by a listing of the
    reservation status of each individual node.
    </p>
    
<li><a NAME="HDS-5"></a>
Chad Barb's avatar
   
Chad Barb committed
947
    <font size='+1'><b>Can I do traffic shaping on my links?</b></font>
948
949
950
951
952
953
954
955
956
    <p>
    Yes! You can specify the delay, bandwidth, and packet loss rate
    between any two nodes in your topology. Bandwidth and delay are
    specified in the NS <tt>duplex-link</tt> statement, while packet
    loss rate is specified with the Emulab <tt>tb-set-link-loss</tt>
    extension to NS. You may also specify delay, bandwidth, and packet
    loss rate between nodes in a regular LAN.
    </p>
    <p>
957
    Please see the
958
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
959
    page for a summary of all Emulab NS extensions, and the
960
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
961
962
    example. 
    </p>
963
964

<li><a NAME="HDS-6"></a>
Chad Barb's avatar
   
Chad Barb committed
965
966
    <font size='+1'><b>Can I modify the 
    traffic shaping parameters on my links?</b></font>
967
968
969
970
971
972
    <p>
    Yes! If your NS file specified traffic shaping on a link, then you
    can subsequently modify those parameters after the experiment has
    been swapped in. Note that you cannot <em>convert</em> a non shaped
    link into a shaped link; you can only modify the traffic shaping
    parameters of a link that is already being shaped.  To modify the
973
974
975
976
977
978
    parameters, go to the Experiment Information page of your
    experiment, and click on the "Control Traffic Shaping" menu
    option. Follow the instructions at the top of the page.
    </p>
    <p>
    An alternative method is to log into <b>users.emulab.net</b> and use the
979
980
981
    <tt>delay_config</tt> program. This program requires that you know
    the symbolic names of the individual links. This information is
    available via the web interface on the Experiment Information
982
983
    page. The command line syntax for <tt>delay_config</tt> will be
    displayed when the <tt>-h</tt> option is given.
984
    </p>
985
986

<li><a NAME="HDS-7"></a>
Chad Barb's avatar
   
Chad Barb committed
987
988
989
    <font size='+1'><b>Are there other 
    traffic shaping parameters besides latency,
    bandwidth, and packet loss rate?</b></font>
990
    <p>
991
992
993
994
995
996
997
    Yes! Please see the
    <a href="tutorial/docwrapper.php3?docname=advanced.html">
    advanced tutorial</a>. Note though, that these other parameters
    can be specified for duplex links only (not lans), and that they
    are not configurable with <tt>delay_config</tt>, but with a
    different testbed utility call <tt>tevc</tt> (also described in
    the advanced tutorial).
998
    </p>    
999

1000
1001
1002
1003
1004
</ul>

<hr>

<a NAME="SWS"></a>
Chad Barb's avatar
   
Chad Barb committed
1005
<font size='+1'><b>Software Setup</b></font>
1006
1007
<ul>
<li><a NAME="SWS-1"></a>
Chad Barb's avatar
   
Chad Barb committed
1008
    <font size='+1'><b>What OS do the nodes run?</b></font>
1009
    <p>
1010
1011
1012
    Please see the <a href="docwrapper.php3?docname=software.html">
    Software Overview</a> page for a description of the Operating
    Systems that can be run on each of the Testbed nodes.
1013
1014
1015
    </p>

<li><a NAME="SWS-2"></a>
Chad Barb's avatar
   
Chad Barb committed
1016
    <font size='+1'><b>How do I select which OS to run on each node?</b></font>
1017
1018
1019
1020
1021
1022
1023
1024
    <p>
    When a choice of OS is available, you may specify which one you
    prefer for each node in the NS file using the Emulab
    <tt>tb-set-node-os</tt> extension to NS. When your experiment is
    configured, the appropriate disk image will be loaded on your
    nodes, and the selected operating system will boot up on each.
    </p>
    <p>
1025
    Please see the
1026
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
1027
    page for a summary of all Emulab NS extensions, and the
1028
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
1029
1030
1031
1032
    example. 
    </p>    
    
<li><a NAME="SWS-3"></a>
Chad Barb's avatar
   
Chad Barb committed
1033
    <font size='+1'><b>Can I load my own software (RPMs) on my nodes?</b></font>
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
    <p>
    Yes! If have an RPM (or more than one) that is appropriate for
    loading on the OS you have selected, you can arrange to have them
    loaded automatically when your experiment is configured. The
    Emulab NS extension <tt>tb-set-node-rpms</tt> is used in the NS
    file to specify a list of RPMS to install. You may specify a
    different list for each node in the experiment. When the node
    first boots after the experiment is configured, each of the RPMs
    will be installed (but only RPMs that have not already been
    installed). 
    </p>
    <p>
1046
    Please see the
1047
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
1048
    page for a summary of all Emulab NS extensions, and the
1049
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
1050
1051
    example. 
    </p>    
1052
    
1053
<li><a NAME="SWS-4"></a>
Chad Barb's avatar
   
Chad Barb committed
1054
1055
    <font size='+1'><b>Can I schedule 
    programs to run automatically when a node boots?</b></font>
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
    <p>
    Yes! You can arrange to run a single program or script when your
    node boots. The script is run as the UID of the experiment
    creator, and is run after all other node configuration (including
    RPM installation) has completed. The exit status of the script (or
    program) is reported back and is made available for you to view in
    Experiment Information link in the menu at your left. The Emulab
    NS extension <tt>tb-set-node-startup</tt> is used in the NS file
    to specify the path of the script (or program) to run. You may
    specify a different program for each node in the experiment.
    </p>
    <p>
1068
    Please see the
1069
    <a href="tutorial/docwrapper.php3?docname=nscommands.html">Extensions</a>
1070
    page for a summary of all Emulab NS extensions, and the
1071
    <a href = "tutorial/tutorial.php3">Emulab Tutorial</a> for an
1072
1073
    example.
    </p>    
1074

1075
<li><a NAME="SWS-5"></a>
Chad Barb's avatar
   
Chad Barb committed
1076
1077
    <font size='+1'><b>How can I turn on routing or set up routes automatically 
    in my nodes?</b></font>
1078
    <p>
1079
    By default, we do not setup any static routes or run any routing daemon
1080
    on nodes in an experiment.  However, we do provide several options for
Jay Lepreau's avatar
spello    
Jay Lepreau committed
1081
    experimenters, which are described in the
1082
1083
1084
    <a href="tutorial/tutorial.php3#Routing">"Setting up IP routing
    between nodes"</a> section of the
    <a href="tutorial/tutorial.php3">Emulab Tutorial.</a>
1085
1086
1087
    </p>
    
<li><a NAME="SWS-6"></a>
Chad Barb's avatar
   
Chad Barb committed
1088
1089
    <font size='+1'><b>How does my software determine when other nodes in my
    experiment are ready?</b></font>
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
    <p>
    If your application requires synchronization to determine when all
    of the nodes in your experiment have started up and are ready to
    proceed, then you can use the Testbed's <i>ready bits</i>
    mechanism. The ready bits are really just a way of determining how
    many nodes have issued the <b>ready</b> command, and is returned
    to the application as a simple N of M string, where N is the
    number that have reported in, and M is the total number of nodes
    in the experiment. Applications can use this as a very simplistic
    form of barrier synchronization, albeit one that can be used just
    once and one that does not actually block!
    </p>
    <p>
    Use of the ready bits is described in more detail in the <a href =
1104
    "tutorial/tutorial.php3">Emulab Tutorial</a> and in the <a href =
1105
    "doc/docwrapper.php3?docname=tmcd.html"> Testbed Master Control
1106
    Daemon</a> documentation.
1107
1108
    </p>
    
1109
<li><a NAME="SWS-7"></a>
Chad Barb's avatar
   
Chad Barb committed
1110
    <font size='+1'><b>Can I run my own Operating System?</b></font>
1111
    <p>
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
    Yes! You can run your own OS (or a customized version of an Emulab
    supported OS) on any of the PCs. You can also run <a href =
    "http://www.cs.utah.edu/flux/oskit/">OSKit</a> kernels on the
    PCs. Each of the PCs is partitioned with two DOS partitions large
    enough to hold the typical OS installation. The 1st and 2nd
    partitions are each 3GB. The 3rd partition is 500MB, and is
    labeled as Linux Swap. The 4th partition is the remainder of the
    disk, and varies in size depending on the pc type. We recommend
    that you use the 1st or 2nd partition; using the 4th partition
    will restrict the number of machines that you can run your OS on
    since it varies in size. Note that you must leave the MBR (Master
    Boot Record) in sector 0 alone, and that your custom partition
    must contain a proper DOS boot record in the first sector.
    </p>

    <p>
    Please note that while users are free to customize their disks and
    install their own operating systems, Emulab staff will not be able
    to offer more than encouragement and advice! We cannot install the
    OS for you, and we cannot load CDROMS, floppy disks, or tape drives!
    We <em>do</em> provide an easy way for you to boot FreeBSD from a
    memory based filesystem (MFS) so that you can more easily work
    with the disk (in case it is not possible to install your OS on a
    live disk). Beyond that, you are pretty much on your own!
    </p>

    <p>
    Many users had great success with customizing an Emulab supported
    OS (FreeBSD or Linux), and then creating a disk image that is
    autoloaded when the experiment is swapped in. We strongly
    encourage people to use this approach whenever possible! There is
    more information available in the <a
    href="tutorial/tutorial.php3#CustomOS">Custom OS</a> section of
    the <a href = "tutorial/tutorial.php3">Emulab Tutorial.</a>
1146
    </p>
1147

1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
<li><a NAME="MovingImage"></a>
    <font size='+1'><b>
    Can I share a disk image between two projects?
    </b></font>
    <p>
    No. At this time you cannot share OS images between projects. We
    are thinking of adding project collaboration support, but that is
    a future project.
    </p>
    <p>
    In the meantime, you will need to create an
    <a href="https://www.emulab.net/newimageid_ez.php3">image
    descriptor</a> in the project that wants to use your image. Fill out
    the form, but leave out the "Node to Obtain Snapshot from". Then
    just copy the image over to the default path it picked for you in
    the form. There is more information available in the <a
    href="tutorial/tutorial.php3#CustomOS">Custom OS</a> section of
    the <a href = "tutorial/tutorial.php3">Emulab Tutorial.</a>
    </p>

1168
<li><a NAME="SWS-8"></a>
Chad Barb's avatar
   
Chad Barb committed
1169
    <font size='+1'><b>What if I need more disk space on my nodes?</b></font>
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
    <p>
    Each node has a partition at the end of the disk that you can use if
    you wish. In Linux, the partition is <code>/dev/hda4</code> ; in FreeBSD,
    it's </code>/dev/ad0s4</code> . There is no filesystem on this partition,
    so you'll need to create it yourself. For example, in Linux:
    <blockquote>
	<code>
	    mkfs /dev/hda4;<br>
	    mount /dev/hda4 /mnt;
	</code>
    </blockquote>
    This partition is only 6 Gigs, the size of the leftover space on our
    smallest drives. If you need more space than this, it would be possible
    to enlarge this partition on some machines (for example, our pc850s
Jay Lepreau's avatar
Nits.    
Jay Lepreau committed
1184
    have 40 GB disks), but that is outside the scope of this FAQ.
1185
1186
    </p>

1187
<li><a NAME="SWS-9"></a>
Chad Barb's avatar
   
Chad Barb committed
1188
1189
    <font size='+1'><b>Are there testbed-specific 
    daemons that could interfere with my experiment?</b></font>
1190
    <p>
Jay Lepreau's avatar
Nits.    
Jay Lepreau committed
1191
    Probably not.
1192
    By default, the testbed startup scripts currently start two daemons in 
1193
1194
1195
1196
1197
    addition to the OS's standard set.  Other daemons may be started depending
    on the network services you ask for in your ns file (see below).
    </p>

    <p>
Chad Barb's avatar
   
Chad Barb committed
1198
    <b>Unconditionally started daemons:</b>
1199
1200
1201
1202
    </p>

    <p>
    <blockquote>
Chad Barb's avatar
   
Chad Barb committed
1203
    <li><code>healthd</code> - A low overhead hardware health monitor.</li>
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
    </blockquote>
    </p>

    <p> 
    This deamon periodically polls the machine's health monitoring
    hardware and sends this information back to our <code>boss</code>
    node for analysis.  The hardware is polled once per second, and a
    status datagram is sent out once every five minutes.
    <code>Healthd</code>'s overhead is quite low, but it can be safely
    killed and disabled from startup if you're worried about possible
1214
1215
    side effects.  It is started by
    <code>/etc/testbed/rc.healthd</code>.
1216
    </p>
1217
1218
1219

    <p>
    <blockquote>
Chad Barb's avatar
   
Chad Barb committed
1220
    <li><code>slothd</code> - A low overhead usage analysis tool.</li>
1221
1222
1223
1224
1225
1226
    </blockquote>
    </p>

    <p>
    <code>Slothd</code> is important to efficient testbed utilization
    and should run on every node whenever possible.  Its overhead is
1227
    almost negligible (essentially less than running <code>'ls -l
1228
    /dev'</code> once every five minutes), and should not interfere with your
1229
1230
1231
1232
1233
1234
1235
    work.  However if your experiment is exceptionally sensitive, then
    you may arrange with us to disable <code>slothd</code>.  Please
    note that we will restart this daemon if it is not running unless
    prior arrangements have been made.
    </p>

    <p>
Chad Barb's avatar
   
Chad Barb committed
1236
    <b>Conditionally started daemons:</b>
1237
1238
1239
1240
    </p>

    <p>
    <blockquote>
Chad Barb's avatar
   
Chad Barb committed
1241
    <li><code>gated</code> - A network routing daemon.</li>
1242
    </blockquote>
1243
1244
    </p>

1245
    <p>
1246
1247
    If you have requested automatic routing on your nodes with  
    <code>$ns&nbsp;rtproto&nbsp;Session</code> in your NS file, this will
1248
1249
    start <code>gated</code> on all of your nodes.
    </p>
1250

1251
1252
1253
1254
1255
1256
    <p>
    We have left all daemons started by the operating systems' default
    configurations (such as <code>cron</code>) enabled, so you should also
    look at them if you are concered about running processes affecting
    your experiment.
    </p>
Mac Newbold's avatar
Mac Newbold committed
1257
1258

<li><a NAME="SWS-10"></a>
Chad Barb's avatar
   
Chad Barb committed
1259
    <font size='+1'><b>Does Emulab support IP Multicast?</b></font>
Mac Newbold's avatar
Mac Newbold committed
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
    <p>
      In short, yes, the local nodes in Emulab (but not all remote
      Netbed nodes) support IP Multicast on the experimental
      network. In order to use it, you must have a kernel that
      supports it, and if you want multicast routing, you'll need to
      enable <code>mrouted</code>. (You can do it manually, or
      automatically via program objects or startup commands, but the
      rtproto commands will not do it.)
    </p>

1270
1271
</ul>

1272
1273
1274
<hr>

<a NAME="SEC"></a>
Chad Barb's avatar
   
Chad Barb committed
1275
<font size='+1'><b>Security Issues</b></font>
1276
1277
<ul>
<li><a NAME="SEC-1"></a>
Chad Barb's avatar
   
Chad Barb committed
1278
    <font size='+1'><b>Is Emulab Firewalled?</b></font>
1279
    <p>
1280
1281
    Yes. Emulab blocks all of the <i>low numbered</i> ports (ports below 1024),
    with the exception of ports 20 and 21 (FTP), 22 (Secure Shell), and 80
Jay Lepreau's avatar
Jay Lepreau committed
1282
    (HTTP). This is for the protection of experimenters, as well as to ensure
1283
1284
1285
1286
    that an errant application cannot become the source of a Denial of Service
    attack to sites outside of Emulab. If your application requires external
    access to other low numbered ports, please contact us to make special
    arrangements.
1287
1288
    </p>
</ul>
1289
1290
1291
1292

<hr>

<a NAME="TR"></a>
Chad Barb's avatar
   
Chad Barb committed
1293
<font size='+1'><b>Troubleshooting</b></font>
1294
1295
<ul>
<li><a NAME="TR-1"></a>
Chad Barb's avatar
   
Chad Barb committed
1296
1297
    <font size='+1'><b>My experiment is set up, but I cannot
	send packets between some of the nodes. Why?</b></font>
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
    <p>
    The most common reason is that your topology
    includes nodes which are not directly connected, and you have
    not setup any routing.  Refer to
    "<a href="#SWS-5">How can I turn on routing or set up routes
    automatically in my nodes?</a>" for details.  If you cannot
    send packets between two machines which are directly connected
    (via a link or a lan), then there are two possibilities:
    either the nodes did not properly negotiate their speed and
    duplex with the Cisco switch, or the physical wire is loose
1308
1309
    or bad.  In these cases, <a href="#GS-7a">you should contact us</a>
    for help.
1310
    </p>
1311
1312

<li><a NAME="TR-2"></a>
Chad Barb's avatar
   
Chad Barb committed
1313
1314
1315
    <font size='+1'><b>I asked for traffic shaping, 
    but everything seems to be going at full LAN speeds.  
    What's wrong?</b></font>
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
    <p>
    The most likely problem is that it is using the unshaped control
    network for the traffic you're looking at. This occurs when it
    tries to contact a node using a "pcXXX" address, like pc76 or
    pc76.emulab.net, or when it tries to ping a fully-qualified name,
    like NodeA.myexpt.myproj.emulab.net , which also resolves to a
    control network address. On one of your nodes, take a look at the
    file /etc/hosts . It shows the IP addresses and aliases that refer
    to the different experimental interfaces. These are the names/IPs
    you can use to see the delays.
    </p>

    <p>
    For instance, let's say I have an experiment that had NodeA and
    NodeB connected with a shaped link. The file /etc/hosts on NodeA
    would have a line for NodeB, with an address like 192.168.1.3, and
    on NodeB there would be an entry for NodeA with the address
    192.168.1.2. These addresses correspond to the delayed link
    between them. Any address outside the 192.168.*.* range that you
    didn't configure manually corresponds to an unshaped link.
    </p>

    <p>
    For a discussion of the way to 'name' interfaces on the control
    and experimental networks, see the the <a href="#UTT-Naming">naming
    section</a> of this document.
    </p>

<li><a NAME="TR-3"></a>