emulab-transition.scrbl 13.2 KB
Newer Older
Mike Hibler's avatar
Mike Hibler committed
1 2 3 4 5 6
#lang scribble/manual
@(require "defs.rkt")

@title[#:tag "emulab-transition" #:version apt-version]{Transitioning from the ``Classic'' Interface to the ``Portal''}

The new Emulab portal interface is a custom version of the
Mike Hibler's avatar
Mike Hibler committed
7 8 9
@hyperlink["http://cloudlab.us"]{CloudLab} interface, so if you are familiar
with that interface, using the Emulab portal will be easy!
The portal currently supports the most commonly used features of the classic
Mike Hibler's avatar
Mike Hibler committed
10
interface, with more features ported on demand.
Mike Hibler's avatar
Mike Hibler committed
11 12 13
@bold{The classic interface should be considered deprecated},
with no new features being added.
It will continue to be supported until all important
Mike Hibler's avatar
Mike Hibler committed
14 15 16
features have been moved to the portal interface.

The remainder of this section covers
Mike Hibler's avatar
Mike Hibler committed
17
@seclink["emulab-cloudlab-features"]{new and improved features}
Mike Hibler's avatar
Mike Hibler committed
18 19 20 21 22 23
that the portal interface offers over the classic interface,
@seclink["emulab-missing-features"]{features currently missing}
from the portal interface, and how to
@seclink["emulab-conversion"]{convert an existing experiment}
into a portal experiment.

Mike Hibler's avatar
Mike Hibler committed
24
@section[#:tag "emulab-cloudlab-features"]{New and Improved Features of the Portal Interface}
Mike Hibler's avatar
Mike Hibler committed
25

Mike Hibler's avatar
Mike Hibler committed
26 27 28 29
Besides a much updated look and feel, the portal interface offers significant
new features and improvements to existing features.

@subsection{Profiles}
Mike Hibler's avatar
Mike Hibler committed
30
One important change that is more than just an interface issue,
Mike Hibler's avatar
Mike Hibler committed
31 32 33 34 35 36 37 38 39 40 41 42 43 44
is the clear delineation between an experiment's @italic{description}
and its @italic{instantiation}. 

In the classic interface, the term ``experiment'' is used to refer to
both: experiment descriptions and instances can be created and destroyed
together or, the experiment can be created just as a description and then
``swapped in'' (instantiated with resources) and ``swapped out''
(resources freed) repeatedly.

Through the portal,
experiment descriptions are now known as @seclink["profiles"]{profiles}
and the term @seclink["experiments"]{experiment} is used to refer to a
running instance of a profile.
Think of a @italic{profile} as a ``swapped out'' classic experiment
Mike Hibler's avatar
Mike Hibler committed
45
and an @italic{experiment} (or @italic{instance}) as a ``swapped in'' classic
Mike Hibler's avatar
Mike Hibler committed
46 47 48 49 50 51 52 53
experiment.

The portal offers a variety of ways in which to create new profiles,
including a web-based GUI and a Python-based scripting library.
See the section on @seclink["creating-profiles"]{creating profiles}
for details.
In addition, the Emulab portal provides its own method for
@seclink["emulab-conversion"]{converting classic experiments to profiles}.
Mike Hibler's avatar
Mike Hibler committed
54

Mike Hibler's avatar
Mike Hibler committed
55
The profile representation of an experiment is different from that of
Mike Hibler's avatar
Mike Hibler committed
56
the classic experiment.
Mike Hibler's avatar
Mike Hibler committed
57
In the classic interface, experiments are described using @tt{ns-2},
Mike Hibler's avatar
Mike Hibler committed
58
a network simulator scripting language based on TCL.
Mike Hibler's avatar
Mike Hibler committed
59 60 61
The portal represents experiments at the base level as GENI
@seclink["rspecs"]{RSpecs} with the Python-based
@seclink["geni-lib"]{geni-lib}
Mike Hibler's avatar
Mike Hibler committed
62
interface available for scripting experiments.
Mike Hibler's avatar
Mike Hibler committed
63
As described in the @seclink["emulab-conversion"]{conversion} section
Mike Hibler's avatar
Mike Hibler committed
64
below, the portal can automatically convert classic ns-2 experiment
Mike Hibler's avatar
Mike Hibler committed
65 66 67 68 69
descriptions into geni-lib based profiles.

In addition to the hardware and software resources required by an experiment,
a profile also contains various metadata.
The Description and Instructions fields permit you to provide additional
Mike Hibler's avatar
Mike Hibler committed
70
information to the user of a profile.
Mike Hibler's avatar
Mike Hibler committed
71 72 73 74
These fields support @seclink["markdown"]{Markdown} formatting allowing more
expressive text.
Classic experiments allow only a short, ASCII-only description.

Mike Hibler's avatar
Mike Hibler committed
75 76
Profiles can also have @italic{parameters} with descriptions and default
values. This permits customized instantiations from a single profile.
Mike Hibler's avatar
Mike Hibler committed
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
For example, you can
have parameters to specify the number or types of nodes in an experiment.
In classic experiments this customization is accomplished by setting TCL
variables in the NS file.
Using a different value for the variables entails either
modifying the experiment while it is swapped out or creating a new experiment
with different values for the variables.
With profile parameters, values are provided through the interface at
instantiation time.

By default, profiles have the same visibility that classic experiments
do---only people in the same project can instantiate an
experiment from a profile.
However, you can also make profiles visible by ``Anyone,'' marking the profile
global and allowing any experimenter in any project to instantiate it.

@subsection{Web-based Interaction with Nodes}

The portal page for an active experiment has a dashboard allowing interaction
with all nodes. You can fire off in-browser console or SSH sessions, reboot,
and view activity graphs.

@subsection{Disk Images}
Mike Hibler's avatar
Mike Hibler committed
100 101 102 103

Custom disk images are now created by taking a @italic{snapshot} of an
experiment node or @italic{cloning} an existing experiment to create
a new profile.
Mike Hibler's avatar
Mike Hibler committed
104 105 106
The disk image format has not changed---the same images are used for
classic or portal experiments---but the way in which images are
named in a profile is different.
Mike Hibler's avatar
Mike Hibler committed
107 108
Images are now identified by URNs.
For example, you would use
Mike Hibler's avatar
Mike Hibler committed
109
@tt{urn:publicid:IDN+emulab.net+image+emulab-ops//FBSD103-64-STD}
Mike Hibler's avatar
Mike Hibler committed
110
in a profile where you would use
Mike Hibler's avatar
Mike Hibler committed
111 112
@tt{emulab-ops/FBSD103-64-STD}
in an NS file.
Mike Hibler's avatar
Mike Hibler committed
113 114
The URN for an image can be obtained from the ``Images'' tab on the user
dashboard.
Mike Hibler's avatar
Mike Hibler committed
115 116
See the section on @seclink["disk-images"]{disk images} for more information.

Mike Hibler's avatar
Mike Hibler committed
117 118
@subsection{Versioning}

Mike Hibler's avatar
Mike Hibler committed
119 120 121 122 123 124 125 126 127 128
Both profiles and disk images are now versioned.
Whenever you make a change to a profile,
you create a new version of that profile.
Whenever you take a snapshot of a node running a custom image,
you create a new version of that image.
When using a profile or image and no explicit version is specified,
you get the most recent version.
You can delete old versions of images and profiles (subject to some
constraints).
Deleting the current version effectively ``rolls back''
Mike Hibler's avatar
Mike Hibler committed
129
the profile or image to the previous version.
Mike Hibler's avatar
Mike Hibler committed
130

Mike Hibler's avatar
Mike Hibler committed
131 132
@subsection{Experiment Extensions}

Mike Hibler's avatar
Mike Hibler committed
133
The portal interface has a more rigorous procedure for @italic{extending}
Mike Hibler's avatar
Mike Hibler committed
134
a running experiment.
Mike Hibler's avatar
Mike Hibler committed
135 136 137 138
In the classic interface you can extend a running experiment by modifying
the ``Max Duration'' setting within a small range (0 - 120 hours).
Longer extensions require an email exchange with the admins.
In the portal interface, you request an extension via the web interface
Mike Hibler's avatar
Mike Hibler committed
139 140
with shorter extensions being approved instantly and longer extensions
automatically forwarded to admins for their consideration.
Mike Hibler's avatar
Mike Hibler committed
141 142
Be aware that even short extensions may not be possible if the extension
would conflict with a future reservation (described below).
Mike Hibler's avatar
Mike Hibler committed
143
See the section on @seclink["extending"]{extending experiments}
Mike Hibler's avatar
Mike Hibler committed
144 145
for more information.

Mike Hibler's avatar
Mike Hibler committed
146
@subsection{Resource Reservations}
Mike Hibler's avatar
Mike Hibler committed
147

Mike Hibler's avatar
Mike Hibler committed
148
A major new feature supported via the portal interface is the ability to
Mike Hibler's avatar
Mike Hibler committed
149
automatically reserve node resources at a future time.
Mike Hibler's avatar
Mike Hibler committed
150 151
Emulab has traditionally been a first-come-first-serve, best-effort facility
and reserving nodes required prior arrangement with Emulab staff.
Mike Hibler's avatar
Mike Hibler committed
152 153
Through the portal, you can now ensure that a given number of nodes of a
specific type will be available during a future time window.
Mike Hibler's avatar
Mike Hibler committed
154
See the section on @seclink["reservations"]{reservations} for details.
Mike Hibler's avatar
Mike Hibler committed
155 156 157

@section[#:tag "emulab-missing-features"]{Classic Features Not Currently Supported in the Portal}

158 159
A number of prominent Emulab features are not currently supported via the
portal interface.
Mike Hibler's avatar
Mike Hibler committed
160
These include:
Mike Hibler's avatar
Mike Hibler committed
161 162

@itemlist[
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185
@item{
    @bold{Events.}
    The Emulab @link["https://wiki.emulab.net/wiki/eventsystem"]{event system}
    is not currently exposed through the portal interface.
    Specifically, you cannot schedule events from a profile.
    There is also no access to the program agent, traffic generator, or
    dynamic traffic shaping as those all use events to control execution.
}
@item{
    @bold{Modifying a Live Experiment.}
    The classic interface allows you to modify the description (e.g.,
    network topology or node OSes) of a running experiment.
    This is not supported under the portal interface.
}
@item{
    @bold{Batch Experiments.}
    Automated execution of experiments via the
    @link["https://wiki.emulab.net/wiki/Tutorial#section-21"]{batch system}
    is not supported.
    Portal experiments can include ``startup services,'' but there is no
    current equivalent to starting an experiment when resources become
    available and then automatically terminating it when done.
}
Mike Hibler's avatar
Mike Hibler committed
186 187 188 189 190 191 192
@item{
    @bold{Asymmetric Link Shaping.}
    The classic interface supports the ability to shape a link with
    different characteristic (BW, latency, loss) in each direction.
    The portal interface only allows specification of one set of
    characteristics that apply in both directions.
}
Mike Hibler's avatar
Mike Hibler committed
193 194 195 196
]

@section[#:tag "emulab-conversion"]{Converting an Emulab experiment to a Profile}

Mike Hibler's avatar
Mike Hibler committed
197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220
As a convenience to users, we provide a mechanism to convert NS-based
experiments into geni-lib profiles.
@margin-note{
    If you have no classic Emulab experiments (swapped in or swapped out)
    then you can skip this section and return to the
    @seclink["getting-started"]{getting started} section.
}
Before walking through an example conversion, there are a couple of
things to be aware of:
@itemlist[
@item{
    The converter does not support all Emulab NS extensions, only those
    most commonly used by users.
}
@item{
    The conversion process is @italic{not} a source-to-source translation,
    rather it works by interpreting the NS script to generate state for
    the Emulab DB and then generating the geni-lib script from that state.
    From a practical standpoint, what this means is that if you use TCL
    loops, conditionals or variables in your original description, those
    will not appear in the resulting geni-lib script--loops will be unrolled,
    conditionals evaluated, and variables bound to fix values.
}
]
Mike Hibler's avatar
Mike Hibler committed
221 222 223 224 225 226 227

When you login to the new portal interface at @url[(apturl)], you will be
taken to your @italic{dashboard} on which there are a number of tabs.
If you have any swapped in experiments you will start on the ``Experiments''
tab, otherwise you will be on the ``Profiles'' tab.
In either case you will be presented with a list of ``Classic Emulab
Experiments.''
228 229 230 231 232
@margin-note{
    On the ``Profiles'' tab, the list of classic experiments will show only
    those that are swapped out.
    On the ``Experiments'' tab, it will show only those that are swapped in.
}
Mike Hibler's avatar
Mike Hibler committed
233 234
On the right hand side of the Name column, there will be an arrow icon:

Mike Hibler's avatar
Mike Hibler committed
235
@screenshot["classic-list.png"]
Mike Hibler's avatar
Mike Hibler committed
236 237 238 239 240 241

Clicking on this icon will start the conversion process for that experiment.
The process will take anywhere from seconds to a minute or more depending
on the complexity of the experiment description.
After conversion, you will see:

Mike Hibler's avatar
Mike Hibler committed
242
@screenshot["classic-conversion1.png"]
Mike Hibler's avatar
Mike Hibler committed
243 244

Typically, all you would do at this point is to click ``Create.''
Mike Hibler's avatar
Mike Hibler committed
245 246 247

However, you may want or need to change the profile name if you don't like
the default
Mike Hibler's avatar
Mike Hibler committed
248 249 250 251 252 253 254
(same as the classic experiment name) or if the creation fails because
the name is already taken.
You may also want to examine, or even modify, the new profile prior to
actually creating it. Use either the ``Edit Code'' or ``Edit Topology''
buttons for this. The former allows you to modify the python/geni-lib script
representation, the latter fires up the embedded @seclink["jacks"]{Jacks} GUI.
  
Mike Hibler's avatar
Mike Hibler committed
255
After clicking ``Create,'' you will be taken to a new page:
Mike Hibler's avatar
Mike Hibler committed
256

Mike Hibler's avatar
Mike Hibler committed
257
@screenshot["classic-conversion2.png"]
Mike Hibler's avatar
Mike Hibler committed
258 259 260 261 262 263

showing details about the new profile (on the left) and allowing you
to further edit the profile, copy or delete it.

On the bottom right is a button to ``Instantiate'' the profile---the
equivalent of ``swapping in'' an experiment in the classic interface.
Mike Hibler's avatar
Mike Hibler committed
264 265 266 267
Clicking that button takes you to the third and final step of the
instantiation wizard:
@margin-note{
    The first step is skipped because the created profile is automatically
268
    chosen and the second step because converted NS experiments
Mike Hibler's avatar
Mike Hibler committed
269 270
    have no parameters.
}
Mike Hibler's avatar
Mike Hibler committed
271

Mike Hibler's avatar
Mike Hibler committed
272
@screenshot["classic-instantiate.png"]
Mike Hibler's avatar
Mike Hibler committed
273 274

where you can optionally name this instance and then click ``Finish.''
Mike Hibler's avatar
Mike Hibler committed
275
Once instantiated the node icons should turn green as they finish configuring
Mike Hibler's avatar
Mike Hibler committed
276
and the experiment state should change to ``ready'' and you will be off and
Mike Hibler's avatar
Mike Hibler committed
277
running!
Mike Hibler's avatar
Mike Hibler committed
278

Mike Hibler's avatar
Mike Hibler committed
279
@screenshot["classic-running.png"]
Mike Hibler's avatar
Mike Hibler committed
280

Mike Hibler's avatar
Mike Hibler committed
281
From the ``Topology View'' you can interact with the experiment.
Mike Hibler's avatar
Mike Hibler committed
282 283
By clicking on a node in the topology view as shown, you can perform a variety
of useful actions including: starting an in-browser shell or console window,
284 285
rebooting, or taking a snapshot.

Mike Hibler's avatar
Mike Hibler committed
286
Going to the ``List View'' tab presents you with a more Emulab-like listing:
287

Mike Hibler's avatar
Mike Hibler committed
288
@screenshot["classic-list-nodes.png"]
289

Mike Hibler's avatar
Mike Hibler committed
290 291 292 293
showing the node names and types as well as a per-node ``Actions'' menu for
console, SSH, rebooting, etc.
The settings drop down in the header allows you to perform certain actions
(e.g., rebooting) on all selected nodes.
Mike Hibler's avatar
Mike Hibler committed
294

Mike Hibler's avatar
Mike Hibler committed
295 296 297 298 299 300
If you are nearing the expiration time for your experiment and need to
keep it for longer, click on the green ``Extend'' button and then use the
slider or ``Requested'' text box to choose an extension time.
Keep in mind that an extension may not be possible or may require admin
approval.

Mike Hibler's avatar
Mike Hibler committed
301
Once you are finished with your experiment, you can ``swap out'' the
Mike Hibler's avatar
Mike Hibler committed
302 303 304
experiment using the red ``Terminate'' button.
This destroys the instance but leaves the profile intact.
If you want to terminate the experiment in the classic sense
Mike Hibler's avatar
Mike Hibler committed
305 306 307 308
(getting rid of all traces of it), terminate the experiment and then
go to the ``Profiles'' tab in your dashboard and delete (all versions of)
the profile as well.
Of course, you can only delete the profile if it belongs to you.