Commit 03098e22 authored by Robert Ricci's avatar Robert Ricci

Merge branch 'master' into 'master'

emulab-manual

See merge request !8
parents 467c8169 6a53d728
Pipeline #2167 passed with stages
in 2 minutes and 51 seconds
......@@ -126,7 +126,7 @@ expires reminding you to copy your data off or request an extension.
@subsection[#:tag "extending"]{Extending Experiments}
If you need more time to run an experiment, you may use the ``extend'' button
If you need more time to run an experiment, you may use the ``Extend'' button
on the experiment's page. You will be presented with a dialog that allows you
to select how much longer you need the experiment. Longer time periods require
more extensive appoval processes. Short extensions are auto-approved, while
......
......@@ -78,7 +78,7 @@
('apt "https://www.aptlab.net/")
('clab "https://www.cloudlab.us/")
('pnet "https://www.phantomnet.org/")
('elab "https://www.emulab.net/")))
('elab "https://www.emulab.net/portal/")))
(define apt-doc-url
(case (tb-mode)
......
#lang scribble/manual
@(require "defs.rkt")
@title[#:tag "hardware" #:version apt-version]{Hardware}
@section[#:tag "emulab-mothership"]{Emulab Cluster}
The Emulab cluster at the University of Utah contains almost 500 servers across
four CPU generations with a total of more than 3,800 cores.
@; XXX don't use apturl right now
More technical details can be found at
@url["https://www.emulab.net/hardware.php"].
@(nodetype "d430" 160 "Haswell, 16 core, 3 disks"
(list "CPU" "Two Intel E5-2630v3 8-Core CPUs at 2.4 GHz (Haswell)")
(list "RAM" "64GB ECC Memory (8x 8 GB DDR4 2133MT/s)")
(list "Disk" "One 200 GB 6G SATA SSD")
(list "Disk" "Two 1 TB 7.2K RPM 6G SATA HDDs")
(list "NIC" "Two or four Intel I350 1GbE NICs")
(list "NIC" "Two or four Intel X710 10GbE NICs"))
@(nodetype "d820" 16 "Sandy Bridge, 32 core, 7 disks"
(list "CPU" "Four Intel E5-4620 8-Core CPUs at 2.2 GHz (Sandy Bridge)")
(list "RAM" "128GB ECC Memory (8x 16 GB DDR3 1333MHz)")
(list "Disk" "One 250 GB 7.2K RPM 3G SATA HDD")
(list "Disk" "Six 600 GB 10K RPM 3G SAS HDDs")
(list "NIC" "Four Intel X520 10GbE NICs"))
@(nodetype "d710" 160 "Nehalem, 4 core, 2 disks"
(list "CPU" "One Intel Xeon E5530 4-Core CPU at 2.4 GHz (Nehalem)")
(list "RAM" "12GB ECC Memory (6x 2 GB DDR2 1066MHz)")
(list "Disk" "One 500 GB 7.2K RPM SATA HDD")
(list "Disk" "One 250 GB 7.2K RPM SATA HDD")
(list "NIC" "Four Broadcom BCM5709 1GbE NICs"))
@(nodetype "pc3000" 160 "Nocona, 1 core, 2 disks"
(list "CPU" "One Intel Xeon single-core CPU at 3.0 GHz (Nocona)")
(list "RAM" "2GB ECC Memory (2x 1 GB DDR2 400MHz)")
(list "Disk" "Two 146 GB 10K RPM SCSI HDD")
(list "NIC" "1-4 Intel 1GbE NICs")
(list "NIC" "1-4 Intel 10/100 NICs"))
All nodes are connected to two networks:
@itemlist[
@item{A 100Mb and 1 Gb @italic{Ethernet} @bold{``control network''}---this network
is used for remote access, experiment management, etc., and is
connected to the public Internet. When you log in to nodes in your
experiment using @code{ssh}, this is the network you are using.
@italic{You should not use this network as part of the experiments you
run in Apt.}
}
@item{A 10/100Mb, 1/10Gb Ethernet
@bold{``experiment network''}--each node has at least four
interfaces on this network. This network is implemented using multiple
Cisco, HP, Arista, and Dell (Force10) switches.
}
]
......@@ -41,8 +41,10 @@ you can apply to start a new project.
@include-section["repeatable-research.scrbl"]
@include-section["creating-profiles.scrbl"]
@include-section["basic-concepts.scrbl"]
@include-section["emulab-transition.scrbl"]
@include-section["reservations.scrbl"]
@include-section["geni-lib.scrbl"]
@include-section["advanced-topics.scrbl"]
@include-section["hardware.scrbl"]
@include-section["emulab-hardware.scrbl"]
@include-section["planned.scrbl"]
@include-section["getting-help.scrbl"]
#lang scribble/manual
@(require "defs.rkt")
@title[#:tag "emulab-transition" #:version apt-version]{Transitioning from the ``Classic'' Interface to the ``Portal''}
The new Emulab portal interface is a custom version of the
@hyperlink["http://cloudlab.us"]{CloudLab} interface, so if you are familiar
with that interface, using the Emulab portal will be easy!
The portal currently supports the most commonly used features of the classic
interface, with more features ported on demand.
@bold{The classic interface should be considered deprecated},
with no new features being added.
It will continue to be supported until all important
features have been moved to the portal interface.
The remainder of this section covers
@seclink["emulab-cloudlab-features"]{new and improved features}
that the portal interface offers over the classic interface,
@seclink["emulab-missing-features"]{features currently missing}
from the portal interface, and how to
@seclink["emulab-conversion"]{convert an existing experiment}
into a portal experiment.
@section[#:tag "emulab-cloudlab-features"]{New and Improved Features of the Portal Interface}
Besides a much updated look and feel, the portal interface offers significant
new features and improvements to existing features.
@subsection{Profiles}
One important change that is more than just an interface issue,
is the clear delineation between an experiment's @italic{description}
and its @italic{instantiation}.
In the classic interface, the term ``experiment'' is used to refer to
both: experiment descriptions and instances can be created and destroyed
together or, the experiment can be created just as a description and then
``swapped in'' (instantiated with resources) and ``swapped out''
(resources freed) repeatedly.
Through the portal,
experiment descriptions are now known as @seclink["profiles"]{profiles}
and the term @seclink["experiments"]{experiment} is used to refer to a
running instance of a profile.
Think of a @italic{profile} as a ``swapped out'' classic experiment
and an @italic{experiment} (or @italic{instance}) as a ``swapped in'' classic
experiment.
The portal offers a variety of ways in which to create new profiles,
including a web-based GUI and a Python-based scripting library.
See the section on @seclink["creating-profiles"]{creating profiles}
for details.
In addition, the Emulab portal provides its own method for
@seclink["emulab-conversion"]{converting classic experiments to profiles}.
The profile representation of an experiment is different from that of
the classic experiment.
In the classic interface, experiments are described using @tt{ns-2},
a network simulator scripting language based on TCL.
The portal represents experiments at the base level as GENI
@seclink["rspecs"]{RSpecs} with the Python-based
@seclink["geni-lib"]{geni-lib}
interface available for scripting experiments.
As described in the @seclink["emulab-conversion"]{conversion} section
below, the portal can automatically convert classic ns-2 experiment
descriptions into geni-lib based profiles.
In addition to the hardware and software resources required by an experiment,
a profile also contains various metadata.
The Description and Instructions fields permit you to provide additional
information to the user of a profile.
These fields support @seclink["markdown"]{Markdown} formatting allowing more
expressive text.
Classic experiments allow only a short, ASCII-only description.
Profiles can also have @italic{parameters} with descriptions and default
values. This permits customized instantiations from a single profile.
For example, you can
have parameters to specify the number or types of nodes in an experiment.
In classic experiments this customization is accomplished by setting TCL
variables in the NS file.
Using a different value for the variables entails either
modifying the experiment while it is swapped out or creating a new experiment
with different values for the variables.
With profile parameters, values are provided through the interface at
instantiation time.
By default, profiles have the same visibility that classic experiments
do---only people in the same project can instantiate an
experiment from a profile.
However, you can also make profiles visible by ``Anyone,'' marking the profile
global and allowing any experimenter in any project to instantiate it.
@subsection{Web-based Interaction with Nodes}
The portal page for an active experiment has a dashboard allowing interaction
with all nodes. You can fire off in-browser console or SSH sessions, reboot,
and view activity graphs.
@subsection{Disk Images}
Custom disk images are now created by taking a @italic{snapshot} of an
experiment node or @italic{cloning} an existing experiment to create
a new profile.
The disk image format has not changed---the same images are used for
classic or portal experiments---but the way in which images are
named in a profile is different.
Images are now identified by URNs.
For example, you would use
@tt{urn:publicid:IDN+emulab.net+image+emulab-ops//FBSD103-64-STD}
in a profile where you would use
@tt{emulab-ops/FBSD103-64-STD}
in an NS file.
The URN for an image can be obtained from the ``Images'' tab on the user
dashboard.
See the section on @seclink["disk-images"]{disk images} for more information.
@subsection{Versioning}
Both profiles and disk images are now versioned.
Whenever you make a change to a profile,
you create a new version of that profile.
Whenever you take a snapshot of a node running a custom image,
you create a new version of that image.
When using a profile or image and no explicit version is specified,
you get the most recent version.
You can delete old versions of images and profiles (subject to some
constraints).
Deleting the current version effectively ``rolls back''
the profile or image to the previous version.
@subsection{Experiment Extensions}
The portal interface has a more rigorous procedure for @italic{extending}
a running experiment.
In the classic interface you can extend a running experiment by modifying
the ``Max Duration'' setting within a small range (0 - 120 hours).
Longer extensions require an email exchange with the admins.
In the portal interface, you request an extension via the web interface
with shorter extensions being approved instantly and longer extensions
automatically forwarded to admins for their consideration.
Be aware that even short extensions may not be possible if the extension
would conflict with a future reservation (described below).
See the section on @seclink["extending"]{extending experiments}
for more information.
@subsection{Resource Reservations}
A major new feature supported via the portal interface is the ability to
automatically reserve node resources at a future time.
Emulab has traditionally been a first-come-first-serve, best-effort facility
and reserving nodes required prior arrangement with Emulab staff.
Through the portal, you can now ensure that a given number of nodes of a
specific type will be available during a future time window.
See the section on @seclink["reservations"]{reservations} for details.
@section[#:tag "emulab-missing-features"]{Classic Features Not Currently Supported in the Portal}
A number of prominent Emulab features are not currently supported via the
portal interface.
These include:
@itemlist[
@item{
@bold{Events.}
The Emulab @link["https://wiki.emulab.net/wiki/eventsystem"]{event system}
is not currently exposed through the portal interface.
Specifically, you cannot schedule events from a profile.
There is also no access to the program agent, traffic generator, or
dynamic traffic shaping as those all use events to control execution.
}
@item{
@bold{Modifying a Live Experiment.}
The classic interface allows you to modify the description (e.g.,
network topology or node OSes) of a running experiment.
This is not supported under the portal interface.
}
@item{
@bold{Batch Experiments.}
Automated execution of experiments via the
@link["https://wiki.emulab.net/wiki/Tutorial#section-21"]{batch system}
is not supported.
Portal experiments can include ``startup services,'' but there is no
current equivalent to starting an experiment when resources become
available and then automatically terminating it when done.
}
@item{
@bold{Asymmetric Link Shaping.}
The classic interface supports the ability to shape a link with
different characteristic (BW, latency, loss) in each direction.
The portal interface only allows specification of one set of
characteristics that apply in both directions.
}
]
@section[#:tag "emulab-conversion"]{Converting an Emulab experiment to a Profile}
As a convenience to users, we provide a mechanism to convert NS-based
experiments into geni-lib profiles.
@margin-note{
If you have no classic Emulab experiments (swapped in or swapped out)
then you can skip this section and return to the
@seclink["getting-started"]{getting started} section.
}
Before walking through an example conversion, there are a couple of
things to be aware of:
@itemlist[
@item{
The converter does not support all Emulab NS extensions, only those
most commonly used by users.
}
@item{
The conversion process is @italic{not} a source-to-source translation,
rather it works by interpreting the NS script to generate state for
the Emulab DB and then generating the geni-lib script from that state.
From a practical standpoint, what this means is that if you use TCL
loops, conditionals or variables in your original description, those
will not appear in the resulting geni-lib script--loops will be unrolled,
conditionals evaluated, and variables bound to fix values.
}
]
When you login to the new portal interface at @url[(apturl)], you will be
taken to your @italic{dashboard} on which there are a number of tabs.
If you have any swapped in experiments you will start on the ``Experiments''
tab, otherwise you will be on the ``Profiles'' tab.
In either case you will be presented with a list of ``Classic Emulab
Experiments.''
@margin-note{
On the ``Profiles'' tab, the list of classic experiments will show only
those that are swapped out.
On the ``Experiments'' tab, it will show only those that are swapped in.
}
On the right hand side of the Name column, there will be an arrow icon:
@screenshot["classic-list.png"]
Clicking on this icon will start the conversion process for that experiment.
The process will take anywhere from seconds to a minute or more depending
on the complexity of the experiment description.
After conversion, you will see:
@screenshot["classic-conversion1.png"]
Typically, all you would do at this point is to click ``Create.''
However, you may want or need to change the profile name if you don't like
the default
(same as the classic experiment name) or if the creation fails because
the name is already taken.
You may also want to examine, or even modify, the new profile prior to
actually creating it. Use either the ``Edit Code'' or ``Edit Topology''
buttons for this. The former allows you to modify the python/geni-lib script
representation, the latter fires up the embedded @seclink["jacks"]{Jacks} GUI.
After clicking ``Create,'' you will be taken to a new page:
@screenshot["classic-conversion2.png"]
showing details about the new profile (on the left) and allowing you
to further edit the profile, copy or delete it.
On the bottom right is a button to ``Instantiate'' the profile---the
equivalent of ``swapping in'' an experiment in the classic interface.
Clicking that button takes you to the third and final step of the
instantiation wizard:
@margin-note{
The first step is skipped because the created profile is automatically
chosen and the second step because converted NS experiments
have no parameters.
}
@screenshot["classic-instantiate.png"]
where you can optionally name this instance and then click ``Finish.''
Once instantiated the node icons should turn green as they finish configuring
and the experiment state should change to ``ready'' and you will be off and
running!
@screenshot["classic-running.png"]
From the ``Topology View'' you can interact with the experiment.
By clicking on a node in the topology view as shown, you can perform a variety
of useful actions including: starting an in-browser shell or console window,
rebooting, or taking a snapshot.
Going to the ``List View'' tab presents you with a more Emulab-like listing:
@screenshot["classic-list-nodes.png"]
showing the node names and types as well as a per-node ``Actions'' menu for
console, SSH, rebooting, etc.
The settings drop down in the header allows you to perform certain actions
(e.g., rebooting) on all selected nodes.
If you are nearing the expiration time for your experiment and need to
keep it for longer, click on the green ``Extend'' button and then use the
slider or ``Requested'' text box to choose an extension time.
Keep in mind that an extension may not be possible or may require admin
approval.
Once you are finished with your experiment, you can ``swap out'' the
experiment using the red ``Terminate'' button.
This destroys the instance but leaves the profile intact.
If you want to terminate the experiment in the classic sense
(getting rid of all traces of it), terminate the experiment and then
go to the ``Profiles'' tab in your dashboard and delete (all versions of)
the profile as well.
Of course, you can only delete the profile if it belongs to you.
......@@ -4,6 +4,15 @@
@title[#:tag "getting-started" #:version apt-version]{Getting Started}
@elab-only{
@bold{Note:} if you are a current user of Emulab and are accustomed to the
@italic{classic} Emulab interface (@url["https://www.emulab.net/"]),
you may first want to read the @seclink["emulab-transition"]{Emulab transition}
section that discusses differences between that interface and this new
CloudLab-based interface and how to convert your current Emulab experiments
for use via the new interface.
}
This chapter will walk you through a simple experiment on @(tb) and
introduce you to some of its @seclink["basic-concepts"]{basic concepts}.
......@@ -202,6 +211,10 @@ Start by pointing your browser at @url[(apturl)].
@item{Making your own profiles is easy: see the
@seclink["creating-profiles"]{chapter on profile creation} for instructions.}
@elab-only{@item{If you want to run an existing Emulab experiment: see the
@seclink["emulab-transition"]{chapter on transitioning from the Emulab Classic
interface} for instructions.}}
@item{If you need help, or have questions or comments about @(tb)'s features,
@seclink["getting-help"]{contact us}!}
]
......@@ -115,7 +115,7 @@ You may have more than one reservation at a time; if you need resources of
more than one type, or on different clusters, you can get this by requesting
mutliple reservations.
@section[#:tag "reservations-making"]{Using a Reservation}
@section[#:tag "reservations-using"]{Using a Reservation}
To use a reservation, simply create experiments as normal. Experiments run
during the duration of the reservation (even those begun before its start
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment