basic-concepts.scrbl

#lang scribble/manual

@(require "defs.rkt")

@title[#:tag "basic-concepts" #:version apt-version]{Basic Concepts}

This chapter covers the basic concepts that you'll need to understand in
order to use @(tb).

@section[#:tag "profiles"]{Profiles}

A @italic{profile} encapsulates everything needed to run an
@seclink["experiments"]{@italic{experiment}}.  It consists of two main parts: a
@seclink["rspecs"]{description of the resources} (hardware, storage, network,
etc.) needed to run the experiment, and the @seclink["disk-images"]{software
artifacts} that run on those resources.

The resource specification is in the @seclink["rspecs"]{RSpec} format. The
RSpec describes an entire @italic{topology}: this includes the nodes (hosts)
that the software will run on, the storage that they are attached to, and the
network that connects them. The nodes may be
@seclink["virtual-machines"]{virtual machines} or
@seclink["physical-machines"]{physical servers}. The RSpec can specify the
properties of these nodes, such as how much RAM they should have, how many
cores, etc., or can directly reference a specific @seclink["hardware"]{class of
hardware} available in one of @(tb)'s clusters. The network topology can include
point to point links, LANs, etc. and may be either built from Ethernet or
Infiniband.

The primary way that software is associated with a profile are through
@seclink["disk-images"]{disk images}. A disk image (often just called an
``image'') is a block-level snapshot of the contents of a real or virtual
disk---and it can be loaded onto either. A disk image in @(tb) typically has an
installation of a full operating system on it, plus additional packages,
research software, data files, etc. that comprise the software environment of
the profile. Each node in the RSpec is associated with a disk image, which it
boots from; more than one node in a given profile may reference the same disk
image, and more than one profile may use the same disk image as well.

Profiles come from two sources: some are provided by @(tb) itself; these tend
to be standard installations of popular operating systems and software
stacks. Profiles may also be provided by @(tb)'s users, as a way for communities
to share research artifacts.

@subsection[#:tag "on-demand-profiles"]{On-demand Profiles}

Profiles in @(tb) may be @italic{on-demand profiles}, which means that they are
designed to be instantiated for a relatively short period of time (hours or
days). Each person instantiating the profile gets their own
@seclink["experiments"]{experiment}, so everyone using the profile is doing so
independently on their own set of resources.

@subsection[#:tag "persistent-profiles"]{Persistent Profiles}

@(tb) also supports @italic{persistent} profiles, which are longer-lived (weeks
or months) and are set up to be shared by multiple users. A persistent profile
can be thought of as a ``testbed within a testbed''---a testbed facility built
on top of @(tb)'s hardware and provisioning capabilities. Examples of persistent
profiles include:

@itemlist[
@item{An instance of a cloud software stack, providing VMs to a large community}
@item{A cluster set up with a specific software stack for a class}
@item{A persistent instance of a database or other resource used by a large
research community}
@item{Machines set up for a contest, giving all participants access to the same
hardware}
@item{An HPC cluster temporarily brought up for the running of a particular
set of jobs}
]

A persistent profile may offer its own user interface, and its users may not
necessarily be aware that they are using @(tb).  For example, a cloud-style
profile might directly offer its own API for provisioning virtual machines.
Or, an HPC-style persistent profile might run a standard cluster scheduler,
which users interact with rather than the @(tb) website.

@apt-only{For the time being, allocations for persistent profiles on @(tb) are
handled by directly @seclink["getting-help"]{contacting the @(tb) staff}.}

@section[#:tag "experiments"]{Experiments}

@margin-note{See the chapter on @seclink["repeatable-research"]{repeatability}
for more information on repeatable experimentation in @(tb).}

An @italic{experiment} is an instantiation of a @seclink["profiles"]{profile}.
An experiment uses resources, @seclink["virtual-machines"]{virtual} or
@seclink["physical-machines"]{physical}, on one or more of the
@seclink["hardware"]{clusters} that @(tb) has access to. In most cases, the
resources used by an experiment are devoted to the individual use of the user
who instantiates the experiment. This means that no one else has an account,
access to the filesystems, etc. In the case of experiments using solely
@seclink["physical-machines"]{physical machines}, this also means strong
performance isolation from all other @(tb) users. @apt-only{In the case of
@seclink["virtual-machines"]{virtual machines}, there is still isolation from a
security and accounting standpoint, but weaker performance isolation.} (The
exceptions to this rule are @seclink["persistent-profiles"]{persistent
profiles}, which may offer resources to many users.)

Running experiments on @(tb) consume @bold{real resources}, which are
@bold{limited}. We ask that you be careful about not holding on to experiments
when you are not actively using them. If you are are holding on to experiments
because getting your working environment set up takes time, consider
@seclink["creating-profiles"]{creating a profile}.

@future-work["planned-persistent-storage"]

The contents of local disk on nodes in an experiment are considered
@italic{ephemeral}---that is, when the experiment ends (either by being
explicitly terminated by the user or expiring), the contents of the disk are
lost. So, you should copy important data off before the experiment ends. A
simple way to do this is through @tt{scp} or @tt{sftp}.  You may also
@seclink["creating-profiles"]{create a profile}, which captures the contents of
the disk in a @seclink["disk-images"]{disk image}.

All experiments have an @italic{expiration time}. By default, the expiration
time is short (a few hours), but users can use the ``Extend'' button on the
experiment page to request an extension. A request for an extension must
be accompanied by a short description that explains the reason for requesting
an extension, which will be reviewed by @(tb) staff.
@apt-only{@seclink["guest-users"]{Guest users} are not permitted to hold
experiments for very long; if you are using @(tb) as a guest, and find yourself
running out of time frequently, we recommend @seclink["register"]{registering
for an account}.} You will receive email a few hours before your experiment
expires reminding you to copy your data off or request an extension.

@subsection[#:tag "extending"]{Extending Experiments}

If you need more time to run an experiment, you may use the ``Extend'' button
on the experiment's page. You will be presented with a dialog that allows you
to select how much longer you need the experiment. Longer time periods require
more extensive appoval processes. Short extensions are auto-approved, while
longer ones require the intervention of @(tb) staff or, in the case of
indefinite extensions, the steering commitee.

@screenshot["extend-experiment.png"]

@section[#:tag "projects"]{Projects}

Users are grouped into @italic{projects}. A project is, roughly speaking, a
group of people working together on a common research or educational goal. This
may be people in a particular research lab, a distributed set of
collaborators, instructors and students in a class, etc.

A project is headed by a @italic{project leader}. We require that project
leaders be faculty, senior research staff, or others in an authoritative
position. This is because we trust the project leader to approve other members
into the project, ultimately making them responsible for the conduct of the
users they approve. If @(tb) staff have questions about a project's activities,
its use of resources, etc., these questions will be directed to the project
leader. Some project leaders run a lot of experiments themselves, while some
choose to approve accounts for others in the project, who run most of the
experiments. Either style works just fine in @(tb).

Permissions for some operations / objects depend on the project that they
belong to. Currently, the only such permission is the ability to make a profile
visible onto to the owning project. We expect to introduce more
project-specific permissions features in the future.

@section[#:tag "virtual-machines"]{Virtual Machines}

@apt-only{
    The default node type in @(tb) is a @italic{virtual machine}, or VM. VMs in
    @(tb) are currently implemented on
    @hyperlink["http://blog.xen.org/index.php/2013/07/09/xen-4-3-0-released/"]{Xen
    4.3} using
    @hyperlink["http://wiki.xenproject.org/wiki/Paravirtualization_(PV)"]{paravirtualization}.
    Users have full root access with their VMs via @tt{sudo}.

    VMs in @(tb) are hosted on shared nodes; this means that while no one else
    has access to your VMs, there are other users on the same hardware whose
    activities may affect the performance of your VMs.  @(tb) currently @italic{does}
    oversubscribe virtual cores, meaning VMs may get less CPU time than would be
    indicated by the number of virtual cores they have, depending on others'
    activity. It @italic{does not} oversubscribe RAM, meaning that all virtual
    RAM is backed by physical pages, though performance can still be affected
    by others' use of the memory bus, which is a shared resource. @(tb) @italic{does
    not} provide performance isolation for disks, so I/O performance may also vary
    depending on use. Finally, @(tb) @italic{does not} oversubscribe bandwidth on
    network links attached to VMs, though variance in performance due to
    fine-grained timing effects is still possible.
}

@clab-only{
    While @(tb) does have the ability to provision virtual machines itself
    (using the Xen hypervisor), we expect that the dominant use of @(tb) is
    that users will provision @seclink["physical-machines"]{physical machines}.
    Users (or the cloud software stacks that they run) may build their own
    virtual machines on these physical nodes using whatever hypervisor they 
    wish.
}

@section[#:tag "physical-machines"]{Physical Machines}

Users of @(tb) may get exclusive, root-level control over @italic{physical
machines}. When allocated this way, no layers of virtualization or indirection
get in the way of the way of performance, and users can be sure that no other
users have access to the machines at the same time. This is an ideal situation
for @seclink["repeatable-research"]{repeatable research}.

Physical machines are @seclink["disk-images"]{re-imaged} between users, so you
can be sure that your physical machines don't have any state left around from
the previous user. You can find descriptions of the
hardware in @(tb)'s clusters in the @seclink["hardware"]{hardware} chapter.

@apt-only{
Physical machines are relatively scarce, and getting access to large numbers of
them, or holding them for a long time, may require
@seclink["getting-help"]{contacting @(tb) staff}. 
    }