Commit a5c04574 authored by Leigh Stoller's avatar Leigh Stoller

Add dataset example.

parent c2f20ad7
Pipeline #1838 failed with stages
in 2 minutes and 28 seconds
"""This profile demonstrates how to mount a copy of a dataset in a file system. You can make changes to the dataset, but they are *temporary** (temporary means any changes will be lost when you terminate your experiment). You must explicitly request that the changes be saved if you want them to persist.
Instructions:
Log into your node, your dataset is mounted at `/mydata`.
"""
# Import the Portal object.
import geni.portal as portal
# Import the ProtoGENI library.
import geni.rspec.pg as pg
# Import the emulab extensions library.
import geni.rspec.emulab
# Create a portal context.
pc = portal.Context()
# Create a Request object to start building the RSpec.
request = pc.makeRequestRSpec()
# Allocate a node and ask for a copy of a dataset to be mounted at /mydata
node = request.RawPC("node")
bs = node.Blockstore("bs", "/mydata")
# A demonstration dataset, there is nothing in the file system.
bs.dataset = "urn:publicid:IDN+emulab.net:testbed+imdataset+pgimdat"
# Print the RSpec to the enclosing page.
pc.printRequestRSpec(request)
"""This profile demonstrates how to add some extra *local* disk space on your node. In general nodes have much more disk space then what you see with `df` when you log in. That extra space is in unallocated partitions or additional disk drives. An *ephemeral blockstore* is how you ask for some of that space to be allocated and mounted as a **temporary** filesystem (temporary means it will be lost when you terminate your experiment).
Instructions:
Log into your node, your **temporary** file system in mounted at `/mydata`.
"""
# Import the Portal object.
import geni.portal as portal
# Import the ProtoGENI library.
import geni.rspec.pg as pg
# Import the emulab extensions library.
import geni.rspec.emulab
# Create a portal context.
pc = portal.Context()
# Create a Request object to start building the RSpec.
request = pc.makeRequestRSpec()
# Allocate a node and ask for a 30GB file system mounted at /mydata
node = request.RawPC("node")
bs = node.Blockstore("bs", "/mydata")
bs.size = "30GB"
# Print the RSpec to the enclosing page.
pc.printRequestRSpec(request)
......@@ -349,6 +349,79 @@ by constructing a @geni-lib["portal.ParameterError"] object, which can be passed
the portal context's @geni-lib["portal.Context.reportError" 'func] method to abort generation of
the RSpec.
@section[#:tag "local-diskpace"]{Add temporary local disk space to a node}
@code-sample["geni-lib-local-diskspace.py"]
This example demonstrates how to request @bold{extra temporary diskspace}
on a node. The extra disk space is allocated from unused disk partitions,
and is mounted at a directory of your choosing. In the example code above,
we are asking for a 30GB file system mounted at @tt{"/mydata"}. Anything
you store in this file system is @bold{temporary}, and will be @bold{lost}
when your experiment is terminated.
The total size of the file systems you can ask for on a node, is obviously
limited to the amount of unused disk space available. The system does its
best to find nodes with enough space to fulfill the request, but in general
you are limited to temporary file systems in the 10s of, or a few hundred
GB.
@section[#:tag "local-dataset"]{Creating a reusable dataset}
@code-sample["geni-lib-local-dataset.py"]
In this example, we demonstrate how to create and use a @italic{dataset}. A
dataset is simply a snapshot of a temporary file system (see the previous
example) that has been saved to permanent storage, and reloaded on a node
(or nodes) in a different experiment. This type of dataset must be
explicitly saved (more on this below) in order to make changes permanent
(and available later). In the example code above, the temporary file system
will be loaded with the dataset specified by the URN.
But before you can use a dataset, you first have to create one using the
following steps:
@itemlist[ #:style 'ordered
@instructionstep["Create an experiment"]{Create an experiment using the
@seclink["local-diskpace"]{local diskspace example} above.}
@instructionstep["Add your data"]{Populate the file system mounted at
@tt{/mydata} with the data you wish to use in other experiments.}
@instructionstep["Fill out the Create Dataset form"]{Click on the
@tt{"Create Dataset"} option in the @tt{Actions} menu. This will bring
up the form to create a new dataset. Choose a name for your dataset and
optionally the project the dataset should be associated with. Be sure
to select @bold{Image Backed} for the type. Then choose the the
experiment, which node in the experiment, and which blockstore on the
node.
@screenshot["create-imdataset.png"]
}
@instructionstep["Click ``Create" #:screenshot "snapshot-dataset.png"]{
When you click the ``Create'' button, the file system on your node
will be unmounted so that we can take a consistent snapshot of the
contents. This process can take several minutes or longer,
depending on the size of the file system. You can watch the
progress on this page. When the progress bar reaches the ``Ready''
stage, your new dataset is ready! It will now show up in your
``List Datasets'' list, and can be used in new experiments.}
@instructionstep["Use your dataset" #:screenshot "show-dataset.png"]{
To use your new dataset, you will need to reference it in your geni
lib script (see the example code above). The @italic{name} of your
dataset is a URN, and can be found on the information page for the
dataset. From the @tt{Actions} menu, click on @tt{"List Datasets"},
find the name of your dataset in the list, and click on it.}
@instructionstep["Update your dataset"]{If you need to make
changes to your dataset, simply start an experiment that uses your
dataset. Make the changes you need to the file system mounted at
@tt{/mydata}, and then use the @tt{"Modify"} button as shown in the
previous step.}
]
@section[#:tag "geni-lib-debugging"]{Debugging @tt{geni-lib} profile scripts}
It is not necessary to instantiate the profile via the portal web interface
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment