repeatable-research.scrbl 2.03 KB
Newer Older
1 2 3
#lang scribble/manual
@(require "defs.rkt")

4
@title[#:tag "repeatable-research" #:version apt-version]{Apt and Repeatable Research}
5 6 7 8 9 10 11

One of Apt's key goals is to enable @italic{repeatable research}---we aim to
make it easier for researchers to get the same software and hardware
environment so that they can repeat or build upon each others' work.

Virtual machines in Apt do @bold{not} provide strong resource guarantees or
performance isolation from other concurrent users. Therefore, they are suitable
Robert Ricci's avatar
Robert Ricci committed
12
primarily under the following conditions:
13 14

@itemlist[
Robert Ricci's avatar
Robert Ricci committed
15
    @item{During initial exploration, development, gathering of preliminary
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
        performance results, etc.}
    @item{When it is the output of the code, rather than the code's performance,
        that is of interest.}
]

We therefore recommend that researchers who want to provide a repeatable 
environment do the following:

@itemlist[#:style 'ordered
    @item{Do their initial development work and gather initial numbers using
        virtual machines. Because a lot of time in this phase is often spent
        on debugging, development, etc., using a full physical machine is often
        an inefficient use of resources.}
    @item{Switch to physical machines to collect numbers of publication. This
        ensures that published numbers are not affected by interference from
        other users of Apt.}
]

Similarly, for those who are repeating or building on the work of others, we
recommend:

@itemlist[#:style 'ordered
    @item{During the initial, exploratory phase of figuring out how to run
        the code, examining it's configuration and parameters, etc., use
        virtual machines.}
    @item{Switch to physical machines when it's time to do real experiments,
        compare performance, etc.}
]

Because the @secref{images} that Apt provides boot on both virtual and physical
Robert Ricci's avatar
Robert Ricci committed
46
machines, in most cases, switching means simply modifying one's profile to
47 48 49 50
request the other type of resource.

@TODO{This suggests we ought to make it easier at instantiation time to pick
VMs or bare metal.}