1. 11 Jul, 2014 1 commit
  2. 26 Mar, 2014 1 commit
    • Mike Hibler's avatar
      Create archive directory (expinfo) entry with unix gid of the project. · d123e03d
      Mike Hibler authored
      This is because at different times, users in different subgroups can
      create an experiment with the same name. If the directory has the
      unix group of the initial experiment with that name, then any other future
      experiment with that name but in a different subgroup will not be able
      to write the directory.
      d123e03d
  3. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  4. 15 Mar, 2012 1 commit
    • Leigh Stoller's avatar
      Add LINUX_FSNODE support contributed by Maarten Dewispelaere at · c8cbdf70
      Leigh Stoller authored
      intec.ugent.be
      
      These are not complete changes, but enough to support the usage these
      guys need. In this case, I think it is a hand crafted linux node that
      allows boss to ssh in and do the exports setup stuff. But hey, it is a
      start on actually supporting a linux FS node.
      c8cbdf70
  5. 25 Oct, 2011 1 commit
  6. 07 Nov, 2009 1 commit
    • Leigh Stoller's avatar
      Change to infodir (/usr/testbed/expinfo) handling; experiment · 41d34103
      Leigh Stoller authored
      directories are now placed in a project subdirectory, to avoid
      blowing out the max number of subdirs (32K in FreeBSD). Dirs are
      now called $pid/$eid/$idx.
      
      Added sanity checks to batchexp, swapexp, and endexp to watch for the
      case that testbed admin installed the new code but did not run the
      fixup script as instructed in doc/UPDATING.
      41d34103
  7. 06 Mar, 2007 1 commit
  8. 14 Jun, 2006 1 commit
    • Leigh Stoller's avatar
      The template "datastore" ... · fe9aa6a4
      Leigh Stoller authored
      Each template has a datastore, which is really just a subdirectory that can
      be populated with files, and committed to the subversion archive.  Note,
      the datastore os specific to the template itself. The Template Archive link
      on the Show Template page takes you to the subdirectory, which by
      convention I am calling "datastore".
      
      The directory actually lives in /proj/pid/exp/eid/TGUID-VERS ... but that
      path is printed out for you on the archive page.
      
      Anyway, put stuff in the datastore directory, and then commit the template
      archive so there is a tag associated with it.
      
      When an instance is created, a checkout of the datastore is placed in the
      experiment directory (/proj/pid/eid/exp/template_datastore). The current
      tag (from above) is stored with the instance so that we can later recreate
      the enviroment for the instance, say for rerun.
      
      Tarfiles and rpms in the datastore can be referenced as xxx://foo.rpm (in
      your NS file).  tarfiles_setup transforms those when the instance is
      swapped in, sorta like it does other URLs, only it does not actually fetch
      them, just need to rewrite the paths so they reference datastore.
      
      The program agent gets another environment variable so you can refer to the
      datastore without hardwiring paths ($DATASTORE). Eventually I want to move
      the checkout someplace else, but it was easy to drop it into the experiment
      directory for now.
      fe9aa6a4
  9. 28 Dec, 2005 1 commit
    • Leigh Stoller's avatar
      A fair number of changes. · 564d958d
      Leigh Stoller authored
      * The rest of the backend support for simplistic experiment duplication,
        either from an existing (current) experiment, or from a specific
        archive revision of a current or terminated experiment.
      
      * Rework the swapmod code so that the archive is committed at the exact
        end of the swapout phase. This required adding (and moving) some code
        from swapexp to tbswap sine that is where the actual swapout/swapin
        happens during a swapmod
      
      * Add a special directory called "archive" to the experiment
        directory, which is a place where users can store stuff they want
        saved away. This will eventually be a user defined set of
        directories, but this was good for getting the basic mechanism in
        place. Note that the when the contents of this directory are copied
        out for placement into the archive, it is an exact copy made with
        rsync.
      
      * No longer "clean" the contents of the temporary store between
        commits of the archive. This was creating a lot of headaches, and
        was also causing the revision history to get messed up. The downside
        of this is that we have to be more careful to explicitly delete
        files that the user no longer uses. I have not solved all these
        issues yet, so in the meantime files will get left in the archive
        even if the user no longer references them.
      564d958d
  10. 06 Dec, 2005 1 commit
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      
      The nitty-gritty:
      
      1. Keep disk image signature files for all nodes in an experiment.
      
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
      
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      
      2. Swapout-time hook for creating swapout image.
      
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
      
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      
      3. Client-side script for creating swapout image.
      
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
      
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      
      4. Auto creation of signature files for new images.
      
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
      
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      
      5. Web changes to set/clear saving of disk state at swapout time.
      
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
      
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
      ed0d25b4
  11. 31 May, 2005 1 commit
  12. 18 May, 2005 1 commit
  13. 10 Dec, 2003 1 commit
  14. 19 Nov, 2003 1 commit
  15. 17 Nov, 2003 1 commit
    • Leigh Stoller's avatar
      Merge the two state machines (batchstate and state) into a single · 2025e0bd
      Leigh Stoller authored
      state machine (state). All of the stuff that was previously handled by
      using batchstate is now embedded into the one state machine. Of
      course, these mostly overlapped, so its not that much of a change,
      except that we also redid the machine, adding more states (for
      example, modify phases are now explicit. To get a picture of the
      actual state machine, on boss:
      
      		stategraph -o newstates EXPTSTATE
      		gv newstates.ps
      
      Things to note:
      
      * The "batchstate" slot of the experiments table is now used solely to
        provide a lock for batch daemon. A secondary change will be to
        change the slot name to something more appropriate, but it can
        happen anytime after this new stuff is installed.
      
      * I have left expt_locked for now, but another later change will be to remove
        expt_locked, and change it to active_busy or some such new state name in
        the state machine. I have removed most uses of expt_locked, except those
        that were necessary until there is a new state to replace it.
      
      * These new changes are an implementation of the new state machine,
        but I have not done anything fancy. Most of the code is the same as
        it was before.
      
      * I suspect that there are races with the batch daemon now, but they
        are going to be rare, and the end result is probably that a
        cancelation is delayed a little bit.
      2025e0bd
  16. 27 Mar, 2003 1 commit
  17. 16 Sep, 2002 1 commit
    • Leigh Stoller's avatar
      Reorg of working directory and log file stuff for start/swap/end · 533dc18f
      Leigh Stoller authored
      experiment. Here is mail to tbops:
      
      * Moved the working directory for experiment setup/swap/end to a new
        directory located on boss instead of over NFS to /proj/$pid/$eid. This
        new location is /usr/testbed/expwork/$pid/$eid.
      
      * Changed the name of the directories we create in /usr/testbed/expinfo to
        $pid-$eid.$index where $index is a new autoincrement field in the DB
        table. I really hated the names that were created before.
      
      * Changed where logs are written from /tmp to the new location in
        /usr/testbed/expwork/$pid/$eid.
      
      Okay, why.
      
      * We no longer operate on NFS mounted directories that might hang. Its
        easier to catch the situation where a copy of the log file over at the
        end of experiment creation fails cause of an NFS problem.
      
      * We no longer have user writable files that are inputs to other parts of
        the system (like top and ptop files).  Not that a user would be bad, but
        it closes a hole.
      
      * We no longer copy user writable files from /proj to boss where we might
        fill up an important filesystem cause the user put a .ndz file in the the
        working directory. Not that a user would be bad, but it closes a hole.
      
      * Its easier to save all the log files this way, for each swap in and
        out.
      
      * Removing a directory over NFS is a royal irritant when someone is CD'ed
        into that directory or looking at a file on the other side (the astute
        observer will peg this as the reason I went down this idiotic path in the
        first place!).
      
      * About 6 other reasons that I can no longer remember. Seriously, I really
        had more reasons I can no longer remember! :-)
      533dc18f
  18. 07 Jul, 2002 1 commit
  19. 16 Oct, 2001 1 commit
  20. 23 Mar, 2001 1 commit
  21. 03 Jan, 2001 1 commit
  22. 06 Dec, 2000 1 commit
    • Leigh Stoller's avatar
      Checkpoint a set of hacky changes to support your own OSKit kernel · ce613288
      Leigh Stoller authored
      via the create-os directive in the NS file.
      tbsetup/ir/handle_os.tcl - Do a valid check for the image given with
      	set-node-os in the NS file, and propogate that information through
              to the IR file. Nothing is added to the DB.
      tbsetup/mkexpdir - Add a tftpboot to the list of experiment directories. The
      	tftpd daemon now allows kernels from /proj.
      tbsetup/os_setup - Very hacky changes to allow for multiboot kernels. Read
      	local images table and cross check against that for nodeos spec.
              Hardwire in "mb" as a special partition tag that says to not try
      	and do too much with it. This should be changed to a DB check of
      	some kind. On reboot, do not wait for these nodes to come alive since
      	there is no way to determine if an oskit kernel (or any foreign)
      	kernel is running.
      ce613288
  23. 01 Dec, 2000 1 commit