1. 03 Aug, 2007 1 commit
  2. 02 Aug, 2007 2 commits
  3. 31 Jul, 2007 1 commit
  4. 30 Jul, 2007 3 commits
  5. 27 Jul, 2007 3 commits
    • Leigh B. Stoller's avatar
      Terriblly bogus hacks to deal with symlinks when doing the cvs import · b2bafa07
      Leigh B. Stoller authored
      of the record. Basically, record them in the XML file, then remove
      them before the import, and restore them after. Did I mention that
      this was totally bogus?
      b2bafa07
    • Leigh B. Stoller's avatar
      Some fixes to deal with older templates that do not have a CVS repo. · f2888e91
      Leigh B. Stoller authored
      On modify, create the repo, but add checks to make sure that we do not
      bother with operations on branches that do not exist cause they
      predate the repo creation. This allows you to move forward after a
      modifying an existing template. Also add checks for instantiating a
      old template that has no repo at all cause it has not been modified since
      adding the repo support.
      
      A few other fixes to make sure that instances are cleaned up after errors.
      f2888e91
    • Leigh B. Stoller's avatar
      * Add new tables to store NS files (and any files they source) in the · f8333ef2
      Leigh B. Stoller authored
        DB alongside the resource records. Previously, we stored only the
        nsfiles for current experiments, and purged them when the experiment
        was terminated. The new approach saves them forever using the resource
        record ID. Note that we do not store copies of NS files, but reference
        them indirectly instead so that we can MD5 them and avoid the dups.
      
        I put a "compressed" bit into the table cause at some point we will
        start compressing the data before storing them into the DB. Or maybe
        we bag this and start using GFS!
      
        Also note that this addresses the problem of losing the NS file
        history when using swapmod, since the NS file is overwritten.
      
      * Add a pmapping table stores the nodes (and their types) used by an
        experiment. This data is also saved forever, alongside the resource
        records, so that we can more accurately replay an experiment. As Rob
        points out, the node names can also be used in conjunction with the
        ptop files that are saved, to get a 100% accurate remap of resources.
      f8333ef2
  6. 25 Jul, 2007 1 commit
  7. 19 Jul, 2007 1 commit
  8. 18 Jul, 2007 1 commit
  9. 17 Jul, 2007 1 commit
    • Leigh B. Stoller's avatar
      Add some sanity checking to make sure that the user is tagging the · 4c278840
      Leigh B. Stoller authored
      highest revision on the branch (or trunk) of the magic .template file,
      since doing otherwise would seem to indicate an inconsistency in
      what the user is doing. Why? Templates are modified via a tag
      operation of the .template file, but splitting off can only be done
      via a branch operation. Subsequent mods after a branch have to happen
      along a straight line.
      
      Also allow for tag moving to initiate a template modify since the
      above implied error can be corrected by doing a cvs update, and then a
      retag with the -F option to move the tag.
      4c278840
  10. 16 Jul, 2007 5 commits
    • David Johnson's avatar
      Support for accessing the v4 NM via a nm-controller slice account. Now, · 88e61ae3
      David Johnson authored
      for each delegated slice we create, we set the 'delegations' attribute to
      'utah_nmcontrol', the name of our nm-controller slice that has permissions
      to talk to the NM.
      
      Also, a few other fixes.
      88e61ae3
    • Leigh B. Stoller's avatar
      Generate a info.xml file in the records/run directory that describes · 055b8769
      Leigh B. Stoller authored
      the run. Includes the bindings for the run and the metadata name/value
      pairs. This is the same data that is dumped by template export, so
      at this point template_export has been completely superceded by the
      CVS repo.
      055b8769
    • David Johnson's avatar
      Several things in this commit: · 3bbd843b
      David Johnson authored
        * Prior to this commit, libplab depended on db state to create slivers
          and slices.  Now it can be done using the regular command line tools
          without the metadata in the db.  This makes development and debugging
          much easier and allows us to use the command-line tools even if state has
          been cleared out of the db (i.e., for sliver garbage collection).
        * Add support for sliver start/stop/restart via the v4 NM.
        * Some support for sliver garbage collection.
        * Various other improvements and cleanup.
      3bbd843b
    • Leigh B. Stoller's avatar
      Remove debugging code. · 4a50330d
      Leigh B. Stoller authored
      4a50330d
    • Leigh B. Stoller's avatar
      The CVS repo will now have two subdirs, one called "setup" which is · 1aaa383c
      Leigh B. Stoller authored
      where the current stuff goes (datastore, nsfile, etc.). The other
      subdir is called "records" and will be populated with stuff that we
      currently put into the archive (the contents of the experiment
      directory).
      
      There is a record for each run, currently named XXX,YY where XXX is
      the instance ID and YY is the run number. I plan to add an XML file to
      map these to the symbolic names, but these are not required to be
      unique, which is why the directories are not named with them.
      1aaa383c
  11. 14 Jul, 2007 2 commits
  12. 13 Jul, 2007 1 commit
  13. 12 Jul, 2007 1 commit
    • Russ Fish's avatar
      Check that the inner elab db schema isn't out of date wrt the outer one, · b54ae4da
      Russ Fish authored
        since we import a bunch of dumped db state from the outer to the inner db.
        Added/removed columns in the schema misalign row data, causing much chaos.
      
      tbsetup/elabinelab.in - Dump and send the outer db schema with the db state.
        Runs mysqldump just like schemacheck does.
      
      tmcd/common/config/rc.mkelab - Add the schema check using schemadiff.
      
      utils/schemadiff - Add an option to ignore table creates and drops.
      b54ae4da
  14. 10 Jul, 2007 1 commit
    • Leigh B. Stoller's avatar
      Changes to switch from using tag to rtag. Note that this commit requires · 7a2ef6f1
      Leigh B. Stoller authored
      a locally modifed version of CVS that does a couple of extra things. The
      hacks are quite simple and currently on users. Briefly, there is a new
      config file option to specify a program to run when the tag operation is
      complete (similar to a module hook for rtag), and in tag.c I run that hook.
      
      Check out a sandbox as before:
      
        cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo rtag mytag XXXXX
      
      To create a new template (along the trunk), use the tag operation instead:
      
      	cd sandbox
      	cvs tag mtag
      
      and what for the email that says the template is created. You can
      continue to work along the trunk in this manner, although note that
      the .template file is changing (by the backend commit), and you might
      find it less distriacting to do a cvs update each time.
      
      To work along a branch:
      
      	cd sandbox
      	cvs tag -r existing_tag -b newtag1
              cvs update -r newtag
              make changes and commits
      	cvs tag newtag2
      
      When working along branched you will want to do a cvs update to get
      the .template file in sync.
      7a2ef6f1
  15. 09 Jul, 2007 1 commit
    • Leigh B. Stoller's avatar
      Checkpoint my cvs interface to the workbench. This first cut uses the · 8371fc79
      Leigh B. Stoller authored
      "rtag" directive to initiate template modify operations. So, to get started
      you do a checkout:
      
        cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo checkout XXXXX
      
      where XXXXX is the part of the guid (10000/1) before the slash. Might try
      and roll all templates into a single project wide repo at some point, to
      avoid the extraneous path stuff, but didn't want to worry that just yet.
      
      Okay, so have a checkout. You can work along the trunk, doing commits. To
      create a new template (a modify of the existing template), you tag the tree
      using rtag:
      
        cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo rtag mytag XXXXX
      
      A template modify is started at the end, and you should probably wait for
      email before continuing. Eventually I will need to add locking of some
      kind, but I have to do the modify in the background, or else I get deadlock
      cause cvs keeps the repo locked, and the modify also needs to access it.
      
      Each time you tag along the trunk, you get a modified template, which in
      the history diagram looks like:
      
        10000/1 --> 10000/2 --> 10000/3 ...
      
      If you want to branch, say at 10000/2 you can create a branch tag using rtag:
      
        cvs -d [cut] rtag -r T10000/2 -b mytag2 XXXXX
      
      You can also use your own tags for -r option, but I also create a TXXXXX/YY
      tag at each template modify, which is easy to remember.
      
      Then update your sandbox to the new branch, commit changes along that
      branch, and then later use rtag again to initiate a template modify
      operation:
      
        cvs update -r mytag2
        cvs commit ...
        cvs -d [cut] rtag -r mytag2 mytag3 XXXXX
      
      And now the history diagram looks like:
      
        10000/1 --> 10000/2 --> 10000/3 ...
                      |
                      |
                      -> 10000/4 ...
      
      You should be able to mix interaction via the web with interaction via the
      cvs interface. I've tested it, although not extensively.
      8371fc79
  16. 20 Jun, 2007 1 commit
    • Leigh B. Stoller's avatar
      Add summary node utilization stats. The initial values are derived by · 21890006
      Leigh B. Stoller authored
      processing the node_history table, but that is *way* too slow to do on
      the fly (say, from the web interface) cause of the number of records,
      so the summary info is stored in the new node_utilization table. I
      generate the summary info as new entries are added to node_history (in
      SetNodeHistory) but if that becomes too messy, we can just as easily
      shift to processing the table on a nightly basis.
      
      Note that I added a new "inception" date field to the nodes table,
      which will get set on new nodes, but for existing nodes I have to
      derive it from the first entry in the node_history table, or else the
      numbers will not make sense.
      21890006
  17. 19 Jun, 2007 1 commit
    • Leigh B. Stoller's avatar
      Big update to the stats gathering code ... · 495f6803
      Leigh B. Stoller authored
      This change attempts to make the stats gathering code more reliable by
      not relying on the testbed_stats records to reconstruct usage
      statistics.  The main source of errors and total confusion in the
      current stats code is that testbed_stats includes all the errors and
      transitions, from which I have to reconstruct what happened in order
      to determine usage by a project or user.
      
      The new stats code still generates the testbed_stats code, but actual
      usage is recorded as it happens, in the experiment_resources table, as
      swapins, swapouts, and swapmods occur. Its also much faster to compute
      the data for the tables in the web interface, not having to scan a
      zillion testbed_stats records in php.
      
      There is a time consuming update to the records that takes place with
      a lot of tables locked.
      495f6803
  18. 15 Jun, 2007 1 commit
  19. 06 Jun, 2007 1 commit
  20. 30 May, 2007 1 commit
  21. 28 May, 2007 1 commit
    • Leigh B. Stoller's avatar
      Implement two requests by Eric: · e12ea1e2
      Leigh B. Stoller authored
      * The checkout directory now includes just the datastore directory and
        tbdata/nsfile.ns. This removes a bunch of clutter.
      
      * You can now run template_commit in any subdir of a checkout; the
        python wrapper will crawl upwards looking for the .template file.
      
        Note that I removed the option that allowed you to do a commit from
        the template tree in /proj/pid/templates since that directory will
        ultimately go away (or at least hide from view), and cause it
        conflicted with the new option and I didn't want to make things any
        messier for no reason.
      e12ea1e2
  22. 24 May, 2007 1 commit
  23. 23 May, 2007 2 commits
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      First cut at template checkout and commit from a checkout. The interface · b674bc7b
      Leigh B. Stoller authored
      described is the one exported to ops via the XMLRPC interface. This is
      just playing aroundl no doubt this stuff is going to change.
      
      * template_checkout guid/vers
      
        Checkout a copy of the template to the current working directory.
      
      * template_commit
      
        Modify the previous template checkout, using the nsfile contained in
        the tbdata directory (subdir of the current directory). In other words,
        the current template is modified, creating a new template in the
        current working directory (the current directory refers to the new
        template).
      
        The datastore subdir is imported into the new template, but that is
        the only directory that is imported at present. Might change that.
      
      So this sounds much cooler then it really is. Why?
      
      * This only works from ops.
      
      * The "current directory" must be one of the standard approved directories
        (/proj, /users, /groups).
      
      * Cause, boss reads and writes that directory via NFS, as told to it
        by the xmlrpc client.
      
      At some point in the future it would be nice to support something
      fancier, using a custom transport, but lets see how this goes.
      b674bc7b
  24. 18 May, 2007 1 commit
  25. 17 May, 2007 2 commits
  26. 16 May, 2007 1 commit
  27. 15 May, 2007 2 commits