1. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  2. 11 Oct, 2010 1 commit
    • Leigh Stoller's avatar
      Work on an optimization to the perl code. Maybe you have noticed, but · 92f83e48
      Leigh Stoller authored
      starting any one of our scripts can take a second or two. That time is
      spent including and compiling 10000s of thousands of lines of perl
      code, both from our libraries and from the perl libraries.
      
      Mostly this is just a maintenance thing; we just never thought about
      it much and we have a lot more code these days.
      
      So I have done two things.
      
      1) I have used SelfLoader() on some of our biggest perl modules.
         SelfLoader delays compilation until code is used. This is not as
         good as AutoLoader() though, and so I did it with just a few 
         modules (the biggest ones).
      
      2) Mostly I reorganized things:
      
        a) Split libdb into an EmulabConstants module and all the rest of
           the code, which is slowly getting phased out.
      
        b) Move little things around to avoid including libdb or Experiment
           (the biggest files).
      
        c) Change "use foo" in many places to a "require foo" in the
           function that actually uses that module. This was really a big
           win cause we have dozens of cases where we would include a
           module, but use it in only one place and typically not all.
      
      Most things are now starting up in 1/3 the time. I am hoping this will
      help to reduce the load spiking we see on boss, and also help with the
      upcoming Geni tutorial (which kill boss last time).
      92f83e48
  3. 05 Dec, 2007 1 commit
  4. 26 Nov, 2007 1 commit
  5. 06 Nov, 2007 1 commit
    • Leigh Stoller's avatar
      This started out as a simple change to turn the datastore into a CVS · c1cff09b
      Leigh Stoller authored
      sandbox, and that I did. It falls back to the older archive when
      the template is older then CVS repos.
      
      But along the way I got annoyed with the fact that template instantiation
      does not provide a logfile to the web interface. The reason is that
      the current logfile stuff is very experiment centric; there has to be an
      experiment and an attached logfile. An instance does not have an experiment
      until really late in the game so the code was just not bothering.
      
      Anyway, I've started to generalize the logfile stuff with a new table
      and the approach that a logfile is named by a random key, and if you
      know the key you can look at the logfile in the web (since without an
      experiment it is hard to do permission checks unless we make logfiles
      uid/gid owned, and I did not want to do that.
      c1cff09b
  6. 25 Sep, 2007 1 commit
  7. 21 Sep, 2007 2 commits
    • Leigh Stoller's avatar
      If the template being modified is "active" then move the active bit to · 22e420d4
      Leigh Stoller authored
      the child template (parent no longer active).
      22e420d4
    • Leigh Stoller's avatar
      Two somewhat related template changes. · 217de8ab
      Leigh Stoller authored
      * Reorg the CVS repo so that records and setup are toplevel modules in
        the repo, instead of directories in a single module named by the
        guid (which is redundant and annoying).
      
      * Some changes to the spewlog stuff. It used to handle only
        experiments, but I really wanted it to handle template create and
        modify. Took a bunch of small changes to a lot of places to make
        this work correctly, but it was worth it.
      
        There are some changes I made that I can retrofit to the other spew
        pages to make it look a little nicer at the top of the page, to use
        less space.
      217de8ab
  8. 16 Jul, 2007 1 commit
  9. 09 Jul, 2007 1 commit
    • Leigh Stoller's avatar
      Checkpoint my cvs interface to the workbench. This first cut uses the · 8371fc79
      Leigh Stoller authored
      "rtag" directive to initiate template modify operations. So, to get started
      you do a checkout:
      
        cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo checkout XXXXX
      
      where XXXXX is the part of the guid (10000/1) before the slash. Might try
      and roll all templates into a single project wide repo at some point, to
      avoid the extraneous path stuff, but didn't want to worry that just yet.
      
      Okay, so have a checkout. You can work along the trunk, doing commits. To
      create a new template (a modify of the existing template), you tag the tree
      using rtag:
      
        cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo rtag mytag XXXXX
      
      A template modify is started at the end, and you should probably wait for
      email before continuing. Eventually I will need to add locking of some
      kind, but I have to do the modify in the background, or else I get deadlock
      cause cvs keeps the repo locked, and the modify also needs to access it.
      
      Each time you tag along the trunk, you get a modified template, which in
      the history diagram looks like:
      
        10000/1 --> 10000/2 --> 10000/3 ...
      
      If you want to branch, say at 10000/2 you can create a branch tag using rtag:
      
        cvs -d [cut] rtag -r T10000/2 -b mytag2 XXXXX
      
      You can also use your own tags for -r option, but I also create a TXXXXX/YY
      tag at each template modify, which is easy to remember.
      
      Then update your sandbox to the new branch, commit changes along that
      branch, and then later use rtag again to initiate a template modify
      operation:
      
        cvs update -r mytag2
        cvs commit ...
        cvs -d [cut] rtag -r mytag2 mytag3 XXXXX
      
      And now the history diagram looks like:
      
        10000/1 --> 10000/2 --> 10000/3 ...
                      |
                      |
                      -> 10000/4 ...
      
      You should be able to mix interaction via the web with interaction via the
      cvs interface. I've tested it, although not extensively.
      8371fc79
  10. 28 May, 2007 1 commit
    • Leigh Stoller's avatar
      Implement two requests by Eric: · e12ea1e2
      Leigh Stoller authored
      * The checkout directory now includes just the datastore directory and
        tbdata/nsfile.ns. This removes a bunch of clutter.
      
      * You can now run template_commit in any subdir of a checkout; the
        python wrapper will crawl upwards looking for the .template file.
      
        Note that I removed the option that allowed you to do a commit from
        the template tree in /proj/pid/templates since that directory will
        ultimately go away (or at least hide from view), and cause it
        conflicted with the new option and I didn't want to make things any
        messier for no reason.
      e12ea1e2
  11. 23 May, 2007 2 commits
    • Leigh Stoller's avatar
    • Leigh Stoller's avatar
      First cut at template checkout and commit from a checkout. The interface · b674bc7b
      Leigh Stoller authored
      described is the one exported to ops via the XMLRPC interface. This is
      just playing aroundl no doubt this stuff is going to change.
      
      * template_checkout guid/vers
      
        Checkout a copy of the template to the current working directory.
      
      * template_commit
      
        Modify the previous template checkout, using the nsfile contained in
        the tbdata directory (subdir of the current directory). In other words,
        the current template is modified, creating a new template in the
        current working directory (the current directory refers to the new
        template).
      
        The datastore subdir is imported into the new template, but that is
        the only directory that is imported at present. Might change that.
      
      So this sounds much cooler then it really is. Why?
      
      * This only works from ops.
      
      * The "current directory" must be one of the standard approved directories
        (/proj, /users, /groups).
      
      * Cause, boss reads and writes that directory via NFS, as told to it
        by the xmlrpc client.
      
      At some point in the future it would be nice to support something
      fancier, using a custom transport, but lets see how this goes.
      b674bc7b
  12. 17 May, 2007 1 commit
  13. 15 May, 2007 2 commits
    • Leigh Stoller's avatar
      Symlink trick to deal with the fact that we switched to archiving the · d4bcc494
      Leigh Stoller authored
      entire experiment directory, which now that we zip files, was putting
      thinks in the wrong place.
      d4bcc494
    • Leigh Stoller's avatar
      Checkpoint changes that have been discussed in the last few weeks: · c4f53202
      Leigh Stoller authored
      * Records are now "help open" when a run is stopped. When the next run
        is started, a check is made to see if the files
        (/project/$pid/exp/$eid) have changed, and if so a new version of the
        archive is committed before the next run is started.
      
      * Change the way swapmod is handled within an instance. A new option
        on the ShowExp page called Modify Resources. The intent is to allow
        an instance to be modified without having to start and stop runs,
        which tends to clutter things up, according to our user base. So, if
        you are within a run, that run is reset (reused) after the swapmod is
        finished. You can do this as many times as you like. If you are
        between runs (last operation was a stoprun), do the swapmod and then
        "speculatively" start a new run. Subsequent modifies reuse the that
        run again, as above.
      
        I think this is what Kevin was after ... there are some UI issues
        that may need to be resolved, will wait to hear what people have to
        say.
      
      * Revising a record is now supported. Export, change in place, and
        then use the Revise link on the ShowRun page. Currently this has to
        happen from the export directory on ops, but eventually allow an
        upload (to correspond to downloaded exports)
      
      * Check to see if export already exists, and give warning. Added a
        checkbox that allows user to overwrite the export.
      
      * A bunch of minor UI changes to the various template pages.
      c4f53202
  14. 07 May, 2007 1 commit
    • Leigh Stoller's avatar
      Mostly this commit is the switch from SVN archives to ZIP archives. · 55d1bb6e
      Leigh Stoller authored
      Other stuff leaked in too ...
      
      I did separate out a lot of tbsetup/libArchive into db/Archive, and
      whats left in libArchive.pm will eventually move over into the
      Template library.
      
      Note that I have dropped archiving of plain experiments; this is not
      really owrth it outside the workbench context, and it just wastes
      space and makes a lot if stuff painful in the web interface.
      55d1bb6e
  15. 10 Apr, 2007 1 commit
    • Kevin Atkinson's avatar
      · 858319bd
      Kevin Atkinson authored
      Add needed method Template::gid_idx.
      858319bd
  16. 09 Apr, 2007 1 commit
  17. 07 Mar, 2007 1 commit
  18. 06 Mar, 2007 1 commit
  19. 21 Feb, 2007 1 commit
  20. 20 Feb, 2007 2 commits
  21. 16 Feb, 2007 1 commit
  22. 14 Feb, 2007 3 commits
  23. 22 Jan, 2007 1 commit
  24. 09 Jan, 2007 1 commit
  25. 05 Jan, 2007 1 commit
    • Kevin Atkinson's avatar
      · 43eac695
      Kevin Atkinson authored
      Location of datastore is not "exp/datastore" not "datastore"
      in exparchive.  Update Template::Instance::CopyDataStore to reflect this.
      43eac695
  26. 08 Dec, 2006 1 commit
    • Leigh Stoller's avatar
      As discussed in meetings and email ... this commit changes what is · b898a8cc
      Leigh Stoller authored
      archived.  Rather then a special "archive" directory in the experiment
      directory, we know archive the entire experiment directory.
      
      This change should be backwards compatable, but let me know if not.
      
      Note that the nsdata directory is gone; the nsfile comes from the
      tbdata, but I know place a copy in nsfile.ns so that the name is well
      known.
      b898a8cc
  27. 09 Nov, 2006 1 commit
  28. 06 Nov, 2006 1 commit
  29. 03 Nov, 2006 1 commit
  30. 20 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      Add compression option to sync option of loghole. When turned on, any file · 4d4a27e1
      Leigh Stoller authored
      greater the 512K is automatically compressed with gzip. Might need to
      make this number bigger; we shall see.
      
      If you run emacs, put this in your .emacs file.
      
      	(load "jka-compr")
      	(jka-compr-install)
      
      and any time you visit a file that ends in one of the standard compression
      extensions, emacs will automatically do the uncompress for you on the data
      in the buffer (not the actual disk file of course). Very convenient.
      
      You can also get your browser to do the same, but I leave that as an
      exercise for the reader.
      4d4a27e1
  31. 18 Oct, 2006 1 commit
  32. 13 Oct, 2006 1 commit
  33. 12 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      By popular demand, give user a choice of where to get the next set of · bb996961
      Leigh Stoller authored
      (initial) parameters for a new run. Three choices right now; from the
      template itself, from the instance, or from the previous run. On the
      web interface this is presented as three buttons. On ops, it is the
      the -y option, which takes one of template,instance,lastrun as its
      argument (you can of course combine the -y option with an XML file to
      override specific params).
      
      At present, there is no default. Lets give it a chance to sink in
      before I pick something that will annoy 50% of the people 75% of the
      time.
      bb996961
  34. 09 Oct, 2006 1 commit