1. 17 Sep, 2015 1 commit
  2. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  3. 30 Aug, 2011 1 commit
  4. 09 Dec, 2008 1 commit
  5. 16 Sep, 2008 1 commit
  6. 26 Nov, 2007 1 commit
  7. 01 Nov, 2007 1 commit
  8. 24 Oct, 2007 1 commit
    • Leigh Stoller's avatar
      Moderate rework of the Emulab UI. When logged in, all of the main · 69ec957f
      Leigh Stoller authored
      menus that used to be on the left side, are now drop down menus along
      the top. The banner across the top will also be smaller and more space
      efficient.
      
      The primary motivation for these changes is to leave more room for
      actual content in the main window.
      
      Tested with Firefox (1.5 and 2.0), Safari (2.0 and 3.0) and on
      Internet Explorer Version 7. IE6 has some visual issues that might
      make it a little annoying, but should be viewed as incentive to
      upgrade to IE7 since it has been out a long time.
      69ec957f
  9. 20 Sep, 2007 1 commit
  10. 26 Apr, 2007 1 commit
  11. 12 Feb, 2007 1 commit
    • Leigh Stoller's avatar
      * Replace the argument processing code in all pages. Currently we rely on · 48acc8e3
      Leigh Stoller authored
        register_globals=1 to turn POST/GET/COOKIES arguments in local variables.
        This is known to be a terrible security risk, and we keep saying we are
        going to fix it, and now I am. In order to accomplish this on a
        transitional basis (since I don't want the entire web interface to stop
        working while I debug it), and because the code just needs the cleanup, I
        am doing it like this: Each page will sport new declarations at the top:
      
      	RequiredPageArguments("experiment", PAGEARG_EXPERIMENT,
                                    "template",   PAGEARG_TEMPLATE,
                                    "instance",   PAGEARG_INSTANCE,
                                    "metadata",   PAGEARG_METADATA,
                                    "osinfo",     PAGEARG_OSINFO,
                                    "image",      PAGEARG_IMAGE,
                                    "project",    PAGEARG_PROJECT,
                                    "group",      PAGEARG_GROUP,
                                    "user",       PAGEARG_USER,
      			      "node",       PAGEARG_NODE,
      			      "yesno",      PAGEARG_BOOLEAN,
      			      "message",    PAGEARG_STRING,
      			      "age",        PAGEARG_INTEGER,
                                    "cost",       PAGEARG_NUMERIC,
                                    "formfields", PAGEARG_ARRAY,
                                    "unknown",    PAGEARG_ANYTHING);
      
      	OptionalPageArguments("canceled", PAGEARG_BOOLEAN);
      
        The first token in each pair is the name of the global variable to
        set, and the second token is the type. So, for "experiment" we look at
        the URL for a pid/eid or exptidx, etc, sanity check them (safe for a
        DB query), and then try to find that experiment in the DB. If it maps
        to an experiment, set global variable $experiment to the object. Since
        its a required argument, produce an error if not supplied. Similar
        treatment for optional arguments, with the obvious difference.
      
        The goal is to have ALL argument processing in one place, consistent,
        and correct. I've found numerous places where we leak unchecked
        arguments into queries. It also cuts out a lot of duplicated code.
      
      * To make the above easier to deal with, I've been replacing lots of
        hardcoded URLS in the code of the form:
      
      	foo.php3?pid=$pid&eid=$eid ...
      
        with
      
              CreateURL("foo", $experiment)
      
        which creates and returns the neccessary url string, by looking at
        the type of its arguments (experiment, template, instance, etc.)
      
        Eventually plan to replace them all so that URL handling throughout
        the code is all defined in one place (all the new URL code is in
        url_defs.php).
      
      * I have cranked up error reporting to tell me anytime a variable is
        used before it is initialized, plus a bunch of other stuff that PHP
        deems improper. Think of it like -Wall ... and boy we get a lot of
        warnings.  A very large percentage of the diffs are to fix all these
        warnings.
      
        The warnings are currently going to /usr/testbed/log/php-errors.log,
        and I'll be adding a script to capture them each night and mail them
        to tbops. This file also gets errors (this will be a change for
        developers; rather then seeing errors and warnings dumped in the
        middle of web pages, they will go to this file instead).
      
      * Major refactoring of the code. More objects (nodes, images, osids).
        Moving tons of queries into the objects in the hopes of someday
        getting to a point where we can split the web interface onto a
        different server.  Lots of general cleanup.
      48acc8e3
  12. 20 Dec, 2006 1 commit
  13. 18 Oct, 2006 1 commit
  14. 12 Oct, 2006 1 commit
    • Leigh Stoller's avatar
      By popular demand, give user a choice of where to get the next set of · bb996961
      Leigh Stoller authored
      (initial) parameters for a new run. Three choices right now; from the
      template itself, from the instance, or from the previous run. On the
      web interface this is presented as three buttons. On ops, it is the
      the -y option, which takes one of template,instance,lastrun as its
      argument (you can of course combine the -y option with an XML file to
      override specific params).
      
      At present, there is no default. Lets give it a chance to sink in
      before I pick something that will annoy 50% of the people 75% of the
      time.
      bb996961
  15. 06 Jun, 2006 1 commit
    • Leigh Stoller's avatar
      A set of cosmetic changes. Actually one consmetic changes in a lot of · ecab2742
      Leigh Stoller authored
      places. I refactored the "busy" logo stuff that is done in various
      scripts in various formats into a couple of functions. I have not
      changed all the code that does the busy thing, but I got a bunch this
      time around. In menu.php I added a couple of functions (that spit out
      little bits of javascript).
      
      	STARTBUSY($msg)
      
      	Display a centered message (ie: "Starting Swapout"), then a
      	plead to be patient, and then the "busy" icon (spinning thing
      	that my Mac does too).
      
      
      	STOPBUSY();
      
      	Clear the busy icon, and replace "Working ..." with "Done"
      
      
      	CLEARBUSY();
      
      	Clear the busy icon and delete "Working ..."
      
      
      	PAGEREPLACE($url)
      
      	Delay one second and then replace the current page with the
      	new page.
      ecab2742
  16. 26 May, 2006 1 commit
    • Leigh Stoller's avatar
      Fix another Eric nit: · 1605bf1e
      Leigh Stoller authored
        + On the swap-in status page (showing the swap-in progress), there's a link
          to the template page but no link to the instance page.  Conversely, on the
          swap-out page ("Terminate Instance"), there's a link to the instance but
          not to the template.
      1605bf1e
  17. 16 May, 2006 1 commit
  18. 05 May, 2006 1 commit
  19. 31 Mar, 2006 1 commit
    • Leigh Stoller's avatar
      More experiment run stuff. Also the NS file parsing support that was · 74bf8d1d
      Leigh Stoller authored
      needed (reparsing of NS file with current bindings).
      
      Mostly I decided that I made a huge mistake in using the exptidx as the
      main cross index. I started to change that, which took a bunch of time.
      Still more tables to fix up, but I think all of this code is gonna need
      a major reorg once I am no longer under the gun to get it done.
      74bf8d1d
  20. 28 Mar, 2006 1 commit
  21. 18 Dec, 2005 1 commit
    • Leigh Stoller's avatar
      Move the bulk of what was in showlogfile, into showlogfile_sup, so · 41f4d6a5
      Leigh Stoller authored
      that it can be "included" into other scripts; showlogfile, beginexp,
      swapexp, modifyexp, endexp.
      
      Each of these scripts now appends the showlog directly to the page,
      instead of providing a link to a different page.
      
      Also changed the showlog code to place the output into an inner iframe
      so that it can be sized to fit in the window, without the outer stuff
      (like the menus) getting scrolled away.
      41f4d6a5
  22. 26 Oct, 2005 1 commit
  23. 17 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      The Emulab Knowledge Base! · 6f08c442
      Leigh Stoller authored
      Okay, I implemented a primitive Knowledge Base! The current contents are
      *all* the existing FAQ entries, which I entered manually. Here are the
      details.
      
      * My reason for doing this is that we need something very simple. The wiki
        is too much of a barrier, and its search capabilities are pathetic.
      
      * The search page for the Knowledge Base is:
      
      	https://www.emulab.net/kb-search.php3
      
        Fairly primitive keyword search. Turns out that mysql 4.0 has a bunch for
        really good text searching functions built in, but we run 3.23 ... so I
        had to roll it myself. So, its a simple keyword (space or comma
        separated) search, no regular expressions.
      
      * Each DB record has a "faq_entry" flag, so creating the current FAQ on the
        fly from the DB is easy. See:
      
      	https://www.emulab.net/kb-faq.php3
      
      * In reddot mode, you can add new KB entries:
      
      	https://www.emulab.net/kb-manage.php3
      
        The form is fairly obvious but here are details anyway:
      
          Section Name: Choose an existing title, or make up a new one.
          Title:        The title of the KB (or FAQ) entry.
          Faq Entry:    Check this box if the new entry should show up in the FAQ.
          X Ref Tag:    A token so you can refer to other KB entries by name,
                        instead of by its index. Within the KB entry you would
                        write: <a href=kb-show.php3?xref_tag=sometag>
          Body:         Whatever you like. I took the existing FAQ entries and
                        stuck them with no changes except for the xref_tag
                        mentioned about (since some entries referenced other
                        entries).
      
      * Once you click on sumbit, you will see the entry as it will appear to
        users, along with a submenu to Modify/Delete/Add entries. You can modify
        the current entry from that menu. Mere users do not see this menu, only
        when in reddot mode.
      
      * The intent here is that we can generate new entries really easy, right
        from email if you like (with appropriate <pre> or <xmp> tags around it).
      
      * I have added sql/knowlbase-create.sql and a makefile target to
        generate that file when creating a distribution. I also added a section
        to install/boss-install to insert the entries into the new DB.
      
      * I hooked the search function into the existing Search Documentation link.
        We know search both the Knowledge Base *and* the Documentation on doc
        searches. This probably needs a little more work to get right.
      
      * I changed a lot of faq links to be more consistent and to reference
        the proper xref_tags (#swapping instead of #UTT-34).
      6f08c442
  24. 12 Jan, 2005 1 commit
  25. 10 Sep, 2004 1 commit
    • Leigh Stoller's avatar
      Small change to suexec code. This change has the potential for creating · 7e731fba
      Leigh Stoller authored
      unanticipated breakage. If that happens, just need to back out the
      changes under the "suexec-stuff" tag. However, the better solution will
      probably be to fix the PHP scripts that break by adding the proper
      groups in the call to suexec (in the web page, see below) or by fixing
      the backend Perl script that breaks.
      
      This fix is primarily to address the problem of some users being in more
      groups (cause of subgroups) then the max number of groups allowed
      (NGROUPS).  The groups that really mattered (say, for creating an
      experiment in a subgroup) could be left out cause they were at the end
      of the list.
      
      * suexec.c: Change how groups are handled. Instead of taking a single
        gid argument (the gid to setgid as), now takes a comma separated list
        of groups. Further, instead of doing a setgroups to the user's entire
        group list as specified in the groups file (getgroups), setgroups to
        just the groups listed on the command line, plus the user's primary
        group from the password file (this is to prevent potential breakage
        with accessing files from the users homedir, although might not really
        be necessary).
      
        This change is somewhat rational in the sense that in our case, suexec
        is not being used to run arbitrary user code (CGIs), but only to run
        specific scripts that we say should be run. The environment for
        running those scripts can be more tightly controlled then it would
        otherwise need to be if running some random CGI the user has in his
        public html directory.
      
      * www: Change the gid argument to SUEXEC() in a number of scripts so
        that the project and subgroup are explicitly given to suexec, as
        described above. For example, in beginexp:
      
      	SUEXEC(gid, "$pid,$unix_gid", ....);
      
        Aside: note that project names (pid) are always one to one with their
        unix group name, but subgroup names are not, and *always* have to be
        looked up in the DB, hence the "unix_gid" argument.
      
        Script breakage should require nothing more then adding the proper
        group to the list as above.
      7e731fba
  26. 13 May, 2004 1 commit
  27. 17 Nov, 2003 1 commit
    • Leigh Stoller's avatar
      Merge the two state machines (batchstate and state) into a single · 2025e0bd
      Leigh Stoller authored
      state machine (state). All of the stuff that was previously handled by
      using batchstate is now embedded into the one state machine. Of
      course, these mostly overlapped, so its not that much of a change,
      except that we also redid the machine, adding more states (for
      example, modify phases are now explicit. To get a picture of the
      actual state machine, on boss:
      
      		stategraph -o newstates EXPTSTATE
      		gv newstates.ps
      
      Things to note:
      
      * The "batchstate" slot of the experiments table is now used solely to
        provide a lock for batch daemon. A secondary change will be to
        change the slot name to something more appropriate, but it can
        happen anytime after this new stuff is installed.
      
      * I have left expt_locked for now, but another later change will be to remove
        expt_locked, and change it to active_busy or some such new state name in
        the state machine. I have removed most uses of expt_locked, except those
        that were necessary until there is a new state to replace it.
      
      * These new changes are an implementation of the new state machine,
        but I have not done anything fancy. Most of the code is the same as
        it was before.
      
      * I suspect that there are races with the batch daemon now, but they
        are going to be rare, and the end result is probably that a
        cancelation is delayed a little bit.
      2025e0bd
  28. 09 Oct, 2003 1 commit
    • Leigh Stoller's avatar
      Reorg of two aspects of node update. · 2641af4d
      Leigh Stoller authored
      * install-rpm, install-tarfile, spewrpmtar.php3, spewrpmtar.in: Pumped
        up even more! The db file we store in /var/db now records both the
        timestamp (of the file, or if remote the install time) and the MD5
        of the file that was installed. Locally, we can get this info when
        accessing the file via NFS (copymode on or off). Remote, we use wget
        to get the file, and so pass the timestamp along in the URL request,
        and let spewrpmtar.in determine if the file has changed. If the
        timestamp it gets is >= to the timestamp of the file, an error code
        of 304 (Not Modifed) is returned. Otherwise the file is returned.
      
        If the timestamps are different (remote, server sends back an actual
        file), the MD5 of the file is compared against the value stored. If
        they are equal, update the timestamp in the db file to avoid
        repeated MD5s (or server downloads) in the future. If the MD5 is
        different, then reinstall the tarball or rpm, and update the db file
        with the new timestamp and MD5. Presto, we have auto update capability!
      
        Caveat: I pass along the old MD5 in the URL, but it is currently
        ignored. I do not know if doing the MD5 on the server is a good
        idea, but obviously it is easy to add later. At the moment it
        happens on the node, which means wasted bandwidth when the timestamp
        has changed, but the file has not (probably not something that will
        happen in typical usage).
      
        Caveat: The timestamp used on remote nodes is the time the tarfile
        is installed (GM time of course). We could arrange to return the
        timestamp of the local file back to the node, but that would mean
        complicating the protocol (or using an http header) and I was not in
        the mood for that. In typical usage, I do not think that people will
        be changing tarfiles and rpms so rapidly that this will make a
        difference, but if it does, we can change it.
      
      * node_update.in, client side watchdog, and various web pages:
        Deflated node_update, removing all of the older ssh code. We now
        assume that all nodes will auto update on a periodic basis, via the
        watchdog that runs on all client nodes, including plab nodes.
      
        Changed the permission check to look for new UPDATE permission (used
        to be UPDATEACCOUNT). As before, it requires local_root or better.
        The reason for this is that node_update now implies more than just
        updating the accounts/mounts. The web pages have been changed to
        explain that in addition to mounts/accounts, rpms and tarfiles will
        also be updated. At the moment, this is still tied to a single
        variable (update_accounts) in the nodes table, but as Kirk requested
        at the meeting, it will probably be nice to split these out in the
        future.
      
        Added the ability to node_update a single node in an experiment (in
        addition to all nodes option on the showexp page). This has been
        added to the shownode webpage menu options.
      
        Changed locking code to use the newer wrapper states, and to move
        the experiment to RUNNING_LOCKED until the update completes. This is
        to prevent mayhem in the rest of the system (which could be dealt
        with, but is not worth the trouble; people have to wait until their
        initiated update is complete, before they can swap out the
        experiment).
      
        Added "short" mode to shownode routine, equiv to the recently added
        short mode for showexp. I use this on the confirmation page for
        updating a single node, giving the user a couple of pertinent (feel
        good) facts before they comfirm.
      2641af4d
  29. 30 Sep, 2003 1 commit
    • Leigh Stoller's avatar
      Up to now we have had two state variables associated with an experiment, · 4269dad1
      Leigh Stoller authored
      plus a lock field. The lock field was a simple "experiment locked, go away"
      slot that is easy to use when you do not care about the actual state that
      an experiment is in, just that it is in "transition" and should not be
      messed with.
      
      The other two state variables are "state" and "batchstate". The former
      (state) is the original variable that Chris added, and was used by the tb*
      scripts to make sure that the experiment was in the state each particular
      script wanted them to be in. But over time (and with the addition of so
      much wrapper goo around them), "state" has leaked out all over the place to
      determine what operations on an experiment are allowed, and if/when it
      should be displayed in various web pages. There are a set of transition
      states in addition to the usual "active", "swapped", etc like "swapping"
      that make testing state a pain in the butt.
      
      I added the other state variable ("batchstate") when I did the batch
      system, obviously! It was intended as a wrapper state to control access to
      the batch queue, and to prevent batch experiments from being messed with
      except when it was really okay (for example, its okay to terminate a
      swapped out batch experiment, but not a swapped in batch experiment since
      that would confuse the batch daemon). There are fewer of these states, plus
      one additional state for "modifying" experiments.
      
      So what I have done is change the system to use "batchstate" for all
      experiments to control entry into the swap system, from the web interface,
      from the command line, and from the batch daemon. The other state variable
      still exists, and will be brutally pushed back under the surface until its
      just a vague memory, used only by the original tb* scripts. This will
      happen over time, and the "batchstate" variable will be renamed once I am
      convinced that this was the right thing to do and that my changes actually
      work as intended.
      
      Only people who have bothered to read this far will know that I also added
      the ability to cancel experiment swapin in progress. For that I am using
      the "canceled" flag (ah, this one was named properly from the start!), and
      I test that at various times in assign_wrapper and tbswap. A minor downside
      right now is that a canceled swapin looks too much like a failed swapin,
      and so tbops gets email about it. I'll fix that at some point (sometime
      after the boss complains).
      
      I also cleaned up various bits of code, replacing direct calls to exec
      with calls to the recently improved SUEXEC interface. This removes
      some cruft from each script that calls an external script.
      
      Cleaned up modifyexp.ph3 quite a bit, reformatting and indenting.
      Also fixed to not run the parser directly! This was very wrong; should
      call nscheck instead. Changed to use "nobody" group instead of group
      flux (made the same change in nscheck).
      
      There is a script in the sql directory called newstates.pl. It needs
      to be run to initialize the batchstate slot of the experiments table
      for all existing experiments.
      4269dad1
  30. 30 Jul, 2003 1 commit
  31. 29 Jul, 2003 1 commit
    • Leigh Stoller's avatar
      Some cleanup on the batch mode stuff. Make it more explicit in the · 29b820b1
      Leigh Stoller authored
      showexp page that its a batch experiment, by the menu options. Same
      deal in the swapexp output, plus some other minor cleanup. The only
      bug I found while trying to figure out the batchmode problem reported
      this morning by the FileMover people, is that the cancelflag is not
      cleared after swaping a running batch experiment out, so even after
      reinjecting it into the queue, it will not run. Still, that does seem
      to be what the FileMover people reported.
      29b820b1
  32. 17 Jul, 2003 1 commit
  33. 28 May, 2003 1 commit
  34. 24 May, 2003 1 commit
    • Mac Newbold's avatar
      Round of changes related to idleswapping and autoswapping. The web and · 02aaf8e4
      Mac Newbold authored
      back end scripts now support 3 different kind of forced swaps:
      
      1. Idle-Swap : this is ths same one we had before. Email message to them
      says it was swapped "because it was idle for too long"
      
      2. Auto-Swap : A new one, typically for user-requested timed swapouts.
      Email says it was swapped "because it was swapped in too long"
      
      3. Force swap: Generic one, for "none of the above" cases. Just says
      Experiment "has been forcibly swapped out by Testbed Operations."
      
      The force swap option on the web now lets you choose which of these three
      you want. Only "Idle-Swap" counts as an idleswap in the stats. Soon
      idleswap and autoswap will be used by idlemail when it does automatic
      swapping.
      02aaf8e4
  35. 22 May, 2003 2 commits
    • Leigh Stoller's avatar
      Reorg the batch system slightly as per Eric's request that batch mode · da97ba35
      Leigh Stoller authored
      experiments look more like regular experiments. Batch mode experiments
      can now be preloaded and swapped. When preloaded, they go into a
      "Pause" state. Swapping a batch mode experiment in puts them into the
      "posted" state so the batch daemon will see them. Swapping out a
      batchmode experiment does the expected; it puts them back into the
      Pause state. Terminating a batch mode experiment does the expected;
      its gone. When a batch mode experiment finishes normally, it goes back
      into the pause state, which allows batches to be reinjected as many
      times as Eric likes.
      da97ba35
    • Mac Newbold's avatar
      Show a big warning if someone tries to forceswap an unswappable expt. · f527cd91
      Mac Newbold authored
      Also do some prep work to carry $idleswap and $autoswap through this
      script like $force is. (For calling the back end with different params,
      when it supports it.)
      f527cd91
  36. 16 Apr, 2003 1 commit
    • Leigh Stoller's avatar
      Add support for idleswapping an experiment as the creator of the · ff5a57de
      Leigh Stoller authored
      experiment, rather than as an administrator, which presents group
      permission problems when the experiment is in a subgroup (requires two
      additional group, whereas suexec adds only one group). That aside, the
      correct approach is to run the swap as the creator. To do that, must
      flip to the user (from the admin person) in the backend using the new
      idleswap script, and then run the normal swapexp. Add new option to
      swapexp (-i) which changes the email slightly to make it clear that
      the experiment was idleswapped, and so that the From: is tbops not the
      user (again, to make it more clear).
      ff5a57de
  37. 18 Dec, 2002 1 commit
    • Leigh Stoller's avatar
      New "restart" or perhaps better if named "replay" mode to swapexp. · d651dd42
      Leigh Stoller authored
      Attempts to replay an experiment by rebooting all the nodes, clearing
      the various startup bits (ready, startstatus, bootstatus, portstats),
      and then restarting the event system. I am dubious that this is a
      workable solution because of the asynchronous nature of the testbed
      (nodes happily cruise from TBRESET to ISUP and beyond without
      stopping), and so its hard to truly replicate the initial lack of
      state that a freshly swapped in experiment has. Still, people
      requested it and I cheerfully provided it cause thats what I do;
      service with a smile and not a wit of complaint. Is anyone reading
      this?
      d651dd42
  38. 16 Sep, 2002 1 commit
  39. 25 Jul, 2002 1 commit