1. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  2. 11 Jul, 2012 1 commit
    • Leigh Stoller's avatar
      Cleanup in the web interface to prevent XSS attacks. · 6cf701f9
      Leigh Stoller authored
      We had a couple of different problems actually.
      
      * We allow users to insert html into many DB fields (say, a project or
        experiment description).
      
      * We did not sanitize that output when displaying back.
      
      * We did not sanitize initial page arguments that were reflected in the
        output (say, in a form).
      
      Since no one has the time to analyze every line of code, I took a couple of
      shortcuts. The first is that I changed the regex table to not allow any <>
      chars to go from the user into the DB. Brutal, but in fact there are only a
      couple of places where a user legitimately needs them. For example, a
      startup command that includes redirection. I handle those as special
      cases. As more come up, we can fix them.
      
      I did a quick pass through all of the forms, and made sure that we run
      htmlspecialchars on everything including initial form args. This was not
      too bad cause of the way all of the forms are structured, with a
      "formfields" array.
      
      I also removed a bunch of obsolete code and added an update script to
      actually remove them from the www directory.
      
      Lastly, I purged some XMLRPC code I did a long time ago in the Begin
      Experiment path. Less complexity, easier to grok and fix.
      
      	modified:   sql/database-fill.sql
      	modified:   sql/dbfill-update.sql
      6cf701f9
  3. 30 Aug, 2011 1 commit
  4. 09 Dec, 2008 1 commit
  5. 21 Oct, 2008 1 commit
  6. 26 Nov, 2007 1 commit
  7. 02 Oct, 2007 1 commit
  8. 21 Jun, 2007 1 commit
  9. 12 Feb, 2007 1 commit
    • Leigh Stoller's avatar
      * Replace the argument processing code in all pages. Currently we rely on · 48acc8e3
      Leigh Stoller authored
        register_globals=1 to turn POST/GET/COOKIES arguments in local variables.
        This is known to be a terrible security risk, and we keep saying we are
        going to fix it, and now I am. In order to accomplish this on a
        transitional basis (since I don't want the entire web interface to stop
        working while I debug it), and because the code just needs the cleanup, I
        am doing it like this: Each page will sport new declarations at the top:
      
      	RequiredPageArguments("experiment", PAGEARG_EXPERIMENT,
                                    "template",   PAGEARG_TEMPLATE,
                                    "instance",   PAGEARG_INSTANCE,
                                    "metadata",   PAGEARG_METADATA,
                                    "osinfo",     PAGEARG_OSINFO,
                                    "image",      PAGEARG_IMAGE,
                                    "project",    PAGEARG_PROJECT,
                                    "group",      PAGEARG_GROUP,
                                    "user",       PAGEARG_USER,
      			      "node",       PAGEARG_NODE,
      			      "yesno",      PAGEARG_BOOLEAN,
      			      "message",    PAGEARG_STRING,
      			      "age",        PAGEARG_INTEGER,
                                    "cost",       PAGEARG_NUMERIC,
                                    "formfields", PAGEARG_ARRAY,
                                    "unknown",    PAGEARG_ANYTHING);
      
      	OptionalPageArguments("canceled", PAGEARG_BOOLEAN);
      
        The first token in each pair is the name of the global variable to
        set, and the second token is the type. So, for "experiment" we look at
        the URL for a pid/eid or exptidx, etc, sanity check them (safe for a
        DB query), and then try to find that experiment in the DB. If it maps
        to an experiment, set global variable $experiment to the object. Since
        its a required argument, produce an error if not supplied. Similar
        treatment for optional arguments, with the obvious difference.
      
        The goal is to have ALL argument processing in one place, consistent,
        and correct. I've found numerous places where we leak unchecked
        arguments into queries. It also cuts out a lot of duplicated code.
      
      * To make the above easier to deal with, I've been replacing lots of
        hardcoded URLS in the code of the form:
      
      	foo.php3?pid=$pid&eid=$eid ...
      
        with
      
              CreateURL("foo", $experiment)
      
        which creates and returns the neccessary url string, by looking at
        the type of its arguments (experiment, template, instance, etc.)
      
        Eventually plan to replace them all so that URL handling throughout
        the code is all defined in one place (all the new URL code is in
        url_defs.php).
      
      * I have cranked up error reporting to tell me anytime a variable is
        used before it is initialized, plus a bunch of other stuff that PHP
        deems improper. Think of it like -Wall ... and boy we get a lot of
        warnings.  A very large percentage of the diffs are to fix all these
        warnings.
      
        The warnings are currently going to /usr/testbed/log/php-errors.log,
        and I'll be adding a script to capture them each night and mail them
        to tbops. This file also gets errors (this will be a change for
        developers; rather then seeing errors and warnings dumped in the
        middle of web pages, they will go to this file instead).
      
      * Major refactoring of the code. More objects (nodes, images, osids).
        Moving tons of queries into the objects in the hopes of someday
        getting to a point where we can split the web interface onto a
        different server.  Lots of general cleanup.
      48acc8e3
  10. 20 Dec, 2006 1 commit
  11. 20 Oct, 2006 1 commit
    • Mike Hibler's avatar
      Wow, this should make me look important! · afa5e919
      Mike Hibler authored
      Two-day boondoggle to support "/scratch", an optional large, shared filesystem
      for users.  To do this, I needed to find all the instances where /proj is used
      and behave accordingly.  The boondoggle part was the decision to gather up all
      the hardwired instances of shared directory names ("/proj", "/users", etc.)
      so that they are set in a common place (via unexposed configure variables).
      This is a boondoggle because:
      
      1. I didn't change the client-side scripts.  They need a different mechanism
         (e.g., tmcd) to get the info, configure is the wrong way.
      
      2. Even if I had done #1 it is likely--no, certain--that something would
         fail if you tried to rename "/proj" to be "/mike".  These names are just
         too ingrained.
      
      3. We may not even use "/scratch" as it turns out.
      
      Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
      To maintain my illusion in the future you should:
      
      1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
         et.al. functions where possible.  If not possible, make sure they run
         through configure and use @PROJROOT_DIR@, etc.
      
      2. Use the configure method for python, C, php and other languages.
      
      3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
         you should call to determine if an NS, template parameter, tarball or
         other file are in "an acceptable location."  Use these functions where
         possible.  They know about the optional "scratch" filesystem.  Note that
         the perl function is over-engineered to handles cases that don't occur
         in nature.
      afa5e919
  12. 12 Oct, 2006 1 commit
  13. 14 Jun, 2006 1 commit
  14. 07 Apr, 2006 1 commit
  15. 29 Mar, 2006 1 commit
    • Robert Ricci's avatar
      Add a few features that Jon and I want to the modifyexp page: · f21102ed
      Robert Ricci authored
      Add a 'Syntax Check' button like the one on beginexp so that you
      can syntax check without loosing all your edits if you had
      syntax errors.
      
      Add a file upload box and an option to use an NS file on ops,
      like the beginexp page has.
      
      Maybe we should try to find a spiffy ajax editor for this page?
      f21102ed
  16. 18 Dec, 2005 1 commit
    • Leigh Stoller's avatar
      Move the bulk of what was in showlogfile, into showlogfile_sup, so · 41f4d6a5
      Leigh Stoller authored
      that it can be "included" into other scripts; showlogfile, beginexp,
      swapexp, modifyexp, endexp.
      
      Each of these scripts now appends the showlog directly to the page,
      instead of providing a link to a different page.
      
      Also changed the showlog code to place the output into an inner iframe
      so that it can be sized to fit in the window, without the outer stuff
      (like the menus) getting scrolled away.
      41f4d6a5
  17. 28 Nov, 2005 1 commit
    • Timothy Stack's avatar
      · 19dbd0fd
      Timothy Stack authored
      Make the netlab client applet available to locals.
      
      	* configure, configure.in: Add xmlrpcpipe.php3.
      
      	* xmlrpc/emulabserver.py.in: Add missing virtual_tables.  Add
      	getareas call to get the list of robot areas.  Add node.getlist
      	and node.typeinfo methods for getting information about the nodes.
      	Add a "nic" argument to node.available to get the count of
      	wireless nodes.  Add "exclude" argument to
      	experiment.virtual_topology so we don't have to download the
      	massive route table, also delete the 'pid'/'eid' fields for the
      	same reason.  Don't return string output of virtual_topology, it's
      	huge.  Return some more info in experiment.getlist().
      
      	* www/GNUmakefile.in: Add xmlrpcpipe.php3.
      
      	* www/beginexp_form.php3, www/modifyexp.php3: Add links to the
      	client gui.
      
      	* www/netlab-client.jar: The client binary.
      19dbd0fd
  18. 26 Oct, 2005 1 commit
  19. 17 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      The Emulab Knowledge Base! · 6f08c442
      Leigh Stoller authored
      Okay, I implemented a primitive Knowledge Base! The current contents are
      *all* the existing FAQ entries, which I entered manually. Here are the
      details.
      
      * My reason for doing this is that we need something very simple. The wiki
        is too much of a barrier, and its search capabilities are pathetic.
      
      * The search page for the Knowledge Base is:
      
      	https://www.emulab.net/kb-search.php3
      
        Fairly primitive keyword search. Turns out that mysql 4.0 has a bunch for
        really good text searching functions built in, but we run 3.23 ... so I
        had to roll it myself. So, its a simple keyword (space or comma
        separated) search, no regular expressions.
      
      * Each DB record has a "faq_entry" flag, so creating the current FAQ on the
        fly from the DB is easy. See:
      
      	https://www.emulab.net/kb-faq.php3
      
      * In reddot mode, you can add new KB entries:
      
      	https://www.emulab.net/kb-manage.php3
      
        The form is fairly obvious but here are details anyway:
      
          Section Name: Choose an existing title, or make up a new one.
          Title:        The title of the KB (or FAQ) entry.
          Faq Entry:    Check this box if the new entry should show up in the FAQ.
          X Ref Tag:    A token so you can refer to other KB entries by name,
                        instead of by its index. Within the KB entry you would
                        write: <a href=kb-show.php3?xref_tag=sometag>
          Body:         Whatever you like. I took the existing FAQ entries and
                        stuck them with no changes except for the xref_tag
                        mentioned about (since some entries referenced other
                        entries).
      
      * Once you click on sumbit, you will see the entry as it will appear to
        users, along with a submenu to Modify/Delete/Add entries. You can modify
        the current entry from that menu. Mere users do not see this menu, only
        when in reddot mode.
      
      * The intent here is that we can generate new entries really easy, right
        from email if you like (with appropriate <pre> or <xmp> tags around it).
      
      * I have added sql/knowlbase-create.sql and a makefile target to
        generate that file when creating a distribution. I also added a section
        to install/boss-install to insert the entries into the new DB.
      
      * I hooked the search function into the existing Search Documentation link.
        We know search both the Knowledge Base *and* the Documentation on doc
        searches. This probably needs a little more work to get right.
      
      * I changed a lot of faq links to be more consistent and to reference
        the proper xref_tags (#swapping instead of #UTT-34).
      6f08c442
  20. 17 Jun, 2005 1 commit
  21. 12 Jan, 2005 1 commit
  22. 10 Sep, 2004 1 commit
    • Leigh Stoller's avatar
      Small change to suexec code. This change has the potential for creating · 7e731fba
      Leigh Stoller authored
      unanticipated breakage. If that happens, just need to back out the
      changes under the "suexec-stuff" tag. However, the better solution will
      probably be to fix the PHP scripts that break by adding the proper
      groups in the call to suexec (in the web page, see below) or by fixing
      the backend Perl script that breaks.
      
      This fix is primarily to address the problem of some users being in more
      groups (cause of subgroups) then the max number of groups allowed
      (NGROUPS).  The groups that really mattered (say, for creating an
      experiment in a subgroup) could be left out cause they were at the end
      of the list.
      
      * suexec.c: Change how groups are handled. Instead of taking a single
        gid argument (the gid to setgid as), now takes a comma separated list
        of groups. Further, instead of doing a setgroups to the user's entire
        group list as specified in the groups file (getgroups), setgroups to
        just the groups listed on the command line, plus the user's primary
        group from the password file (this is to prevent potential breakage
        with accessing files from the users homedir, although might not really
        be necessary).
      
        This change is somewhat rational in the sense that in our case, suexec
        is not being used to run arbitrary user code (CGIs), but only to run
        specific scripts that we say should be run. The environment for
        running those scripts can be more tightly controlled then it would
        otherwise need to be if running some random CGI the user has in his
        public html directory.
      
      * www: Change the gid argument to SUEXEC() in a number of scripts so
        that the project and subgroup are explicitly given to suexec, as
        described above. For example, in beginexp:
      
      	SUEXEC(gid, "$pid,$unix_gid", ....);
      
        Aside: note that project names (pid) are always one to one with their
        unix group name, but subgroup names are not, and *always* have to be
        looked up in the DB, hence the "unix_gid" argument.
      
        Script breakage should require nothing more then adding the proper
        group to the list as above.
      7e731fba
  23. 12 Feb, 2004 1 commit
  24. 10 Feb, 2004 1 commit
  25. 17 Nov, 2003 1 commit
    • Leigh Stoller's avatar
      Merge the two state machines (batchstate and state) into a single · 2025e0bd
      Leigh Stoller authored
      state machine (state). All of the stuff that was previously handled by
      using batchstate is now embedded into the one state machine. Of
      course, these mostly overlapped, so its not that much of a change,
      except that we also redid the machine, adding more states (for
      example, modify phases are now explicit. To get a picture of the
      actual state machine, on boss:
      
      		stategraph -o newstates EXPTSTATE
      		gv newstates.ps
      
      Things to note:
      
      * The "batchstate" slot of the experiments table is now used solely to
        provide a lock for batch daemon. A secondary change will be to
        change the slot name to something more appropriate, but it can
        happen anytime after this new stuff is installed.
      
      * I have left expt_locked for now, but another later change will be to remove
        expt_locked, and change it to active_busy or some such new state name in
        the state machine. I have removed most uses of expt_locked, except those
        that were necessary until there is a new state to replace it.
      
      * These new changes are an implementation of the new state machine,
        but I have not done anything fancy. Most of the code is the same as
        it was before.
      
      * I suspect that there are races with the batch daemon now, but they
        are going to be rare, and the end result is probably that a
        cancelation is delayed a little bit.
      2025e0bd
  26. 30 Sep, 2003 1 commit
    • Leigh Stoller's avatar
      Up to now we have had two state variables associated with an experiment, · 4269dad1
      Leigh Stoller authored
      plus a lock field. The lock field was a simple "experiment locked, go away"
      slot that is easy to use when you do not care about the actual state that
      an experiment is in, just that it is in "transition" and should not be
      messed with.
      
      The other two state variables are "state" and "batchstate". The former
      (state) is the original variable that Chris added, and was used by the tb*
      scripts to make sure that the experiment was in the state each particular
      script wanted them to be in. But over time (and with the addition of so
      much wrapper goo around them), "state" has leaked out all over the place to
      determine what operations on an experiment are allowed, and if/when it
      should be displayed in various web pages. There are a set of transition
      states in addition to the usual "active", "swapped", etc like "swapping"
      that make testing state a pain in the butt.
      
      I added the other state variable ("batchstate") when I did the batch
      system, obviously! It was intended as a wrapper state to control access to
      the batch queue, and to prevent batch experiments from being messed with
      except when it was really okay (for example, its okay to terminate a
      swapped out batch experiment, but not a swapped in batch experiment since
      that would confuse the batch daemon). There are fewer of these states, plus
      one additional state for "modifying" experiments.
      
      So what I have done is change the system to use "batchstate" for all
      experiments to control entry into the swap system, from the web interface,
      from the command line, and from the batch daemon. The other state variable
      still exists, and will be brutally pushed back under the surface until its
      just a vague memory, used only by the original tb* scripts. This will
      happen over time, and the "batchstate" variable will be renamed once I am
      convinced that this was the right thing to do and that my changes actually
      work as intended.
      
      Only people who have bothered to read this far will know that I also added
      the ability to cancel experiment swapin in progress. For that I am using
      the "canceled" flag (ah, this one was named properly from the start!), and
      I test that at various times in assign_wrapper and tbswap. A minor downside
      right now is that a canceled swapin looks too much like a failed swapin,
      and so tbops gets email about it. I'll fix that at some point (sometime
      after the boss complains).
      
      I also cleaned up various bits of code, replacing direct calls to exec
      with calls to the recently improved SUEXEC interface. This removes
      some cruft from each script that calls an external script.
      
      Cleaned up modifyexp.ph3 quite a bit, reformatting and indenting.
      Also fixed to not run the parser directly! This was very wrong; should
      call nscheck instead. Changed to use "nobody" group instead of group
      flux (made the same change in nscheck).
      
      There is a script in the sql directory called newstates.pl. It needs
      to be run to initialize the batchstate slot of the experiments table
      for all existing experiments.
      4269dad1
  27. 09 Sep, 2003 1 commit
  28. 06 Aug, 2003 1 commit
    • Leigh Stoller's avatar
      Clean up temporary files used in modify. The temp dirs were being · 05bd80ff
      Leigh Stoller authored
      created in /tmp and left behind. I've moved them to the expwork
      directory instead, and added a routine in the library to clear them
      out.
      
      Clear out the nsfile (stored in /tmp) used in modify. The web page was
      creating a temp file, but never removing it. swapexp now copies the
      nsfile in so that the web page can remove the temporary after the
      script exits. The temp is placed in the expwork directory as well, but
      left behind for debugging.
      
      When swapmod fails, send along the nsfile in the email message.
      05bd80ff
  29. 30 Jul, 2003 1 commit
  30. 14 Jul, 2003 1 commit
  31. 06 Jun, 2003 1 commit
    • Chad Barb's avatar
      · 2fad09a4
      Chad Barb authored
      NetBuild modify. Woot.
      
      Available only to admins for now.
      
      Link is available off "modify experiment page" in admin mode.
      2fad09a4
  32. 21 May, 2003 1 commit
    • Chad Barb's avatar
      · f3a7da49
      Chad Barb authored
      Changed submit button to work on NS4.7
      (by using <input type='submit'> instead of <button>)
      for our retrocomputing friends.
      f3a7da49
  33. 14 May, 2003 1 commit
    • Chad Barb's avatar
      · 84bffaa3
      Chad Barb authored
      Modify experiment is now deployed/documented...
      We can announce it tomorrow.
      84bffaa3
  34. 17 Apr, 2003 1 commit
    • Chad Barb's avatar
      · 4233af4e
      Chad Barb authored
      For the benefit of our users,
      added 'reboot nodes in experiment' checkbox,
      on by default, with a stern warning.
      4233af4e
  35. 03 Apr, 2003 1 commit
    • Chad Barb's avatar
      · 765de560
      Chad Barb authored
      Added new feature 'Experiment Modify'.
      Now available (to admins only for now) from the showexp page.
      
      Warning! doing a modify which alters the topology will probably
      require a "reboot all nodes" afterwards.
      (There will be a checkbox soon in the modify experiment page.)
      
      Adding/removing delay nodes seems to work fine without reboots, though.
      
      Warning! If the new version of the experiment cannot be mapped
       (not enough nodes available, for instance) the experiment will be
       swapped out! This will get fixed later.
      
      Prerun backs up the experiment topology, so using a bad NS
      file doesn't result in experiment termination.
      
      As part of this, added library functions to libdb to
      delete, backup, and restore both virtual and physical experiment state.
      765de560