1. 12 Feb, 2007 1 commit
    • Leigh Stoller's avatar
      * Replace the argument processing code in all pages. Currently we rely on · 48acc8e3
      Leigh Stoller authored
        register_globals=1 to turn POST/GET/COOKIES arguments in local variables.
        This is known to be a terrible security risk, and we keep saying we are
        going to fix it, and now I am. In order to accomplish this on a
        transitional basis (since I don't want the entire web interface to stop
        working while I debug it), and because the code just needs the cleanup, I
        am doing it like this: Each page will sport new declarations at the top:
      
      	RequiredPageArguments("experiment", PAGEARG_EXPERIMENT,
                                    "template",   PAGEARG_TEMPLATE,
                                    "instance",   PAGEARG_INSTANCE,
                                    "metadata",   PAGEARG_METADATA,
                                    "osinfo",     PAGEARG_OSINFO,
                                    "image",      PAGEARG_IMAGE,
                                    "project",    PAGEARG_PROJECT,
                                    "group",      PAGEARG_GROUP,
                                    "user",       PAGEARG_USER,
      			      "node",       PAGEARG_NODE,
      			      "yesno",      PAGEARG_BOOLEAN,
      			      "message",    PAGEARG_STRING,
      			      "age",        PAGEARG_INTEGER,
                                    "cost",       PAGEARG_NUMERIC,
                                    "formfields", PAGEARG_ARRAY,
                                    "unknown",    PAGEARG_ANYTHING);
      
      	OptionalPageArguments("canceled", PAGEARG_BOOLEAN);
      
        The first token in each pair is the name of the global variable to
        set, and the second token is the type. So, for "experiment" we look at
        the URL for a pid/eid or exptidx, etc, sanity check them (safe for a
        DB query), and then try to find that experiment in the DB. If it maps
        to an experiment, set global variable $experiment to the object. Since
        its a required argument, produce an error if not supplied. Similar
        treatment for optional arguments, with the obvious difference.
      
        The goal is to have ALL argument processing in one place, consistent,
        and correct. I've found numerous places where we leak unchecked
        arguments into queries. It also cuts out a lot of duplicated code.
      
      * To make the above easier to deal with, I've been replacing lots of
        hardcoded URLS in the code of the form:
      
      	foo.php3?pid=$pid&eid=$eid ...
      
        with
      
              CreateURL("foo", $experiment)
      
        which creates and returns the neccessary url string, by looking at
        the type of its arguments (experiment, template, instance, etc.)
      
        Eventually plan to replace them all so that URL handling throughout
        the code is all defined in one place (all the new URL code is in
        url_defs.php).
      
      * I have cranked up error reporting to tell me anytime a variable is
        used before it is initialized, plus a bunch of other stuff that PHP
        deems improper. Think of it like -Wall ... and boy we get a lot of
        warnings.  A very large percentage of the diffs are to fix all these
        warnings.
      
        The warnings are currently going to /usr/testbed/log/php-errors.log,
        and I'll be adding a script to capture them each night and mail them
        to tbops. This file also gets errors (this will be a change for
        developers; rather then seeing errors and warnings dumped in the
        middle of web pages, they will go to this file instead).
      
      * Major refactoring of the code. More objects (nodes, images, osids).
        Moving tons of queries into the objects in the hopes of someday
        getting to a point where we can split the web interface onto a
        different server.  Lots of general cleanup.
      48acc8e3
  2. 20 Dec, 2006 1 commit
  3. 20 Oct, 2006 1 commit
    • Mike Hibler's avatar
      Wow, this should make me look important! · afa5e919
      Mike Hibler authored
      Two-day boondoggle to support "/scratch", an optional large, shared filesystem
      for users.  To do this, I needed to find all the instances where /proj is used
      and behave accordingly.  The boondoggle part was the decision to gather up all
      the hardwired instances of shared directory names ("/proj", "/users", etc.)
      so that they are set in a common place (via unexposed configure variables).
      This is a boondoggle because:
      
      1. I didn't change the client-side scripts.  They need a different mechanism
         (e.g., tmcd) to get the info, configure is the wrong way.
      
      2. Even if I had done #1 it is likely--no, certain--that something would
         fail if you tried to rename "/proj" to be "/mike".  These names are just
         too ingrained.
      
      3. We may not even use "/scratch" as it turns out.
      
      Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
      To maintain my illusion in the future you should:
      
      1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
         et.al. functions where possible.  If not possible, make sure they run
         through configure and use @PROJROOT_DIR@, etc.
      
      2. Use the configure method for python, C, php and other languages.
      
      3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
         you should call to determine if an NS, template parameter, tarball or
         other file are in "an acceptable location."  Use these functions where
         possible.  They know about the optional "scratch" filesystem.  Note that
         the perl function is over-engineered to handles cases that don't occur
         in nature.
      afa5e919
  4. 18 Jul, 2006 1 commit
    • Leigh Stoller's avatar
      Changes necessary for moving most of the stuff in the node_types · 624a0364
      Leigh Stoller authored
      table, into a new table called node_type_attributes, which is intended
      to be a more extensible way of describing nodes.
      
      The only things left in the node_types table will be type,class and the
      various isXXX boolean flags, since we use those in numerous joins all over
      the system (ie: when discriminating amongst nodes).
      
      For the most part, all of that other stuff is rarely used, or used in
      contexts where the information is needed, but not for type descrimination.
      Still, it made for a lot of queries to change!
      
      Along the way I added a NodeType library module that represents the type
      info as a perl object. I also beefed up the existing Node module, and
      started using it in more places. I also added an Interfaces module, but I
      have not done much with that yet.
      
      I have not yet removed all the slots from the node_types table; I plan to
      run the new code for a few days and then remove the slots.
      
      Example using the new NodeType object:
      
      	use NodeType;
      
      	my $typeinfo = NodeType->Lookup($type);
      
              if ($typeinfo->control_interface(\$control_iface) ||
                  !$control_iface) {
        	    warn "No control interface for $type is defined in the DB!\n";
              }
      
      or using the Node:
      
      	use Node;
      
              my $nodeobject = Node->Lookup($node_id);
              my $imageable  = $nodeobject->NodeTypeInfo()->imageable();
      or
              my $rebootable = $nodeobject->isrebootable();
      or
              $nodeobject->NodeTypeAttribute("control_interface", \$control_iface);
      
      Lots of way to accomplish the same thing, but the main point is that the
      Node is able to override the NodeType (if it wants to), which I think is
      necessary for flexibly describing one/two of a kind things like switches, etc.
      624a0364
  5. 08 Mar, 2005 1 commit
  6. 08 Dec, 2004 1 commit
  7. 01 Nov, 2004 1 commit
  8. 27 Jul, 2004 1 commit
  9. 30 May, 2003 1 commit
  10. 26 Mar, 2003 1 commit
    • Leigh Stoller's avatar
      Add "gid" slot to the images table for changing permission scheme from · 4c56daf6
      Leigh Stoller authored
      only pid, to pid/gid like most other things in the testbed. Also add a
      "global" slot to denote images that are globally available to all
      projects (system images). The older "shared" attribute is now used to
      denote images that are shared within a project (available to all
      subgroups in the project). The migration path for existing DBs is
      given in the migrate file. Be sure to run those commands on an
      existing testbed or things will break!
      
      www/newimageid, www/newimageid_ez: A bunch of changes for
      shared/global attributes. Added a group menu to the form so users can
      create images in subgroups. Beefed up the Java code that constructs
      the path name to use the gid, shared, and global attributes of the
      form to give the user the best possible path that we can. Improved the
      pathname checking code so that we do not allow just any old path in
      case the user elects to disregard the path we carefully constructed
      for them. Also check the proj/group membership, and setup defaults for
      users that have permission in just one pid/gid to create images.
      
      libdb.in: Changed permission check in TBImageIDAccessCheck() to
      reflect shared/global attribute changes.
      
      os_load: Get rid of test that checked path of the image. The path
      checking is done in the web interface anyway, so why duplicate in 4
      places. Other minor changes reflecting shared->global name change.
      Also note that images can come from the group directory now.
      
      create_image: Get rid of test that checked path of the image. The path
      checking is done in the web interface anyway, so why duplicate in 4
      places. Also note that images can come from the group directory now.
      
      www/dbdefs: Changed permission check in TBImageIDAccessCheck() to
      reflect shared/global attribute changes.
      
      www/showimageid_list, www/showstuff: Minor global/shared attribute
      changes.
      
      www/menu: Change osids/imageids pointer to point to the image list,
      not the osid list. This is more reasonable for mere users who have
      access to the EZ form, and thus never really need to concern
      themselves with osids.
      
      www/editimageid: Add proper pathname checking. There were no checks at
      all before!
      4c56daf6
  11. 02 Dec, 2002 1 commit
  12. 07 Jul, 2002 1 commit
  13. 08 Feb, 2002 1 commit
    • Leigh Stoller's avatar
      Big round of image/osid changes. This is the first cut (final cut?) at · a73e627e
      Leigh Stoller authored
      supporting autocreating and autoloading images. The imageid form now
      sports a field to specify a nodeid to create the image from; If set,
      the backend create_image script is invoked. Thats the easy part.
      Slightly harder is autoloading images based on the osid specified in
      the NS file. To support this, I have added a new DB table called
      osidtoimageid, which holds the mapping from osid/pctype to imageid.
      When users create images, they must specify what node types that image
      is good for. Obviously, the mappings have to be unique or it would be
      impossible to figure it out! Anyway, once that image mapping is
      in place and the image created, the user can specify that ID in the NS
      file. I've changed os_setup to to look for IDs that are not loaded,
      and to try and find one in the osidtoimageid. If found, it invokes
      os_load. To keep things running in parallel as much as possible,
      os_setup issues all the loads/reboots (could be more than a single set
      of loads is multiple IDs are in the NS file) at once, and waits for
      all the children to exit. I've hacked up os_load a bit to try and be
      more robust in the face of PXE failures, which still happen and are
      rather troublsesome. Need an event system!
      
      Contained in this revision are unrelated changed to make the OS and
      Image IDs per-project unique instead of globally unique, since thats a
      pain for the users. This turns out to be very messy, since underneath
      we do not want to pass around pid/ID in all the various places its
      used. Rather, I create a globally unique name and extened the OS and
      Image tables to include pid/name/ID. The user selects pid/name, and I
      create the globally unique ID. For the most part this is invisible
      throughout the system, except where we interface with the user, say in
      the web pages; the user should see his chosen name where possible, and
      the should invoke scripts (os_load, create_image, etc) using his/her
      name not the internal ID. Also, in the front end the NS file should
      use the user name not the ID. All in all, this accounted for a number
      of annoying changes and some special cases that are unavoidable.
      a73e627e
  14. 18 Oct, 2001 1 commit
  15. 16 Oct, 2001 1 commit
  16. 25 Sep, 2001 1 commit
  17. 24 Sep, 2001 1 commit