GitLab

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* statements, the licensing status of
the source files was unclear.

This commit is intended to clear up that confusion.

Most Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the Affero GNU General Public License, version 3
(AGPLv3).

Most Utah-copyrighted files related to ProtoGENI are distributed under
the terms of the GENI Public License, which is a BSD-like open-source
license.

Some Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the GNU Lesser General Public License, version 2.1
(LGPL).

We had a couple of different problems actually.

* We allow users to insert html into many DB fields (say, a project or
  experiment description).

* We did not sanitize that output when displaying back.

* We did not sanitize initial page arguments that were reflected in the
  output (say, in a form).

Since no one has the time to analyze every line of code, I took a couple of
shortcuts. The first is that I changed the regex table to not allow any <>
chars to go from the user into the DB. Brutal, but in fact there are only a
couple of places where a user legitimately needs them. For example, a
startup command that includes redirection. I handle those as special
cases. As more come up, we can fix them.

I did a quick pass through all of the forms, and made sure that we run
htmlspecialchars on everything including initial form args. This was not
too bad cause of the way all of the forms are structured, with a
"formfields" array.

I also removed a bunch of obsolete code and added an update script to
actually remove them from the www directory.

Lastly, I purged some XMLRPC code I did a long time ago in the Begin
Experiment path. Less complexity, easier to grok and fix.

	modified:   sql/database-fill.sql
	modified:   sql/dbfill-update.sql

redirect to the login page rather than printing a message with a link
to the page.  Otherwise send a "403 Forbidden" to keep robots from
indexing the page.  Also send appreciate HTTP responses on other
precheck errors to keep a robot from indexing the page.  In order to
do this the PAGEHEADER call needed to be moved to after
CheckLoginOrDie and Required/OptionalPageArguments on many pages.  A
warning will be printed if either CheckLoginOrDie or
Required/OptionalPageArguments detects that PAGEHEADER was already
called.

Also change the redirect in kb-show to be a permanent redirect (301)
rather than a temporary one (302) which is the default unless a status
code is given.

  register_globals=1 to turn POST/GET/COOKIES arguments in local variables.
  This is known to be a terrible security risk, and we keep saying we are
  going to fix it, and now I am. In order to accomplish this on a
  transitional basis (since I don't want the entire web interface to stop
  working while I debug it), and because the code just needs the cleanup, I
  am doing it like this: Each page will sport new declarations at the top:

	RequiredPageArguments("experiment", PAGEARG_EXPERIMENT,
                              "template",   PAGEARG_TEMPLATE,
                              "instance",   PAGEARG_INSTANCE,
                              "metadata",   PAGEARG_METADATA,
                              "osinfo",     PAGEARG_OSINFO,
                              "image",      PAGEARG_IMAGE,
                              "project",    PAGEARG_PROJECT,
                              "group",      PAGEARG_GROUP,
                              "user",       PAGEARG_USER,
			      "node",       PAGEARG_NODE,
			      "yesno",      PAGEARG_BOOLEAN,
			      "message",    PAGEARG_STRING,
			      "age",        PAGEARG_INTEGER,
                              "cost",       PAGEARG_NUMERIC,
                              "formfields", PAGEARG_ARRAY,
                              "unknown",    PAGEARG_ANYTHING);

	OptionalPageArguments("canceled", PAGEARG_BOOLEAN);

  The first token in each pair is the name of the global variable to
  set, and the second token is the type. So, for "experiment" we look at
  the URL for a pid/eid or exptidx, etc, sanity check them (safe for a
  DB query), and then try to find that experiment in the DB. If it maps
  to an experiment, set global variable $experiment to the object. Since
  its a required argument, produce an error if not supplied. Similar
  treatment for optional arguments, with the obvious difference.

  The goal is to have ALL argument processing in one place, consistent,
  and correct. I've found numerous places where we leak unchecked
  arguments into queries. It also cuts out a lot of duplicated code.

* To make the above easier to deal with, I've been replacing lots of
  hardcoded URLS in the code of the form:

	foo.php3?pid=$pid&eid=$eid ...

  with

        CreateURL("foo", $experiment)

  which creates and returns the neccessary url string, by looking at
  the type of its arguments (experiment, template, instance, etc.)

  Eventually plan to replace them all so that URL handling throughout
  the code is all defined in one place (all the new URL code is in
  url_defs.php).

* I have cranked up error reporting to tell me anytime a variable is
  used before it is initialized, plus a bunch of other stuff that PHP
  deems improper. Think of it like -Wall ... and boy we get a lot of
  warnings.  A very large percentage of the diffs are to fix all these
  warnings.

  The warnings are currently going to /usr/testbed/log/php-errors.log,
  and I'll be adding a script to capture them each night and mail them
  to tbops. This file also gets errors (this will be a change for
  developers; rather then seeing errors and warnings dumped in the
  middle of web pages, they will go to this file instead).

* Major refactoring of the code. More objects (nodes, images, osids).
  Moving tons of queries into the objects in the hopes of someday
  getting to a point where we can split the web interface onto a
  different server.  Lots of general cleanup.

converting to locally unique ids and later globally unique ids.

- Added 'Country' to users table
- Changed "Zip" to "ZIP/Postal Code"
- Reformatted Postal Address Forms

finally! These have been in the DB for a long time, but never used.
As Eric pointed out, its easier to get realistic address info from
people if we provide the right forms. The downside is that all current
users will have to adjust their info the next time they edit their
info. Thats okay. I thought about forcing all users to do it the next
time they log in, but I figured people would scream.

   - Performing an approval action (postpone/approve/deny/nuke) on a
     user in a subgroup results in an implicit version of that same
     approval action on the user in the
     default group (for approve, $trust='user'.)

   - Tidied up a table in approveuser_form.php3,
     but form is otherwise unchanged.

More rework on the groups system.

* BESTOWGROUPROOT permission added to dbdefs.

* Permissions criteria for group operations changed in dbdefs
  (consult code for full explanation.)

* Approveuser and Editgroup now check for BESTOWGROUPROOT
  permissions before allowing changes to group_root.

* approveuser_form and editgroup_form do not show "Group Root"
  as an option unless you are allowed to set it (or it is already set.)

* editgroup does not UPDATE rows where trust has not been changed.

* showgroup does a correct check to see whether to show the
  "group options" subpage.

* Altered consistency checks to treat any root as equivalent
  (so, if you're project_root in the default group, but group_root in
   a group, that won't be a problem)

* Moved consistency checks, which were done in two different places into
  dbdefs TBCheckGroupTrustConsistency()

* Added preemptive checks, so if 'user' or '*_root' are not valid
  trusts, they aren't displayed as options in editgroup_form and
  approveuser_form (using above function)

* In approveuser, a new approval may now be sent to group_root.

This query was duplicated in the menu page (which I just fixed) to
determine if a "new" icon needed to be displayed.

Fix nit with table formatting. Spread addr across table since we don't use addr2/city/state/zip anyway.

The big one.

New look;
most of the changes are in menu.php3.

A lot of the changes in other files are s/<TD>/<TH>/
for table headers.

Also closed some tags, tweaked some table styles, etc..

No actual functionality should have changed.

Will be installing soon...

already been approved into another project will show up.

* Cleanup! A lot of the structure derived from the early frame days,
  which had a noticable (and bad) effect on how I wrote the stuff.  I
  cleaned up most of that yuckyness.

* In process, optimize a little bit on the queries. The old code did
  about 9 queries just to write out the menu options, and then
  repeated most of those queries again in the page guts. I've
  consolidated the queries as much as possible (to 3) and cache all
  the results.

* Fix up problem with users who forget their passwords before
  verification. Basically, I fixed the more general problem of not
  being able to update your user info before verification/approval;
  users now get that menu option no matter their status.

* Fix up problem of users being able to access pages before
  verification (but after approval) by going around the menu options.
  The page level check (after the menu is drawn) now checks all
  conditions (password expired, unverified, unapproved, timedout, and
  also nologins()).

* Minor change in approveuser; do not show the new account to the
  project leader until the new user has verified his account.

* Change verification method, as reqwuested by Dave.  In addition to
  providing the key, also provide a web link to take the user straight
  to verification. I actually take them direct to the login page, and
  pass the key in as an argument. If the user is already logged in,
  bypass and go directly to the verify page (not the form page of
  course).  If the user is not logged in, let him log in, and then
  forward the key onward to the verify page. Basically, bypass the
  form all the time, and just do the verification.

* Minor change in showuser; Do not show pid/groups not approved in,
  and if the count is zero, do not draw the table headings.

and complaining about this week.

1. editgroup: You can now edit the trust levels for existing group
   members (default group too), and you can specify trust levels when
   adding users to subgroups.

2. approveusers: When approving users in the approval page, you can
   specify different levels of trust. Before, I invisibly set all the
   trust values the same. I also added some ordering to the DB query
   to group users together.

3. I added a great deal of error checking to the processing pages for
   both forms. I split things up into a pre/post pass. The prepass
   goes through all of the form args and checks them for consistency
   and correctness. Nothing is changed in the DB unless all checks
   pass for all args. Then I do a second pass and make the changes.
   Both scripts set the ignore_user_abort() flag to prevent the user
   from stopping the script and causing a DB inconsistency.

4. Added trust consistency checks as well. Rather than allow the
   project or group leader to set inconsistent trust levels, I look
   for those and just plain disallow them. You may not give different
   trust levels in different subgroups of the *same* project, and you
   may not give a user a higher trust level in the default group than
   in the subgroups. Both edit and approve make these checks, and the
   code is absolutely awful.

    From: "Patrick Tullmann 'tullmann'" <tullmann@cs.utah.edu>
    Subject: Re: TESTBED: aclement janos Project Join Request

    First the reply-to: address for approval mails should be
    testbed-ops@fast (right?).

    Second, Austin isn't listed on my testbed user approval page.  I
    assume Mike or a testbed person approved him (which is good because
    who knows when I'd get around to it. :)

    An option like "remove" or "ignore" or something like that for just
    nuking requests without a reply would be useful (I've got some guy
    from yahoo.com who wants to join Janos).

    Also, the date of the join request would be nice to know (e.g., for
    the above, I think he tried joining 4 or 5 months ago).

    he documentation above the table is out of synch with the pull-down
    boxes.

who the user is instead of passing ?uid to every page all the way down.
Update login timeout with each useful operation (done in checklogin).
Put default user name in the login box when visiting the page.

of all that duplicated goo).
Clean up all the formerror code, calling standard FORMERROR() function.
Get rid of all the inline DB specific constants, moving them to dbdefs.php3
include file.

multiple projects.