usually handled via uid cookie we get back from the browser, but if
the user Clicks stop or maybe has cookies off, we don't that info.

zillions of DB fields that we have to set. My solution was to add a
meta table that describes what is a legal value for each table/slot
for which we take from user input. The table looks like this right
now, but is likely to adapt as we get more experience with this
approach (or it might get tossed if it turns out to be a pain in the
ass!).

	CREATE TABLE table_regex (
	  table_name varchar(64) NOT NULL default '',
	  column_name varchar(64) NOT NULL default '',
	  column_type enum('text','int','float') default NULL,
	  check_type enum('regex','function','redirect') default NULL,
	  check tinytext NOT NULL,
	  min int(11) NOT NULL default '0',
	  max int(11) NOT NULL default '0',
	  comment tinytext,
	  UNIQUE KEY table_name (table_name,column_name)
	) TYPE=MyISAM;

Entries in this table look like this:

	('virt_nodes','vname','text','regex','^[-\\w]+$',1,32,NULL);

Which says that the vname slot of the virt_nodes table (which we trust the
user to give us in some form) is a text field to be checked with the given
regex (perlre of course), and that the min/max length of the text field is
1 and 32 chars respectively.

Now, you wouldn't want to write the same regex over and over, and since we
use the same fields in many tables (like pid, eid, vname, etc) there is an
option to redirect to another entry (recursively). So, for "PID" I do this:

        ('eventlist','pid','text','redirect','projects:pid',0,0,NULL);

which redirects to:

	('projects','pid','text','regex','^[a-zA-Z][-\\w]+$',2,12,NULL);

And, for many fields you just want to describe generically what could go
into it. For that I have defined some default fields. For example, a user
description:

        ('experiment,'usr_name','text','redirect','default:tinytext',0,0,NULL);

which redirects to:

	('default','tinytext','text','regex','^[\\040-\\176]*$',0,256,NULL);

and this says that a tinytext (in our little corner of the database
universe) field can have printable characters (but not a newline), and
since its a tinytext field, its maxlen is 256 chars.

You also have integer fields, but these are little more irksome in the
details.

	('default','tinyint,'int,'regex','^[\\d]+$',-128,127,NULL);

and you would use this anyplace you do not care about the min/max values
being something specific in the tinyint range. The range for a float is of
course stated as an integer, and thats kinda bogus, but we do not have many
floats, and they generally do not take on specific values anyway.

A note about the min/max fields and redirecting. If the initial entry has
non-zero min/max fields, those are the min mac fields used. Otherwise they
come from the default. So for example, you can do this:

    ('experiments','mem_usage','int','redirect','default:tinyint',0,5,NULL);

So, you can redirect to the standard "tinyint" regular expression, but you
still get to define min/max for the specific field.

Isn't this is really neat and really obtuse too? Sure, you can say it.

Anyway, xmlconvert now sends all of its input through these checks (its
all wrapped up in library calls), and if a slot does not have an entry, it
throws an error so that we are forced to define entries for new slots as we
add them.

In the web page, I have changed all of the public pages (login, join
project, new project, and a couple of others) to also use these checks.
As with the perl code, its all wrapped up in a library. Lots more code
needs to be changed of course, but this is a start.

of the CHECKEMAIL function as a result.

for key upload). Add more regexes and addslashes().

Users can, via, moduserinfo, set a preferred shell.
One of {tcsh, bash, csh, sh}.
When users are created, they are given tcsh.
All users which already exist have been given tcsh.

The first three are aggregate tables, while the experiment stats table
gets a record for each new experiment, and is updated when an
experiment is swapped in/out/modify or terminated. Look at the table
to see what is tracked. Once the experiment_stats record is updated,
the aggregate tables are updated as necessary. There are a bunch of
ugly changes to assign_wrapper to get the stats. Note that pnodes is
not incremented until an experiment sucessfully swaps in. This is in
leu of getting status codes; I'm not tracking failed operations yet,
nor creating the log file that Jay wants. I'll do that in the next
round of changes when we see how useful these numbers are.

Most of the changes are to create/delete table entries where
appropriate, and to display the records. Display is only under admin
mode, and the display is raw; just a dump of the assoc tables in php.
The last 100 experiment stats records are available via the Experiment
List page, using the "Stats" show option at the top. Bad place, but
will do for now.

our foreign clientele.

Nit; remove legend for "+" (recommended) on fields,
since there are no recommended fields!

- Added 'Country' to users table
- Changed "Zip" to "ZIP/Postal Code"
- Reformatted Postal Address Forms

finally! These have been in the DB for a long time, but never used.
As Eric pointed out, its easier to get realistic address info from
people if we provide the right forms. The downside is that all current
users will have to adjust their info the next time they edit their
info. Thats okay. I thought about forcing all users to do it the next
time they log in, but I figured people would scream.

we can send links that cause those fields to be filled in for people.
Add those links to email generated by newproject and newgroup pages
so that they can be saved by the leaders.

Add a bit of referrer magic to login page. When clicked from the join
or new project pages, tell login page to pass along the referrer page
so that when login is complete, user is zapped back to the original
page. This is especially nice when combined with the above change to
joinproject, whereby we send along the target pid/gid, but the user
has not yet logged in and remembers to do so via the link at the top
of the page.

Account to make it more clear.

a certain person who shall remain nameless unhappy!

protected page except those that are explicitly deemed okay for a
webonly user. This makes me feel better and safer!

see if user actually has an account (by checking user status user
table). Avoids trying to run suexec as a user that does not actuall
exist on boss cause they do not have an account (since we allow users
to edit personal info before being approved and getting an account).
For addpubkey, we have to run the program as someone, so when the user
does not have an account, run it as nobody.

things. Now whenever someone joins a project, it goes to everyone that
can approve them, and when an approval happens, it gets cc'd to the same
list, so they all know it happened and what permissions were given.

of a random number, as suggested in the php manual. This number
is stashed in the database, in the new verify_key column in the
users table.

Rename the functions that generate and get the keys, and move from
defs.php3 to dbdefs.php3, since they're now DB operations.

getting into the system.

Improved error reporting style.. also added image for uky,
though right now it is the same as the standard image
(will edit it soon.)

to an external perl script, and use ssh-keygen to attempt conversion
off SSH2/SECSH key formats. This is actually a simplification of the
php code, which is not generally very good at this kind of thing (or
maybe I mean perl is just better at it). The parsing and error
handling it also much improved.

table and create a new table to hold user_pubkeys, indexed by the
comment field of the key. Change mkacct to insert newly created Emulab
keys into that table, and to regen the users authorized_keys file
from the DB. Users should no longer edit their own authorized_keys
file or the changes will be lost (I put a comment in their files).

Change the three pages that deal with keys. join/new project can now
take a file of multiple keys; each is inserted. Moved the key stuff
that was in the update user info page into a new pubkeys page that
allows users to add/sub keys easily. New key additions are password
protected.

Unrelated change: Add an audit mode to mkacct to log its output and
send it to the tblogs email. Previously, warnings and errors tended to
get lost.

this last part! Senility is setting in. Anyway, change join project to
not send the email to the projleader when the user is new; let that
happen out of the verify page when the newuser does the verification.
Basically, duplicate some code to generate the email message.

XXX: I have not fixed up approveuser.php3 to check to make sure that
the user has been verified. Basically, for this to happen the proj
leader would have to generate the URL by hand, and thats not likely,
and not really a dangerous thing anyway. However, it is confusing.

* Cleanup! A lot of the structure derived from the early frame days,
  which had a noticable (and bad) effect on how I wrote the stuff.  I
  cleaned up most of that yuckyness.

* In process, optimize a little bit on the queries. The old code did
  about 9 queries just to write out the menu options, and then
  repeated most of those queries again in the page guts. I've
  consolidated the queries as much as possible (to 3) and cache all
  the results.

* Fix up problem with users who forget their passwords before
  verification. Basically, I fixed the more general problem of not
  being able to update your user info before verification/approval;
  users now get that menu option no matter their status.

* Fix up problem of users being able to access pages before
  verification (but after approval) by going around the menu options.
  The page level check (after the menu is drawn) now checks all
  conditions (password expired, unverified, unapproved, timedout, and
  also nologins()).

* Minor change in approveuser; do not show the new account to the
  project leader until the new user has verified his account.

* Change verification method, as reqwuested by Dave.  In addition to
  providing the key, also provide a web link to take the user straight
  to verification. I actually take them direct to the login page, and
  pass the key in as an argument. If the user is already logged in,
  bypass and go directly to the verify page (not the form page of
  course).  If the user is not logged in, let him log in, and then
  forward the key onward to the verify page. Basically, bypass the
  form all the time, and just do the verification.

* Minor change in showuser; Do not show pid/groups not approved in,
  and if the count is zero, do not draw the table headings.

* tmcd/ron: A new directory of client code, based on the freebsd
  client code, but scaled back to the bare minimum. Does only account
  and group file maintenance. I redid the account stuff so that only
  emulab accounts are operated on. Does not require a stub file, but
  instead keeps a couple of local dbm files recording what groups and
  accounts were added by Emulab. There is a ton of paranoia checking
  to make sure that local accounts are not touched.

  The update script that runs on the client node detaches so that the
  ssh from boss returns immediately. update can also be run from the
  node periodically and at boottime. The script is installed setuid
  root, but checks to make sure that *only* root or "emulabman" has
  invoked it.

* utils/sshremote: New file. For remote nodes, instead of using sshtb,
  use sshremote, which ssh's in as "emulabman", which needs to be a
  local non-root user, but with an authorized_keys file containing
  boss' public key.

* web interface changes: Allow user to specify his own public key in
  addition to the emulab key.

  Add option in showexp page to update accounts on nodes in the
  experiment. I was originally intending to do this from approveuser,
  but this was easier and faster. I will add an option to do it on the
  approveuser page later.

* libdb.pm: Add a TBIsNodeRemote() query to see if a node is in the
  local testbed or a pcRemote node. Currently, this test is hardwired
  to a check for class=pcRemote, but this will need to change to a
  node_types property at some point.

* node_update: Reorg so that there is a maximum number of children
  created. Previously, a child was forked for each node, but that
  could chew up too many processes, especially for remote nodes which
  might hang up. For the same reason, we need to "lock" the experiment
  so that it cannot be terminated while a node_update is in progress.
  Might be to relax that, but this was easy for now. Also add
  distinction between local and remote, since for remote we use
  sshremote insted of sshtb. Various cleanup stuff

* mkacct; When generating a new account, include user supplied pub key
  in the authorized keys file, in addition to the eumlab generated
  key. Both keys are stored in the DB in the users table. Anytime we
  update an account, get a fresh copy of the emulab pub key, in case
  user changes it.

when backing up (cause of an error that needs to be fixed) since not
all browsers handle this the same. Instead, redraw the form with all
of the original info and a list of error messages at the top.
Conceptually simple change, but it turns out to be a pain to implement
since you need to combine the form and processing code in one page
(well, its just a lot easier to do that), and then change all of the
forms to deal with a "default" value. That is, each different kind of
input tag (text, radio, select, checkbox, etc.) requires slightly
different changes to do that. Lots of forms, lots of entries on the
forms, and its a long slow tedious process. Much nicer though, although
the code is a bit harder to grok. At the same time, I added a lot more
sanity checks of the information being passed in.

The other change is to deal with how browsers handle the back button
on a form thats been properly submitted. Not all browsers use
the cache directives the same, and I was often typing back, only to
have some form get reposted. Thats a major pain in the butt. The way
to deal with that is to have the processor send out a Location header,
which modifies the browser history so that the post is no longer in
the history. You back up straight to the unposted form (if its in the
cache). I've done this to only some forms, since its a bit of a pain
to rework things so that you can jump ahead to a page that spits out
the requisite warm fuzzies for the specific operation just completed.

I've done newproject, joinproject, update user info, newimageid, and
newimaged_dz forms.