GitLab

on the Utah main emulab only.

"modes".  Also some machinations in libplab and the federation to support
not only object synch, but also "operations".

database is only used for Utah's emulab.  In the process many other
changes where made to dumperrorlog to fix bugs and make it more
correct/robust.  I also avoided the need to explicitly lock any tables.

they add a .in file somewhere

Set the charset for all emulab pages to utf-8 which is needed to
display foreign names correctly.  Also added three rows to table_regex:
tinytext_utf8, text_utf8, fulltext_utf8

Enhanced form_defs.php:

  - Add Support for Textarea

  - Add Support for a vertical list, for example see Category in
    submitpub.php

  - Add support for dumping the form values in plain text, used in the
    email I send to testbed-ops.

  - Change Error Reporting text for subfields such as in a list so
    they also include the text from the outer field, for example in
    newosid "ping" will become "OS Features, ping".  I needed this
    since some of my subfields didn't have labels and it seamed like
    the right thing to do.

  - Set #return_value to 1 in FormRenderCheckBox if it not already
    specified

  - In FormRenderSelect, if the #value is not in the list add it to
    the end.

  - Add support for "display" type, to just display some text in the
    table

  - Possibly some other little things I forgot about.

New site variable PUBSUPPORT.

Implement FS#201: Provide users a way to add a paper that used Emulab
see (https://users.emulab.net/flyspray/?do=details&id=201), summary of
changes:

  - Controlled via the new site variable PUBSUPPORT.

  - The emulab_papers spread sheet is now frozen all new changes
    should be made using the new system.

  - New pages submitpub.php for users to submit a new publication
    with different views for admins and non-admins.

  - New page expubs.php to replace doc/expubs.html, lists publications
    in database.  Also allows admins to edit publications.

  - Project members can edit papers associated with the project as
    long as they have local_root or higher privileges. Papers for a
    project are listed under a new tab in the showproject page. I keep
    track of the last person to edit the paper submission in addition
    to who initially submitted it.

  - Users can modify papers they submitted which is needed if the
    project was N/A. Papers users submitted now appear in a new tab in
    the showuser page.

  - Script to import spreadsheet to the database is in
    utah/bibliography/emulab_pubs-import.pl.

  - Changes web pages that point to doc/expubs.html to instead point
    to expubs.php

config.h for tmcd.c

and non-admins in projects that have users in both categories (mostly
emulab-ops, tbres and testbed).  The new order says that if the
swapper of an active experiment is an admin, only other admins have
their accounts (and homedirs) propagated to the experiment.  Analogous
for non-admin experiments.

Add DYNAMICROOTPASSWORDS, default to true, which causes the root
passwords on emulab images to be changed each time they are allocated
to an experiment. This is to prevent the same password on every node
for years at a time!

now, this is keyed off nodetype.  Lots of hardcoded constants and config
stuff moved to attributes in the db.  You can now set per-PLC and
per-slice attributes, so you can (for instance) use different auth info
whenever you want.  Experiments can use preexisting slices if somebody
sets up the db before swapin.  Also, we no longer have to rely on
slices.xml to sync up nodes/sites with PLC... can use xmlrpc instead.

Lots of code cleanup, improved some abstractions, etc.

Makefile is not ready yet.

fully integrated Trac. I put a new installation in /usr/local/www/data/trac
and I added all the hooks for adding users and doing the cross machine
login. Only STUDLY() users will actually see the new option in the collab
dropdown menu.

I have not done anything to make the trac installation look like Emulab.

     www/newimageid_ez.php3 - The reworked PHP page.  Calls Image::NewImageId with ez=1.
     www/newimageid.php3 - Call Image::NewImageId with ez=0.
     www/imageid_defs.php - Re-use the Image::NewImageId class method, adding an 'ez' arg.
     backend/{newimageid_ez,GNUmakefile}.in configure configure.in - New backend script.
         After checks, calls OSinfo->Create and Image->Create with the same XML args array.
     db/Image.pm.in - Re-use the Image->Create method, adding an imageid over-ride arg.
     db/OSinfo.pm.in - Filter out extraneous db slot args from XML in the Create method.
     db/libdb.pm.in - Add TB_{OS,IMAGE}ID_* constants from dbdefs.php3 .
     sql/database-fill.sql - Add OS entries to the table_regex 'images' pattern set.

 * added new tmcd directory with Ubuntu (really, Debian) specifics
 * fixed up GNUmakefiles to not do "-g wheel" when creating directories
 * other, relatively minor tweaks

     www/editgroup.php3 - The reworked PHP page.
     www/editgroup_form.php3 - Removed, form merged into editgroup.php3 .
     www/showgroup.php3 - Link to editgroup.php3, rather than editgroup_form.php3 .
     www/group_defs.php - Add an Image::EditGroup class method
                            bridging to the script via XML.
     backend/{editgroup,GNUmakefile}.in configure configure.in - New backend script.
     db/Group.pm.in - Add an EditGroup worker class method for script arg checking.
                      Also the missing NonMemberList and CheckTrustConsistency methods,
                      and a GETTRUST flag to MemberList, as in the PHP version.
     db/User.pm.in - Add the missing but tiny {Set,Get}TempData methods.
     sql/database-fill.sql - Add gid_idx to the table_regex 'groups' checking patterns.

     www/newimageid.php3 - The reworked PHP page.
     www/imageid_defs.php - Add an Image::NewImageId class method
                            bridging to the script via XML.
     backend/{newimageid,GNUmakefile}.in configure configure.in - New backend script.
     db/Image.pm.in - Add a Create class method for script arg checking.
     sql/database-fill.sql - Add to the table_regex 'images' checking patterns.

     www/editsitevars.php3 - The reworked PHP page, including
       a SetSiteVar function bridging to the script via XML.
     backend/{editsitevars,GNUmakefile}.in configure configure.in - New backend script.
     sql/database-fill.sql - Add table_regex 'sitevariables' checking patterns.

       Note: new_node attributes have not yet been converted from
       names for osid's and imageid's to new-style integer indices.
     www/editnodetype.php3 - The reworked PHP page, including
       an EditNodeType function bridging to the script via XML.
     backend/{editnodetype,GNUmakefile}.in configure configure.in - New backend script,
       including wildcarding on %xmlfields table entries to handle dynamic attributes.
     sql/database-fill.sql - Add table_regex 'node_types' checking patterns.

     www/editimageid.php3 - The reworked PHP page.
     www/imageid_defs.php - Add an Experiment::EditImageid class method
                            bridging to the script via XML.
     backend/{editimageid,GNUmakefile}.in configure configure.in - New backend script.
     db/Image.pm.in - Add an EditImageid worker class method for script arg checking.
     sql/database-fill.sql - Add to the table_regex 'images' checking patterns.

     www/editexp.php3 - The reworked PHP page.
     backend/{editexp,GNUmakefile}.in configure configure.in - New backend script.
     www/experiment_defs.php - Add an Experiment::EditExp class method
                               bridging to the script via XML.
     db/Experiment.pm.in - Add an EditExp worker class method for script arg checking,
                           And a missing description method.
     sql/database-fill.sql - Add to the table_regex 'experiments' checking patterns.

     www/newmmlist.php3 - The reworked PHP page.
     backend/{newmmlist,GNUmakefile}.in configure configure.in - New backend script.
     sql/database-fill.sql - Add table_regex 'mailman_lists' checking patterns.

     www/newgroup.php3 - The reworked PHP page.
     www/newgroup_form.php3 - Removed, form merged into newgroup.php3 .
     www/group_defs.php - Add a Group::Create method bridging to the script via XML.
     www/showproject.php3 - Link to newgroup.php3, rather than newgroup_form.php3 .
     configure configure.in  - Add the newgroup backend script.
     backend/{newgroup,GNUmakefile}.in - Add the Perl script.
     db/Group.pm.in - Update the AccessCheck method to allow TB_PROJECT_LEADGROUP.
     sql/database-fill.sql - Add table_regex 'groups' checking patterns.

 GNUmakefile.in configure configure.in  - Add the testbed/backend directory.
 www/moduserinfo.php3 - The reworked PHP page.
 www/user_defs.php - Add a ModUserInfo method bridging to the script via XML,
     and remove the ChangeProfile method that is being replaced.
 backend/{moduserinfo,GNUmakefile}.in - Add the Perl script.
 db/User.pm.in - Add a ModUserInfo worker class method for script arg checking.
     Also SetUserInterface, SetWindowsPassword, and AccessCheck methods,
     and a copy of the escapeshellarg fn.
 sql/database-fill.sql - Add some to the table_regex 'users' checking patterns.

Support stuff:
 account/tbacct.in - Update the UpdateWindowsPassword() function.
 db/libdb.pm.in - Add TBDB_USER_INTERFACE_EMULAB and TBDB_USER_INTERFACE_PLAB().
 tbsetup/libtestbed.pm.in - Add TB*EMAIL, TBMAIL_* vars (OPS, WWW, AUDIT).

widearea reloading support.

* New slot in the images table to store an access key which remote
  sites must provide in order to download an image (via https).

* tmcd returns a different kind of ADDRESS field from doloadinfo.
  Instead of the multicast stuff, return a URL that points to boss'
  web server. The URL is of the form:

   https://www.myemulab.net/spewimage.php?imageid=10013&access_key=abcdef

  which as you can see is fully specified; the client does not need
  to know anything else.

* New webpage and backend scripts appropriately called "spewimage"
  which also includes support for the http HEAD request (from wget) to
  avoid downloading images that are already on the node. I just
  learned about this HEAD request stuff today ... but otherwise these
  operate as expected, spewing the image if the access key is provided.

* Changes to rc.frisbee to deal with remote loading. In addition to
  URL support, I also added support for simple paths, the intent being
  that we will probably distribute images offline (say, at night) so
  that when a node reboots it doesn't actually have to wait 60 minutes
  for an image to download. I have not added any server side support
  for this yet though. Maybe later this week.

* Other bits and pieces and fixes to make this work.

"rtag" directive to initiate template modify operations. So, to get started
you do a checkout:

  cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo checkout XXXXX

where XXXXX is the part of the guid (10000/1) before the slash. Might try
and roll all templates into a single project wide repo at some point, to
avoid the extraneous path stuff, but didn't want to worry that just yet.

Okay, so have a checkout. You can work along the trunk, doing commits. To
create a new template (a modify of the existing template), you tag the tree
using rtag:

  cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo rtag mytag XXXXX

A template modify is started at the end, and you should probably wait for
email before continuing. Eventually I will need to add locking of some
kind, but I have to do the modify in the background, or else I get deadlock
cause cvs keeps the repo locked, and the modify also needs to access it.

Each time you tag along the trunk, you get a modified template, which in
the history diagram looks like:

  10000/1 --> 10000/2 --> 10000/3 ...

If you want to branch, say at 10000/2 you can create a branch tag using rtag:

  cvs -d [cut] rtag -r T10000/2 -b mytag2 XXXXX

You can also use your own tags for -r option, but I also create a TXXXXX/YY
tag at each template modify, which is easy to remember.

Then update your sandbox to the new branch, commit changes along that
branch, and then later use rtag again to initiate a template modify
operation:

  cvs update -r mytag2
  cvs commit ...
  cvs -d [cut] rtag -r mytag2 mytag3 XXXXX

And now the history diagram looks like:

  10000/1 --> 10000/2 --> 10000/3 ...
                |
                |
                -> 10000/4 ...

You should be able to mix interaction via the web with interaction via the
cvs interface. I've tested it, although not extensively.

elvin_gateway with the daemon_wrapper

(linux), which is probably ok, but not optimal.