GitLab

table that will prevent an experiment from being swapped/modified. The
toggle is on the showexp page, and the toggle is *not* admin
over-ridable; you must turn the toggle off (and of course, you must be
an admin to do that).

unanticipated breakage. If that happens, just need to back out the
changes under the "suexec-stuff" tag. However, the better solution will
probably be to fix the PHP scripts that break by adding the proper
groups in the call to suexec (in the web page, see below) or by fixing
the backend Perl script that breaks.

This fix is primarily to address the problem of some users being in more
groups (cause of subgroups) then the max number of groups allowed
(NGROUPS).  The groups that really mattered (say, for creating an
experiment in a subgroup) could be left out cause they were at the end
of the list.

* suexec.c: Change how groups are handled. Instead of taking a single
  gid argument (the gid to setgid as), now takes a comma separated list
  of groups. Further, instead of doing a setgroups to the user's entire
  group list as specified in the groups file (getgroups), setgroups to
  just the groups listed on the command line, plus the user's primary
  group from the password file (this is to prevent potential breakage
  with accessing files from the users homedir, although might not really
  be necessary).

  This change is somewhat rational in the sense that in our case, suexec
  is not being used to run arbitrary user code (CGIs), but only to run
  specific scripts that we say should be run. The environment for
  running those scripts can be more tightly controlled then it would
  otherwise need to be if running some random CGI the user has in his
  public html directory.

* www: Change the gid argument to SUEXEC() in a number of scripts so
  that the project and subgroup are explicitly given to suexec, as
  described above. For example, in beginexp:

	SUEXEC(gid, "$pid,$unix_gid", ....);

  Aside: note that project names (pid) are always one to one with their
  unix group name, but subgroup names are not, and *always* have to be
  looked up in the DB, hence the "unix_gid" argument.

  Script breakage should require nothing more then adding the proper
  group to the list as above.

plus a lock field. The lock field was a simple "experiment locked, go away"
slot that is easy to use when you do not care about the actual state that
an experiment is in, just that it is in "transition" and should not be
messed with.

The other two state variables are "state" and "batchstate". The former
(state) is the original variable that Chris added, and was used by the tb*
scripts to make sure that the experiment was in the state each particular
script wanted them to be in. But over time (and with the addition of so
much wrapper goo around them), "state" has leaked out all over the place to
determine what operations on an experiment are allowed, and if/when it
should be displayed in various web pages. There are a set of transition
states in addition to the usual "active", "swapped", etc like "swapping"
that make testing state a pain in the butt.

I added the other state variable ("batchstate") when I did the batch
system, obviously! It was intended as a wrapper state to control access to
the batch queue, and to prevent batch experiments from being messed with
except when it was really okay (for example, its okay to terminate a
swapped out batch experiment, but not a swapped in batch experiment since
that would confuse the batch daemon). There are fewer of these states, plus
one additional state for "modifying" experiments.

So what I have done is change the system to use "batchstate" for all
experiments to control entry into the swap system, from the web interface,
from the command line, and from the batch daemon. The other state variable
still exists, and will be brutally pushed back under the surface until its
just a vague memory, used only by the original tb* scripts. This will
happen over time, and the "batchstate" variable will be renamed once I am
convinced that this was the right thing to do and that my changes actually
work as intended.

Only people who have bothered to read this far will know that I also added
the ability to cancel experiment swapin in progress. For that I am using
the "canceled" flag (ah, this one was named properly from the start!), and
I test that at various times in assign_wrapper and tbswap. A minor downside
right now is that a canceled swapin looks too much like a failed swapin,
and so tbops gets email about it. I'll fix that at some point (sometime
after the boss complains).

I also cleaned up various bits of code, replacing direct calls to exec
with calls to the recently improved SUEXEC interface. This removes
some cruft from each script that calls an external script.

Cleaned up modifyexp.ph3 quite a bit, reformatting and indenting.
Also fixed to not run the parser directly! This was very wrong; should
call nscheck instead. Changed to use "nobody" group instead of group
flux (made the same change in nscheck).

There is a script in the sql directory called newstates.pl. It needs
to be run to initialize the batchstate slot of the experiments table
for all existing experiments.

showexp page that its a batch experiment, by the menu options. Same
deal in the swapexp output, plus some other minor cleanup. The only
bug I found while trying to figure out the batchmode problem reported
this morning by the FileMover people, is that the cancelflag is not
cleared after swaping a running batch experiment out, so even after
reinjecting it into the queue, it will not run. Still, that does seem
to be what the FileMover people reported.

backend, but given that you have to mighty quick on the click, I never
bothered to put in the link. But I hear Mike is the quickest clicker
in the group.

as requested by Jay.

but simply entered into the DB record for the experiment until we know
what to do with them. Add to batchexp script arguments, since all that
stuff is done outside the web interface. Add a swapexp perl script to
swap an an experiment in/out form the command line. Add web links on
the Experiment Information page to do this from the web interface. A
bunch of locking changes. Previously expt_terminating in the
experiment record prevented multiple calls to terminate an experiment,
but now we have a more general locking problem with
start,swapin,swapout, and terminate, so change expt_terminating to
expt_locked (still a datetime field) and add locking to all of
startexp, swapexp, and endexp. Note that batch experiments cannot be
swapped yet because of locking issues still to be resolved. Minor
cleanup in tbreport to make email message look better.

experiment code. No longer uses another table. Rather, the experiment
record contains a couple of extra fields for the batch system. Also
combined some of the backend code (no longer a killbatch script).
Also added scriptable experiments; the batchexp program in the bin
directory can start an experiment from the command line, and in fact
is used from the web page for both batch experiments and immediate
experiments (-i option). All of the DB code that was in the web
interfaces was moved to batchexp.

of problems.

Change to startexp/endexp, which are almost scriptable now (can be
called directly in addition from the web page). Add front ends to
these for the web page (webstartexp and webendexp). These changes are
mostly support for batch mode.

termination, setup now exits immediately and sends email to the user
when the experiment is fully configured.

by the web server forks a child to do the actual work of calling tbend
and other stuff. The parent returns right away and the script ends.
When the experiment termination (child) ends, an email message is sent
to the user that issued the termination request. To prevent multiple
clicks, I added a DB field called expt_terminating that is a DATETIME
field.  If the field is set, the script fails and the user is told to
be more patient. I used a DATETIME field mostly for debugging purposes
so we can track and future problems.

option list, use a plain list instead. Simplify the DB queries now
that I'm a joining fool, and put in a proper confirmation step for
ending an experiment. Yippie.

who the user is instead of passing ?uid to every page all the way down.
Update login timeout with each useful operation (done in checklogin).
Put default user name in the login box when visiting the page.

webserver, as the uid of the experimentor. Not fully debugged yet
since the testbed has been kinda broken all day, but this needs to get
in if anyone else works on it.

of all that duplicated goo).
Clean up all the formerror code, calling standard FORMERROR() function.
Get rid of all the inline DB specific constants, moving them to dbdefs.php3
include file.

This just adds and deletes the database field entries (experiments table)
so that other things will work correctly.

EIDs no longer concatenate the PID into the string, but instead there
are independent PID/EID fields to make experiments unique. Also, there
were changes to the tb* scripts.