GitLab

generally improve the clarity and appearance of the maps.  The scale factor
buttons numbered 1-5 actually correspond to scale factors of 1, 1.5, 2, 2.5,
and 3.

Notice that the floorimages table in the database now has an integer "scale"
column to distinguish the cached zoomed-in images with a suffix "-[1-5].jpg" .

I use jpeg files instead of the png files that were used before.  They look as
good or better, and are a third the size.  Panning around at a given zoom
factor just involves cropping out a different rectangle of the scaled-up
images.

The 0311X064_[1-4]fl.pdf floor map files we got from Michael Kay started
out life in Adobe Illustrator.  Although GhostScript/gv crash while
trying to draw the pdf files, extracting them into PostScript files in
Acrobat on Coke allows GhostScript to render them flawlessly.

I started out using gs to render jpeg files in the shell, and then
found that ImageMagick uses the same GhostScript renderer, and nicely
handles cropping, drawing and annotations, and format conversion as well.

The PostScript rendering gets slow at higher zoom factors.  So I cache the
zoomed floor images, scaled up by rendering the PostScript files at a finer
dot-pitch (density) in GhostScript, and then cropping out the right rectangle
to register them properly.

For readability, and to avoid having the thin lines and bitmap fonts used in
some parts of the PostScript map from falling between the pixel cracks, I
actually render at twice the pixel density and filter down to the final image
with a gaussian -sharpen option to keep it from looking blurry.

ImageMagick is callable from both the shell and as the Image::Magick package
in Perl.  I converted the Perl logic in floormap.in to use Image::Magick
instead of the GD graphics library, resulting in anti-aliased (smoothed)
drawing of the wireless node dots and labels on floor maps for a better
appearance.  Then I added optional scaling and centering arguments which
select the proper cached zoomed-in image, and drive the logic for cropping and
drawing.

I modified the floormap.php3 file to wrap an input form around a set of zoom
control buttons and the floormap image.  The zoom buttons set the scale
argument to the Perl floormap script called on the server.  The floor map
image itself is wrapped in a graphical submit button, so clicking on it
returns the mouse coordinates within the image, which are then sent to the
centering argument of the Perl script.

ops as the user, but that breaks for admin people who have real shells
on boss and no unencrypted key in the .ssh dir. So, switched it to a
root ssh, and a simple proxy on the other side that flips to the user
and invokes the run_linktest stuff.

the user changes the quantities, add the usual mess of javascript to
make that happen.

Table values are included here too, switching from png to jpg files
and adding the scaled-up maps.

before turning them into xmlrpc dates

unanticipated breakage. If that happens, just need to back out the
changes under the "suexec-stuff" tag. However, the better solution will
probably be to fix the PHP scripts that break by adding the proper
groups in the call to suexec (in the web page, see below) or by fixing
the backend Perl script that breaks.

This fix is primarily to address the problem of some users being in more
groups (cause of subgroups) then the max number of groups allowed
(NGROUPS).  The groups that really mattered (say, for creating an
experiment in a subgroup) could be left out cause they were at the end
of the list.

* suexec.c: Change how groups are handled. Instead of taking a single
  gid argument (the gid to setgid as), now takes a comma separated list
  of groups. Further, instead of doing a setgroups to the user's entire
  group list as specified in the groups file (getgroups), setgroups to
  just the groups listed on the command line, plus the user's primary
  group from the password file (this is to prevent potential breakage
  with accessing files from the users homedir, although might not really
  be necessary).

  This change is somewhat rational in the sense that in our case, suexec
  is not being used to run arbitrary user code (CGIs), but only to run
  specific scripts that we say should be run. The environment for
  running those scripts can be more tightly controlled then it would
  otherwise need to be if running some random CGI the user has in his
  public html directory.

* www: Change the gid argument to SUEXEC() in a number of scripts so
  that the project and subgroup are explicitly given to suexec, as
  described above. For example, in beginexp:

	SUEXEC(gid, "$pid,$unix_gid", ....);

  Aside: note that project names (pid) are always one to one with their
  unix group name, but subgroup names are not, and *always* have to be
  looked up in the DB, hence the "unix_gid" argument.

  Script breakage should require nothing more then adding the proper
  group to the list as above.

just say that it is mapped but don't try to resolve the name.
Maybe someday I'll fix this...

of signal descriptions.  This caused us to never match and revealed a
bug when we hit the end of the list: strcasecmp blows up when passed a
NULL argument.

Now we defined our own list of signal names (with no NULL entries :-)

directly to ops via opsdir.

* Force tty allocation in the control script. Also force protocol 1 and
  ignore the users .ssh/config file.

* Do not start up linktest on delay nodes!

link to full show page.

       nextosid mechinism of 1.114 making it possible to map a generic *-STD
       OSID based on the time in which an experiment is created.  This
       provides backward compatibility for old experiments when the standard
       images are changed.

       The osid_map table lookup is triggered when the value of the nextosid
       field is set to 'MAP:osid_map'.  The nextosid also continues to behave
       as before: if it contains a valid osid, that OSID value is used to map
       independent of the experiment creation time.  The two styles can also
       be mixed, for example FBSD-JAIL has a nextosid of FBSD-STD which in
       turn is looked up and redirects to the osid_map and selects one of
       FBSD47-STD or FBSD410-STD depending on the time.

	CREATE TABLE osid_map (
	  osid varchar(35) NOT NULL default '',
	  btime datetime NOT NULL default '1000-01-01 00:00:00',
	  etime datetime NOT NULL default '9999-12-31 23:59:59',
	  nextosid varchar(35) default NULL,
	  PRIMARY KEY  (osid,btime,etime)
	) TYPE=MyISAM;

       Yeah, yeah, I'm using another magic date as a sentinel value.
       Tell ya what, in 7995 years, find out where I'm buried, dig me up,
       and kick my ass for being so short-sighted...

       The following commands are not strictly needed, they just give
       an example, default population of the table.  They cause the standard
       images to be revectored through the table and then remapped, based on
       two time ranges, to the exact same image.  Obviously, the second set
       would normally be mapped to a different set of images (say RHL90 and
       FBSD410):

	INSERT INTO osid_map (osid,etime,nextosid) VALUES \
	  ('RHL-STD','2004-09-08 08:59:59','emulab-ops-RHL73-STD');
	INSERT INTO osid_map (osid,etime,nextosid) VALUES \
	  ('FBSD-STD','2004-09-08 08:59:59','emulab-ops-FBSD47-STD');

	INSERT INTO osid_map (osid,btime,nextosid) VALUES \
	  ('RHL-STD','2004-09-08 09:00:00','emulab-ops-RHL73-STD');
	INSERT INTO osid_map (osid,btime,nextosid) VALUES \
	  ('FBSD-STD','2004-09-08 09:00:00','emulab-ops-FBSD47-STD');

	UPDATE os_info SET nextosid='MAP:osid_map' \
	  WHERE osname IN ('RHL-STD','FBSD-STD');

* When generating the initial ssh ley, use -C option to keygen so that
  the comment field is rational. Now set to $user@$domain.

* Add -f (force) option to use in conjunction with -i (inituser)
  option to regenerate the initial (unencrypted) ssh key. The user's
  auth_keys are files are regenerated as well.

  The bad thing about all this is that you have to go remove any old
  keys by hand via the web interface since we do not mark the key we
  generate in the DB.

to create them with a perl open() call.

the big power down when I changed sshtb to not invoke a subshell
wrapper, but to exec ssh directly. Built into tbacct was an extra pair
of \\ escapes to protect the outer double quotes from that extra
subshell.  When I removed that subshell, the extra escapes wreaked
havoc.

Needless to say, I really want to change how accounts are built on ops
to use tmcd like a regular experimental node. We can almost do that
now, except for the little detail that sending over 800 users would be
a lot of traffic for single updates. I've been meaning to extend the
protocol to allow for single updates, but have not had time yet!

* SSL based server (sslxmlrpc_server.py) that wraps the existing Python
  classes (what we export via the existing ssh XMLRPC server). I also have a
  demo client that is analogous the ssh demo client (sslxmlrpc_client.py).
  This client looks for an ssl cert in the user's .ssl directory, or you can
  specify one on the command line. The demo client is installed on ops, and
  is in the downloads directory with the rest of the xmlrpc stuff we export
  to users. The server runs as root, forking a child for each connection and
  logs connections to /usr/testbed/log/sslxmlrpc.log via syslog.

* New script (mkusercert) generates SSL certs for users. Two modes of
  operation; when called from the account creation path, generates a
  unencrypted private key and certificate for use on Emulab nodes (this is
  analagous to the unencrypted SSH key we generate for users). The other mode
  of operation is used to generate an encrypted private key so that the user
  can drag a certificate to their home/desktop machine.

* New webpage (gensslcert.php3) linked in from the My Emulab page that
  allows users to create a certificate. The user is prompted for a pass
  phrase to encrypt the private key, as well as the user's current Emulab
  login password. mkusercert is called to generate the certificate, and the
  result is stored in the user's ~/.ssl directory, and spit back to the user
  as a text file that can be downloaded and placed in the users homedir on
  their local machine.

* The server needs to associate a certificate with a user so that it can
  flip to that user in the child after it forks. To do that, I have stored
  the uid of the user in the certificate. When a connection comes in, I grab
  the uid out of the certificate and check it against the DB. If there is a
  match (see below) the child does the usual setgid,setgroups,setuid to the
  user, instantiates the Emulab server class, and dispatches the method. At
  the moment, only one request per connection is dispatched. I'm not sure
  how to do a persistant connection on the SSL path, but probably not a big
  deal right now.

* New DB table user_sslcerts that stores the PEM formatted certificates and
  private keys, as well as the serial number of the certificate, for each
  user. I also mark if the private key is encrypted or not, although not
  making any use of this data. At the moment, each user is allowed to get
  one unencrypted cert/key pair and one encrypted cert/key pair. No real
  reason except that I do not want to spend too much time on this until we
  see how/if it gets used. Anyway, the serial number is used as a crude form
  of certificate revocation. When the connection is made, I suck the serial
  number and uid out of the certificate, and look for a match in the table.
  If cert serial number does not match, the connection is rejected. In other
  words, revoking a certificate just means removing its entry from the DB
  for that user. I could also compare the certificate itself, but I am not
  sure what purpose that would serve since that is what the SSL handshake is
  supposed to take of, right?

* Updated the documentation for the XMLRPC server to mention the existence
  of the SSL server and client, with a pointer into the downloads directory
  where users can pick up the client.

use/test the event system while logins are turned off.

* The per-experiment event scheduler now runs on ops instead of boss.
  Boss still runs elvind and uses events internally, but the user part
  of the event system has moved.

* Part of the guts of eventsys_control moved to new script, eventsys.proxy,
  which runs on ops and fires off the event scheduler. The only tricky part
  of this is that the scheduler runs as the user, but killing it has to be
  done as root since a different person might swap out the experiment. So,
  the proxy is a perl wrapper invoked from a root ssh from boss, which
  forks, writes the pid file into /var/run/emulab/evsched/$pid_$eid.pid,
  then flips to the user and execs the event scheduler (which is careful
  not to fork). Obviously, if the kill is done as root, the pid file has to
  be stored someplace the user is not allowed to write.

* The event scheduler has been rewritten to use Tim's C++ interface to the
  sshxmlrpc server on boss. Actually, I reorg'ed the scheduler so that it
  can be built either as a mysql client, or as RPC client. Note that it can
  also be built to use the SSL version of the XMLRPC server, but that will
  not go live until I finish the server stuff up. Also some goo for dealing
  with building the scheduler with C++.

* Changes to several makefiles to install the ops binaries over NFS to
  /usr/testbed/opsdir. Makes life easier, but only if boss and ops are
  running the same OS. For now, using static linking on the event scheduler
  until ops upgraded to same rev as boss.

* All of the event clients got little tweaks for dealing with the new CNAME
  for the event system server (event-sever). Will need to build new images
  at some point. Old images and clients will continue to work cause of an
  inetd hack on boss that uses netcat to transparently redirect elvind
  connections to ops.

* Note that eventdebug needs some explaining. In order to make the inetd
  redirect work, elvind cannot be listening on the standard port. So, the
  boss event system uses an alternate port since there are just a few
  subsystems on boss that use the server, and its easy to propogate changes
  on boss. Anyway, the default for eventdebug is to connect to the standard
  port on localhost, which means it will work as expected on ops, but will
  require -b argument on boss.

* Linktest changes were slightly more involved. No longer run linktest on
  boss when called from the experiment swapin path, but ssh over to ops to
  fire it off. This is done as the user of course, and there are some
  tricks to make it possible to kill a running linktest and its ssh when
  experiment swapin is canceled (or from the command line) by forcing
  allocation of a tty. I will probably revisit this at some point, but I
  did not want to spend a bunch of time on linktest.

* The upgrade path detailed in doc/UPDATING is necessarily complicated and
  bound to cause consternation at remote sites doing an upgrade.