GitLab

brought on by too much thinking and not enough hacking.

Grant permission to use types/class of nodes. If given a specific
"type", then grant permission to use just that type. If its a class, then
permission to use all of the types in that class, the class itself, and any
aux nodetypes for the type/class (node_types_auxtypes table).

For example:

	wap grantnodetype -p testbed pc2000
	wap grantnodetype -p testbed pcvm
	wap grantnodetype -p testbed pcplab

Also added webnsgen as a frontend for web pages that need to call
nsgen.

 - fix bad indenting to a uniform 4 spaces (before was 2, 4 and 8 mixed)
 - Move ping-for-isup functionality into a separate script
 - Make sure every transition triggered by stated (directly or indirectly)
   sends an event, instead of taking shortcuts.

This called for a new script, eventping, which just pings until the node
is pingable, then sends an ISUP event. Stated runs this in the background
where necessary, and nothing else should run it.

Adding eventping meant modifying configure and the utils makefile, too.

history).

to an external perl script, and use ssh-keygen to attempt conversion
off SSH2/SECSH key formats. This is actually a simplification of the
php code, which is not generally very good at this kind of thing (or
maybe I mean perl is just better at it). The parsing and error
handling it also much improved.

flips the uid/gid to nobody/nobody. It would be good to run this in a
chroot shell, but that would be difficult given that we cannot easily
rebuild cvsupd (modula-3). The right solution is to either run it in a
jail or to move it to ops.

Note, files in the sup tree obviously have to be world readable for
cvsupd to send them off.

commands to fill a few tables that should be common across most
sites, so that this data can be distributed to them.

The simple single-column table exported_tables controls which tables
are output.

links that do not need to get created in the ops install target.

sshtb also checks for unified control nodes, and if the destination
host is one of them, just runs the command, rather than invoking it
remotely via ssh.

admin privs.  The idea is that you have to be explicit about when you
want to make use of your super powers, to prevent accidents.

Use the new withadminprivs script to get your admin privs, as in:
withadminprivs inuse
or
withadminprivs node_reboot -e testbed,foo

script that checks the database to see if local or remote. The problem
with this is that the ssh syntax makes it hard to determine the host
name by inspection. Would need to parse all the ssh args (bad idea),
ot work backwards and try to figure out the difference between the
command (which is not a string but a sequence of args) and the host
and the preceeding ssh args. Hell with that! Changed sshtb to require
a specific -host argument. Read the args and look for it. Error out of
not found, to catch improper usage.

The moral of this update: "sshtb [ssh args] -host <host> [more args ...]

* tmcd/ron: A new directory of client code, based on the freebsd
  client code, but scaled back to the bare minimum. Does only account
  and group file maintenance. I redid the account stuff so that only
  emulab accounts are operated on. Does not require a stub file, but
  instead keeps a couple of local dbm files recording what groups and
  accounts were added by Emulab. There is a ton of paranoia checking
  to make sure that local accounts are not touched.

  The update script that runs on the client node detaches so that the
  ssh from boss returns immediately. update can also be run from the
  node periodically and at boottime. The script is installed setuid
  root, but checks to make sure that *only* root or "emulabman" has
  invoked it.

* utils/sshremote: New file. For remote nodes, instead of using sshtb,
  use sshremote, which ssh's in as "emulabman", which needs to be a
  local non-root user, but with an authorized_keys file containing
  boss' public key.

* web interface changes: Allow user to specify his own public key in
  addition to the emulab key.

  Add option in showexp page to update accounts on nodes in the
  experiment. I was originally intending to do this from approveuser,
  but this was easier and faster. I will add an option to do it on the
  approveuser page later.

* libdb.pm: Add a TBIsNodeRemote() query to see if a node is in the
  local testbed or a pcRemote node. Currently, this test is hardwired
  to a check for class=pcRemote, but this will need to change to a
  node_types property at some point.

* node_update: Reorg so that there is a maximum number of children
  created. Previously, a child was forked for each node, but that
  could chew up too many processes, especially for remote nodes which
  might hang up. For the same reason, we need to "lock" the experiment
  so that it cannot be terminated while a node_update is in progress.
  Might be to relax that, but this was easy for now. Also add
  distinction between local and remote, since for remote we use
  sshremote insted of sshtb. Various cleanup stuff

* mkacct; When generating a new account, include user supplied pub key
  in the authorized keys file, in addition to the eumlab generated
  key. Both keys are stored in the DB in the users table. Anytime we
  update an account, get a fresh copy of the emulab pub key, in case
  user changes it.

supporting autocreating and autoloading images. The imageid form now
sports a field to specify a nodeid to create the image from; If set,
the backend create_image script is invoked. Thats the easy part.
Slightly harder is autoloading images based on the osid specified in
the NS file. To support this, I have added a new DB table called
osidtoimageid, which holds the mapping from osid/pctype to imageid.
When users create images, they must specify what node types that image
is good for. Obviously, the mappings have to be unique or it would be
impossible to figure it out! Anyway, once that image mapping is
in place and the image created, the user can specify that ID in the NS
file. I've changed os_setup to to look for IDs that are not loaded,
and to try and find one in the osidtoimageid. If found, it invokes
os_load. To keep things running in parallel as much as possible,
os_setup issues all the loads/reboots (could be more than a single set
of loads is multiple IDs are in the NS file) at once, and waits for
all the children to exit. I've hacked up os_load a bit to try and be
more robust in the face of PXE failures, which still happen and are
rather troublsesome. Need an event system!

Contained in this revision are unrelated changed to make the OS and
Image IDs per-project unique instead of globally unique, since thats a
pain for the users. This turns out to be very messy, since underneath
we do not want to pass around pid/ID in all the various places its
used. Rather, I create a globally unique name and extened the OS and
Image tables to include pid/name/ID. The user selects pid/name, and I
create the globally unique ID. For the most part this is invisible
throughout the system, except where we interface with the user, say in
the web pages; the user should see his chosen name where possible, and
the should invoke scripts (os_load, create_image, etc) using his/her
name not the internal ID. Also, in the front end the NS file should
use the user name not the ID. All in all, this accounted for a number
of annoying changes and some special cases that are unavoidable.

testbed boot process and the freebsd pxeboot/MFS kernel. I'm hoping
this will be useful enough for users to load their own disk images,
but we shall see. Usage is simple:

	/usr/testbed/bin/node_admin on|off pcXXX

The "on" state boots the MFS, and the "off" states sets it back to
normal.

This uses the pxe booted freebsd kernel and MFS. In addition, I use
the standard testbed mechanism of specifying a startup command to
run, which will do the imagezip to NFS mounted /proj/<pid>/.... The
controlling script on paper sets up the database, reboots the node,
and then waits for the startstatus to change. Then it resets the DB
and reboots the node so that it returns back to its normal OS. The
format of operation is:

	create_image <node> <imageid> <filename>

Node must be under the user's control of course. The filename must
reside in the node's project (/proj/<pid>/whatever) since thats the
directory that is mounted by the testbed config software when the
machine boots. The imageid already exists in the DB, and is used to
determine what part of the disk to zip up (say, using the slice option
to the zipper). Since this operation is rather time consuming, it does
the usual trick of going to background and sending email status later.

sshtb is a _very_ simple shell script that runs ssh with a few commandline
parameters, which make it play nicer in an script environment. These
parameters can be changed with the '--with-ssh-args' argument, but default to:
'-q -o "BatchMode yes" -o "StrictHostKeyChecking no"'
All ssh calls now use this script.

to mere users.

simple autogen of the symlinks to plasticwrap on the control node, and
added control-install targets in several more directories.

Usage: if2port <node | node:if | node if>
Example:
<ricci@paper:~>./if2port tbpc01
+--------+-----+--------+-----+-----+
|node_id1|card1|node_id2|card2|port2|
+--------+-----+--------+-----+-----+
|tbpc01  |    0|cisco   |    3|   25|
|tbpc01  |    1|cisco   |    3|   27|
|tbpc01  |    2|cisco   |    3|   29|
|tbpc01  |    3|cisco   |    3|   31|
|tbpc01  |    4|cisco2  |    3|    1|
+--------+-----+--------+-----+-----+
5 rows processed

<ricci@paper:~>./if2port tbpc01:0
+--------+-----+--------+-----+-----+
|node_id1|card1|node_id2|card2|port2|
+--------+-----+--------+-----+-----+
|tbpc01  |    0|cisco   |    3|   25|
+--------+-----+--------+-----+-----+
1 row processed

<ricci@paper:~>

the testbed list.