GitLab

Ignore a couple of plab nodes with duplicate MAC addrs until they are fixed.
These nodes won't be available for allocation in the meantime.

A few fixes for plab.

assign_wrapper:

Assign unique sshd port numbers.  This was only being done for pcvms for some
reason.

libplab.py:

Make 'nobody' the contact uid for newly createed plab nodes (instead of
Brent Chun).

rc.inplab"

Create /local/logs inside plab vservers.

The actual dump routine still needs to be added.

Updates to to Error Logging API Code.

You should start seeing much better error messages coming from my
system.  Errors coming from parse.proxy and assign (the two most
frequent sources of errors) should now be concise and to the point.
Errors coming from libosload/libreboot (the next most frequent source
of errors) should now also be much better, but not perfect.  Getting
perfect errors will likely a rework of how errors are handled in
libosload/libreboot, just adding tberror/tbwarn/tbnotice calls is not
enough.  I can do this at a latter date if necessary.

A few minor database changes.

Some changes to the API.  A few bug fixes. Lots of tberror/tbwarn/tbnotice
added to scripts.

Since assign is a C program, and at this time my API is perl only, I wrote a
second wrapper around assign, assign_wrapper2.  When assign fails errors are
now parsed in assign_wrapper2, sent to stderr and logged.  This means that
RunAssign() just returns when assign fails rather than echoing some of
assign.log output and then quiting.  The output to the activity log remains
unchanged.

Since "parse.proxy" is run from ops I couldn't use my API in it, even though
it is a perl program.  Instead I parse the errors coming form it in
parse-ns.

into the main NS files via the "source" command, and stuffing them
into the experiment archive.

would be no big deal, except that we want to retain user_stats for
deleted users, and rather then a deleted_user_stats table, I want to
retain stats for deleted users in the user_stats table, since that
is a more natural place for them.

The main problem is that we use the login (uid) as the cross table
reference slot all over the DB, which is fundamentally incorrect, if
we want to be able reuse uids and still know what historical data
refers to.

So, I have taken a few baby steps towards weaning us off the uid, and
towards permanently unique key for users, using the unix_uid integer
for now, but probably something slightly different later.

The user_stats is now indexed on this new key (called uid_idx in the
users_stats table) instead of the plain uid.

The unix_uid slot in the users table is no longer an auto_increment
field, but instead uses the emulab_indicies table for the next
available index.

(actually just stats gathering right now, no images are produced)

that it can be "included" into other scripts; showlogfile, beginexp,
swapexp, modifyexp, endexp.

Each of these scripts now appends the showlog directly to the page,
instead of providing a link to a different page.

Also changed the showlog code to place the output into an inner iframe
so that it can be sized to fit in the window, without the outer stuff
(like the menus) getting scrolled away.

array for every 32-byte entry.  When building a hashfile with 100,000+
entries, this caused quite a bit of moving memory around!

It is still stupid, I just realloc after every 256KB or so.  Need to
eliminate the need to have the whole thing contiguous and maybe the need
to have the whole thing in memory at once!

allows takes you to new (admin only) page to select a project that
node will be reserved for.

* The node is not actually *reserved*, it is *pre* reserved! The node
  stays in the free pool, and is available only for use in the project
  to which it is reserved.

* The node can already be reserved to some other project when you
  pre-reserve it. It is not until the current owner releases the node
  that the pre-reservation takes effect.

* The node free counts (on the web pages) count a free a node with a
  pre-reservation, as allocated. This way people do not see a free
  count that includes a node they will never be able to get.

(as generated by imagehash).  This is Mike's piss-all-over-it version of
Prashanth's implementation of Mike's algorithm.  Delta images are created
by specifying "-H <sigfile>" on the command line.

This is still experimental.

Fix plab rootball distribution scripts.

working on getting a feeling of whether or not it's feasible to
implement assign in Comet. I think if it is, it would be a huge win in
terms of maintainability and extensibility.

So far, it's looking very promising. I'm struggling a bit with the
syntax, and the lack of an API reference. But, once I figure out how
to do what I need, it's making a lot of sense.

All this program does so far is try to respect the number of slots on
physical nodes - ie. not assign more than one vnode to a pnode, but it
can do arbitrary numbers instead of just 1.

It does not read top/ptop files, doesn't know about types, doesn't try
to minimize any score, doesn't do features/desires, doesn't know about
links at all, etc. Like I said, it's the beginnings of a toy program.

It looks to me like what we're going to end up doing is writing a
program to produce Comet programs. One of the cool things about this
is that it means we can leave out any constraints, etc. that are not
relevant to the particular mapping problem at hand. And, it should
make it easy for us to plug in other search techniques (ie. write the
obvious ones for certain pathological cases, like snakes and
trees.)

Fix up autoswap units in advanced form.

Add note about updating the node/ssh_pubkey sitevar.

With some work, we might be able to further reduce the image size.

Create a hosts file containing entries for boss and ops
for plab slivers and put it into the rootball.  This
will (hopefully) avoid flakiness due to flaky DNS servers.

files are being added to the archive at the correct places, but it
seems to work.

the experiment from ops' db to boss'.  File names are stored in the
accessed_files table and are associated with the experiment via the
fs_resources table.

into the main NS files via the "source" command.

Bump the plab rootball version up.

The revived Plab interface is here!

Lots of updates to the plab backend, including improved plab <-> elab node
id translation and update handling.  Includes support for the current PLC
API, and the new pl_conf node manager interface API.  Several more db library
routines were ported from the perl library to the python one to support the
new code (mostly the node_id tracking stuff).  Fixes to the client side and
also a rootball creation cleanup (binaries removed from the CVS repo).

There are also enhancements to the experiment view page for experiments
including plab nodes: site and widearea hostname are now displayed along
with the other node information.

Note that the way setup timeout for vnodes is calculated has been changed a
bit.  Instead of using a hardwired base timeout, the base timeout is now
based on the reload_waittime database field, which comes from the 'OS'
(e.g., FBSD-JAIL, RHL-PLAB) the vnode runs.

The default max duration for a plab slice created through the plab_ez interface
is set to 1 year, and linktest is currently disabled and hidden through
the ez interface.

There is still work to do, but this checkin brings with it a functional
plab portal!

archive!