GitLab

manifest we have store in the DB. Basically a highly stripped down
version of the status page.

1. Show current reservations on the admin extend page (if any) for the
   user who started the experiment.

2. Add a reservation history page, to see historical reservations for a
   user.

3. Changes to the reservation listing page.

4. And then the main content of this commit is that for the pages above,
   show the experiment usage history for the project and the user who
   created the reservation. This takes the form of a time line of
   allocation changes so that we can graph node usage against the
   reservation bounds, to show graphically how well utilized the
   reservation is.

I spent a fair amount of improving error handling along the RPC path,
as well making the code more consistent across the various files. Also
be more consistent in how the web interface invokes the backend and gets
errors back, specifically for errors that are generated when taking to a
remote cluster.

Add checks before every RPC to make sure the cluster is not disabled in
the database. Also check that we can actually reach the cluster, and
that the cluster is not offline (NoLogins()) before we try to do
anything. I might have to relax this a bit, but in general it takes a
couple of seconds to check, which is a small fraction of what most RPCs
take. Return precise errors for clusters that are not available, to the
web interface and show them to user.

Use webtasks more consistently between the web interface and backend
scripts. Watch specifically for scripts that exit abnormally (exit
before setting the exitcode in the webtask) which always means an
internal failure, do not show those to users.

Show just those RPC errors that would make sense users, stop spewing
script output to the user, send it just to tbops via the email that is
already generated when a backend script fails fatally.

But do not spew email for clusters that are not reachable or are
offline. Ditto for several other cases that were generating mail to
tbops instead of just showing the user a meaningful error message.

Stop using ParRun for single site experiments; 99% of experiments.

For create_instance, a new "async" mode that tells CreateSliver() to
return before the first mapper run, which is typically very quickly.
Then watch for errors or for the manifest with Resolve or for the slice
to disappear. I expect this to be bounded and so we do not need to worry
so much about timing this wait out (which is a problem on very big
topologies). When we see the manifest, the RedeemTicket() part of the
CreateSliver is done and now we are into the StartSliver() phase.

For the StartSliver phase, watch for errors and show them to users,
previously we mostly lost those errors and just sent the experiment into
the failed state. I am still working on this.

ajax entrypoint, since we need guest access enabled for that.

could let a guest user gain entry is now gone. I will clean up
straggling guest code over time. This closes issue #352.

warning modal to schedule cancel in 24 hours. Also a new control to
cancel cancellation.

us to reactivate them first.

to warn user to use their reservation or die.

1. Allow uuids to be used to specify reservations, change pretty much
   everything in the web interface to use uuid's so we stop exporting
   databases indexes to the client side.

2. Change RPC path to return a blob of data when approving a
   reservation. Ditto for initial creation, so that we can see precisely
   what the local cluster has done.

3. When a reservation is created/approved, insert an announcement in the
   announce system for that user, set to go off 24 hours ahead of
   reservation. Update that announcement when reservation is modified.

1. Get rid of the old myexperiments page and change all uses to the user
   dashboard or landing page.

   Replaced with admins only page for listing all experiments, pending
   extensions, locked down, expires, and older then 45 days.

2. New page for approving projects using modern technology, as per Rob's
   request in issue #304.

You can find and change the admin notes on the admin extension page,
since is the page where we show lots of admin only stuff. Might need
to rename this page at some point.

want to have a project list on the project page.

Redo the rspectogenilib converter with the goal of supporting both
translation *and* regression testing. A new library is responsible for
taking the output of libXML and creating a data structure representing
the rspec. While this is being done, we look for any constructs or
attributes we cannot handle with geni-lib and report that as an error;
we are not going to convert an rspec unless we can do it correctly. and
completely.

Regression testing is done with another part of this library, that knows
how to compare each element of two rspecs. Basically start at the root
and compare all the way down, failing if the two "parses" of the XML are
not equal at any level.

rspectogenilib now has an option that does regression testing by running
the new geni-lib and comparing the resulting rspec against the original.

On the UI side, there is a new button on existing rspec based
profiles (currently only for admin and studs) called "Convert to
geni-lib" that runs the converter to convert the profile to geni-lib.
The user does not have to accept the new script of course.

However, a converted profile is marked in the database, and the user can
still use Jacks on it, we just run rspectogenilib geni-lib again on the
new rspec. If the user edits the geni-lib, we switch back to normal
geni-lib (clear the flag) when the new version is saved, and Jacks is
once again read-only. This is explained in the UI, and is one of the
things people need to give comment on.

There is also a mode on the Create Profile page for converting new rspec
based profiles to geni-lib, but that is fully turned off for now, we can
get to that later.

1. Finally, ajaxify the manage profile page. I never got around to doing
   this, and it made doing the thing I really wanted to do easier,

2. Add a checkbox modal to ask the user if any accounts have been added,
   when doing initial clone of an experiment. We already did this with
   snapshot on the status page, but as above, it was going to be
   annoying to add to this page without reworking it.

3. Improvements to repo-based profiles; when a push hook is executed, we
   were already updating the Repository Panel and the branch/tag list,
   but I wasn't updating the source code in the web page. Now we do that.

We are running another apache server on boss, on port 51369, which
invokes a backend perl script that maps the URL path argument to the
profile, and then calls out to manage_profile to pull from the
repository and update the profile to reflect the new HEAD branch.
Using mod_rewrite in the apache config to restrict URLs to exactly
the one URL that is accepted, modulo the value of the secret token.

I had to refactor a bunch of code in manage_profile to make it easier to
add a new entrypoint for modification from a git repo. This needed to be
done for a long time, I had never cleaned up the original profile
creation code.

On the edit profile web page, there is a new row in the Repository panel
providing the Push URL, and an explanatory help modal.

There is a new slow polling timer that looks for a change to the repo
hash and causes the web page to update in place from the repo, as when a
push hook is invoked and changes the repo.

The main point of the changes are to require that the old password is
provided when changing your password. This holds true even for admins
changing their own password in red-dot. Note though, that when an admin
changes another user's password in red-dot, old password is not
required, which is somewhat in conflict with the overall goal, but hey,
we want to be practical too.

I ended up removing password modification from the profile page, and
use the already existing changepswd page, which I cleaned up and turned
into a first class ajax citizen to make the page operate smoother.

The control flow is kinda confusing, cause of the desire to hold off on
converting the geni-lib script (reducing the time we are updating from the
origin repo in the first please wait modal). Not really happy with how it
turned out.

I also need to change the code that does the initial fetch of a new repo,
and then a delete. The only reason for this is so we do not leave dangling
repos on our file system if the user wanders away before submitting the
form. Need a better way to deal with this, many options available, but
probably need to do something soon since the overhead (second fetch) on
larger repos will be annoying.

slight delay filling in that table value, since we ask the backend
cluster(s) and fill in the value when it comes back.

1. Add a Reload icon on the Graphs tab, to reload the cluster data and
   redraw the graphs.

2. Implement a Reload Topology function, with button on status page, to
   sync the portal topology with the clusters current manifests. Currently
   available to studly users only. This closes ticket #92.

* Add a "lockdown" checkbox to status page (red-dot) next to the lockout
  checkbox.

* Change Lockdown on adminextend page to the an active checkbox (like
  lockout).

* Add openstackgraphs.js, not in use yet.

expiration #days in the future, clears the lockdown, sets the lockout (no
free extensions) and sends the text in the box to the user.

* More on issue #54; watch for openstack experiments and try to download
  the new openstack stats file via the fast XMLRPC path. Show this as a
  text blob in a new tab on the status page, still need to graph the data.
  The apt_daemon handles the periodic request for the data (every 10
  minutes), which we store in the apt_instances table.

* Addition for Rob on the admin extend page; Add a "more info" button that
  sends the contents of the text box as an email message requesting more
  info and stores that in the ongoing interaction log. Responses from the
  user are not stored though, might look at that someday.

* Another addition for Rob; on the extensions list page, also show expired,
  locked down experiments. Note the sorting; at the top of the list are
  actual extension request (status='ready') while the bottom of the list
  are status='expired'.

* Add a "graphs" tab to the status page, which shows the same idle stats
  graphs that were added to the admin extend page. Most of this change is
  refactoring the code and sharing it between the two pages.