1. 07 Mar, 2007 1 commit
    • Leigh Stoller's avatar
      Changes for how we distribute the initial set of imageids and osids. · 3c1678d6
      Leigh Stoller authored
      * install/dump-descriptors <filename> will write out a set of insert
        statements for the images and os_info table, slightly munged. In
        fact, what I do is create temporary tables called temp_images and
        temp_os_info, clean them a bit, and then write out the insert
        statements to load them into new tables of the same name.
      
        There are some arrays at the top of this script that says what images and
        osids to write out.
      
      * install/load-descriptors <filename> takes the output of
        dump-descriptors, creates the two temporary tables and loads the
        data into them. Then it (optionally) updates those tables with the
        local indicies of elabman and the emulab-ops project and group.
      
        Then it computes an osidtoimageid table for all class='pc' types. On a
        new testbed this is a reasonable approach, in my opinion.
      
        Next it takes the contents of the two temp tables and moves them across
        to the actual tables.
      
      * install/descriptors.sql is the current data set which has everything
        contained in sql/database-fill-supplemental.sql and install/images/*
      3c1678d6
  2. 06 Mar, 2007 2 commits
  3. 02 Mar, 2007 1 commit
  4. 21 Feb, 2007 1 commit
  5. 20 Feb, 2007 1 commit
  6. 16 Feb, 2007 3 commits
  7. 16 Jan, 2007 1 commit
    • Leigh Stoller's avatar
      Move the bulk (or guts) of newuser and newproject from the web · 16aaa101
      Leigh Stoller authored
      interface to the backend. There are new scripts that can be called
      from the command line:
      
      	newuser xmlfile
      	newproj xmlfile
      
      They both run from small xmlfiles that are generated by the web
      interface from the form data. I also moved user verification to the
      backend so that we do not have duplicated email functions, but that
      was a small change.
      
      Upon error, the xmlfile is saved and sent to tbops so that we can
      rerun the command by hand, rather then force user to fill out form
      again. I also do a better job of putting the form back up intact when
      there are internal errors.
      
      If the user provides an initial public key, that is put into the xml
      file as well and addpubkey is called from newuser instead of the web
      interface. A more general change to addpukey is that it is now
      *always* called as "nobody". This script was a morass of confusion
      cause of having to call it as nobody before the user actually
      exists. In fact, another of my ongoing projects is to reduce the
      number of scripts called as a particular user, but thats a story for
      another day. Anyway, the script is always called as nobody, but we
      pass along the implied user in the environment so that it can do
      permission checks.
      16aaa101
  8. 10 Jan, 2007 1 commit
  9. 09 Jan, 2007 2 commits
  10. 20 Dec, 2006 1 commit
  11. 04 Dec, 2006 1 commit
  12. 30 Nov, 2006 1 commit
  13. 29 Nov, 2006 1 commit
    • Leigh Stoller's avatar
      Keith Sklower's changes to support a single control network model for · 08737f67
      Leigh Stoller authored
      ElabInElab experiments, so as not to consume an interface.
      
      I actually modified Keith's changes so that we can dynmaically choose
      the single or dual model in the NS file (Keith's changes hardwired the
      system to only single control network) since the single network model
      has the disadvantage of not allowing new nodes to be added to an inner
      elab, which would make it impossible to test some things (like the
      newnode path!).
      
      To choose the single control network model, place this in your NS file:
      
      	tb-elabinelab-singlenet
      
      other it defaults to the older dual network model.
      08737f67
  14. 27 Nov, 2006 2 commits
  15. 03 Nov, 2006 1 commit
    • Leigh Stoller's avatar
      Big set of changes intended to solve a couple of problems with long · ff9061d4
      Leigh Stoller authored
      term archiving of firstclass objects like users, projects, and of
      course templates.
      
      * Projects, Users, and Groups are now uniquely identified inside the
        DB by a index value that will not be reused. If necessary, this
        could easily be a globally unique identifier, but without federation
        there is no reason to do that yet.
      
      * Currently, pid, gid, and uid still need to be locally unique until
        all of the changes are in place (which is going to take a fairly
        long time since the entire system operates in terms of those, except
        for the few places that I had to change to get the ball rolling).
      
      * We currently archive deleted users to the deleted_users table (their
        user_stats are kept forever since they are indexed by the new index
        column). Eventually do the same with projects (not sure about
        groups) but since we rarely if ever delete a project, there is no
        rush on this one.
      
      * At the same time, I have started a large reorg of the code, to move
        all of the user, group, project code into modules, both in php and
        perl, turning them into first class "objects" (as far as that goes
        in php and perl). Eventually, the number of query statements
        scattered around the code will be manageable, or so I hope.
      
      * Another related part of this reorg is to make it easier to move the
        new user/project/group code in the perl backend so that it can be
        made available via the xmlrpc interface (without duplication of the
        code).
      ff9061d4
  16. 26 Oct, 2006 3 commits
    • Kevin Atkinson's avatar
      · a364f40c
      Kevin Atkinson authored
      Give explicit widths to integers in sql/tbreport.sql as per Keith
      Sklower:
        The sql/tbreport.sql file does not have the same width of integers
        as database_create.sql.  This causes schema_check to burp when you have
        updgraded an existing database, but at least the output can be fed back in.
      a364f40c
    • Kevin Atkinson's avatar
      Various tblog changes: · 95a3a6a7
      Kevin Atkinson authored
        Make an attempt to discover what the error was before an swap-* was
        canceled, if any.  Both the main error (canceled) and the other error
        are stored in the error table.  To support this a new column in the
        error table is added "rank".  The primary error has a rank 0 while the
        other error has a rank 1.
      
        Make an attempt to determine when an error is a "me too" error or the
        real cause of the problem.  "Me too" errors are errors which are
        generally reported when the callie script determined that the caller
        script fails.  The caller script should have reported the error, but
        in some cases the error didn't make it into the database.  Thus if a
        "me too" is reported as the cause of a "swap-*" more info is needed to
        determine the true cause.  When a "me too" error is reported it is
        followed by a "..." on it's own line.  It is also recorded in the
        errors table under the new column "need_more_info".
      
        Add inferred column to the errors table.  This is the same value as
        the inferred variable in tblog_find_error.
      
        Add revision column to errors table to make it easy to tell which
        algorithm was used to determine the error.
      95a3a6a7
    • Leigh Stoller's avatar
      Bug fixes from Keith Sklower. · 4a21ca5b
      Leigh Stoller authored
      4a21ca5b
  17. 24 Oct, 2006 1 commit
  18. 18 Oct, 2006 1 commit
  19. 12 Oct, 2006 2 commits
  20. 08 Oct, 2006 1 commit
  21. 05 Oct, 2006 2 commits
  22. 28 Sep, 2006 1 commit
  23. 27 Sep, 2006 2 commits
  24. 26 Sep, 2006 1 commit
    • Leigh Stoller's avatar
      Add a table to record node failures during start/stop run. This is so · c0fc3d6e
      Leigh Stoller authored
      we have a record of what nodes failed to respond during the
      synchronous parts of the start and stop experiment run. Data is stored
      for now, but not sure how it will be used later. The main goal is to
      avoid the problems we see with plab nodes where they do not respond,
      and start/stop run hang and fail.
      c0fc3d6e
  25. 25 Sep, 2006 2 commits
  26. 20 Sep, 2006 1 commit
    • Kirk Webb's avatar
      · 6887cc00
      Kirk Webb authored
      Add bwlimit column to widearea_nodeinfo table.
      6887cc00
  27. 12 Sep, 2006 1 commit
    • Leigh Stoller's avatar
      This started out as a simple little hack to add a StopRun "ns" event, but · cbdc4178
      Leigh Stoller authored
      it got more complicated as it progressed.
      
      The bulk of the change was changing template_exprun so that it can take a
      pid/eid as an alternative to eid/guid. This is a big convenience since its
      easy to find the template from a running experiment, and it makes it
      possible to invoke from the event scheduler, which has never heard of a
      template before (and its not something I wanted to teach it about).  Its
      also easier on users.
      
      Anyway, back to the stoprun event. You can now do this:
      
      	$ns at 100 "$ns stoprun"
      or
      	tevc -e pid/eid now ns stoprun
      
      You can add the -w option to wait for the completion event that is sent,
      but this brings me to the glaring problems with this whole thing.
      
      * First, the scheduler has to fire off the stoprun in the background,
        since if it waits, we get deadlock. Why? Cause the implementation of
        stoprun uses the event system (SNAPSHOT event, other things), and if
        the scheduler is sitting and waiting, nothing happens.
      
        Okay, the solution to this was to generate a COMPLETION event from
        template_exprun once the stop operation is complete. This brings me
        to the second problem ...
      
      * Worse, is that the "ns" events that are sent to implement stoprun (like
        snapshot) send their own completion events, and that confuses anyone
        waiting on the original stoprun event (it returns early).
      
        So what to do about this? There is a "token" field in the completion
        event structure, which I presume is to allow you to match things up.  But
        there is no way to set this token using tevc (and then wait for it), and
        besides, the event scheduler makes them up anyway and sticks them into
        the event. So, the seed of a fix are already germinating in my mind, but
        I wanted to get this commit in so that Mike would have fun reading this
        commit log.
      cbdc4178
  28. 05 Sep, 2006 1 commit
    • Leigh Stoller's avatar
      A bunch of template changes resulting from meetings last week. · 087dbfff
      Leigh Stoller authored
      * Add XMLRPC interface for template swapin,stoprun,startrun,swapout and
        add the appropriate wrappers to the script_wrapper on ops.
      
      * Allow parameter descriptions in NS files. This is probably not in its
        final form since its a bit confusing as to what has priority; something
        in the NS file or a metadata item. Anyway, you can do this in your NS
        file:
      
      	$ns define-template-parameter GUID "0/0" "The GUID to be analyzed"
      
        The rules are currently that the NS file description has priority and
        is copied to child templates, unless the user has modified a description
        via the web interface, in which case the NS file description is ignored.
        I know, sounds awful, but for the most part people are going to use the
        NS file anyway.
      
      * Add "clear" option when starting a new experiment run; the per
        experiment DB at the logholes are cleared. Note that this is *not* the
        default behaviour; you have to either check the checkbox on the web form
        or use the -c option to the script wrapper, or clear=yes if talking
        directly to the XMLRPC server.
      
      * Fix up how email is generated for template_swapin and template_create,
        so that Kevin can debug tblog/tbreport stuff, but also so that we maintain
        mail logs as before. I have made some improvements to libaudit so as to
        centralize the mail goo, and avoid duplicating all that stuff.
      
      * Minor fixes to the program agent so that the new environment strings are
        sent before the program agent exits and reloads them!
      
      * Other minor little things.
      087dbfff
  29. 31 Aug, 2006 1 commit
    • Kirk Webb's avatar
      · a48210ac
      Kirk Webb authored
      Change types of latitude and longitude columns in widearea_nodeinfo to
      float (instead of float(6,5)) to fix problem with values getting adjusted
      improperly during insert.  Apparently the definition of the float(n,m)
      type is non-standard, and has changed since sql 3.x.  Regular float
      columns will do fine here.
      a48210ac