1. 08 Jun, 2004 1 commit
  2. 04 Jun, 2004 1 commit
    • Leigh B. Stoller's avatar
      Back out previous revision that changed the XML output format to RPC · f8ed2e3b
      Leigh B. Stoller authored
      format instead of plain old well formed XML. Note this change does not
      affect the parser in any way; the parser continues to spit out its own
      brand of XML, and xmlconvert treats that special when it reads it.
      
      The XML I spit out now is better formed and makes more sense when you
      look at it. It also helps that its much faster to generate, and the
      resulting output is much smaller. For example:
      
      	time /usr/testbed/libexec/xmlconvert testbed jail-2440
      	real    2m54.229s
      	user    1m7.492s
      	sys     1m15.927s
      
      	time /usr/testbed/devel/stoller/libexec/xmlconvert testbed jail-2440
      	real    0m14.738s
      	user    0m5.755s
      	sys     0m5.156s
      
      	/tmp/bar*.xml
      	-rw-rw-r--  1 stoller  wheel  15595161 Jun  4 11:16 /tmp/rpc.xml
      	-rw-rw-r--  1 stoller  wheel   5285690 Jun  4 11:17 /tmp/plain.xml
      
      Thats for a big experiment. For a small experiment:
      
      	time /usr/testbed/libexec/xmlconvert testbed jail-416
      	real    0m9.346s
      	user    0m5.083s
      	sys     0m2.675s
      
      	time /usr/testbed/devel/stoller/libexec/xmlconvert testbed jail-416
      	real    0m2.751s
      	user    0m1.377s
      	sys     0m1.183s
      
      	ll /tmp/bar*.xml
      	-rw-rw-r--  1 stoller  wheel  3135518 Jun  4 11:19 /tmp/rpc.xml
      	-rw-rw-r--  1 stoller  wheel  1015745 Jun  4 11:19 /tmp/plain.xml
      
      As you can see, the RPC code is rather non-linear in its performance!
      
      Since this affects Tim and netlab-client, I have added an alternate
      XMLRPC routine called virtual_topology_xml() that returns the new format,
      since Tim will need to transition to this new format (parsing it into a
      Java data structure). The old format is left in the RPC server until that
      is done.
      f8ed2e3b
  3. 20 May, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add EventFork() to event.pm (perl interface to event system) and to · 116539b6
      Leigh B. Stoller authored
      the tail file of course. Called from TBdbfork() in libdb, EventFork
      resets the event handle so that the child does a reconnect. Note that
      I do not disconnect in the child since I have no idea what that is
      going to do to the parents connection to the elvind, as Elvin makes no
      mention of what to do in the presence of a process that forks after
      connecting to the event server. At the least, this avoids a bunch of
      warnings and errors from vnodesetup!
      116539b6
  4. 05 May, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add new script to change the uid of a user, since this seems to be · 9f73d582
      Leigh B. Stoller authored
      something we are continually being asked to do! This script will only
      operate in UNAPPROVED users; once a user is approved it becomes too
      difficult to deal with cause of existing directories which might be
      exported to active experiments. We can probably deal with this, but we
      rarely do this to an approved user, so its not a big deal.
      
      Obvious: This script can get out of sync with the DB, but of course.
      9f73d582
  5. 29 Apr, 2004 1 commit
    • Leigh B. Stoller's avatar
      Fix for the problem of not setting default values when inserting · 6ab03d5f
      Leigh B. Stoller authored
      experiment table rows after a parse. It used to be that slots that
      were not explicitly given values by the parser would be left as
      whatever they were in the DB. Now I get the default values by doing
      a "describe experiments" query, inserting those values for any slot
      that is not set by the parser.
      6ab03d5f
  6. 23 Apr, 2004 1 commit
    • Leigh B. Stoller's avatar
      I have changed xmlconvert to do the following (note that I have not changed · a3ad59cd
      Leigh B. Stoller authored
      the parser; it still spits out the simplistic xml stuff and xmlconvert
      still reads that):
      
      * When generating an XML representation of an experiment, I read the
        virtual tables and build a *flat* data structure.
      
      	{"settings" => {pid:testbed, eid:foobar ....}.
      	 "nodes"    => [ {vname:nodeA, osid:RHL-STD ... },  ],
      	 "lans"     => [ {vname:link0, member:nodeA ... },  ],
               ...
      
        So, nodes is a list of node hash tables, lans is a list of hash tables,
        etc. All of the other virt_tables are represented as well.  You can find
        everything you want as long as you the names of the fields!
      
      * Once I have the above data structure, I use the XML-RPC lib to turn it
        into an XML document, and spit out the resulting XML string (a very long
        string!) Note that XML-RPC has its own format that allows you to convert
        data structures into XML and back; thats how you issue an RPC from your
        python client to a perl server.
      
      * Our python XMLRPC server basically uses xmlconvert as a backend so it can
        ship the above data structure back to client. We could also allow the
        client to ship the data structure back, and have xmlconvert upload the
        changes back to the DB.
      
        The reason for doing it in xmlconvert as a backend is so that this XML
        stuff is in just one place; whoever reads/writes this stuff has to know a
        lot of internal detail about an experiment (like what things should not
        be exported and what things we should not allow to be imported). I prefer
        to keep that all in one place since its easy to pass XML strings around
        (as long as everyone is talking the same dialect).
      
      Future work is to pick an XML representation that is not tied to
      XML-RPC. Ideally, we want to take a data structure in Perl, spit out
      the XML tree (with a DTD) and be able to reconstitute that in another
      program, using the XML Parser library (based on expat, which is common
      to Perl, Python, C).
      a3ad59cd
  7. 19 Apr, 2004 1 commit
  8. 12 Apr, 2004 1 commit
  9. 09 Apr, 2004 1 commit
  10. 08 Apr, 2004 1 commit
  11. 31 Mar, 2004 3 commits
  12. 25 Mar, 2004 1 commit
  13. 23 Mar, 2004 3 commits
    • Kirk Webb's avatar
      Couple of items: · ed848fd5
      Kirk Webb authored
      * Small fix to DBQueryFatal in libdb.py: () is a valid return value,
        don't fail on it(insert/replace); do fail if DBQuery returns None though.
      
      * Fix up libplab.py to not choke on new plab_slices column.
      ed848fd5
    • Kirk Webb's avatar
      Snapshot. · fd6d8cc9
      Kirk Webb authored
      * incompatible option handling and use removed from gen purpose libs
      * Global PLC mutex implemented, but currently disabled
      * plabmonitord parallelization cut in half (for now)
      
      I'm still very frustrated with option handling/passing.  Needs more thought,
      but the primary issue is that there really isn't a global variable space in
      python (global to file, yes, but not global to interpreter invocation).
      
      I've learned that __builtin__ might work for this, but it seems hacky..
      fd6d8cc9
    • Leigh B. Stoller's avatar
      Fix my last revision to that DBQueryFatal() and DBQuery() really do · af79be8f
      Leigh B. Stoller authored
      what I want them to do and properly mirror the perl impl.
      af79be8f
  14. 19 Mar, 2004 3 commits
  15. 17 Mar, 2004 1 commit
    • Kirk Webb's avatar
      Snapshot. · 856c2509
      Kirk Webb authored
      * Changed the way options are parsed in the python scripts so that modules
        can easily add and use their own options independent of top-level scripts.
      
      * Added --noIS and --pollNodes module options.
      
      * Added batch option to vnode_setup (degree of parallelization)
        - defaults to 10
      
      * Major updates to plamonitord
        - batches testing, currently to 40
      856c2509
  16. 16 Mar, 2004 2 commits
  17. 08 Mar, 2004 1 commit
  18. 26 Feb, 2004 1 commit
  19. 25 Feb, 2004 1 commit
    • Kirk Webb's avatar
      Kirk takes the weed whacker to the plab code. This is the first pass result. · ae2eec76
      Kirk Webb authored
      I'll come along for a closer cut in the future.
      
      * Modularized the plab communications 'adaptor' interface and moved the
        dslice- and PLC-specific code into their own modules.
      
      * Wrote an API definition README
      
      * Separated out generic routines from libplab into their own library modules
        (libtestbed.py and libdb.py)
      
      Functionally, not much has changed - this was just a massive re-org with some
      other cleanup.  Should be much easier to code up new PLAB interfaces as the
      plab folks flail around in their attempt to standardize on something.
      
      XXX: may want to re-think where the generic library modules should go.  If
      more python code enters Elab, we'll probably want to move 'em to more standard
      locations.
      
      This isn't the end of the cleanup - I would eventually like to go back and
      rethink the class structures, beef up the comments, and extend the API.
      ae2eec76
  20. 23 Feb, 2004 1 commit
  21. 16 Feb, 2004 2 commits
  22. 11 Feb, 2004 3 commits
  23. 10 Feb, 2004 1 commit
  24. 09 Feb, 2004 1 commit
  25. 06 Feb, 2004 1 commit
  26. 05 Feb, 2004 1 commit
  27. 02 Feb, 2004 1 commit
  28. 28 Jan, 2004 1 commit
  29. 16 Jan, 2004 2 commits