1. 13 Sep, 2004 1 commit
  2. 10 Sep, 2004 1 commit
  3. 01 Sep, 2004 1 commit
  4. 26 Aug, 2004 1 commit
  5. 25 Aug, 2004 1 commit
  6. 19 Aug, 2004 2 commits
  7. 18 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      Some rather crude privledge level hacks to allow admin people (real · c2a4acd4
      Leigh B. Stoller authored
      shells on boss) to use the rpc server without an agent running.
      Using the no-passphrase key, these changes allow us to use the server
      from ops in a very restricted manner. This change is temporary, until
      I have something better in place. In the meantime, admin people change
      their auth keys files on *boss* as such:
      
      command="/usr/testbed/sbin/sshxmlrpc_server.py -ro",from="ops.emulab.net" ... rest of emulab generated key ...
      
      Note the -ro argument; very important!
      c2a4acd4
  8. 13 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add three new methods to return all the event system goo to the event · 7d91c544
      Leigh B. Stoller authored
      scheduler running on ops.
      
      Okay, so this is probably not an ideal mechanism for wrapping up what are
      essentially DB queries, since there are no xmlrpc-over-ssh bindings for
      C. So we have to run an external client (sshxmlrpc_client) to talk to the
      server. This is not so bad in itself, but returning the eventlist as a
      single response seems like an unnatural approach for returning what is
      essentialy a stream of data. Besides, unless we expect people might want to
      write their own schedulers (highly unlikely), exporting this stuff via the
      XMLRPC server is a silly thing to do.
      
      Instead, some kind of protected DB access proxy might be more appropriate.
      I have no idea how this might operate, but its something to think about.
      7d91c544
  9. 10 Aug, 2004 1 commit
  10. 09 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      Major rework of the script interface to Emulab. Up to now we have been · 5ef8f70a
      Leigh B. Stoller authored
      supporting both a shell script driven interface, plus the newer XMLRPC
      interface. This change removes the script driven interface from boss,
      replacing it with just the XMLRPC interface. Since we like to maintain
      backwards compatability with interfaces we have advertised to users (and
      which we know are being used), I have implemented a script wrapper that
      exports the same interface, but which converts the operations into XMLRPC
      requests to the server. This wrapper is written in python and uses our
      locally grown xmlrpc-over-ssh library. Like the current "demonstation"
      client, you can take this wrapper to your machine that has python and ssh
      installed, and use it there; you do not need to use these services from
      just users.emulab.net. Other things to note:
      
      * The wrapper is a single python script that has a "class" for each wrapped
        script. Running the wrapper without any arguments will list all of the
        operations it supports. You can invoke the wrapper with the operation as
        its argument:
      
          {987} stoller$ script_wrapper.py swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
          where:
               -w   - Wait for experiment to finish swapping
               -e   - Project and Experiment ID
               in   - Swap experiment in  (must currently be swapped out)
              out   - Swap experiment out (must currently be swapped in)
      
          Wrapper Options:
              --help      Display this help message
              --server    Set the server hostname
              --login     Set the login id (defaults to $USER)
              --debug     Turn on semi-useful debugging
      
         But more convenient is to create a set of symlinks so that you can just
         invoke the operation by its familiar scriptname. This is what I have
         done on users.emulab.net.
      
          {987} stoller$ /usr/tesbed/bin/swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
      
      
      * For those of you talking directly to the RPC server from python, I have
        added a wrapper class so that you can issue requests to any of the
        modules from a single connection. Instead using /xmlrpc/modulename, you
        can use just /xmlrpc, and use method names of the form experiment.swapexp,
        node.reboot, etc.
      
        Tim this should be useful for the netlab client which I think opens up
        multiple ssh connections?
      
      * I have replaced the paperbag shell with a stripped down xmlrpcbag shell
        that is quite a bit simpler since we no longer allow access to anything
        but the RPC server. No interactive mode, no argument processing, no
        directory changing, etc. My main reason for reworking the bag is to make
        it easier to understand, maintain, and verify that it is secure. The new
        bag also logs all connections to syslog (something we should have done in
        the orginal). I also added some setrlimit calls (core, maxcpu). I also
        thought about niceing the server down, but that would put RPC users at a
        disadvantage relative to web interface users. When we switch the web
        interface to use the XMLRPC backend, we can add this (reniceing from the
        web server would be a pain cause of its scattered implementation).
      5ef8f70a
  11. 03 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      A couple more minor changes before I turn the new stuff loose. · 8fddf3ce
      Leigh B. Stoller authored
      * Added a wrapper class so that you can invoke methods as
        experiment.swapexp or node.reboot. So instead of invoking as
        /XMLRPC/experiment can calling swapexp, you can call the server as
        /XMLRPC and call experiment.swapexp. This allows you to use a single
        connection to talk to different parts of the API. Note this is standard
        (or is it defacto) syntax in XMLRPC.
      
      * Changed the demonstration client to talk the server this way.
      
      * Changed paperbag to allow this as well; the xmlrpc server is invoked with
        no args, which tells it to export the wrapper interface instead of a
        specific module interface.
      
      * A few more cleanups in the server, more permission checks, etc.
      8fddf3ce
  12. 30 Jul, 2004 1 commit
    • Leigh B. Stoller's avatar
      Preparation for replacing script interface on ops with XMLRPC only · c0cb83e4
      Leigh B. Stoller authored
      interface (which will have its own frontend to emulate the original
      script interface!)
      
      * Refactor the experiment permission checks into a subroutine.
      
      * Cleanup some parameter names that did not make sense.
      
      * Add portstats, readycount, node_admin, eventsys_control, savelogs,
      
      * A couple of little fixes.
      c0cb83e4
  13. 26 Jul, 2004 1 commit
  14. 15 Jul, 2004 1 commit
    • Timothy Stack's avatar
      · e9b599df
      Timothy Stack authored
      Fix experiment.info() to conditionally add the "idle" field to the
      mapping aspect.
      e9b599df
  15. 12 Jul, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add a "statewait" method to the experiment module, which allows a · 28a7aaa0
      Leigh B. Stoller authored
      client to wait for an experiment to reach a desired state (active,
      swapped, etc). Use the optional timeout parameter to terminate the
      wait early. See the documentation for a full descriptione of the
      interface.
      
      This is implemented using the just added "poll" interface to the
      python event interface library
      28a7aaa0
  16. 04 Jun, 2004 1 commit
    • Leigh B. Stoller's avatar
      Back out previous revision that changed the XML output format to RPC · f8ed2e3b
      Leigh B. Stoller authored
      format instead of plain old well formed XML. Note this change does not
      affect the parser in any way; the parser continues to spit out its own
      brand of XML, and xmlconvert treats that special when it reads it.
      
      The XML I spit out now is better formed and makes more sense when you
      look at it. It also helps that its much faster to generate, and the
      resulting output is much smaller. For example:
      
      	time /usr/testbed/libexec/xmlconvert testbed jail-2440
      	real    2m54.229s
      	user    1m7.492s
      	sys     1m15.927s
      
      	time /usr/testbed/devel/stoller/libexec/xmlconvert testbed jail-2440
      	real    0m14.738s
      	user    0m5.755s
      	sys     0m5.156s
      
      	/tmp/bar*.xml
      	-rw-rw-r--  1 stoller  wheel  15595161 Jun  4 11:16 /tmp/rpc.xml
      	-rw-rw-r--  1 stoller  wheel   5285690 Jun  4 11:17 /tmp/plain.xml
      
      Thats for a big experiment. For a small experiment:
      
      	time /usr/testbed/libexec/xmlconvert testbed jail-416
      	real    0m9.346s
      	user    0m5.083s
      	sys     0m2.675s
      
      	time /usr/testbed/devel/stoller/libexec/xmlconvert testbed jail-416
      	real    0m2.751s
      	user    0m1.377s
      	sys     0m1.183s
      
      	ll /tmp/bar*.xml
      	-rw-rw-r--  1 stoller  wheel  3135518 Jun  4 11:19 /tmp/rpc.xml
      	-rw-rw-r--  1 stoller  wheel  1015745 Jun  4 11:19 /tmp/plain.xml
      
      As you can see, the RPC code is rather non-linear in its performance!
      
      Since this affects Tim and netlab-client, I have added an alternate
      XMLRPC routine called virtual_topology_xml() that returns the new format,
      since Tim will need to transition to this new format (parsing it into a
      Java data structure). The old format is left in the RPC server until that
      is done.
      f8ed2e3b
  17. 17 May, 2004 2 commits
    • Leigh B. Stoller's avatar
      Oops, typo'ed the name of the table. · d7780943
      Leigh B. Stoller authored
      d7780943
    • Leigh B. Stoller's avatar
      Redo virtual_topology call to be faster: · 6fb0009d
      Leigh B. Stoller authored
      For tbres/resweb (75 nodes):
      
      	Old time: 42 seconds.
      	New time: 19 seconds.
      
      For testbed/BigLan (600 nodes):
      
      	Old time: 120 seconds.
      	New time: 40  seconds.
      
      So, still needs some work, but for the average case its not so bad
      anymore.
      
      I also turn on compression to level 5 by default in our sshxmlrpc.py
      library, thereby overriding the user's config file for that option.
      6fb0009d
  18. 12 May, 2004 1 commit
  19. 11 May, 2004 1 commit
  20. 03 May, 2004 1 commit
  21. 22 Apr, 2004 2 commits
    • Leigh B. Stoller's avatar
      Add virtual_topology() method to experiments interface; arguments are · cd8cbadc
      Leigh B. Stoller authored
      pid and eid, and it returns a giant data structure consisting of all
      of the virtual tables.
      
      At some point we want to combine the physical representation into the
      structure (for mapped experiments).
      cd8cbadc
    • Timothy Stack's avatar
      · e5a12dee
      Timothy Stack authored
      Add five new modules:
      
        imageid, osid - Get listings of image/OS IDs.
        fs - Get directory listings for the emulab file system.
        user - Get information about the user.
        emulab - Get information about emulab as a whole.
      
      Added new method response, RESPONSE_REFUSED, which is returned
      whenever emulab is down or otherwise unavailable.
      
      Arguments to methods can now be strings or typed XML-RPC data.  For
      example, a boolean parameter will accept (in python) True or the
      string "true".
      
      Added osid and status values to the "mapping" aspect of the
      "experiment.info" method.  Also added "experiment.constraints" for
      getting the idle threshold and "experiment.getlist" for getting the
      list of experiments where the user is the head.
      
      Added "node.console" method to get ACL data for connecting to a node's
      console.
      e5a12dee
  22. 07 Apr, 2004 1 commit
  23. 25 Mar, 2004 1 commit
  24. 19 Mar, 2004 1 commit
    • Leigh B. Stoller's avatar
      Add "state" method to return the current state of an experiment. · 3cc879c9
      Leigh B. Stoller authored
      Add "info" method to return stuff that we currently get via the
      tbreport script. The difference is that the info is returned as hashed
      arrays (or arrays) instead of as a big print statement. Currently just
      the node mapping and link mapping functions (of tbreport) are
      supported.
      
      At Jay's request, purge pid/eid/gid from documentation and use/allow
      "proj", "exp", "group" instead. Note that pid/eid/gid are still
      accepted by the server (cause I won't be able to stop using them!).
      3cc879c9
  25. 18 Mar, 2004 2 commits
  26. 17 Mar, 2004 1 commit
  27. 16 Mar, 2004 1 commit
  28. 15 Mar, 2004 1 commit