1. 01 Nov, 2004 1 commit
  2. 29 Oct, 2004 1 commit
  3. 25 Oct, 2004 1 commit
    • Leigh B. Stoller's avatar
      Minor fix to stats code; Watch for a swapmod on a swapped out · 3595089a
      Leigh B. Stoller authored
      experiment, and not count its pnodes or duration as if it was swapped
      in! Thanks Kirk.
      The actual fix is simple. The script to fix up the existing stats
      records was a bear and took me all morning to get right! Even so, its
      probably no perfect, but close enough. That script in the sql directory
      and mentioned in doc/UPDATING.
  4. 01 Sep, 2004 1 commit
    • Leigh B. Stoller's avatar
      SSL version of the XMLRPC server. · a9c1045e
      Leigh B. Stoller authored
      * SSL based server (sslxmlrpc_server.py) that wraps the existing Python
        classes (what we export via the existing ssh XMLRPC server). I also have a
        demo client that is analogous the ssh demo client (sslxmlrpc_client.py).
        This client looks for an ssl cert in the user's .ssl directory, or you can
        specify one on the command line. The demo client is installed on ops, and
        is in the downloads directory with the rest of the xmlrpc stuff we export
        to users. The server runs as root, forking a child for each connection and
        logs connections to /usr/testbed/log/sslxmlrpc.log via syslog.
      * New script (mkusercert) generates SSL certs for users. Two modes of
        operation; when called from the account creation path, generates a
        unencrypted private key and certificate for use on Emulab nodes (this is
        analagous to the unencrypted SSH key we generate for users). The other mode
        of operation is used to generate an encrypted private key so that the user
        can drag a certificate to their home/desktop machine.
      * New webpage (gensslcert.php3) linked in from the My Emulab page that
        allows users to create a certificate. The user is prompted for a pass
        phrase to encrypt the private key, as well as the user's current Emulab
        login password. mkusercert is called to generate the certificate, and the
        result is stored in the user's ~/.ssl directory, and spit back to the user
        as a text file that can be downloaded and placed in the users homedir on
        their local machine.
      * The server needs to associate a certificate with a user so that it can
        flip to that user in the child after it forks. To do that, I have stored
        the uid of the user in the certificate. When a connection comes in, I grab
        the uid out of the certificate and check it against the DB. If there is a
        match (see below) the child does the usual setgid,setgroups,setuid to the
        user, instantiates the Emulab server class, and dispatches the method. At
        the moment, only one request per connection is dispatched. I'm not sure
        how to do a persistant connection on the SSL path, but probably not a big
        deal right now.
      * New DB table user_sslcerts that stores the PEM formatted certificates and
        private keys, as well as the serial number of the certificate, for each
        user. I also mark if the private key is encrypted or not, although not
        making any use of this data. At the moment, each user is allowed to get
        one unencrypted cert/key pair and one encrypted cert/key pair. No real
        reason except that I do not want to spend too much time on this until we
        see how/if it gets used. Anyway, the serial number is used as a crude form
        of certificate revocation. When the connection is made, I suck the serial
        number and uid out of the certificate, and look for a match in the table.
        If cert serial number does not match, the connection is rejected. In other
        words, revoking a certificate just means removing its entry from the DB
        for that user. I could also compare the certificate itself, but I am not
        sure what purpose that would serve since that is what the SSL handshake is
        supposed to take of, right?
      * Updated the documentation for the XMLRPC server to mention the existence
        of the SSL server and client, with a pointer into the downloads directory
        where users can pick up the client.
  5. 30 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      The bulk of the event system changes. · 9aa6b5ca
      Leigh B. Stoller authored
      * The per-experiment event scheduler now runs on ops instead of boss.
        Boss still runs elvind and uses events internally, but the user part
        of the event system has moved.
      * Part of the guts of eventsys_control moved to new script, eventsys.proxy,
        which runs on ops and fires off the event scheduler. The only tricky part
        of this is that the scheduler runs as the user, but killing it has to be
        done as root since a different person might swap out the experiment. So,
        the proxy is a perl wrapper invoked from a root ssh from boss, which
        forks, writes the pid file into /var/run/emulab/evsched/$pid_$eid.pid,
        then flips to the user and execs the event scheduler (which is careful
        not to fork). Obviously, if the kill is done as root, the pid file has to
        be stored someplace the user is not allowed to write.
      * The event scheduler has been rewritten to use Tim's C++ interface to the
        sshxmlrpc server on boss. Actually, I reorg'ed the scheduler so that it
        can be built either as a mysql client, or as RPC client. Note that it can
        also be built to use the SSL version of the XMLRPC server, but that will
        not go live until I finish the server stuff up. Also some goo for dealing
        with building the scheduler with C++.
      * Changes to several makefiles to install the ops binaries over NFS to
        /usr/testbed/opsdir. Makes life easier, but only if boss and ops are
        running the same OS. For now, using static linking on the event scheduler
        until ops upgraded to same rev as boss.
      * All of the event clients got little tweaks for dealing with the new CNAME
        for the event system server (event-sever). Will need to build new images
        at some point. Old images and clients will continue to work cause of an
        inetd hack on boss that uses netcat to transparently redirect elvind
        connections to ops.
      * Note that eventdebug needs some explaining. In order to make the inetd
        redirect work, elvind cannot be listening on the standard port. So, the
        boss event system uses an alternate port since there are just a few
        subsystems on boss that use the server, and its easy to propogate changes
        on boss. Anyway, the default for eventdebug is to connect to the standard
        port on localhost, which means it will work as expected on ops, but will
        require -b argument on boss.
      * Linktest changes were slightly more involved. No longer run linktest on
        boss when called from the experiment swapin path, but ssh over to ops to
        fire it off. This is done as the user of course, and there are some
        tricks to make it possible to kill a running linktest and its ssh when
        experiment swapin is canceled (or from the command line) by forcing
        allocation of a tty. I will probably revisit this at some point, but I
        did not want to spend a bunch of time on linktest.
      * The upgrade path detailed in doc/UPDATING is necessarily complicated and
        bound to cause consternation at remote sites doing an upgrade.
  6. 16 Aug, 2004 1 commit
  7. 09 Aug, 2004 1 commit
    • Leigh B. Stoller's avatar
      Major rework of the script interface to Emulab. Up to now we have been · 5ef8f70a
      Leigh B. Stoller authored
      supporting both a shell script driven interface, plus the newer XMLRPC
      interface. This change removes the script driven interface from boss,
      replacing it with just the XMLRPC interface. Since we like to maintain
      backwards compatability with interfaces we have advertised to users (and
      which we know are being used), I have implemented a script wrapper that
      exports the same interface, but which converts the operations into XMLRPC
      requests to the server. This wrapper is written in python and uses our
      locally grown xmlrpc-over-ssh library. Like the current "demonstation"
      client, you can take this wrapper to your machine that has python and ssh
      installed, and use it there; you do not need to use these services from
      just users.emulab.net. Other things to note:
      * The wrapper is a single python script that has a "class" for each wrapped
        script. Running the wrapper without any arguments will list all of the
        operations it supports. You can invoke the wrapper with the operation as
        its argument:
          {987} stoller$ script_wrapper.py swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
               -w   - Wait for experiment to finish swapping
               -e   - Project and Experiment ID
               in   - Swap experiment in  (must currently be swapped out)
              out   - Swap experiment out (must currently be swapped in)
          Wrapper Options:
              --help      Display this help message
              --server    Set the server hostname
              --login     Set the login id (defaults to $USER)
              --debug     Turn on semi-useful debugging
         But more convenient is to create a set of symlinks so that you can just
         invoke the operation by its familiar scriptname. This is what I have
         done on users.emulab.net.
          {987} stoller$ /usr/tesbed/bin/swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
      * For those of you talking directly to the RPC server from python, I have
        added a wrapper class so that you can issue requests to any of the
        modules from a single connection. Instead using /xmlrpc/modulename, you
        can use just /xmlrpc, and use method names of the form experiment.swapexp,
        node.reboot, etc.
        Tim this should be useful for the netlab client which I think opens up
        multiple ssh connections?
      * I have replaced the paperbag shell with a stripped down xmlrpcbag shell
        that is quite a bit simpler since we no longer allow access to anything
        but the RPC server. No interactive mode, no argument processing, no
        directory changing, etc. My main reason for reworking the bag is to make
        it easier to understand, maintain, and verify that it is secure. The new
        bag also logs all connections to syslog (something we should have done in
        the orginal). I also added some setrlimit calls (core, maxcpu). I also
        thought about niceing the server down, but that would put RPC users at a
        disadvantage relative to web interface users. When we switch the web
        interface to use the XMLRPC backend, we can add this (reniceing from the
        web server would be a pain cause of its scattered implementation).
  8. 05 Aug, 2004 1 commit
  9. 28 Jul, 2004 1 commit
  10. 13 Jul, 2004 1 commit
    • Leigh B. Stoller's avatar
      20040713: · 5d1c858e
      Leigh B. Stoller authored
      	For new distributed routing support, must first run this script as
              root to generate a data file for each existing experiment.
      		sudo sql/gentopmaps.pl
  11. 23 Jun, 2004 1 commit
    • Leigh B. Stoller's avatar
      Switch to "dynamic" creation of certain virtual nodes (jail/pcvm and · 4d5620c2
      Leigh B. Stoller authored
      vserver/pcplab). Rather then 10000s of fixed node entries in the DB,
      create node entries on the fly as needed (assign_wrapper), and then
      delete them when they are no longer used (nfree). When new nodes check
      in, no longer create all those nodes table entries (utils/newnode.in
      and tbsetup/plab/libplab.py.in).
      I've added a new library: db/Node.pm which is something I started a
      while back, and decided to commit, along with the support for creating
      and deleting virtual nodes. CreateVnodes() creates a new set of nodes,
      choosing non-conflicting names in the DB, and then immediately
      reserves them to the pid/eid specified. DeleteVnodes takes a list of
      vnodes and deletes them from nodes,reserved,etc. This library does a
      few other things which I am going to be playing with, so you might
      want to go read the comment at the top of the file. Feel free to speak
      up. CreateVnodes() is from called assign_wrapper when a node type has
      the "isdynamic" property. Otherwise does the existing avail/nalloc
      stuff. DeleteVnodes() is called from nfree when the node type has the
      isdynamic property.
      I've added a script (sql/delvnodes.pl) to run after updating the DB
      and software. All free pcvm and pcplab virtual nodes are deleted from
      the DB; reserved ones will get deleted whenever their experiment ends.
      I've noted all of this in doc/UPDATING, including setting the
      isdynamic property on pcvm in the node_types table.
      I've left tbsetup/plab/libplab.py.in to create a single pcplab node
      for the management sliver (still called -20). We can worry about this
      All this for modelnet?
  12. 22 Jun, 2004 1 commit
  13. 15 Jun, 2004 1 commit
  14. 14 Jun, 2004 1 commit
  15. 17 Apr, 2004 1 commit
  16. 03 Mar, 2004 1 commit
  17. 02 Mar, 2004 1 commit
  18. 26 Feb, 2004 1 commit
  19. 21 Jan, 2004 1 commit
  20. 20 Jan, 2004 1 commit
  21. 12 Jan, 2004 1 commit
  22. 15 Dec, 2003 1 commit