1. 09 Feb, 2006 1 commit
  2. 22 Mar, 2005 1 commit
  3. 10 Dec, 2004 1 commit
  4. 15 Nov, 2004 1 commit
    • Timothy Stack's avatar
      · aaff0953
      Timothy Stack authored
      Minor improvement to the node.available() method so you can get the
      number of free nodes of a particular type (e.g. pc850).
      
        * xmlrpc/GNUmakefile.in: Add node_avail symlink.
      
        * xmlrpc/emulabserver.in: Add some optional parameters to
          node.available() so you can specify the class/type of nodes as
          well as project credentials.
      
        * xmlrpc/script_wrapper.py.in: Add node_avail class that calls
          node.available() with the given parameters.
      aaff0953
  5. 09 Nov, 2004 1 commit
    • Leigh Stoller's avatar
      Here is a fun little change. Lacking native perl SSL XMLRPC tools, I · a7482569
      Leigh Stoller authored
      put together a little library that provides the illusion of nativeness.
      
      sslxmlrpc_client.py.in: New "rawmode" (-r) option. Instead of the
      usual command line operation, input raw XMLRPC goo and send that over
      to the server. The raw XMLRPC reply goo is spit out on stdout. In
      other words, it is up to the caller to generate the XML stuff, and
      convert back from XML to a reply structure.
      
      libxmlrpc.pm.in: A new perl library that exports one real method
      called, interestingly enough, CallMethod($$$). The first and second
      arguments are the module and method to invoke in the RPC server. The
      third argument is an arbitrary perl data structure to convert into XML
      and pass to the server. For example:
      
      	libxmlrpc::CallMethod("experiment", "state",
      	                      {"proj" => "testbed", "exp" => "myemulab"});
      
      The return value of CallMethod is whatever data structure the server
      returned, or undef if there is an internal error or if the RPC fails
      with a transport error (one of the errors in emulabclient.py).
      
      In case it is not obvious, CallMethod converts the argument to XML
      using the RPC:XML perl module, forks off a child to run
      sslxmlrpc_client.py.in in rawmode, sends it the XML on its stdin,
      reads back the XML for the reply from its stdout, and converts that to
      a perl data structure to return to the caller.
      
      The more interesting use of this new goo is to invoke the new
      "elabinelab" module in the RPC server, which exports some new methods
      to support elabinelab. The idea is that the inner boss will invoke
      routines (like setup/destroy vlans, or power cycle) using the RPC
      server, and the SSL key of the creator of the inner emulab. This will
      be described in more detail when I check in those changes.
      
      There is also a Config() method that is used to set the SSL cert path,
      debugging, verbosity, etc. You can take a look if you are interested.
      
      This can be arbitrarily fancy, but I don't need this for many things.
      a7482569
  6. 01 Nov, 2004 1 commit
  7. 01 Sep, 2004 1 commit
    • Leigh Stoller's avatar
      SSL version of the XMLRPC server. · a9c1045e
      Leigh Stoller authored
      * SSL based server (sslxmlrpc_server.py) that wraps the existing Python
        classes (what we export via the existing ssh XMLRPC server). I also have a
        demo client that is analogous the ssh demo client (sslxmlrpc_client.py).
        This client looks for an ssl cert in the user's .ssl directory, or you can
        specify one on the command line. The demo client is installed on ops, and
        is in the downloads directory with the rest of the xmlrpc stuff we export
        to users. The server runs as root, forking a child for each connection and
        logs connections to /usr/testbed/log/sslxmlrpc.log via syslog.
      
      * New script (mkusercert) generates SSL certs for users. Two modes of
        operation; when called from the account creation path, generates a
        unencrypted private key and certificate for use on Emulab nodes (this is
        analagous to the unencrypted SSH key we generate for users). The other mode
        of operation is used to generate an encrypted private key so that the user
        can drag a certificate to their home/desktop machine.
      
      * New webpage (gensslcert.php3) linked in from the My Emulab page that
        allows users to create a certificate. The user is prompted for a pass
        phrase to encrypt the private key, as well as the user's current Emulab
        login password. mkusercert is called to generate the certificate, and the
        result is stored in the user's ~/.ssl directory, and spit back to the user
        as a text file that can be downloaded and placed in the users homedir on
        their local machine.
      
      * The server needs to associate a certificate with a user so that it can
        flip to that user in the child after it forks. To do that, I have stored
        the uid of the user in the certificate. When a connection comes in, I grab
        the uid out of the certificate and check it against the DB. If there is a
        match (see below) the child does the usual setgid,setgroups,setuid to the
        user, instantiates the Emulab server class, and dispatches the method. At
        the moment, only one request per connection is dispatched. I'm not sure
        how to do a persistant connection on the SSL path, but probably not a big
        deal right now.
      
      * New DB table user_sslcerts that stores the PEM formatted certificates and
        private keys, as well as the serial number of the certificate, for each
        user. I also mark if the private key is encrypted or not, although not
        making any use of this data. At the moment, each user is allowed to get
        one unencrypted cert/key pair and one encrypted cert/key pair. No real
        reason except that I do not want to spend too much time on this until we
        see how/if it gets used. Anyway, the serial number is used as a crude form
        of certificate revocation. When the connection is made, I suck the serial
        number and uid out of the certificate, and look for a match in the table.
        If cert serial number does not match, the connection is rejected. In other
        words, revoking a certificate just means removing its entry from the DB
        for that user. I could also compare the certificate itself, but I am not
        sure what purpose that would serve since that is what the SSL handshake is
        supposed to take of, right?
      
      * Updated the documentation for the XMLRPC server to mention the existence
        of the SSL server and client, with a pointer into the downloads directory
        where users can pick up the client.
      a9c1045e
  8. 09 Aug, 2004 1 commit
    • Leigh Stoller's avatar
      Major rework of the script interface to Emulab. Up to now we have been · 5ef8f70a
      Leigh Stoller authored
      supporting both a shell script driven interface, plus the newer XMLRPC
      interface. This change removes the script driven interface from boss,
      replacing it with just the XMLRPC interface. Since we like to maintain
      backwards compatability with interfaces we have advertised to users (and
      which we know are being used), I have implemented a script wrapper that
      exports the same interface, but which converts the operations into XMLRPC
      requests to the server. This wrapper is written in python and uses our
      locally grown xmlrpc-over-ssh library. Like the current "demonstation"
      client, you can take this wrapper to your machine that has python and ssh
      installed, and use it there; you do not need to use these services from
      just users.emulab.net. Other things to note:
      
      * The wrapper is a single python script that has a "class" for each wrapped
        script. Running the wrapper without any arguments will list all of the
        operations it supports. You can invoke the wrapper with the operation as
        its argument:
      
          {987} stoller$ script_wrapper.py swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
          where:
               -w   - Wait for experiment to finish swapping
               -e   - Project and Experiment ID
               in   - Swap experiment in  (must currently be swapped out)
              out   - Swap experiment out (must currently be swapped in)
      
          Wrapper Options:
              --help      Display this help message
              --server    Set the server hostname
              --login     Set the login id (defaults to $USER)
              --debug     Turn on semi-useful debugging
      
         But more convenient is to create a set of symlinks so that you can just
         invoke the operation by its familiar scriptname. This is what I have
         done on users.emulab.net.
      
          {987} stoller$ /usr/tesbed/bin/swapexp --help
          swapexp -e pid,eid in|out
          swapexp pid eid in|out
      
      
      * For those of you talking directly to the RPC server from python, I have
        added a wrapper class so that you can issue requests to any of the
        modules from a single connection. Instead using /xmlrpc/modulename, you
        can use just /xmlrpc, and use method names of the form experiment.swapexp,
        node.reboot, etc.
      
        Tim this should be useful for the netlab client which I think opens up
        multiple ssh connections?
      
      * I have replaced the paperbag shell with a stripped down xmlrpcbag shell
        that is quite a bit simpler since we no longer allow access to anything
        but the RPC server. No interactive mode, no argument processing, no
        directory changing, etc. My main reason for reworking the bag is to make
        it easier to understand, maintain, and verify that it is secure. The new
        bag also logs all connections to syslog (something we should have done in
        the orginal). I also added some setrlimit calls (core, maxcpu). I also
        thought about niceing the server down, but that would put RPC users at a
        disadvantage relative to web interface users. When we switch the web
        interface to use the XMLRPC backend, we can add this (reniceing from the
        web server would be a pain cause of its scattered implementation).
      5ef8f70a
  9. 03 Aug, 2004 1 commit
  10. 22 Apr, 2004 1 commit
    • Timothy Stack's avatar
      · e5a12dee
      Timothy Stack authored
      Add five new modules:
      
        imageid, osid - Get listings of image/OS IDs.
        fs - Get directory listings for the emulab file system.
        user - Get information about the user.
        emulab - Get information about emulab as a whole.
      
      Added new method response, RESPONSE_REFUSED, which is returned
      whenever emulab is down or otherwise unavailable.
      
      Arguments to methods can now be strings or typed XML-RPC data.  For
      example, a boolean parameter will accept (in python) True or the
      string "true".
      
      Added osid and status values to the "mapping" aspect of the
      "experiment.info" method.  Also added "experiment.constraints" for
      getting the idle threshold and "experiment.getlist" for getting the
      list of experiments where the user is the head.
      
      Added "node.console" method to get ACL data for connecting to a node's
      console.
      e5a12dee
  11. 19 Mar, 2004 1 commit
  12. 18 Mar, 2004 1 commit
  13. 17 Mar, 2004 2 commits
  14. 16 Mar, 2004 1 commit
  15. 15 Mar, 2004 1 commit
  16. 10 Mar, 2004 1 commit
  17. 04 Dec, 2003 1 commit
    • Leigh Stoller's avatar
      New script. · f06d9a59
      Leigh Stoller authored
      Grant permission to use types/class of nodes. If given a specific
      "type", then grant permission to use just that type. If its a class, then
      permission to use all of the types in that class, the class itself, and any
      aux nodetypes for the type/class (node_types_auxtypes table).
      
      For example:
      
      	wap grantnodetype -p testbed pc2000
      	wap grantnodetype -p testbed pcvm
      	wap grantnodetype -p testbed pcplab
      f06d9a59
  18. 26 Nov, 2003 1 commit
  19. 14 Jul, 2003 1 commit
  20. 07 Mar, 2003 1 commit
    • Mac Newbold's avatar
      A few changes to stated: · 92fa4ae2
      Mac Newbold authored
       - fix bad indenting to a uniform 4 spaces (before was 2, 4 and 8 mixed)
       - Move ping-for-isup functionality into a separate script
       - Make sure every transition triggered by stated (directly or indirectly)
         sends an event, instead of taking shortcuts.
      
      This called for a new script, eventping, which just pings until the node
      is pingable, then sends an ISUP event. Stated runs this in the background
      where necessary, and nothing else should run it.
      
      Adding eventping meant modifying configure and the utils makefile, too.
      92fa4ae2
  21. 13 Feb, 2003 1 commit
  22. 26 Aug, 2002 2 commits
    • Leigh Stoller's avatar
      Rework all of the ssh key handling. Moved the parsing and verification · ae77bdb6
      Leigh Stoller authored
      to an external perl script, and use ssh-keygen to attempt conversion
      off SSH2/SECSH key formats. This is actually a simplification of the
      php code, which is not generally very good at this kind of thing (or
      maybe I mean perl is just better at it). The parsing and error
      handling it also much improved.
      ae77bdb6
    • Leigh Stoller's avatar
      Minor reorg of cvsupd startup. It now gets started by perl script that · 541a3586
      Leigh Stoller authored
      flips the uid/gid to nobody/nobody. It would be good to run this in a
      chroot shell, but that would be difficult given that we cannot easily
      rebuild cvsupd (modula-3). The right solution is to either run it in a
      jail or to move it to ops.
      
      Note, files in the sup tree obviously have to be world readable for
      cvsupd to send them off.
      541a3586
  23. 07 Jul, 2002 1 commit
  24. 01 Jul, 2002 1 commit
  25. 24 Jun, 2002 1 commit
  26. 13 Jun, 2002 1 commit
  27. 05 Jun, 2002 1 commit
    • Leigh Stoller's avatar
      Changes to sshtb. Remove sshremote, and convert sshtb into a perl · 231fc2b1
      Leigh Stoller authored
      script that checks the database to see if local or remote. The problem
      with this is that the ssh syntax makes it hard to determine the host
      name by inspection. Would need to parse all the ssh args (bad idea),
      ot work backwards and try to figure out the difference between the
      command (which is not a string but a sequence of args) and the host
      and the preceeding ssh args. Hell with that! Changed sshtb to require
      a specific -host argument. Read the args and look for it. Error out of
      not found, to catch improper usage.
      
      The moral of this update: "sshtb [ssh args] -host <host> [more args ...]
      231fc2b1
  28. 01 Apr, 2002 1 commit
    • Leigh Stoller's avatar
      First cut at supporting RON (or more generally, remote nodes). · bd587829
      Leigh Stoller authored
      * tmcd/ron: A new directory of client code, based on the freebsd
        client code, but scaled back to the bare minimum. Does only account
        and group file maintenance. I redid the account stuff so that only
        emulab accounts are operated on. Does not require a stub file, but
        instead keeps a couple of local dbm files recording what groups and
        accounts were added by Emulab. There is a ton of paranoia checking
        to make sure that local accounts are not touched.
      
        The update script that runs on the client node detaches so that the
        ssh from boss returns immediately. update can also be run from the
        node periodically and at boottime. The script is installed setuid
        root, but checks to make sure that *only* root or "emulabman" has
        invoked it.
      
      * utils/sshremote: New file. For remote nodes, instead of using sshtb,
        use sshremote, which ssh's in as "emulabman", which needs to be a
        local non-root user, but with an authorized_keys file containing
        boss' public key.
      
      * web interface changes: Allow user to specify his own public key in
        addition to the emulab key.
      
        Add option in showexp page to update accounts on nodes in the
        experiment. I was originally intending to do this from approveuser,
        but this was easier and faster. I will add an option to do it on the
        approveuser page later.
      
      * libdb.pm: Add a TBIsNodeRemote() query to see if a node is in the
        local testbed or a pcRemote node. Currently, this test is hardwired
        to a check for class=pcRemote, but this will need to change to a
        node_types property at some point.
      
      * node_update: Reorg so that there is a maximum number of children
        created. Previously, a child was forked for each node, but that
        could chew up too many processes, especially for remote nodes which
        might hang up. For the same reason, we need to "lock" the experiment
        so that it cannot be terminated while a node_update is in progress.
        Might be to relax that, but this was easy for now. Also add
        distinction between local and remote, since for remote we use
        sshremote insted of sshtb. Various cleanup stuff
      
      * mkacct; When generating a new account, include user supplied pub key
        in the authorized keys file, in addition to the eumlab generated
        key. Both keys are stored in the DB in the users table. Anytime we
        update an account, get a fresh copy of the emulab pub key, in case
        user changes it.
      bd587829
  29. 08 Feb, 2002 1 commit
    • Leigh Stoller's avatar
      Big round of image/osid changes. This is the first cut (final cut?) at · a73e627e
      Leigh Stoller authored
      supporting autocreating and autoloading images. The imageid form now
      sports a field to specify a nodeid to create the image from; If set,
      the backend create_image script is invoked. Thats the easy part.
      Slightly harder is autoloading images based on the osid specified in
      the NS file. To support this, I have added a new DB table called
      osidtoimageid, which holds the mapping from osid/pctype to imageid.
      When users create images, they must specify what node types that image
      is good for. Obviously, the mappings have to be unique or it would be
      impossible to figure it out! Anyway, once that image mapping is
      in place and the image created, the user can specify that ID in the NS
      file. I've changed os_setup to to look for IDs that are not loaded,
      and to try and find one in the osidtoimageid. If found, it invokes
      os_load. To keep things running in parallel as much as possible,
      os_setup issues all the loads/reboots (could be more than a single set
      of loads is multiple IDs are in the NS file) at once, and waits for
      all the children to exit. I've hacked up os_load a bit to try and be
      more robust in the face of PXE failures, which still happen and are
      rather troublsesome. Need an event system!
      
      Contained in this revision are unrelated changed to make the OS and
      Image IDs per-project unique instead of globally unique, since thats a
      pain for the users. This turns out to be very messy, since underneath
      we do not want to pass around pid/ID in all the various places its
      used. Rather, I create a globally unique name and extened the OS and
      Image tables to include pid/name/ID. The user selects pid/name, and I
      create the globally unique ID. For the most part this is invisible
      throughout the system, except where we interface with the user, say in
      the web pages; the user should see his chosen name where possible, and
      the should invoke scripts (os_load, create_image, etc) using his/her
      name not the internal ID. Also, in the front end the NS file should
      use the user name not the ID. All in all, this accounted for a number
      of annoying changes and some special cases that are unavoidable.
      a73e627e
  30. 15 Oct, 2001 1 commit
    • Leigh Stoller's avatar
      Add node_admin command for users. Simply switches between the normal · 85b512e7
      Leigh Stoller authored
      testbed boot process and the freebsd pxeboot/MFS kernel. I'm hoping
      this will be useful enough for users to load their own disk images,
      but we shall see. Usage is simple:
      
      	/usr/testbed/bin/node_admin on|off pcXXX
      
      The "on" state boots the MFS, and the "off" states sets it back to
      normal.
      85b512e7
  31. 18 Sep, 2001 1 commit
  32. 01 Aug, 2001 1 commit
    • Leigh Stoller's avatar
      An attempt at making image creation an easy/automatic operation. HA! · 27f26d99
      Leigh Stoller authored
      This uses the pxe booted freebsd kernel and MFS. In addition, I use
      the standard testbed mechanism of specifying a startup command to
      run, which will do the imagezip to NFS mounted /proj/<pid>/.... The
      controlling script on paper sets up the database, reboots the node,
      and then waits for the startstatus to change. Then it resets the DB
      and reboots the node so that it returns back to its normal OS. The
      format of operation is:
      
      	create_image <node> <imageid> <filename>
      
      Node must be under the user's control of course. The filename must
      reside in the node's project (/proj/<pid>/whatever) since thats the
      directory that is mounted by the testbed config software when the
      machine boots. The imageid already exists in the DB, and is used to
      determine what part of the disk to zip up (say, using the slice option
      to the zipper). Since this operation is rather time consuming, it does
      the usual trick of going to background and sending email status later.
      27f26d99
  33. 26 Jun, 2001 1 commit
    • Robert Ricci's avatar
      New script: sshtb · 9de266c3
      Robert Ricci authored
      sshtb is a _very_ simple shell script that runs ssh with a few commandline
      parameters, which make it play nicer in an script environment. These
      parameters can be changed with the '--with-ssh-args' argument, but default to:
      '-q -o "BatchMode yes" -o "StrictHostKeyChecking no"'
      All ssh calls now use this script.
      9de266c3
  34. 30 May, 2001 1 commit
  35. 11 Apr, 2001 1 commit
  36. 20 Mar, 2001 1 commit
  37. 08 Jan, 2001 2 commits