1. 06 Mar, 2007 1 commit
  2. 05 Jan, 2007 2 commits
    • Leigh Stoller's avatar
      As per Jay's request, send a join request to new leader for old · 3f2d40c7
      Leigh Stoller authored
      leader, when changing the leader of a project when its created.
      3f2d40c7
    • Leigh Stoller's avatar
      Move the core approval code from the web interface to the backend so · 5e25aa17
      Leigh Stoller authored
      that we can run project approval from the command line. Part of the
      ongoing push to get stuff out of php and into the backend ...
      
      The command line is now:
      
      	mkproj [-s] [-h leader_uid] [-m <message> | -f <file>] <pid>
      	switches and arguments:
      	  -s         - silent; do not send approval email to leader
        	  -h <uid>   - switch project leader to specified uid
      	  -m <text>  - Include text in approval email message
      	  -f <file>  - Include text from file in approval email message
      	  <pid>      - project to approve.
      
      Notes:
      
      * The leader can be switched to a new user only at initial project creation.
        Once a project is actually approved (created), its too late. We need
        more stuff in place to change the leader after that, and that code
        is not written yet.
      
      * Email is now sent from the backend script, so easier to recover from
        problems. When invoked from the web interface, the message text will
        be appended to the tberror email if the backend fails for some
        reason.  This should avoid the problem of that text getting lost and
        not being able to recover it.
      
      * The web interface still handles part of project denial internally.
        Move that later.
      5e25aa17
  3. 20 Oct, 2006 1 commit
    • Mike Hibler's avatar
      Wow, this should make me look important! · afa5e919
      Mike Hibler authored
      Two-day boondoggle to support "/scratch", an optional large, shared filesystem
      for users.  To do this, I needed to find all the instances where /proj is used
      and behave accordingly.  The boondoggle part was the decision to gather up all
      the hardwired instances of shared directory names ("/proj", "/users", etc.)
      so that they are set in a common place (via unexposed configure variables).
      This is a boondoggle because:
      
      1. I didn't change the client-side scripts.  They need a different mechanism
         (e.g., tmcd) to get the info, configure is the wrong way.
      
      2. Even if I had done #1 it is likely--no, certain--that something would
         fail if you tried to rename "/proj" to be "/mike".  These names are just
         too ingrained.
      
      3. We may not even use "/scratch" as it turns out.
      
      Note, I also didn't fix any of the .html documentation.  Anyway, it is done.
      To maintain my illusion in the future you should:
      
      1. Have perl scripts include "use libtestbed" and use the defined PROJROOT(),
         et.al. functions where possible.  If not possible, make sure they run
         through configure and use @PROJROOT_DIR@, etc.
      
      2. Use the configure method for python, C, php and other languages.
      
      3. There are perl (TBValidUserDir) and php (VALIDUSERPATH) functions which
         you should call to determine if an NS, template parameter, tarball or
         other file are in "an acceptable location."  Use these functions where
         possible.  They know about the optional "scratch" filesystem.  Note that
         the perl function is over-engineered to handles cases that don't occur
         in nature.
      afa5e919
  4. 01 Jun, 2006 1 commit
    • Leigh Stoller's avatar
      Add suport for building per project, group, experiment DBs on ops. At · adbcfd47
      Leigh Stoller authored
      present the per-experiment stuff is not hooked in, but will be for
      templates later. Anyway, each user gets a mysql account on ops, with
      password set to the same as their mailman password (which is also
      their jabber password, etc). Each project gets a DB named by the
      project, and each group gets a DB named by pid,gid. Users are placed
      on the access lists for the DBs as you would expect.
      
      There is a little bit of complexity to make sure that we can create
      DBs on ops outside the Emulab path and grant access to them, without
      Emulab getting confused or mucking things up.
      
      I'll get a news item done ...
      adbcfd47
  5. 28 Mar, 2006 1 commit
  6. 06 Dec, 2005 1 commit
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      
      The nitty-gritty:
      
      1. Keep disk image signature files for all nodes in an experiment.
      
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
      
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      
      2. Swapout-time hook for creating swapout image.
      
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
      
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      
      3. Client-side script for creating swapout image.
      
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
      
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      
      4. Auto creation of signature files for new images.
      
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
      
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      
      5. Web changes to set/clear saving of disk state at swapout time.
      
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
      
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
      ed0d25b4
  7. 19 Sep, 2005 1 commit
    • Leigh Stoller's avatar
      Move all modification of the group_membership table to the backend, · cfba1ac7
      Leigh Stoller authored
      into a single new script call modgroups. Usage:
      
      	modgroups [-a pid:gid:trust[,pid:gid:trust]...]
                        [-m pid:gid:trust[,pid:gid:trust]...]
                        [-r pid:gid[,pid:gid]...] user
      
      So, -a to add groups, -r to remove groups, and -m to modify the trust
      value for a member of a group.
      
      The reason for doing this is that previously, we had no idea in the
      backend what group changes actually happened; we just knew what the
      current groups are. This make it hard to add and remove users from
      mailing lists, chat server buddy lists, etc. This is cleaner ...
      cfba1ac7
  8. 02 Sep, 2005 1 commit
  9. 15 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      The bulk of the mailman support. Still not turned on by default (cause · a64593f3
      Leigh Stoller authored
      Jay has "comments"), but I do not want it hanging around in my source
      tree. Here is my mail message:
      
      * The "My Mailing Lists" is context sensitive (copied from Tim's
        changes to the My Bug Databases). It takes you to the *archives* for
        the current project (or subgroup) list. Or it takes you to your
        first joined project.
      
      * The showproject and showgroup pages have direct links to the project
        and group specific archives. If you are in reddot mode, you also
        get a link to the admin page for the list. Note that project and
        group leaders are just plain members of these lists.
      
      * The interface to create a new "user" list is:
      
      	https://www.emulab.net/dev/stoller/newmmlist.php3
      
        We do not store the password, but just fire it over in the list
        creation process.
      
        Anyone can create their own mailing lists. They are not associated
        with projects, but just the person creating the list. That person
        is the list administrator and is given permission to access the
        configuration page.
      
        This page is not hooked in yet; not sure where.
      
      * Once you have your own lists, you user profile page includes a link
        in the sub menu: Show Mailman Lists. From this page you can delete
        lists, zap to the admin page, or change the admin password (which is
        really just a subpage of the admin page).
      
      * As usual, in reddot mode you can mess with anyone else's mailman lists,
        (via the magic of mailman cookies).
      
      * Note on cross machine login. The mailman stuff has a really easy way
        to generate the right kind of cookie to give users access. You can
        generate a cookie to give user access, or to the admin interface for
        a list (a different cookie). Behind the scenes, I ssh over and get
        the cookie, and set it in the user's browser from boss. When the
        browser is redirected over to ops, that cookie goes along and gives
        the user the requested access. No passwords need be sent around,
        since we do the authentication ourselves.
      a64593f3
  10. 20 Jul, 2005 1 commit
  11. 07 Jul, 2005 1 commit
    • Leigh Stoller's avatar
      Oh, such a silly little project ... Added CVS support to Emulab. When · 9b17b075
      Leigh Stoller authored
      enabled in the defs file:
      
      	CVSSUPPORT=1
      
      each project gets a stub CVS tree created (using 'cvs init') in
      /proj/$pid/CVS. It is up to users obviously to do something with
      that tree, and of course they have to either set their CVSROOT
      env variable, or use the -d option to cvs.
      
      The showproject page gets a link to the per-project CVS tree, using
      the cvsweb interface, which I hacked up a bit to allow restricted
      access to specific project trees, via a ?pid=$pid argument to the URL.
      Without the ?pid argument, it falls back to normal behaviour, which is
      check the cvsallowed bit in the users table, and provide access to the
      Emulab source repo.
      
      If you are curious, go here:
      
      	https://www.emulab.net/cvsweb/cvsweb.php3/?pid=testbed
      9b17b075
  12. 31 May, 2005 1 commit
  13. 13 May, 2005 1 commit
    • Leigh Stoller's avatar
      Automate initial user/project setup from setup-db.txt. Rather then · dd1b57bc
      Leigh Stoller authored
      have the user go through a set of hard to explain steps, just push
      them through it using the web interface.
      
      * New sitevars to control a little state machine used by the web
        interface.
      
      * When first setting up a testbed, the sitevar value will force the
        web interface to present the user with a single menu option "Create
        New Project" and the "Home" link will take the user to that page.
        The user is instructed to login is as elabman.
      
      * The user fills in the form as directed in setup-ops.txt. Even though
        he is logged in as elabman, the newproject form has been altered to
        operate as if no one is logged in. I also default a bunch more of
        the fields in this case.
      
      * The user submits the form. Rather then pend the new project, just
        jump straight into approveproject. That grinds along as usual, and
        when it is done, the elabman account is frozen and the user logged
        out. The user gets a link inviting him to log back in as the user
        just created.
      
      * Side effects of this new process:
      
      	* The user is made an admin user (admin=1) automatically.
      	* The user is added to the emulab-ops project as group_root.
      	* The user verification process is skipped.
      	* The user is added to the unixgroups wheel and tbadmin.
      
      * I reworked this entire section of setup-db.txt ...
      
      * The user still needs to give himself a real shell and password on
        boss, but I left that for the user to do explicitly. I also drop in
        a pointer to the shellonboss.txt. I might automate this part too at
        some point. Not sure yet.
      dd1b57bc
  14. 21 Mar, 2005 1 commit
  15. 25 Jan, 2005 1 commit
  16. 05 Dec, 2003 1 commit
    • Leigh Stoller's avatar
      Move setting the node permission table for a project from the web · 4931fecf
      Leigh Stoller authored
      interface to the backend. mkproj now looks at the pcremote_ok set
      and makes the proper calls to grantnodetype. This reduces the amount
      of hardwired goo in the web interface.
      
      Still, there is a bit of hardwired stuff in mkproj. At present we do
      not form a relationship between a phys node type and the types we
      assign to the virtual nodes. Thats is, nothing says that a pcplabphys
      implies the right to use pcplabinet, etc. With only 3 remote phys
      types, I just hardwired it into mkproj calling grantnodetype with type
      pcplab (the class for the virtnodes) for pcplabphys. Same for pcron
      and pcwa, (both get pcvwa). Ultimately we need a better type system.
      In general the type system is pretty screwy.
      4931fecf
  17. 16 Oct, 2003 1 commit
  18. 14 Aug, 2003 1 commit
  19. 27 Mar, 2003 1 commit
  20. 13 Feb, 2003 1 commit
  21. 24 Jan, 2003 1 commit
  22. 30 Dec, 2002 1 commit
  23. 05 Dec, 2002 1 commit
  24. 16 Sep, 2002 1 commit
    • Leigh Stoller's avatar
      Reorg of working directory and log file stuff for start/swap/end · 533dc18f
      Leigh Stoller authored
      experiment. Here is mail to tbops:
      
      * Moved the working directory for experiment setup/swap/end to a new
        directory located on boss instead of over NFS to /proj/$pid/$eid. This
        new location is /usr/testbed/expwork/$pid/$eid.
      
      * Changed the name of the directories we create in /usr/testbed/expinfo to
        $pid-$eid.$index where $index is a new autoincrement field in the DB
        table. I really hated the names that were created before.
      
      * Changed where logs are written from /tmp to the new location in
        /usr/testbed/expwork/$pid/$eid.
      
      Okay, why.
      
      * We no longer operate on NFS mounted directories that might hang. Its
        easier to catch the situation where a copy of the log file over at the
        end of experiment creation fails cause of an NFS problem.
      
      * We no longer have user writable files that are inputs to other parts of
        the system (like top and ptop files).  Not that a user would be bad, but
        it closes a hole.
      
      * We no longer copy user writable files from /proj to boss where we might
        fill up an important filesystem cause the user put a .ndz file in the the
        working directory. Not that a user would be bad, but it closes a hole.
      
      * Its easier to save all the log files this way, for each swap in and
        out.
      
      * Removing a directory over NFS is a royal irritant when someone is CD'ed
        into that directory or looking at a file on the other side (the astute
        observer will peg this as the reason I went down this idiotic path in the
        first place!).
      
      * About 6 other reasons that I can no longer remember. Seriously, I really
        had more reasons I can no longer remember! :-)
      533dc18f
  25. 07 Jul, 2002 1 commit
  26. 12 Feb, 2002 1 commit
  27. 03 Jan, 2002 2 commits
  28. 27 Dec, 2001 1 commit
    • Leigh Stoller's avatar
      Another set of group changes. As discussed in email and meetings, · 8404af03
      Leigh Stoller authored
      group directories are now created in a different tree than the
      the project directory so that they can be exported independently of
      the project tree to the nodes in a group experiment. The tree is
      routed at /groups on boss/users and on nodes.
      
      1. mkgroup,rmgroup,mkproj - Minor changes to reflect new group
         directory location (/groups). We leave a symlink in the old spot to
         maintain compatability, and to reduce the number of different
         directories that a person needs to worry about. So, when a group is
         made, you get a real directory /groups/pid/gid, and a symlink
         /proj/pid/groups/gid that points to the former.
      
      2. tmcd/tmcd.c - Minor change to add the additional group directory mount
         in the mounts command. Only done when pid!=gid for the experiment.
      
      3. tmcd/libsetup.pm and friends - Minor changes to fix the fact that
         mkdir does not create subdirs along the way unless the -p option is
         specified. Needed to create the local directory for the mounts
         returned by tmcd for group dirs. Pushed them out to the sup trees,
         although 6.2 images older than the most recent one are not going to
         work right. No one is using those images though, and we should just
         flush the sup trees.
      
      4. exports_setup.in - Ah, the crux of the issue. I really dislike NFS
         at this point. The original idea was to export a third set of
         directories to nodes that were part of a group experiment. Those
         nodes would get /groups/pid/gid exported, and /proj/pid read-only.
         Well, no such luck. On users, /groups and /proj are both really on
         /q, and the old restriction of mountd not allowing an IP to
         specified more than once on the right hand side for any FS, reared
         its ugly head again. As far as mountd is concerned, /q/groups and
         /q/proj are the same thing, and so it bombed when I tried to export
         them on different lines, since that meant an IP was repeated twice.
         So, I reworked exports_setup, and now for any node that is part of
         a group experiment, it gets this:
      
      	/q/proj/pid /q/groups/pid/gid -maproot=root 155.101.132.26
      
         which at least allows the individual group dirs to be protected
         from each other, but does not allow /proj/pid to be exported read
         only. Sigh.
      8404af03
  29. 26 Dec, 2001 1 commit
    • Leigh Stoller's avatar
      A bunch o' account managment script schanges. I have reworked · 46068860
      Leigh Stoller authored
      mkprojdir, mkacct-cntrl, mkgroup, and group-update into a set of new
      scripts that are more specific to their intended operation, and strive
      to do less work.
      
      1. mkacct - Replaces mkacct-cntrl. This script no longer does any
         group stuff. All it does is create new accounts, or update the
         password and gecos fields of existing accounts. Usage is the same
         as it was: "mkacct <userid>", and is typically invoked from the web
         interface via the approveuser form.
      
      2. mkgroup - Replaces group-update. This script creates new groups,
         either for the main project when it is approved, or for subgroup
         creation. This script does not alter the group membership. Usage
         is typically from the web interface, but mkgroup can be invoked
         from the command line: "mkgroup [-b | -a] <pid> <gid>" where -b
         puts it in the background and sends email later, while -a just
         captures the log and emails. This "audit" feature is going to find
         its way into more scripts as soon as I figure out a neat and clean
         perl mechanism to make it easy.
      
      3. setgroups - Replaces group-update. This script modifies the group
         membership of either specific users, or all the users in a
         project. It is typically invoked from the web interface when a
         project leader edits the subgroup membership or when a user is
         first approved to a project or subgroup. Command line usage is:
      
      	setgroups [-b | -a] -p <pid> [user ...]
              setgroups [-b | -a] [user ...]\n
      
         The first form is mostly a means to speed things up. The web
         interfaces knows exactly what users have need to be changed, but a
         global project update is nice too.
      
      4. mkproj - Replaces mkprojdir. Actually, mkproj still has all that
         directory code, but it also handles creating the groups and the
         account for the project leader. Part of my policy to move as much
         random code out of the web interface and into the PERL backend
         where it belongs.
      46068860