1. 12 Jan, 2006 1 commit
    • Leigh Stoller's avatar
      Checkpoint changes to the Archive code. · aca4f452
      Leigh Stoller authored
      * Add support for linking to the NS file that will be used, from the
        begin experiment page, when duplicating or branching an experiment.
        Ultimately we want to separate things so that user can first edit
        the NS file and then proceed to branching.
      
      * In discussion we agreed to use the convention that a directory called
        "archive" in experiment directory, will always be saved and restored.
        This has been implemented.
      
      * Add more of the support for branching an experiment (the archive).
        Batchexp takes a couple of new arguments:
      
      	-c pid,eid[:tag]  or
      	-c exptidx[:tag]
      
        The above specifies what and where to duplicate or branch. Simply
        giving pid,eid does not use the archive, but just copies right out
        of the existing experiment directory.
      
        Adding the -b option says to branch instead of duplicate.
      aca4f452
  2. 06 Jan, 2006 1 commit
  3. 04 Jan, 2006 1 commit
  4. 21 Dec, 2005 1 commit
  5. 19 Dec, 2005 1 commit
    • Leigh Stoller's avatar
      Add support for moving deleted users to a deleted users table. This · b4231fbf
      Leigh Stoller authored
      would be no big deal, except that we want to retain user_stats for
      deleted users, and rather then a deleted_user_stats table, I want to
      retain stats for deleted users in the user_stats table, since that
      is a more natural place for them.
      
      The main problem is that we use the login (uid) as the cross table
      reference slot all over the DB, which is fundamentally incorrect, if
      we want to be able reuse uids and still know what historical data
      refers to.
      
      So, I have taken a few baby steps towards weaning us off the uid, and
      towards permanently unique key for users, using the unix_uid integer
      for now, but probably something slightly different later.
      
      The user_stats is now indexed on this new key (called uid_idx in the
      users_stats table) instead of the plain uid.
      
      The unix_uid slot in the users table is no longer an auto_increment
      field, but instead uses the emulab_indicies table for the next
      available index.
      b4231fbf
  6. 16 Dec, 2005 1 commit
    • Leigh Stoller's avatar
      Add support for pre-reserving nodes. New link off the ShowNode page · e49e2f9a
      Leigh Stoller authored
      allows takes you to new (admin only) page to select a project that
      node will be reserved for.
      
      * The node is not actually *reserved*, it is *pre* reserved! The node
        stays in the free pool, and is available only for use in the project
        to which it is reserved.
      
      * The node can already be reserved to some other project when you
        pre-reserve it. It is not until the current owner releases the node
        that the pre-reservation takes effect.
      
      * The node free counts (on the web pages) count a free a node with a
        pre-reservation, as allocated. This way people do not see a free
        count that includes a node they will never be able to get.
      e49e2f9a
  7. 15 Dec, 2005 1 commit
    • Kirk Webb's avatar
      · 41c54939
      Kirk Webb authored
      The revived Plab interface is here!
      
      Lots of updates to the plab backend, including improved plab <-> elab node
      id translation and update handling.  Includes support for the current PLC
      API, and the new pl_conf node manager interface API.  Several more db library
      routines were ported from the perl library to the python one to support the
      new code (mostly the node_id tracking stuff).  Fixes to the client side and
      also a rootball creation cleanup (binaries removed from the CVS repo).
      
      There are also enhancements to the experiment view page for experiments
      including plab nodes: site and widearea hostname are now displayed along
      with the other node information.
      
      Note that the way setup timeout for vnodes is calculated has been changed a
      bit.  Instead of using a hardwired base timeout, the base timeout is now
      based on the reload_waittime database field, which comes from the 'OS'
      (e.g., FBSD-JAIL, RHL-PLAB) the vnode runs.
      
      The default max duration for a plab slice created through the plab_ez interface
      is set to 1 year, and linktest is currently disabled and hidden through
      the ez interface.
      
      There is still work to do, but this checkin brings with it a functional
      plab portal!
      41c54939
  8. 06 Dec, 2005 1 commit
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      
      The nitty-gritty:
      
      1. Keep disk image signature files for all nodes in an experiment.
      
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
      
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      
      2. Swapout-time hook for creating swapout image.
      
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
      
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      
      3. Client-side script for creating swapout image.
      
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
      
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      
      4. Auto creation of signature files for new images.
      
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
      
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      
      5. Web changes to set/clear saving of disk state at swapout time.
      
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
      
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
      ed0d25b4
  9. 09 Nov, 2005 1 commit
  10. 20 Oct, 2005 2 commits
    • Kirk Webb's avatar
      · df699f1c
      Kirk Webb authored
      Hide the Attributes view when there are none to display.
      df699f1c
    • Kirk Webb's avatar
      · 5326988f
      Kirk Webb authored
      New node_attributes facility and table.
      
      Auxiliary node attributes, such as service tag #, BIOS version, etc., are
      should now be placed into the node_attributes table.  This can be accomplished
      by either using the node_attributes command line tool, or by using the
      modnodeattributes_form.php3 form (not linked in anywhere yet, but will be
      in a moment).  Attribute names and values are checked for sanity using
      table_regex entries.  Also note that I started with the nodecontrol stuff
      as a template.
      
      The command line tool and web form (which simply calls the command line tool
      to actually do the modifications) can add, delete, and/or remove attributes.
      
      Finally, note that the bios_version column has been moved from the nodes
      table to the node_attributes table.  The Node Information page will show
      the list of current attributes at the bottom of the info table.
      5326988f
  11. 17 Oct, 2005 1 commit
    • Kirk Webb's avatar
      · 31090a2d
      Kirk Webb authored
      Add the ability to set the BIOS version, serial number, and service tag from
      the nodecontrol page and command line util.  Must be admin.
      31090a2d
  12. 14 Oct, 2005 2 commits
    • Mike Hibler's avatar
      Mike's yearly PHP hacking: · 39b1477e
      Mike Hibler authored
       * Add a "showclass" param to shownodes so that the
         node listing will include only nodes of the indicated class.
         Prepend "no-" to exclude nodes of a particular class.  So:
              showclass=pc,mote
         would show just pcs and motes.
      
       * Now the real reason I added the above: change the hwdown experiment
         node list to include "showclass=no-pcplabphys" by default so I don't have
         to flip through 600+ dead plab nodes.  Change the URL to "showclass=all"
         if you want to see plab nodes in hwdown.
      39b1477e
    • Leigh Stoller's avatar
      Add Wikionly tag to SHOWUSER(). · fc570caa
      Leigh Stoller authored
      fc570caa
  13. 12 Oct, 2005 1 commit
  14. 25 Sep, 2005 1 commit
  15. 08 Sep, 2005 1 commit
  16. 31 Aug, 2005 1 commit
  17. 30 Aug, 2005 1 commit
  18. 17 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      The Emulab Knowledge Base! · 6f08c442
      Leigh Stoller authored
      Okay, I implemented a primitive Knowledge Base! The current contents are
      *all* the existing FAQ entries, which I entered manually. Here are the
      details.
      
      * My reason for doing this is that we need something very simple. The wiki
        is too much of a barrier, and its search capabilities are pathetic.
      
      * The search page for the Knowledge Base is:
      
      	https://www.emulab.net/kb-search.php3
      
        Fairly primitive keyword search. Turns out that mysql 4.0 has a bunch for
        really good text searching functions built in, but we run 3.23 ... so I
        had to roll it myself. So, its a simple keyword (space or comma
        separated) search, no regular expressions.
      
      * Each DB record has a "faq_entry" flag, so creating the current FAQ on the
        fly from the DB is easy. See:
      
      	https://www.emulab.net/kb-faq.php3
      
      * In reddot mode, you can add new KB entries:
      
      	https://www.emulab.net/kb-manage.php3
      
        The form is fairly obvious but here are details anyway:
      
          Section Name: Choose an existing title, or make up a new one.
          Title:        The title of the KB (or FAQ) entry.
          Faq Entry:    Check this box if the new entry should show up in the FAQ.
          X Ref Tag:    A token so you can refer to other KB entries by name,
                        instead of by its index. Within the KB entry you would
                        write: <a href=kb-show.php3?xref_tag=sometag>
          Body:         Whatever you like. I took the existing FAQ entries and
                        stuck them with no changes except for the xref_tag
                        mentioned about (since some entries referenced other
                        entries).
      
      * Once you click on sumbit, you will see the entry as it will appear to
        users, along with a submenu to Modify/Delete/Add entries. You can modify
        the current entry from that menu. Mere users do not see this menu, only
        when in reddot mode.
      
      * The intent here is that we can generate new entries really easy, right
        from email if you like (with appropriate <pre> or <xmp> tags around it).
      
      * I have added sql/knowlbase-create.sql and a makefile target to
        generate that file when creating a distribution. I also added a section
        to install/boss-install to insert the entries into the new DB.
      
      * I hooked the search function into the existing Search Documentation link.
        We know search both the Knowledge Base *and* the Documentation on doc
        searches. This probably needs a little more work to get right.
      
      * I changed a lot of faq links to be more consistent and to reference
        the proper xref_tags (#swapping instead of #UTT-34).
      6f08c442
  19. 15 Aug, 2005 1 commit
    • Leigh Stoller's avatar
      The bulk of the mailman support. Still not turned on by default (cause · a64593f3
      Leigh Stoller authored
      Jay has "comments"), but I do not want it hanging around in my source
      tree. Here is my mail message:
      
      * The "My Mailing Lists" is context sensitive (copied from Tim's
        changes to the My Bug Databases). It takes you to the *archives* for
        the current project (or subgroup) list. Or it takes you to your
        first joined project.
      
      * The showproject and showgroup pages have direct links to the project
        and group specific archives. If you are in reddot mode, you also
        get a link to the admin page for the list. Note that project and
        group leaders are just plain members of these lists.
      
      * The interface to create a new "user" list is:
      
      	https://www.emulab.net/dev/stoller/newmmlist.php3
      
        We do not store the password, but just fire it over in the list
        creation process.
      
        Anyone can create their own mailing lists. They are not associated
        with projects, but just the person creating the list. That person
        is the list administrator and is given permission to access the
        configuration page.
      
        This page is not hooked in yet; not sure where.
      
      * Once you have your own lists, you user profile page includes a link
        in the sub menu: Show Mailman Lists. From this page you can delete
        lists, zap to the admin page, or change the admin password (which is
        really just a subpage of the admin page).
      
      * As usual, in reddot mode you can mess with anyone else's mailman lists,
        (via the magic of mailman cookies).
      
      * Note on cross machine login. The mailman stuff has a really easy way
        to generate the right kind of cookie to give users access. You can
        generate a cookie to give user access, or to the admin interface for
        a list (a different cookie). Behind the scenes, I ssh over and get
        the cookie, and set it in the user's browser from boss. When the
        browser is redirected over to ops, that cookie goes along and gives
        the user the requested access. No passwords need be sent around,
        since we do the authentication ourselves.
      a64593f3
  20. 07 Jul, 2005 1 commit
    • Leigh Stoller's avatar
      Oh, such a silly little project ... Added CVS support to Emulab. When · 9b17b075
      Leigh Stoller authored
      enabled in the defs file:
      
      	CVSSUPPORT=1
      
      each project gets a stub CVS tree created (using 'cvs init') in
      /proj/$pid/CVS. It is up to users obviously to do something with
      that tree, and of course they have to either set their CVSROOT
      env variable, or use the -d option to cvs.
      
      The showproject page gets a link to the per-project CVS tree, using
      the cvsweb interface, which I hacked up a bit to allow restricted
      access to specific project trees, via a ?pid=$pid argument to the URL.
      Without the ?pid argument, it falls back to normal behaviour, which is
      check the cvsallowed bit in the users table, and provide access to the
      Emulab source repo.
      
      If you are curious, go here:
      
      	https://www.emulab.net/cvsweb/cvsweb.php3/?pid=testbed
      9b17b075
  21. 28 Jun, 2005 1 commit
  22. 02 Jun, 2005 1 commit
  23. 28 Mar, 2005 1 commit
    • Leigh Stoller's avatar
      Implement a cross machine login so that user is automatically logged · 9310dedb
      Leigh Stoller authored
      into the Wiki when clicking on the My Wiki's link. Works like this:
      
      * The My Wiki's link points to new page, gotowiki.php3, on boss.
      
      * The gotowiki page looks for a new cookie in the user's browser
        which holds a key (the usual random data run through md5).
      
      * If the key does not exist, generate it and store it in the user
        browser (expires when browser is closed or emulab login times out).
        Also invoke backend script wikixlogin, which will send the key over
        to the wiki server (via ssh), which will write the key into a file
        named by the user account.
      
      * The user's browser is redirected to the wiki server's login script
        (twiki/bin/newlogon), but instead of username and password, we send
        over username and key (as well as redurl= parameter which is the
        page on the wiki server to redirect to later).
      
      * The new login script looks for this case, and opens the file named
        by the user and compares the key it gets with what is in the file.
        If they match, the user login succeeds and the browser is once again
        redirected, but this time to the page it wants on the wiki server.
        If the key does not match, the browser is redirected to the login
        page (so user can enter username password normally). The redurl
        parameter is passed along as well.
      
      * Subsequent clicks on My Wiki's will not need to invoke the backend
        script, since the cookie will be in the browser.
      9310dedb
  24. 21 Mar, 2005 1 commit
  25. 22 Feb, 2005 1 commit
    • Leigh Stoller's avatar
      Okay, first attempt to deal with os_setup waittimes on a per node_type · facc7acd
      Leigh Stoller authored
      and per OSID basis.
      
      * Added bios_waittime to node_types table and reboot_waittime to
        os_info table. Initialized them as follows:
      
              update node_types set bios_waittime=60 where class='pc';
              update os_info set reboot_waittime=150 where OS='Linux' or
      	  OS='FreeBSD' or OS='NetBSD';
              update os_info set reboot_waittime=180 where OS=Windows';
      
      * The bios waittime can be edited via the web interface.
      
      * The reboot waittime can be set only by admin people right now; this
        is another case of something that maybe the user should not see
        cause its too much stuff? Instead, default values are established in
        www/osiddefs.php3.
      
      * os_setup computes its per-node waitime as:
      
      	(bios_waittime + reboot_waittime) * 2
      
        as per Mike's suggestion. If either value is not defined in the DB,
        it defaults the original 7 minute value.
      facc7acd
  26. 31 Jan, 2005 1 commit
    • Leigh Stoller's avatar
      Tweaks to robot display: · 3acfb53d
      Leigh Stoller authored
      * If there is a destination for a rebot, display that as a shadow circle.
        Draw a light grey line from source to destination.
      
      * If an orientation is in the location_info table, the current location
        circle gets a stick drawn in that direction.
      
      * If the destination has an orientation in the nodes table, the shadow
        circle gets a stick drawn in that direction.
      
      * I added a little bit of code so that labels are drawn on top if the
        orientation stick would draw through it. I'm sure that will interact with
        cropping badly if a node is right at the edge of the picture.
      3acfb53d
  27. 14 Jan, 2005 1 commit
  28. 13 Jan, 2005 1 commit
  29. 12 Jan, 2005 1 commit
  30. 07 Jan, 2005 1 commit
  31. 06 Jan, 2005 1 commit
    • Leigh Stoller's avatar
      A bunch of boot changes. Read carefully. · 94ccc3f4
      Leigh Stoller authored
      * Add boot_errno to the nodes table so that nodes can report in a
        subcode to indicate what went wrong. At present, we do not report any
        real error codes; that is going to take some time to work out since it
        will reqiure a bunch of changes to the boot scripts.
      
      * Add new table node_bootlogs to store logs provided by the nodes. Not
        a full console log, but a log of the tmcd client side part. We can
        make it a full log if we want though; just means mucking about with
        the boot phase a bit.
      
      * Add new state transition to NORMALv2 and PCVM state machines. "TBFAILED"
        is a new state that is sent (after TBSETUP) if a node fails somewhere in
        the tmcd client side.
      
      * Change TBNodeStateWait() to take a list of states (instead of single
        state) and an optional pass by reference parameter to return the actual
        state that the node landed in. Change all calls to TBNodeStateWait() of
        course.
      
      * Change os_setup (and libreboot in wait mode) to look for both TBFAILED
        and ISUP. If a TBFAILED event is seen, we can terminate the wait early
        and not retry os_setup on physical nodes (although still retry virtual
        nodes). The nice thing about this is that the wait should terminate much
        earlier (rather then waiting for timeout), especially for virtual nodes
        which can take a really long time when there are a couple of hundred.
      
      * Add new routines dobooterrno() and dobootlog() to tmcd. Bump version
        number and increase the buffer size to allow for the larger packets that
        a console log wikk generate (added MAXTMCDPACKET variable, set to 0x4000).
      
      * Add new -f option to tmcc to specify a datafile to send along as the last
        argument to tmcd. This is more pleasing then trying to send a console log
        in on the command line. For example: "tmcc -f /tmp/log BOOTLOG" will send
        a BOOTLOG command along with the contents of /tmp/log.
      
        Also close the write side of the pipe so that server sees EOF on
        read. See aside comment below.
      
      * Changes to rc.bootsetup:
           1. Use perl tricks to capture all output, duping to the console and to
              a log file in /var/emulab/logs.
           2. On any error, send a status code (boot_errno) and the bootlog to
              tmcd.
           3. Generate a TBFAILED state transition.
      
      * Changes to rc.injail:
           1. Same as rc.bootsetup, but do not send log files; that would pummel
              boss. Leave them on the physical node.
      
      * Change vnodesetup (which calls mkjail) to watch for any error and send a
        TBFAILED state transition. This should catch almost all errors, and
        dramatically reduce waiting when something fails.
      
      * Changes to rc.cdboot are essentially the same as rc.bootsetup, although a
        bootlog is sent all the time (success or failure), and I do not generate
        a boot_errno yet. Also, instead of TBFAILED, generate a PXEFAILED state
        since the CDROM is actually operating within the PXEFBSD opmode. I have
        yet to work this into the rest of the system though; waiting to get a new
        CD built and actually experiment with it.
      
      * Add new menu option and web page to display the node bootlog. We store
        only the lastest bootlog, but maybe someday store more then one. Display
        boot_errno on node page.
      
      Aside: I made a big mistake in the tmcd protocol; I did not envision
      passing more then a small amount of data (one fragment) and so I do not
      include a record terminator (ie: close of the write side on the client
      sends EOF) or a size field at the beginning. No big deal since small
      requests are sent in one fragment and the server sees the entire
      thing. Well, with a large console log, that will end up as multiple
      fragments, and the server will often not get the entire thing on the first
      read, and there are no subsequent reads (with no EOF or known size, it
      would block forever). Well, fixing this in a backwards compatable manner
      (for old images) was way too much pain. Instead, tmcc now closes the write
      side, and the server does subsequent reads *only* in the new dobbootlog()
      routine. Note that it *is* possible to fix this in a backwards compatable
      manner, but I did not want to go down that path just yet.
      94ccc3f4
  32. 03 Jan, 2005 1 commit
  33. 15 Dec, 2004 1 commit
    • Robert Ricci's avatar
      Blinky light support for motes on stargates. · 9323263d
      Robert Ricci authored
      Display a new 'Blinky Lights' button on the showexp page. In order to
      do this, I have to get a list of which classes/types are in use in
      the experiment.
      
      This leads to moteleds.php3, which displays the blink lights using
      Tim's cool Java applet.
      9323263d
  34. 14 Dec, 2004 1 commit
    • Leigh Stoller's avatar
      Add back some of Russ's code that allowed for clicking on the map, and · 98348a99
      Leigh Stoller authored
      getting a crosshair painted in. Also use those coords and the
      pixels_per_meter conversion from the DB to print out for the user
      where he just clicked.
      
      Change the SHOWNODE function to print out the coordinates (so when you
      click on a robot in the map, you get a line that says what its coords
      are).
      
      Also generate table of all robot positions to place below the image.
      98348a99
  35. 13 Dec, 2004 2 commits
    • Russ Fish's avatar
    • Timothy Stack's avatar
      · a83bc7d2
      Timothy Stack authored
      Start of an applet/php hack for showing mote LED status on web pages:
      
      	* www/BlinkenLichten.java, www/BlinkenLichten.class: Applet that
      	connects back to the server and changes its color based on output
      	from a php page.
      
      	* www/GNUmakefile.in: Install java classes.
      
      	* www/ledpipe.php3: Draft page for driving the BlinkenLichten
      	applet, it just alternates on/off and doesn't read the real
      	status.
      
      	* www/showstuff.php3: Add a function that outputs the html magic
      	to embed the applet.
      a83bc7d2
  36. 09 Dec, 2004 1 commit
  37. 01 Oct, 2004 1 commit
    • Mike Hibler's avatar
      Oft-desired (by me anyway) hack to print out most recent log entry in · 24938e57
      Mike Hibler authored
      the hwdown experiment listing.  Just a simple hack I though.  Ye Gads!
      That was painful.  On the plus side, I now know the different between all
      those SQL join types :-)
      
      Anyway, added a showlastlog variable in the SHOWNODES function.  Currently,
      it only gets set if the pid/eid is emulab-ops/hwdown.  Eventually it could
      be a parameter so it could be selected for any experiment.
      24938e57