1. 30 Mar, 2006 1 commit
  2. 13 Mar, 2006 1 commit
  3. 22 Feb, 2006 1 commit
  4. 13 Feb, 2006 1 commit
  5. 06 Feb, 2006 1 commit
  6. 10 Jan, 2006 1 commit
  7. 06 Jan, 2006 1 commit
  8. 03 Jan, 2006 1 commit
  9. 27 Dec, 2005 2 commits
    • Timothy Stack's avatar
      0b9f692f
    • Mike Hibler's avatar
      Imagezip groks (BSD) inodes. Since I have not enabled the "more dubious" · a24cc15b
      Mike Hibler authored
      optimization, this will only affect UFS2 filesystems.
      
      > /*
      >  * If DO_INODES is defined, we look at the inode list in each cylinder group
      >  * and try to make further space reducing optimizations.  If there are
      >  * uninitialized inodes (UFS2 only) we add those blocks to the skip list.
      >  *
      >  * If CLEAR_FREE_INODES is also defined, we make a more dubious optimization.
      >  * Initialized but free inodes will go into the image data, but we first zero
      >  * everything except the (usually randomized) generation number in an attempt
      >  * to reduce the compressed size of the data.
      >  */
      > #define DO_INODES
      > #undef CLEAR_FREE_INODES
      a24cc15b
  10. 21 Dec, 2005 1 commit
  11. 16 Dec, 2005 8 commits
  12. 09 Dec, 2005 1 commit
  13. 08 Dec, 2005 1 commit
  14. 06 Dec, 2005 2 commits
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      
      The nitty-gritty:
      
      1. Keep disk image signature files for all nodes in an experiment.
      
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
      
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      
      2. Swapout-time hook for creating swapout image.
      
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
      
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      
      3. Client-side script for creating swapout image.
      
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
      
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      
      4. Auto creation of signature files for new images.
      
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
      
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      
      5. Web changes to set/clear saving of disk state at swapout time.
      
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
      
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
      ed0d25b4
    • Mike Hibler's avatar
      Minor tweak to free block handling: wait til we have accumulated and merged · dc7b2389
      Mike Hibler authored
      all free blocks before throwing out ones that are too small (-F).  There
      were a fair number of cases where a small free chunk was adjacent to a larger
      one and we were tossing out the smaller.  This does increase the risk that
      we will run out of memory building the free list.  If that happens, we can
      make an incremental cleanup pass.
      dc7b2389
  15. 03 Dec, 2005 1 commit
  16. 02 Dec, 2005 3 commits
  17. 30 Nov, 2005 2 commits
  18. 22 Nov, 2005 1 commit
  19. 17 Nov, 2005 3 commits
    • Mike Hibler's avatar
      682c31e8
    • Mike Hibler's avatar
      1. Beef up "admin mode" support. · 4ec701e7
      Mike Hibler authored
      * Add libadminmfs.pm with routines for entering/exiting and executing
        commands in, the admin MFS.  Node admin and firewall swapout (see
        below) now use this, the image creation process does not yet.
      
      * Add swapout time hooks for running an admin mode process, likely to
        be used to collect swapout time state.  Currently controlled globally
        by two new sitevars.
      
      * Modified node_admin to use the library and added a "-c <command>"
        option to have nodes go into admin mode and run a command.  I don't
        really expect this to be useful, it was just a testing vehicle for
        the library.
      
      2. Improved the swapout process for firewalled experiments.  Largely
         just generalized what we already did for paniced experiments.
         At swapout, firewalled nodes are:
      
         - powered off
         - set to boot into admin mode and run a disk zapper
         - powered on
      
        The swapout process then waits for all nodes to successfully complete
        disk zapage, at which point the nodes are nfree'ed as usual.  Any
        failure of the above process, marks the experiment as panic'ed (to
        ensure that we are involved in cleanup) and sends mail to testbed-ops
        describing the state of the nodes.
      
      3. Added the aforementioned disk zapper, a little C program in the MFS
         which zeroes out the MBR and partition boot blocks (but not the MBR
         partition table or FS superblocks).  This is added insurance that if
         a node somehow gets diverted after being nfree'd but before getting
         the disk reloaded (e.g., goes to hwdown), that we cannot accidentally
         boot from the disk.  This program gets installed in the admin MFS.
      
      4. Related to firewalls, modified swapin to use the new documented
         "snmpit -N" to get the firewall VLAN number rather than parsing the
         output that was a side-effect of VLAN creation.
      4ec701e7
    • Mike Hibler's avatar
      Make imagehash create the .sig file with the same modtime as the image. · 34ec01b2
      Mike Hibler authored
      This is a cheap verification mechanism.  Also install imagehash now with
      all the other image tools, it will be used soon.
      34ec01b2
  20. 16 Nov, 2005 3 commits
  21. 03 Nov, 2005 1 commit
  22. 17 Oct, 2005 1 commit
  23. 29 Sep, 2005 1 commit
    • Mike Hibler's avatar
      Random stuff: · b3af99a3
      Mike Hibler authored
       * Merge Kevin's changes to ensure chunk data is not modified
       * Add hack retry code on reading (if using -r option) to avoid NFS problem
       * Clean up The Sacred Loop of Decompression, the gawdawful heart of
         inflate_subblock
      b3af99a3
  24. 08 Sep, 2005 1 commit