1. 06 Dec, 2006 1 commit
  2. 01 Dec, 2006 2 commits
    • Mike Hibler's avatar
      stick in copyright · 1141d18c
      Mike Hibler authored
    • Mike Hibler's avatar
      Bug fixes from Annette DeSchon <deschon@ISI.EDU> and Keith Sklower · 8ea641c1
      Mike Hibler authored
      <sklower@vangogh.CS.Berkeley.EDU> for the following, related to the -z (zero)
      option in imageunzip/frisbee:
        1. For the case where a full-disk image is smaller than
           the disk the image is being unzipped onto, we added
           code to zero the area between the end of the image and
           the end of the disk.
        2. During the unzipping process, when zeros are being written
           at the end of a chunk, a write() that returned a length
           different from the expected value previously caused an
           infinite loop.  We noticed this problem at ISI, on a number
           of pc733s, which we suspect may have (relatively minor)
           hardware disk problems.
      The latter addressed a Mike-o that has existed for 4 years.  Call it failure
      resilient computing or just plain denial, but because of a botched conditional,
      I was ignoring failed writes to the disk.  This lead to one of those infinite loop
      thingees if you actually had a bad disk.
  3. 21 Nov, 2006 1 commit
  4. 05 Oct, 2006 1 commit
    • Leigh B. Stoller's avatar
      More work on "recording" template events. · e9607a77
      Leigh B. Stoller authored
      * New version of template_record just for ops, since so much is
        different about ops, not bothering to maintain a single version.
      * Various fixes to how the recorded events are stored and reconstituted.
        The big fix is to wrap them in a sequence to that they get fired
        properly (waiting for completion of previous event in recording).
      * New buttons to Pause and Continue event time, which is used when
        adding recorded events. This allows users to pause time while they
        "think" so when an event is recorded, the thinking time is not actually
        in the timeline. Eventually hope to figure this out automatically, but
        that will take some real, uh, thinking.
      * Add a new event editor (linked off the template page) that allows
        you to delete and change the recordings. Note that you can only edit
        the events at the template level; you cannot edit the events of an
        instance (swapped in experiment), and you can only edit the recorded
        events, not any other events. Not sure its useful to be able to do
        either of these yet, but probably not too hard to add at some point.
  5. 04 Oct, 2006 1 commit
  6. 13 Sep, 2006 1 commit
  7. 28 Aug, 2006 1 commit
  8. 25 Aug, 2006 1 commit
    • Leigh B. Stoller's avatar
      Add support for dynamic registration of ports on experimental nodes so · 73102ef8
      Leigh B. Stoller authored
      that clients and servers can avoid using hardwired ports on those
      experimental nodes. I have added the following tmcd operation:
      	tmcc portregister <service> [<port>]
      where we assume its the control network IP (from the DB), and the pid/eid
      of the node the experiment belongs to. The given port is entered into
      the port_registration table for the experiment, using the service as the
      tag. Supplying port=0 clears the registration from the table.
      When called like:
      	tmcc portregister <service>
      we return the registered port, or nothing.
      I hacked up a little C library module in libtb so that there is something
      that looks like a C interface to this:
       	PortRegister(char *service, int port);
       	PortLookup(char *service, char *hostname, int namelen, int *port);
      The above routines call out to tmcc of course.
      Lastly, I changed the sync server and client to use the new port
      registration, via the library calls above.
      There are other emulab services that need to be changed as well, but
      they can be done on an as needed basis.
  9. 14 Aug, 2006 1 commit
    • Leigh B. Stoller's avatar
      Checkpoint my dynamic event stuff, crude as it is. The idea for this first · 9d021a07
      Leigh B. Stoller authored
      draft is that the user will at the end of an experiment run, log into one
      of his nodes and perform some analysis which is intended to be repeated at
      the end of the next run, and in future instantiations of the template.
      A new table called experiment_template_events holds the dynamic events for
      the template. Right now I am supporting just program events, but it will be
      easy to support arbitrary events later. As an absurd example:
      	node6> /usr/local/bin/template_analyze ~/data_analyze arg arg ...
      The user is currently responsible for making sure the output goes into a
      file in the archive. I plan to make the template_analyze wrapper handle
      that automatically later, but for now what you really want is to invoke a
      script that encapsulates that, redirecting output to $ARCHIVE (this
      variable is installed in the environment template_analyze.
      The wrapper script will save the current time, and then run the program.
      If the program terminates with a zero exit status, it will ssh over to ops
      and invoke an xmlrpc routine to tell boss to add a program event to both
      the eventlist for the current instance, and to the template_eventlist for
      future instances. The time of the event is the relative start time that was
      saved above (remember, each experiment run replays the event stream from
      time zero).
      For the future, we want to allow this to be done on ops as well, but
      that will take more infrastructure, to run "program agents" on ops.
      It would be nice to install the ssl xmlrpc client side on our images so
      that we do not have to ssh to ops to invoke the client.
  10. 30 Mar, 2006 1 commit
  11. 13 Mar, 2006 1 commit
  12. 22 Feb, 2006 1 commit
  13. 13 Feb, 2006 1 commit
  14. 06 Feb, 2006 1 commit
  15. 10 Jan, 2006 1 commit
  16. 06 Jan, 2006 1 commit
  17. 03 Jan, 2006 1 commit
  18. 27 Dec, 2005 2 commits
    • Timothy Stack's avatar
    • Mike Hibler's avatar
      Imagezip groks (BSD) inodes. Since I have not enabled the "more dubious" · a24cc15b
      Mike Hibler authored
      optimization, this will only affect UFS2 filesystems.
      > /*
      >  * If DO_INODES is defined, we look at the inode list in each cylinder group
      >  * and try to make further space reducing optimizations.  If there are
      >  * uninitialized inodes (UFS2 only) we add those blocks to the skip list.
      >  *
      >  * If CLEAR_FREE_INODES is also defined, we make a more dubious optimization.
      >  * Initialized but free inodes will go into the image data, but we first zero
      >  * everything except the (usually randomized) generation number in an attempt
      >  * to reduce the compressed size of the data.
      >  */
      > #define DO_INODES
      > #undef CLEAR_FREE_INODES
  19. 21 Dec, 2005 1 commit
  20. 16 Dec, 2005 8 commits
  21. 09 Dec, 2005 1 commit
  22. 08 Dec, 2005 1 commit
  23. 06 Dec, 2005 2 commits
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      The nitty-gritty:
      1. Keep disk image signature files for all nodes in an experiment.
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      2. Swapout-time hook for creating swapout image.
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      3. Client-side script for creating swapout image.
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      4. Auto creation of signature files for new images.
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      5. Web changes to set/clear saving of disk state at swapout time.
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
    • Mike Hibler's avatar
      Minor tweak to free block handling: wait til we have accumulated and merged · dc7b2389
      Mike Hibler authored
      all free blocks before throwing out ones that are too small (-F).  There
      were a fair number of cases where a small free chunk was adjacent to a larger
      one and we were tossing out the smaller.  This does increase the risk that
      we will run out of memory building the free list.  If that happens, we can
      make an incremental cleanup pass.
  24. 03 Dec, 2005 1 commit
  25. 02 Dec, 2005 3 commits
  26. 30 Nov, 2005 2 commits
  27. 22 Nov, 2005 1 commit