1. 23 Feb, 2004 1 commit
  2. 09 Feb, 2004 2 commits
  3. 06 Feb, 2004 1 commit
  4. 20 Jan, 2004 1 commit
  5. 14 Jan, 2004 1 commit
  6. 12 Dec, 2003 1 commit
  7. 17 Nov, 2003 3 commits
  8. 05 Nov, 2003 1 commit
  9. 24 Oct, 2003 4 commits
    • Mike Hibler's avatar
      Make clean target work correctly too! · 30d21c3b
      Mike Hibler authored
      30d21c3b
    • Mike Hibler's avatar
    • Mike Hibler's avatar
      Introducing imagezip redux! · 0063d2c4
      Mike Hibler authored
      Ok, not really any fundamental changes, mostly a source reorganization,
      but some new features as well:
      
      1. Split off FS specific code for all FS types into subdirectories.
         imagezip.c was just getting unwieldy.  You can now (in theory, I
         haven't enabled this in any easy way) build a version of imagezip
         with only the FSes you care about.
      
      2. As part of the above, localize the filesystem specific headers in
         the subdirs instead of relying on the installed system headers.
         This allows easy building of, for example, BSD FS support under
         Linux.
      
      3. Hacked up the GNUmakefiles some so that imagezip with all FS support
         will build (in the Emulab context) under Linux and FreeBSD 5.1 as
         well as FreeBSD 4.x.  Still a hack, for Linux you need to first do:
      	setenv LINUX 1
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         and for FreeBSD 5.x:
      	setenv HAVE_GCC3 1
      	setenv HAVE_LOCALE 1
         these trigger ifdefs in some of the Makefiles to DTRT.  Imagezip
         and frisbee still need a standalone configure...
      
      4. Added support for UFS2 (aka FreeBSD 5.x default filesystems).
      
      5. Added "-S dos-type" option to imagezip.  This is the general form
         of the -b, -l, and -n options to force imagezip to treat the device
         as though it contains a filesystem of the indicated type.
      0063d2c4
    • Mike Hibler's avatar
      Added -i (ignore v1 features) support to imagedump. Prevents it · ccf23ccb
      Mike Hibler authored
      from looking at the embedded (and deprecated) blockindex field in
      the headers.  Now imagedump won't complain if you run it on a
      piece of a split image.
      ccf23ccb
  10. 23 Oct, 2003 1 commit
  11. 21 Oct, 2003 1 commit
  12. 17 Oct, 2003 1 commit
  13. 15 Oct, 2003 1 commit
    • Mike Hibler's avatar
      Uniform syslog'ing. Change everything I could find to use a syslog facility · cc6d6fa7
      Mike Hibler authored
      as defined in the defs-* file (e.g. "TBLOGFACIL=local2").  The default is
      "local5" which is what we are setup to use so you shouldn't need to mess
      with your defs- file!
      
      perl scripts just get this value configured in when configure is run.
      C programs get the value in two ways.  For programs that are intimate with
      the testbed infrastructure, and include "config.h", they just get it from
      that file.  For programs that we sometimes use outside the Emulab build
      environment (e.g., frisbee, capture) and that don't include config.h,
      the value is set via a "-DLOG_TESTBED=..." in the GNUmakefile build line.
      If the value isn't set, it defaults to what it used to be (usually LOG_USER).
      
      Still to do: healthd, hmcd (whose build doesn't seem to be completely
      integrated) and plabdaemon.in (since its icky python :-)
      cc6d6fa7
  14. 09 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Reorg of two aspects of node update. · 2641af4d
      Leigh B. Stoller authored
      * install-rpm, install-tarfile, spewrpmtar.php3, spewrpmtar.in: Pumped
        up even more! The db file we store in /var/db now records both the
        timestamp (of the file, or if remote the install time) and the MD5
        of the file that was installed. Locally, we can get this info when
        accessing the file via NFS (copymode on or off). Remote, we use wget
        to get the file, and so pass the timestamp along in the URL request,
        and let spewrpmtar.in determine if the file has changed. If the
        timestamp it gets is >= to the timestamp of the file, an error code
        of 304 (Not Modifed) is returned. Otherwise the file is returned.
      
        If the timestamps are different (remote, server sends back an actual
        file), the MD5 of the file is compared against the value stored. If
        they are equal, update the timestamp in the db file to avoid
        repeated MD5s (or server downloads) in the future. If the MD5 is
        different, then reinstall the tarball or rpm, and update the db file
        with the new timestamp and MD5. Presto, we have auto update capability!
      
        Caveat: I pass along the old MD5 in the URL, but it is currently
        ignored. I do not know if doing the MD5 on the server is a good
        idea, but obviously it is easy to add later. At the moment it
        happens on the node, which means wasted bandwidth when the timestamp
        has changed, but the file has not (probably not something that will
        happen in typical usage).
      
        Caveat: The timestamp used on remote nodes is the time the tarfile
        is installed (GM time of course). We could arrange to return the
        timestamp of the local file back to the node, but that would mean
        complicating the protocol (or using an http header) and I was not in
        the mood for that. In typical usage, I do not think that people will
        be changing tarfiles and rpms so rapidly that this will make a
        difference, but if it does, we can change it.
      
      * node_update.in, client side watchdog, and various web pages:
        Deflated node_update, removing all of the older ssh code. We now
        assume that all nodes will auto update on a periodic basis, via the
        watchdog that runs on all client nodes, including plab nodes.
      
        Changed the permission check to look for new UPDATE permission (used
        to be UPDATEACCOUNT). As before, it requires local_root or better.
        The reason for this is that node_update now implies more than just
        updating the accounts/mounts. The web pages have been changed to
        explain that in addition to mounts/accounts, rpms and tarfiles will
        also be updated. At the moment, this is still tied to a single
        variable (update_accounts) in the nodes table, but as Kirk requested
        at the meeting, it will probably be nice to split these out in the
        future.
      
        Added the ability to node_update a single node in an experiment (in
        addition to all nodes option on the showexp page). This has been
        added to the shownode webpage menu options.
      
        Changed locking code to use the newer wrapper states, and to move
        the experiment to RUNNING_LOCKED until the update completes. This is
        to prevent mayhem in the rest of the system (which could be dealt
        with, but is not worth the trouble; people have to wait until their
        initiated update is complete, before they can swap out the
        experiment).
      
        Added "short" mode to shownode routine, equiv to the recently added
        short mode for showexp. I use this on the confirmation page for
        updating a single node, giving the user a couple of pertinent (feel
        good) facts before they comfirm.
      2641af4d
  15. 03 Oct, 2003 3 commits
  16. 02 Oct, 2003 2 commits
    • Mike Hibler's avatar
      Fix bug in computation of max input sector when there are extended partitions. · 4b6a7925
      Mike Hibler authored
      Changes to "info" option
      4b6a7925
    • Mike Hibler's avatar
      "Support for extended partition tables is easy!" he says... · 4301b8be
      Mike Hibler authored
      "Piece of cake!" he says...
      "15 minutes" he says...
      
      Well, a day later, we can now deal with extended partition tables.
      It was almost easy.  The offsets in an extended partition table are
      relative to the location of that table...mostly.  The exception is if
      an extended partition table contains an entry for another extended table.
      Then the offset provided is relative to the *first* extended partition
      table found, not the current one.
      
      This was also complicated by the NTFS library code which needs to be
      fed a device special file corresponding to the filesystem partition.
      So if I am cruising alone creating a whole disk image using /dev/ad0
      and I hit a partition containing an NTFS, I have to figure out the
      BSD name for that particular partition: "hmm..I'm looking at partition 3?
      That would be /dev/ad0s3".  Its easy for the first 4 partitions, but for
      extended partitions, I had to figure out how BSD mapped those partitions
      into device names.  Once I became aware of the DIOCGSLICEINFO ioctl, this
      was easy as it gave back all the info needed.
      4301b8be
  17. 27 Sep, 2003 1 commit
  18. 25 Sep, 2003 2 commits
  19. 24 Sep, 2003 1 commit
    • Leigh B. Stoller's avatar
      Convert install-rpm/install-tarfile to use the web server instead of · 0a259517
      Leigh B. Stoller authored
      tmcd (which is bad, since tying up the tmcd threads blocks all nodes
      in the testbed). The old functionality is left in tmcd for now.
      
      On the server side, a new web page (www/spewrpmtar.php3) receives a
      request for a file, along with the nodeid (pcXXX) making the request,
      and the secret key that is generated for each new experiment and
      transfered to the node via tmcd. If the key matches, the operation is
      handed off to tbsetup/spewrpmtar.in which verifies that the file is in
      the list of rpm/tar files for that node, and then spits it out to
      stdout. The web page uses fpassthru() to send the file out to the
      client. The client is using wget, and is required to use https (the
      web page checks).
      
      At present, the external script is run as the creator of the
      experiment, and gid of the experiment. Perhaps this is not a good
      idea. In any event, the file must be in the list of rpm/tarfiles,
      either owned by the experiment creator or with a group of the
      experiment, and the file must reside in either /proj or /groups.
      I use the realpath() function to make sure there are no symlink tricks
      pointing to outside those filesystems. I use the standard NFS read goo to
      prevent transient mount problems that we all know and love.
      0a259517
  20. 17 Sep, 2003 2 commits
  21. 09 Sep, 2003 1 commit
  22. 03 Sep, 2003 1 commit
  23. 14 Aug, 2003 1 commit
  24. 05 Aug, 2003 2 commits
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      The rest of the sync server additions: · 212cc781
      Leigh B. Stoller authored
      * Parser: Added new tb command to set the name of the sync server:
      
      	tb-set-sync-server <node>
      
        This initializes the sync_server slot of the experiment entry to the
        *vname* of the node that should run the sync server for that
        experiment. In other words, the sync server is per-experiment, runs
        on a node in the experiment, and the user gets to chose which node
        it runs on.
      
      * tmcd and client side setup. Added new syncserver command which
        returns the name of the syncserver and whether the requesting node
        is the lucky one to run the daemon:
      
          SYNCSERVER SERVER='nodeG.syncserver.testbed.emulab.net' ISSERVER=1
      
        The name of the syncserver is written to /var/emulab/boot/syncserver
        on the nodes so that clients can easily figure out where the server
        is.
      
        Aside: The ready bits are now ignored (no DB accesses are made) for
        virtual nodes; they are forced to use the new sync server.
      
      * New os/syncd directory containing the daemon and the client. The
        daemon is pretty simple. It waits for TCP (and UDP, although that
        path is not complete yet) connections, and reads in a little
        structure that gives the name of the "barrier" to wait for, and an
        optional count of clients in the group (this would be used by the
        "master" who initializes barriers for clients). The socket is saved
        (no reply is made, so the client is blocked) until the count reaches
        zero. Then all clients are released by writting back to the
        sockets, and the sockets are closed. Obviously, the number of
        clients is limited by the numbed of FDs (open sockets), hence the
        need for a UDP variant, but that will take more work.
      
        The client has a simple command line interface:
      
          usage: emulab-sync [options]
          -n <name>         Optional barrier name; must be less than 64 bytes long
          -d                Turn on debugging
          -s server         Specify a sync server to connect to
          -p portnum        Specify a port number to connect to
          -i count          Initialize named barrier to count waiters
          -u                Use UDP instead of TCP
      
          The client figures out the server by looking for the file created
          above by libsetup (/var/emulab/boot/syncserver). If you do not
          specify a barrier "name", it uses an internal default. Yes, the
          server can handle multiple barriers (differently named of course)
          at once (non-overlapping clients obviously).
      
          Clients can wait before a barrier in "initialized." The count on
          the barrier just goes negative until someone initializes the
          barrier using the -i option, which increments the count by the
          count. Therefore, the master does not have to arrange to get there
          "first." As an example, consider a master and one client:
      
      	nodeA> /usr/local/etc/emulab/emulab-sync -n mybarrier
      	nodeB> /usr/local/etc/emulab/emulab-sync -n mybarrier -i 1
      
          Node A waits until Node B initializes the barrier (gives it a
          count).  The count is the number of *waiters*, not including the
          master. The master is also blocked until all of the waiters have
          checked in.
      
          I have not made an provision for timeouts or crashed clients. Lets
          see how it goes.
      212cc781
  25. 29 Jul, 2003 1 commit
  26. 02 Jul, 2003 1 commit
    • Leigh B. Stoller's avatar
      Remove some Linux FS hardwired restrictions and assumptions about the · 2ddfb904
      Leigh B. Stoller authored
      block size. I was only allowing 4K block size FS's, and assumed that
      the group descriptor map was in the second block (not true when the
      blocksize if 1K), and that the number of blocks in a descriptor as 8
      times the block size. Also moved some sanity checks around, and
      cleaned up a couple of seeks. Now works on the baleep image, which is
      where we first noticed the problem.
      2ddfb904
  27. 01 Jul, 2003 1 commit
  28. 23 Jun, 2003 1 commit
    • Mike Hibler's avatar
      A couple of new options inspired by home use: · 39b1f04c
      Mike Hibler authored
      1. From the "Thank you DOS for all your stupid limits" department
         we have an enhancement to the -I option to ignore a slice.
         The slice number can be followed by a BSD partition letter
         (e.g., "-I1g") to ignore a BSD partition within a slice.
         Ditto for the -R options which forces raw compression.
      
      2. From the "Gee, I shoulda taken that grinding noise more seriously"
         department we have the -X option to force imagezip to try e(X)tremely
         hard (yeah, yeah, its the only letter I could come up with) to read
         the data off the disk, even faking (zero) data if a read fails after
         10 tries.
      39b1f04c