1. 22 Jul, 2009 10 commits
  2. 02 Jul, 2009 7 commits
  3. 15 Jun, 2009 1 commit
  4. 11 Jun, 2009 4 commits
  5. 10 Jun, 2009 18 commits
    • Shin Hong's avatar
      Btrfs: init worker struct fields before kthread-run · fd0fb038
      Shin Hong authored
      This patch fixes a bug which may result race condition
      between btrfs_start_workers() and worker_loop().
      btrfs_start_workers() executed in a parent thread writes
      on workers->worker and worker_loop() in a child thread
      reads workers->worker. However, there is no synchronization
      enforcing the order of two operations.
      This patch makes btrfs_start_workers() fill workers->worker
      before it starts a child thread with worker_loop()
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Hisashi Hifumi's avatar
      Btrfs: pin buffers during write_dev_supers · 4eedeb75
      Hisashi Hifumi authored
      write_dev_supers is called in sequence.  First is it called with wait == 0,
      which starts IO on all of the super blocks for a given device.  Then it is
      called with wait == 1 to make sure they all reach the disk.
      It doesn't currently pin the buffers between the two calls, and it also
      assumes the buffers won't go away between the two calls, leading to
      an oops if the VM manages to free the buffers in the middle of the sync.
      This fixes that assumption and updates the code to return an error if things
      are not up to date when the wait == 1 run is done.
      Signed-off-by: default avatarHisashi Hifumi <hifumi.hisashi@oss.ntt.co.jp>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: avoid races between super writeout and device list updates · e5e9a520
      Chris Mason authored
      On multi-device filesystems, btrfs writes supers to all of the devices
      before considering a sync complete.  There wasn't any additional
      locking between super writeout and the device list management code
      because device management was done inside a transaction and
      super writeout only happened  with no transation writers running.
      With the btrfs fsync log and other async transaction updates, this
      has been racey for some time.  This adds a mutex to protect
      the device list.  The existing volume mutex could not be reused due to
      transaction lock ordering requirements.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Al Viro's avatar
      Fix btrfs when ACLs are configured out · 7df336ec
      Al Viro authored
      ... otherwise generic_permission() will allow *anything* for all
      files you don't own and that have some group permissions.
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Hisashi Hifumi's avatar
      Btrfs: fdatasync should skip metadata writeout · 524724ed
      Hisashi Hifumi authored
      In btrfs, fdatasync and fsync are identical, but
      fdatasync should skip committing transaction when
      inode->i_state is set just I_DIRTY_SYNC and this indicates
      only atime or/and mtime updates.
      Following patch improves fdatasync throughput.
      --file-block-size=4K --file-total-size=16G --file-test-mode=rndwr
      --file-fsync-mode=fdatasync run
      Test execution summary:
          total time:                          1980.6540s
          total number of events:              10001
          total time taken by event execution: 1192.9804
          per-request statistics:
               min:                            0.0000s
               avg:                            0.1193s
               max:                            15.3720s
               approx.  95 percentile:         0.7257s
      Threads fairness:
          events (avg/stddev):           625.0625/151.32
          execution time (avg/stddev):   74.5613/9.46
      Test execution summary:
          total time:                          1695.9118s
          total number of events:              10000
          total time taken by event execution: 871.3214
          per-request statistics:
               min:                            0.0000s
               avg:                            0.0871s
               max:                            10.4644s
               approx.  95 percentile:         0.4787s
      Threads fairness:
          events (avg/stddev):           625.0000/131.86
          execution time (avg/stddev):   54.4576/8.98
      Signed-off-by: default avatarHisashi Hifumi <hifumi.hisashi@oss.ntt.co.jp>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • David Woodhouse's avatar
      Btrfs: remove crc32c.h and use libcrc32c directly. · 163e783e
      David Woodhouse authored
      There's no need to preserve this abstraction; it used to let us use
      hardware crc32c support directly, but libcrc32c is already doing that for us
      through the crypto API -- so we're already using the Intel crc32c
      acceleration where appropriate.
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Christoph Hellwig's avatar
      Btrfs: implement FS_IOC_GETFLAGS/SETFLAGS/GETVERSION · 6cbff00f
      Christoph Hellwig authored
      Add support for the standard attributes set via chattr and read via
      lsattr.  Currently we store the attributes in the flags value in
      the btrfs inode, but I wonder whether we should split it into two so
      that we don't have to keep converting between the two formats.
      Remove the btrfs_clear_flag/btrfs_set_flag/btrfs_test_flag macros
      as they were confusing the existing code and got in the way of the
      new additions.
      Also add the FS_IOC_GETVERSION ioctl for getting i_generation as it's
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: autodetect SSD devices · c289811c
      Chris Mason authored
      During mount, btrfs will check the queue nonrot flag
      for all the devices found in the FS.  If they are all
      non-rotating, SSD mode is enabled by default.
      If the FS was mounted with -o nossd, the non-rotating
      flag is ignored.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: add mount -o ssd_spread to spread allocations out · 451d7585
      Chris Mason authored
      Some SSDs perform best when reusing block numbers often, while
      others perform much better when clustering strictly allocates
      big chunks of unused space.
      The default mount -o ssd will find rough groupings of blocks
      where there are a bunch of free blocks that might have some
      allocated blocks mixed in.
      mount -o ssd_spread will make sure there are no allocated blocks
      mixed in.  It should perform better on lower end SSDs.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: avoid allocation clusters that are too spread out · c6044801
      Chris Mason authored
      In SSD mode for data, and all the time for metadata the allocator
      will try to find a cluster of nearby blocks for allocations.  This
      commit adds extra checks to make sure that each free block in the
      cluster is close to the last one.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: Add mount -o nossd · 3b30c22f
      Chris Mason authored
      This allows you to turn off the ssd mode via remount.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: avoid IO stalls behind congested devices in a multi-device FS · d644d8a1
      Chris Mason authored
      The btrfs IO submission threads try to service a bunch of devices with a small
      number of threads.  They do a congestion check to try and avoid waiting
      on requests for a busy device.
      The checks make sure we've sent a few requests down to a given device just so
      that we aren't bouncing between busy devices without actually sending down
      any IO.  The counter used to decide if we can switch to the next device
      is somewhat overloaded.  It is also being used to decide if we've done
      a good batch of requests between the WRITE_SYNC or regular priority lists.
      It may get reset to zero often, leaving us hammering on a busy device
      instead of moving on to another disk.
      This commit adds a new counter for the number of bios sent while
      servicing a device.  It doesn't get reset or fiddled with.  On
      multi-device filesystems, this fixes IO stalls in streaming
      write workloads.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: don't allow WRITE_SYNC bios to starve out regular writes · d84275c9
      Chris Mason authored
      Btrfs uses dedicated threads to submit bios when checksumming is on,
      which allows us to make sure the threads dedicated to checksumming don't get
      stuck waiting for requests.  For each btrfs device, there are
      two lists of bios.  One list is for WRITE_SYNC bios and the other
      is for regular priority bios.
      The IO submission threads used to process all of the WRITE_SYNC bios first and
      then switch to the regular bios.  This commit makes sure we don't completely
      starve the regular bios by rotating between the two lists.
      WRITE_SYNC bios are still favored 2:1 over the regular bios, and this tries
      to run in batches to avoid seeking.  Benchmarking shows this eliminates
      stalls during streaming buffered writes on both multi-device and
      single device filesystems.
      If the regular bios starve, the system can end up with a large amount of ram
      pinned down in writeback pages.  If we are a little more fair between the two
      classes, we're able to keep throughput up and make progress on the bulk of
      our dirty ram.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: fix metadata dirty throttling limits · 585ad2c3
      Chris Mason authored
      Once a metadata block has been written, it must be recowed, so the
      btrfs dirty balancing call has a check to make sure a fair amount of metadata
      was actually dirty before it started writing it back to disk.
      A previous commit had changed the dirty tracking for metadata without
      updating the btrfs dirty balancing checks.  This commit switches it
      to use the correct counter.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: reduce mount -o ssd CPU usage · 2c943de6
      Chris Mason authored
      The block allocator in SSD mode will try to find groups of free blocks
      that are close together.  This commit makes it loop less on a given
      group size before bumping it.
      The end result is that we are less likely to fill small holes in the
      available free space, but we don't waste as much CPU building the
      large cluster used by ssd mode.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: balance btree more often · cfbb9308
      Chris Mason authored
      With the new back reference code, the cost of a balance has gone down
      in terms of the number of back reference updates done.  This commit
      makes us more aggressively balance leaves and nodes as they become
      less full.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: stop avoiding balancing at the end of the transaction. · b3612421
      Chris Mason authored
      When the delayed reference code was added, some checks were added
      to avoid extra balancing while the delayed references were being flushed.
      This made for less efficient btrees, but it reduced the chances of
      loops where no forward progress was made because the balances made
      more delayed ref updates.
      With the new dead root removal code and the mixed back references,
      the extent allocation tree is no longer using precise back refs, and
      the delayed reference updates don't carry the risk of looping forever
      anymore.  So, the balance avoidance is no longer required.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Yan Zheng's avatar
      Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE) · 5d4f98a2
      Yan Zheng authored
      This commit introduces a new kind of back reference for btrfs metadata.
      Once a filesystem has been mounted with this commit, IT WILL NO LONGER
      When a tree block in subvolume tree is cow'd, the reference counts of all
      extents it points to are increased by one.  At transaction commit time,
      the old root of the subvolume is recorded in a "dead root" data structure,
      and the btree it points to is later walked, dropping reference counts
      and freeing any blocks where the reference count goes to 0.
      The increments done during cow and decrements done after commit cancel out,
      and the walk is a very expensive way to go about freeing the blocks that
      are no longer referenced by the new btree root.  This commit reduces the
      transaction overhead by avoiding the need for dead root records.
      When a non-shared tree block is cow'd, we free the old block at once, and the
      new block inherits old block's references. When a tree block with reference
      count > 1 is cow'd, we increase the reference counts of all extents
      the new block points to by one, and decrease the old block's reference count by
      This dead tree avoidance code removes the need to modify the reference
      counts of lower level extents when a non-shared tree block is cow'd.
      But we still need to update back ref for all pointers in the block.
      This is because the location of the block is recorded in the back ref
      We can solve this by introducing a new type of back ref. The new
      back ref provides information about pointer's key, level and in which
      tree the pointer lives. This information allow us to find the pointer
      by searching the tree. The shortcoming of the new back ref is that it
      only works for pointers in tree blocks referenced by their owner trees.
      This is mostly a problem for snapshots, where resolving one of these
      fuzzy back references would be O(number_of_snapshots) and quite slow.
      The solution used here is to use the fuzzy back references in the common
      case where a given tree block is only referenced by one root,
      and use the full back references when multiple roots have a reference
      on a given block.
      This commit adds per subvolume red-black tree to keep trace of cached
      inodes. The red-black tree helps the balancing code to find cached
      inodes whose inode numbers within a given range.
      This commit improves the balancing code by introducing several data
      structures to keep the state of balancing. The most important one
      is the back ref cache. It caches how the upper level tree blocks are
      referenced. This greatly reduce the overhead of checking back ref.
      The improved balancing code scales significantly better with a large
      number of snapshots.
      This is a very large commit and was written in a number of
      pieces.  But, they depend heavily on the disk format change and were
      squashed together to make sure git bisect didn't end up in a
      bad state wrt space balancing or the format change.
      Signed-off-by: default avatarYan Zheng <zheng.yan@oracle.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>