1. 11 Sep, 2009 2 commits
    • Chris Mason's avatar
      Btrfs: use larger nr_to_write for larger extents · a97adc9f
      Chris Mason authored
      When btrfs fills a large delayed allocation extent, it is a good idea
      to try and convince the write_cache_pages caller to go ahead and
      write a good chunk of that extent.  The extra IO is basically free
      because we know it is contiguous.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: optimize set extent bit · 40431d6c
      Chris Mason authored
      The Btrfs set_extent_bit call currently searches the rbtree
      every time it needs to find more extent_state objects to fill
      the requested operation.
      This adds a simple test with rb_next to see if the next object
      in the tree was adjacent to the one we just found.  If so,
      we skip the search and just use the next object.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  2. 10 Jun, 2009 1 commit
  3. 27 Apr, 2009 1 commit
  4. 24 Apr, 2009 1 commit
  5. 20 Apr, 2009 3 commits
    • Chris Mason's avatar
      Btrfs: fix oops on page->mapping->host during writepage · 11c8349b
      Chris Mason authored
      The extent_io writepage call updates the writepage index in the inode
      as it makes progress.  But, it was doing the update after unlocking the page,
      which isn't legal because page->mapping can't be trusted once the page
      is unlocked.
      This lead to an oops, especially common with compression turned on.  The
      fix here is to update the writeback index before unlocking the page.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: add a priority queue to the async thread helpers · d313d7a3
      Chris Mason authored
      Btrfs is using WRITE_SYNC_PLUG to send down synchronous IOs with a
      higher priority.  But, the checksumming helper threads prevent it
      from being fully effective.
      There are two problems.  First, a big queue of pending checksumming
      will delay the synchronous IO behind other lower priority writes.  Second,
      the checksumming uses an ordered async work queue.  The ordering makes sure
      that IOs are sent to the block layer in the same order they are sent
      to the checksumming threads.  Usually this gives us less seeky IO.
      But, when we start mixing IO priorities, the lower priority IO can delay
      the higher priority IO.
      This patch solves both problems by adding a high priority list to the async
      helper threads, and a new btrfs_set_work_high_prio(), which is used
      to make put a new async work item onto the higher priority list.
      The ordering is still done on high priority IO, but all of the high
      priority bios are ordered separately from the low priority bios.  This
      ordering is purely an IO optimization, it is not involved in data
      or metadata integrity.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: use WRITE_SYNC for synchronous writes · ffbd517d
      Chris Mason authored
      Part of reducing fsync/O_SYNC/O_DIRECT latencies is using WRITE_SYNC for
      writes we plan on waiting on in the near future.  This patch
      mirrors recent changes in other filesystems and the generic code to
      use WRITE_SYNC when WB_SYNC_ALL is passed and to use WRITE_SYNC for
      other latency critical writes.
      Btrfs uses async worker threads for checksumming before the write is done,
      and then again to actually submit the bios.  The bio submission code just
      runs a per-device list of bios that need to be sent down the pipe.
      This list is split into low priority and high priority lists so the
      WRITE_SYNC IO happens first.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  6. 03 Apr, 2009 1 commit
  7. 24 Mar, 2009 1 commit
    • Chris Mason's avatar
      Btrfs: leave btree locks spinning more often · b9473439
      Chris Mason authored
      btrfs_mark_buffer dirty would set dirty bits in the extent_io tree
      for the buffers it was dirtying.  This may require a kmalloc and it
      was not atomic.  So, anyone who called btrfs_mark_buffer_dirty had to
      set any btree locks they were holding to blocking first.
      This commit changes dirty tracking for extent buffers to just use a flag
      in the extent buffer.  Now that we have one and only one extent buffer
      per page, this can be safely done without losing dirty bits along the way.
      This also introduces a path->leave_spinning flag that callers of
      btrfs_search_slot can use to indicate they will properly deal with a
      path returned where all the locks are spinning instead of blocking.
      Many of the btree search callers now expect spinning paths,
      resulting in better btree concurrency overall.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  8. 12 Feb, 2009 1 commit
  9. 04 Feb, 2009 3 commits
    • Chris Mason's avatar
      Btrfs: don't return congestion in write_cache_pages as often · 9b0d3ace
      Chris Mason authored
      On fast devices that go from congested to uncongested very quickly, pdflush
      is waiting too often in congestion_wait, and the FS is backing off to
      easily in write_cache_pages.
      For now, fix this on the btrfs side by only checking congestion after
      some bios have already gone down.  Longer term a real fix is needed
      for pdflush, but that is a larger project.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: Change btree locking to use explicit blocking points · b4ce94de
      Chris Mason authored
      Most of the btrfs metadata operations can be protected by a spinlock,
      but some operations still need to schedule.
      So far, btrfs has been using a mutex along with a trylock loop,
      most of the time it is able to avoid going for the full mutex, so
      the trylock loop is a big performance gain.
      This commit is step one for getting rid of the blocking locks entirely.
      btrfs_tree_lock takes a spinlock, and the code explicitly switches
      to a blocking lock when it starts an operation that can schedule.
      We'll be able get rid of the blocking locks in smaller pieces over time.
      Tracing allows us to find the most common cause of blocking, so we
      can start with the hot spots first.
      The basic idea is:
      btrfs_tree_lock() returns with the spin lock held
      btrfs_set_lock_blocking() sets the EXTENT_BUFFER_BLOCKING bit in
      the extent buffer flags, and then drops the spin lock.  The buffer is
      still considered locked by all of the btrfs code.
      If btrfs_tree_lock gets the spinlock but finds the blocking bit set, it drops
      the spin lock and waits on a wait queue for the blocking bit to go away.
      Much of the code that needs to set the blocking bit finishes without actually
      blocking a good percentage of the time.  So, an adaptive spin is still
      used against the blocking bit to avoid very high context switch rates.
      btrfs_clear_lock_blocking() clears the blocking bit and returns
      with the spinlock held again.
      btrfs_tree_unlock() can be called on either blocking or spinning locks,
      it does the right thing based on the blocking bit.
      ctree.c has a helper function to set/clear all the locked buffers in a
      path as blocking.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: disable leak debugging checks in extent_io.c · 3935127c
      Chris Mason authored
      extent_io.c has debugging code to report and free leaked extent_state
      and extent_buffer objects at rmmod time.  This helps track down
      leaks and it saves you from rebooting just to properly remove the
      kmem_cache object.
      But, the code runs under a fairly expensive spinlock and the checks to
      see if it is currently enabled are not entirely consistent.  Some use
      #ifdef and some #if.
      This changes everything to #if and disables the leak checking.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  10. 21 Jan, 2009 2 commits
  11. 05 Jan, 2009 3 commits
  12. 17 Dec, 2008 1 commit
    • Chris Mason's avatar
      Btrfs: shift all end_io work to thread pools · cad321ad
      Chris Mason authored
      bio_end_io for reads without checksumming on and btree writes were
      happening without using async thread pools.  This means the extent_io.c
      code had to use spin_lock_irq and friends on the rb tree locks for
      extent state.
      There were some irq safe vs unsafe lock inversions between the delallock
      lock and the extent state locks.  This patch gets rid of them by moving
      all end_io code into the thread pools.
      To avoid contention and deadlocks between the data end_io processing and the
      metadata end_io processing yet another thread pool is added to finish
      off metadata writes.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  13. 08 Dec, 2008 2 commits
    • Chris Mason's avatar
      Btrfs: Use map_private_extent_buffer during generic_bin_search · 934d375b
      Chris Mason authored
      It is possible that generic_bin_search will be called on a tree block
      that has not been locked.  This happens because cache_block_block skips
      locking on the tree blocks.
      Since the tree block isn't locked, we aren't allowed to change
      the extent_buffer->map_token field.  Using map_private_extent_buffer
      avoids any changes to the internal extent buffer fields.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: move data checksumming into a dedicated tree · d20f7043
      Chris Mason authored
      Btrfs stores checksums for each data block.  Until now, they have
      been stored in the subvolume trees, indexed by the inode that is
      referencing the data block.  This means that when we read the inode,
      we've probably read in at least some checksums as well.
      But, this has a few problems:
      * The checksums are indexed by logical offset in the file.  When
      compression is on, this means we have to do the expensive checksumming
      on the uncompressed data.  It would be faster if we could checksum
      the compressed data instead.
      * If we implement encryption, we'll be checksumming the plain text and
      storing that on disk.  This is significantly less secure.
      * For either compression or encryption, we have to get the plain text
      back before we can verify the checksum as correct.  This makes the raid
      layer balancing and extent moving much more expensive.
      * It makes the front end caching code more complex, as we have touch
      the subvolume and inodes as we cache extents.
      * There is potentitally one copy of the checksum in each subvolume
      referencing an extent.
      The solution used here is to store the extent checksums in a dedicated
      tree.  This allows us to index the checksums by phyiscal extent
      start and length.  It means:
      * The checksum is against the data stored on disk, after any compression
      or encryption is done.
      * The checksum is stored in a central location, and can be verified without
      following back references, or reading inodes.
      This makes compression significantly faster by reducing the amount of
      data that needs to be checksummed.  It will also allow much faster
      raid management code in general.
      The checksums are indexed by a key with a fixed objectid (a magic value
      in ctree.h) and offset set to the starting byte of the extent.  This
      allows us to copy the checksum items into the fsync log tree directly (or
      any other tree), without having to invent a second format for them.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  14. 02 Dec, 2008 2 commits
  15. 20 Nov, 2008 1 commit
  16. 19 Nov, 2008 2 commits
    • Chris Mason's avatar
      Btrfs: Fixes for 2.6.28-rc API changes · 15916de8
      Chris Mason authored
      * open/close_bdev_excl -> open/close_bdev_exclusive
      * blkdev_issue_discard takes a GFP mask now
      * Fix blkdev_issue_discard usage now that it is enabled
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: Avoid writeback stalls · d2c3f4f6
      Chris Mason authored
      While building large bios in writepages, btrfs may end up waiting
      for other page writeback to finish if WB_SYNC_ALL is used.
      While it is waiting, the bio it is building has a number of pages with the
      writeback bit set and they aren't getting to the disk any time soon.  This
      lowers the latencies of writeback in general by sending down the bio being
      built before waiting for other pages.
      The bio submission code tries to limit the total number of async bios in
      flight by waiting when we're over a certain number of async bios.  But,
      the waits are happening while writepages is building bios, and this can easily
      lead to stalls and other problems for people calling wait_on_page_writeback.
      The current fix is to let the congestion tests take care of waiting.
      sync() and others make sure to drain the current async requests to make
      sure that everything that was pending when the sync was started really get
      to disk.  The code would drain pending requests both before and after
      submitting a new request.
      But, if one of the requests is waiting for page writeback to finish,
      the draining waits might block that page writeback.  This changes the
      draining code to only wait after submitting the bio being processed.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  17. 11 Nov, 2008 1 commit
  18. 10 Nov, 2008 3 commits
  19. 06 Nov, 2008 2 commits
    • Chris Mason's avatar
      Btrfs: enforce metadata allocation clustering · 3b7885bf
      Chris Mason authored
      The allocator uses the last allocation as a starting point for metadata
      allocations, and tries to allocate in clusters of at least 256k.
      If the search for a free block fails to find the expected block, this patch
      forces a new cluster to be found in the free list.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    • Chris Mason's avatar
      Btrfs: Optimize compressed writeback and reads · 771ed689
      Chris Mason authored
      When reading compressed extents, try to put pages into the page cache
      for any pages covered by the compressed extent that readpages didn't already
      Add an async work queue to handle transformations at delayed allocation processing
      time.  Right now this is just compression.  The workflow is:
      1) Find offsets in the file marked for delayed allocation
      2) Lock the pages
      3) Lock the state bits
      4) Call the async delalloc code
      The async delalloc code clears the state lock bits and delalloc bits.  It is
      important this happens before the range goes into the work queue because
      otherwise it might deadlock with other work queue items that try to lock
      those extent bits.
      The file pages are compressed, and if the compression doesn't work the
      pages are written back directly.
      An ordered work queue is used to make sure the inodes are written in the same
      order that pdflush or writepages sent them down.
      This changes extent_write_cache_pages to let the writepage function
      update the wbc nr_written count.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  20. 31 Oct, 2008 1 commit
    • Chris Mason's avatar
      Btrfs: Compression corner fixes · 70b99e69
      Chris Mason authored
      Make sure we keep page->mapping NULL on the pages we're getting
      via alloc_page.  It gets set so a few of the callbacks can do the right
      thing, but in general these pages don't have a mapping.
      Don't try to truncate compressed inline items in btrfs_drop_extents.
      The whole compressed item must be preserved.
      Don't try to create multipage inline compressed items.  When we try to
      overwrite just the first page of the file, we would have to read in and recow
      all the pages after it in the same compressed inline items.  For now, only
      create single page inline items.
      Make sure we lock pages in the correct order during delalloc.  The
      search into the state tree for delalloc bytes can return bytes before
      the page we already have locked.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  21. 30 Oct, 2008 2 commits
    • Yan Zheng's avatar
      Btrfs: Add fallocate support v2 · d899e052
      Yan Zheng authored
      This patch updates btrfs-progs for fallocate support.
      fallocate is a little different in Btrfs because we need to tell the
      COW system that a given preallocated extent doesn't need to be
      cow'd as long as there are no snapshots of it.  This leverages the
      -o nodatacow checks.
      Signed-off-by: default avatarYan Zheng <zheng.yan@oracle.com>
    • Yan Zheng's avatar
      Btrfs: Fix bookend extent race v2 · 6643558d
      Yan Zheng authored
      When dropping middle part of an extent, btrfs_drop_extents truncates
      the extent at first, then inserts a bookend extent.
      Since truncation and insertion can't be done atomically, there is a small
      period that the bookend extent isn't in the tree. This causes problem for
      functions that search the tree for file extent item. The way to fix this is
      lock the range of the bookend extent before truncation.
      Signed-off-by: default avatarYan Zheng <zheng.yan@oracle.com>
  22. 29 Oct, 2008 2 commits
    • Josef Bacik's avatar
      Btrfs: nuke fs wide allocation mutex V2 · 25179201
      Josef Bacik authored
      This patch removes the giant fs_info->alloc_mutex and replaces it with a bunch
      of little locks.
      There is now a pinned_mutex, which is used when messing with the pinned_extents
      extent io tree, and the extent_ins_mutex which is used with the pending_del and
      extent_ins extent io trees.
      The locking for the extent tree stuff was inspired by a patch that Yan Zheng
      wrote to fix a race condition, I cleaned it up some and changed the locking
      around a little bit, but the idea remains the same.  Basically instead of
      holding the extent_ins_mutex throughout the processing of an extent on the
      extent_ins or pending_del trees, we just hold it while we're searching and when
      we clear the bits on those trees, and lock the extent for the duration of the
      operations on the extent.
      Also to keep from getting hung up waiting to lock an extent, I've added a
      try_lock_extent so if we cannot lock the extent, move on to the next one in the
      tree and we'll come back to that one.  I have tested this heavily and it does
      not appear to break anything.  This has to be applied on top of my
      find_free_extent redo patch.
      I tested this patch on top of Yan's space reblancing code and it worked fine.
      The only thing that has changed since the last version is I pulled out all my
      debugging stuff, apparently I forgot to run guilt refresh before I sent the
      last patch out.  Thank you,
      Signed-off-by: default avatarJosef Bacik <jbacik@redhat.com>
    • Chris Mason's avatar
      Btrfs: Add zlib compression support · c8b97818
      Chris Mason authored
      This is a large change for adding compression on reading and writing,
      both for inline and regular extents.  It does some fairly large
      surgery to the writeback paths.
      Compression is off by default and enabled by mount -o compress.  Even
      when the -o compress mount option is not used, it is possible to read
      compressed extents off the disk.
      If compression for a given set of pages fails to make them smaller, the
      file is flagged to avoid future compression attempts later.
      * While finding delalloc extents, the pages are locked before being sent down
      to the delalloc handler.  This allows the delalloc handler to do complex things
      such as cleaning the pages, marking them writeback and starting IO on their
      * Inline extents are inserted at delalloc time now.  This allows us to compress
      the data before inserting the inline extent, and it allows us to insert
      an inline extent that spans multiple pages.
      * All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
      are changed to record both an in-memory size and an on disk size, as well
      as a flag for compression.
      From a disk format point of view, the extent pointers in the file are changed
      to record the on disk size of a given extent and some encoding flags.
      Space in the disk format is allocated for compression encoding, as well
      as encryption and a generic 'other' field.  Neither the encryption or the
      'other' field are currently used.
      In order to limit the amount of data read for a single random read in the
      file, the size of a compressed extent is limited to 128k.  This is a
      software only limit, the disk format supports u64 sized compressed extents.
      In order to limit the ram consumed while processing extents, the uncompressed
      size of a compressed extent is limited to 256k.  This is a software only limit
      and will be subject to tuning later.
      Checksumming is still done on compressed extents, and it is done on the
      uncompressed version of the data.  This way additional encodings can be
      layered on without having to figure out which encoding to checksum.
      Compression happens at delalloc time, which is basically singled threaded because
      it is usually done by a single pdflush thread.  This makes it tricky to
      spread the compression load across all the cpus on the box.  We'll have to
      look at parallel pdflush walks of dirty inodes at a later time.
      Decompression is hooked into readpages and it does spread across CPUs nicely.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  23. 29 Sep, 2008 1 commit
    • Chris Mason's avatar
      Btrfs: add and improve comments · d352ac68
      Chris Mason authored
      This improves the comments at the top of many functions.  It didn't
      dive into the guts of functions because I was trying to
      avoid merging problems with the new allocator and back reference work.
      extent-tree.c and volumes.c were both skipped, and there is definitely
      more work todo in cleaning and commenting the code.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
  24. 26 Sep, 2008 1 commit
    • Zheng Yan's avatar
      Btrfs: extent_map and data=ordered fixes for space balancing · 5b21f2ed
      Zheng Yan authored
      * Add an EXTENT_BOUNDARY state bit to keep the writepage code
      from merging data extents that are in the process of being
      relocated.  This allows us to do accounting for them properly.
      * The balancing code relocates data extents indepdent of the underlying
      inode.  The extent_map code was modified to properly account for
      things moving around (invalidating extent_map caches in the inode).
      * Don't take the drop_mutex in the create_subvol ioctl.  It isn't
      * Fix walking of the ordered extent list to avoid races with sys_unlink
      * Change the lock ordering rules.  Transaction start goes outside
      the drop_mutex.  This allows btrfs_commit_transaction to directly
      drop the relocation trees.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>