1. 25 Jan, 2008 11 commits
    • Steven Whitehouse's avatar
      [GFS2] Allow page migration for writeback and ordered pages · e5d9dc27
      Steven Whitehouse authored
      To improve performance on NUMA, we use the VM's standard page
      migration for writeback and ordered pages. Probably we could
      also do the same for journaled data, but that would need a
      careful audit of the code, so will be the subject of a later
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Bob Peterson's avatar
      [GFS2] Remove function gfs2_get_block · e9e1ef2b
      Bob Peterson authored
      This patch is just a cleanup.  Function gfs2_get_block() just calls
      function gfs2_block_map reversing the last two parameters.  By
      reversing the parameters, gfs2_block_map() may be called directly
      and function gfs2_get_block may be eliminated altogether.
      Since this function is done for every block operation,
      this streamlines the code and makes it a little bit more efficient.
      Signed-off-by: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Use correct include file in ops_address.c · 47e83b50
      Steven Whitehouse authored
      Something changed in the upstream kernel, and it needs this
      one-liner to allow ops_address.c to build.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Don't hold page lock when starting transaction · c41d4f09
      Steven Whitehouse authored
      This is an addendum to the new AOPs work which moves the point
      at which we take the page lock so that we don't get it until
      the last possible moment. This resolves a conflict between
      starting transactions and the page lock.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Add writepages for GFS2 jdata · b8e7cbb6
      Steven Whitehouse authored
      This patch resolves a lock ordering issue where we had been getting
      a transaction lock in the wrong order with respect to the page lock.
      By using writepages rather than just writepage, it is then possible
      to start a transaction before locking the page, and thus matching the
      locking order elsewhere in the code.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Split gfs2_writepage into three cases · 9ff8ec32
      Steven Whitehouse authored
      This patch splits gfs2_writepage into separate functions for each of
      the three cases: writeback, ordered and journalled. As a result
      it becomes a lot easier to see what each one is doing. The common
      code is moved into gfs2_writepage_common.
      This fixes a performance bug where we were doing more work than
      strictly required in the ordered write case.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Introduce gfs2_set_aops() · 5561093e
      Steven Whitehouse authored
      Just like ext3 we now have three sets of address space operations
      to cover the cases of writeback, ordered and journalled data
      writes. This means that the individual operations can now become
      less complicated as we are able to remove some of the tests for
      file data mode from the code.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Add gfs2_is_writeback() · bf36a713
      Steven Whitehouse authored
      This adds a function "gfs2_is_writeback()" along the lines of the
      existing "gfs2_is_jdata()" in order to clean up the code and make
      the various tests for the inode mode more obvious. It also fixes
      the PageChecked() logic where we were resetting the flag too early
      in the case of an error path.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Remove useless i_cache from inodes · f91a0d3e
      Steven Whitehouse authored
      The i_cache was designed to keep references to the indirect blocks
      used during block mapping so that they didn't have to be looked
      up continually. The idea failed because there are too many places
      where the i_cache needs to be freed, and this has in the past been
      the cause of many bugs.
      In addition there was no performance benefit being gained since the
      disk blocks in question were cached anyway. So this patch removes
      it in order to simplify the code to prepare for other changes which
      would otherwise have had to add further support for this feature.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Use ->page_mkwrite() for mmap() · 3cc3f710
      Steven Whitehouse authored
      This cleans up the mmap() code path for GFS2 by implementing the
      page_mkwrite function for GFS2. We are thus able to use the
      generic filemap_fault function for our ->fault() implementation.
      This now means that shared writable mappings will be much more
      efficiently shared across the cluster if there is a reasonable
      proportion of read activity (the greater proportion, the better).
      As a side effect, it also reduces the size of the code, removes
      special cases from readpage and readpages, and makes the code
      path easier to follow.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Clean up internal read function · 51ff87bd
      Steven Whitehouse authored
      As requested by Christoph, this patch cleans up GFS2's internal
      read function so that it no longer uses the do_generic_mapping_read
      function. This function is obsolete and GFS2 is the last user of it.
      As a side effect the internal read code gets smaller and easier
      to read and gfs2_readpage is split into two. One function has the locking
      and the other function has the rest of the logic.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
      Cc: Christoph Hellwig <hch@infradead.org>
  2. 16 Oct, 2007 1 commit
  3. 10 Oct, 2007 5 commits
    • Steven Whitehouse's avatar
    • Wendy Cheng's avatar
      [GFS2] Data corruption fix · de986e85
      Wendy Cheng authored
      * GFS2 has been using i_cache array to store its indirect meta blocks.
      Its flush routine doesn't correctly clean up all the entries. The
      problem would show while multiple nodes do simultaneous writes to the
      same file. Upon glock exclusive lock transfer, if the file is a sparse
      file with large file size where the indirect meta blocks span multiple
      array entries with "zero" entries in between. The flush routine
      prematurely stops the flushing that leaves old (stale) entries around.
      This leads to several nasty issues, including data corruption.
      * Fix gfs2_get_block_noalloc checking to correctly return EIO upon
      unmapped buffer.
      Signed-off-by: default avatarWendy Cheng <wcheng@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Clean up journaled data writing · 16615be1
      Steven Whitehouse authored
      This patch cleans up the code for writing journaled data into the log.
      It also removes the need to allocate a small "tag" structure for each
      block written into the log. Instead we just keep count of the outstanding
      I/O so that we can be sure that its all been written at the correct time.
      Another result of this patch is that a number of ll_rw_block() calls
      have become submit_bh() calls, closing some races at the same time.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Clean up ordered write code · d7b616e2
      Steven Whitehouse authored
      The following patch removes the ordered write processing from
      databuf_lo_before_commit() and moves it to log.c. This has the effect of
      greatly simplyfying databuf_lo_before_commit() and well as potentially
      making the ordered write code more efficient.
      As a side effect of this, its now possible to remove ordered buffers
      from the ordered buffer list at any time, so we now make use of this in
      invalidatepage and releasepage to ensure timely release of these
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Clean up invalidatepage/releasepage · bb3b0e3d
      Steven Whitehouse authored
      This patch fixes some bugs relating to journaled data files by cleaning
      up the gfs2_invalidatepage() and gfs2_releasepage() functions. We now
      never block during gfs2_releasepage(), instead we always either release
      or refuse to release depending on the status of the buffers.
      This fixes Red Hat bugzillas #248969 and #252392.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
      Cc: Bob Peterson <rpeterso@redhat.com>
  4. 14 Aug, 2007 1 commit
  5. 19 Jul, 2007 1 commit
    • Nick Piggin's avatar
      mm: merge populate and nopage into fault (fixes nonlinear) · 54cb8821
      Nick Piggin authored
      Nonlinear mappings are (AFAIKS) simply a virtual memory concept that encodes
      the virtual address -> file offset differently from linear mappings.
      ->populate is a layering violation because the filesystem/pagecache code
      should need to know anything about the virtual memory mapping.  The hitch here
      is that the ->nopage handler didn't pass down enough information (ie.  pgoff).
       But it is more logical to pass pgoff rather than have the ->nopage function
      calculate it itself anyway (because that's a similar layering violation).
      Having the populate handler install the pte itself is likewise a nasty thing
      to be doing.
      This patch introduces a new fault handler that replaces ->nopage and
      ->populate and (later) ->nopfn.  Most of the old mechanism is still in place
      so there is a lot of duplication and nice cleanups that can be removed if
      everyone switches over.
      The rationale for doing this in the first place is that nonlinear mappings are
      subject to the pagefault vs invalidate/truncate race too, and it seemed stupid
      to duplicate the synchronisation logic rather than just consolidate the two.
      After this patch, MAP_NONBLOCK no longer sets up ptes for pages present in
      pagecache.  Seems like a fringe functionality anyway.
      NOPAGE_REFAULT is removed.  This should be implemented with ->fault, and no
      users have hit mainline yet.
      [akpm@linux-foundation.org: cleanup]
      [randy.dunlap@oracle.com: doc. fixes for readahead]
      [akpm@linux-foundation.org: build fix]
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      Signed-off-by: default avatarRandy Dunlap <randy.dunlap@oracle.com>
      Cc: Mark Fasheh <mark.fasheh@oracle.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
  6. 09 Jul, 2007 7 commits
    • Steven Whitehouse's avatar
      [GFS2] Use zero_user_page() in stuffed_readpage() · 2840501a
      Steven Whitehouse authored
      As suggested by Robert P. J. Day <rpjday@mindspring.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
      Cc: Robert P. J. Day <rpjday@mindspring.com>
    • Robert Peterson's avatar
      [GFS2] Journaled file write/unstuff bug · 8fb68595
      Robert Peterson authored
      This patch is for bugzilla bug 283162, which uncovered a number of
      bugs pertaining to writing to files that have the journaled bit on.
      These bugs happen most often when writing to the meta_fs because
      the files are always journaled.  So operations like gfs2_grow were
      particularly vulnerable, although many of the problems could be
      recreated with normal files after setting the journaled bit on.
      The problems fixed are:
      -GFS2 wasn't ever writing unstuffed journaled data blocks to their
       in-place location on disk. Now it does.
      -If you unmounted too quickly after doing IO to a journaled file,
       GFS2 was crashing because you would discard a buffer whose bufdata
       was still on the active items list.  GFS2 now deals with this
      -GFS2 was losing track of the bufdata for journaled data blocks,
       and it wasn't getting freed, causing an error when you tried to
       unmount the module.  GFS2 now frees all the bufdata structures.
      -There was a memory corruption occurring because GFS2 wrote
       twice as many log entries for journaled buffers.
      -It was occasionally trying to write journal headers in buffers
       that weren't currently mapped.
      Signed-off-by: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarBenjamin Marzinski <bmarzins@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Benjamin Marzinski's avatar
      [GFS2] fix jdata issues · ddf4b426
      Benjamin Marzinski authored
      This is a patch for the first three issues of RHBZ #238162
      The first issue is that when you allocate a new page for a file, it will not
      start off uptodate. This makes sense, since you haven't written anything to that
      part of the file yet.  Unfortunately, gfs2_pin() checks to make sure that the
      buffers are uptodate.  The solution to this is to mark the buffers uptodate in
      gfs2_commit_write(), after they have been zeroed out and have the data written
      into them.  I'm pretty confident with this fix, although it's not completely
      obvious that there is no problem with marking the buffers uptodate here.
      The second issue is simply that you can try to pin a data buffer that is already
      on the incore log, and thus, already pinned. This patch checks to see if this
      buffer is already on the log, and exits databuf_lo_add() if it is, just like
      buf_lo_add() does.
      The third issue is that gfs2_log_flush() doesn't do it's block accounting
      correctly.  Both metadata and journaled data are logged, but gfs2_log_flush()
      only compares the number of metadata blocks with the number of blocks to commit
      to the ondisk journal.  This patch also counts the journaled data blocks.
      Signed-off-by: default avatarBenjamin Marzinski <bmarzins@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Clean up inode number handling · dbb7cae2
      Steven Whitehouse authored
      This patch cleans up the inode number handling code. The main difference
      is that instead of looking up the inodes using a struct gfs2_inum_host
      we now use just the no_addr member of this structure. The tests relating
      to no_formal_ino can then be done by the calling code. This has
      advantages in that we want to do different things in different code
      paths if the no_formal_ino doesn't match. In the NFS patch we want to
      return -ESTALE, but in the ->lookup() path, its a bug in the fs if the
      no_formal_ino doesn't match and thus we can withdraw in this case.
      In order to later fix bz #201012, we need to be able to look up an inode
      without knowing no_formal_ino, as the only information that is known to
      us is the on-disk location of the inode in question.
      This patch will also help us to fix bz #236099 at a later date by
      cleaning up a lot of the code in that area.
      There are no user visible changes as a result of this patch and there
      are no changes to the on-disk format either.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Robert Peterson's avatar
      [GFS2] Addendum patch 2 for gfs2_grow · cd81a4ba
      Robert Peterson authored
      This addendum patch 2 corrects three things:
      1. It fixes a stupid mistake in the previous addendum that broke gfs2.
         Ref: https://www.redhat.com/archives/cluster-devel/2007-May/msg00162.html
      2. It fixes a problem that Dave Teigland pointed out regarding the
         external declarations in ops_address.h being in the wrong place.
      3. It recasts a couple more %llu printks to (unsigned long long)
         as requested by Steve Whitehouse.
      I would have loved to put this all in one revised patch, but there was
      a rush to get some patches for RHEL5.	Therefore, the previous patches
      were applied to the git tree "as is" and therefore, I'm posting another
      addendum.  Sorry.
      Signed-off-by: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Robert Peterson's avatar
      [GFS2] Kernel changes to support new gfs2_grow command (part 2) · 6c53267f
      Robert Peterson authored
      To avoid code redundancy, I separated out the operational "guts" into
      a new function called read_rindex_entry.  Then I made two functions:
      the closer-to-original gfs2_ri_update (without the special condition
      checks) and gfs2_ri_update_special that's designed with that condition
      in mind.  (I don't like the name, but if you have a suggestion, I'm
      all ears).
      Oh, and there's an added benefit:  we don't need all the ugly gotos
      anymore.  ;)
      This patch has been tested with gfs2_fsck_hellfire (which runs for
      three and a half hours, btw).
      Signed-off-By: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Robert Peterson's avatar
      [GFS2] kernel changes to support new gfs2_grow command · 7ae8fa84
      Robert Peterson authored
      This is another revision of my gfs2 kernel patch that allows
      gfs2_grow to function properly.
      Steve Whitehouse expressed some concerns about the previous
      patch and I restructured it based on his comments.
      The previous patch was doing the statfs_change at file close time,
      under its own transaction.  The current patch does the statfs_change
      inside the gfs2_commit_write function, which keeps it under the
      umbrella of the inode transaction.
      I can't call ri_update to re-read the rindex file during the
      transaction because the transaction may have outstanding unwritten
      buffers attached to the rgrps that would be otherwise blown away.
      So instead, I created a new function, gfs2_ri_total, that will
      re-read the rindex file just to total the file system space
      for the sake of the statfs_change.  The ri_update will happen
      later, when gfs2 realizes the version number has changed, as it
      happened before my patch.
      Since the statfs_change is happening at write_commit time and there
      may be multiple writes to the rindex file for one grow operation.
      So one consequence of this restructuring is that instead of getting
      one kernel message to indicate the change, you may see several.
      For example, before when you did a gfs2_grow, you'd get a single
      message like:
      GFS2: File system extended by 247876 blocks (968MB)
      Now you get something like:
      GFS2: File system extended by 207896 blocks (812MB)
      GFS2: File system extended by 39980 blocks (156MB)
      This version has also been successfully run against the hours-long
      "gfs2_fsck_hellfire" test that does several gfs2_grow and gfs2_fsck
      while interjecting file system damage.  It does this repeatedly
      under a variety Resource Group conditions.
      Signed-off-By: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
  7. 01 May, 2007 2 commits
    • Steven Whitehouse's avatar
      [GFS2] Patch to fix mmap of stuffed files · bf126aee
      Steven Whitehouse authored
      If a stuffed file is mmaped and a page fault is generated at some offset
      above the initial page, we need to create a zero page to hang the buffer
      heads off before we can unstuff the file. This is a fix for bz #236087
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Josef Whiter's avatar
      [GFS2] Fix bz 231380, unlock page before dequeing glocks in gfs2_commit_write · 1de91390
      Josef Whiter authored
      If we are writing a file, and in the middle of writing the file
      another node attempts to get a shared lock on that file (by doing a du for
      example) the process doing the writing will hang waiting on lock_page.  The
      reason for this is because when we have waiters on a exclusive glock, we will go
      through and flush out all dirty pages associated with that inode and release the
      lock.  The problem is that when we flush the dirty pages, we could hit a page
      that we have locked durring the generic_file_buffered_write part of this
      operation.  This patch unlocks the page before we go to dequeue the lock and
      locks it immediatly afterwards, since generic_file_buffered_write needs the page
      locked when the commit_write is completed.  This patch resolves the problem,
      however if somebody sees a better way to do this please don't hesistate to yell.
      Signed-off-by: default avatarJosef Whiter <jwhiter@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
  8. 07 Mar, 2007 1 commit
    • Josef Whiter's avatar
      [GFS2] fix hangup when multiple processes are trying to write to the same file · a13cbe37
      Josef Whiter authored
      This fixes a problem I encountered while running bonnie++.  When you have one
      thread that opens a file and starts to write to it, and then another thread that
      tries to open and write to the same file, the second thread will loop forever
      trying to grab the inode lock for that inode.  Basically we come in through
      generic_buffered_file_write, which calls gfs2_prepare_write, which then attempts
      to grab the glock.  Because we don't own the lock, gfs2_prepare_write gets
      GLR_TRYFAILED, which returns AOP_TRUNCATED_PAGE to generic_buffered_file_write.
      At this point generic_buffered_file_write loops around again and immediately
      retries the prepare_write.  This means that the second process never gets off of
      the processor in order to allow the process that holds the lock to finish its
      work and let go of the lock.  This patch makes gfs2_glock_nq schedule() if it
      gets back a GLR_TRYFAILED, which resolves this problem.
      Signed-off-by: default avatarJosef Whiter <jwhiter@redhat.com>
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
  9. 07 Feb, 2007 2 commits
  10. 05 Feb, 2007 3 commits
    • Steven Whitehouse's avatar
      [GFS2] Add writepages for "data=writeback" mounts · a8d638e3
      Steven Whitehouse authored
      It occurred to me that although a gfs2 specific writepages for ordered
      writes and journaled data would be tricky, by hooking writepages only
      for "data=writeback" mounts we could take advantage of not needing
      buffer heads (we don't use them on the read side, nor have we for some
      time) and create much larger I/Os for the block layer.
      Using blktrace both before and after, its possible to see that for large
      I/Os, most of the requests generated through writepages are now 1024
      sectors after this patch is applied as opposed to 8 sectors before.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Fail over to readpage for stuffed files · e1d5b18a
      Steven Whitehouse authored
      This is partially derrived from a patch written by Russell Cattelan.
      It fixes a bug where there is a race between readpages and truncate
      by ignoring readpages for stuffed files. This is ok because a stuffed
      file will never be more than one block (minus sizeof(struct gfs2_dinode))
      in size and block size is always less than page size, so we do not lose
      anything efficiency-wise by not doing readahead for stuffed files. They
      will have already been "read ahead" by the action of reading the inode
      in, in the first place.
      This is the remaining part of the fix for Red Hat bugzilla #218966
      which had not yet made it upstream.
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
      Cc: Russell Cattelan <cattelan@redhat.com>
    • Steven Whitehouse's avatar
      [GFS2] Fix DIO deadlock · c7b33834
      Steven Whitehouse authored
      This patch fixes Red Hat bugzilla #212627 in which a deadlock occurs
      due to trying to take the i_mutex while holding a glock. The correct
      locking order is defined as i_mutex -> glock in all cases.
      I've left dealing with allocating writes. I know that we need to do
      that, but for now this should do the trick. We don't need to take the
      i_mutex on write, because the VFS has already taken it for us. On read
      we don't need it since the glock is enough protection. The reason that
      I've made some of the checks into a separate function is that we'll need
      to do the checks again in the allocating write case eventually, so this
      is partly in preparation for this. Likewise the return value test of !=
      1 might look a bit odd and thats because we'll need a third return value
      in case of requiring an allocation.
      I've made the change to deferred mode on the glock to ensure flushing
      read caches on other nodes. I notice that (using blktrace to look at
      whats going on) we appear to do a better job of large I/Os than ext3
      after this patch (in terms of not splitting up the I/Os).
      Signed-off-by: default avatarSteven Whitehouse <swhiteho@redhat.com>
      Cc: Wendy Cheng <wcheng@redhat.com>
  11. 30 Nov, 2006 6 commits