unlock_buffer(), like unlock_page(), must not clear the lock without
ensuring that the critical section is closed.

Mingming later sent the same patch, saying:

  We are running SDET benchmark and saw double free issue for ext3 extended
  attributes block, which complains the same xattr block already being freed (in
  ext3_xattr_release_block()).  The problem could also been triggered by
  multiple threads loop untar/rm a kernel tree.

  The race is caused by missing a memory barrier at unlock_buffer() before the
  lock bit being cleared, resulting in possible concurrent h_refcounter update.
  That causes a reference counter leak, then later leads to the double free that
  we have seen.

  Inside unlock_buffer(), there is a memory barrier is placed *after* the lock
  bit is being cleared, however, there is no memory barrier *before* the bit is
  cleared.  On some arch the h_refcount update instruction and the clear bit
  instruction could be reordered, thus leave the critical section re-entered.

  The race is like this: For example, if the h_refcount is initialized as 1,

  cpu 0:                                   cpu1
  --------------------------------------   -----------------------------------
  lock_buffer() /* test_and_set_bit */
  clear_buffer_locked(bh);
                                          lock_buffer() /* test_and_set_bit */
  h_refcount = h_refcount+1; /* = 2*/     h_refcount = h_refcount + 1; /*= 2 */
                                          clear_buffer_locked(bh);
  ....                                    ......

  We lost a h_refcount here. We need a memory barrier before the buffer head lock
  bit being cleared to force the order of the two writes.  Please apply.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>