1. 27 Oct, 2010 24 commits
    • Theodore Ts'o's avatar
      ext4: use search_dirblock() in ext4_dx_find_entry() · 7845c049
      Theodore Ts'o authored
      Use the search_dirblock() in ext4_dx_find_entry().  It makes the code
      easier to read, and it takes advantage of common code.  It also saves
      100 bytes or so of text space.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      Cc: Brad Spengler <spender@grsecurity.net>
    • Theodore Ts'o's avatar
      ext4: avoid uninitialized memory references in ext3_htree_next_block() · 8941ec8b
      Theodore Ts'o authored
      If the first block of htree directory is missing '.' or '..' but is
      otherwise a valid directory, and we do a lookup for '.' or '..', it's
      possible to dereference an uninitialized memory pointer in
      We avoid this by moving the special case from ext4_dx_find_entry() to
      ext4_find_entry(); this also means we can optimize ext4_find_entry()
      slightly when NFS looks up "..".
      Thanks to Brad Spengler for pointing a Clang warning that led me to
      look more closely at this code.  The warning was harmless, but it was
      useful in pointing out code that was too ugly to live.  This warning was
      also reported by Roman Borisov.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      Cc: Brad Spengler <spender@grsecurity.net>
    • Eric Sandeen's avatar
      ext4: remove unused ext4_sb_info members · 640e9396
      Eric Sandeen authored
      Not that these take up a lot of room, but the structure is long enough
      as it is, and there's no need to confuse people with these various
      undocumented & unused structure members...
      Signed-off-by: default avatarEric Sandeen <sandeen@redaht.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Eric Sandeen's avatar
      ext4: queue conversion after adding to inode's completed IO list · c999af2b
      Eric Sandeen authored
      By queuing the io end on the unwritten workqueue before adding it
      to our inode's list of completed IOs, I think we run the risk
      of the work getting completed, and the IO freed, before we try
      to add it to the inode's i_completed_io_list.
      It should be safe to add it to the inode's list of completed
      IOs, and -then- queue it for completion, I think.
      Thanks to Dave Chinner for pointing out the race.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Reviewed-by: default avatarJiaying Zhang <jiayingz@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Eric Sandeen's avatar
      ext4: don't use ext4_allocation_contexts for tracing · 3e1e5f50
      Eric Sandeen authored
      Many tracepoints were populating an ext4_allocation_context
      to pass in, but this requires a slab allocation even when
      tracepoints are off.  In fact, 4 of 5 of these allocations
      were only for tracing.  In addition, we were only using a
      small fraction of the 144 bytes of this structure for this
      We can do away with all these alloc/frees of the ac and
      simply pass in the bits we care about, instead.
      I tested this by turning on tracing and running through
      xfstests on x86_64.  I did not actually do anything with
      the trace output, however.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Eric Sandeen's avatar
      ext4: fix oops in trace_ext4_mb_release_group_pa · 4d547616
      Eric Sandeen authored
      Our QA reported an oops in the ext4_mb_release_group_pa tracing,
      and Josef Bacik pointed out that it was because we may have a
      non-null but uninitialized ac_inode in the allocation context.
      I can reproduce it when running xfstests with ext4 tracepoints on, 
      on a CONFIG_SLAB_DEBUG kernel.
      We call trace_ext4_mb_release_group_pa from 2 places, 
      ext4_mb_discard_group_preallocations and 
      In both cases we allocate an ac as a container just for tracing (!)
      and never fill in the ac_inode.  There's no reason to be assigning,
      testing, or printing it as far as I can see, so just remove it from
      the tracepoint.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Reviewed-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Toshiyuki Okajima's avatar
      ext4: fix potential infinite loop in ext4_da_writepages() · 0c9169cc
      Toshiyuki Okajima authored
      On linux-2.6.36-rc2, if we execute the following script, we can hang
      the system when the /bin/sync command is executed:
      echo -n "HANG UP TEST: "
      /bin/dd if=/dev/zero of=/tmp/img bs=1k count=1 seek=1M 2> /dev/null
      /sbin/mkfs.ext4 -Fq /tmp/img
      /bin/mount -o loop -t ext4 /tmp/img /mnt
      /bin/dd if=/dev/zero of=/mnt/file bs=1 count=1 \
      seek=$((16*1024*1024*1024*1024-4096)) 2> /dev/null
      /bin/umount /mnt
      echo "DONE"
      exit 0
      We can see the following backtrace if we get the kdump when this
      hangup occurs:
      => bdi_writeback_thread()
         => wb_do_writeback()
            => wb_writeback()
               => writeback_inodes_wb()
                  => writeback_sb_inodes()
                     => writeback_single_inode()
                        => ext4_da_writepages()  ---+ 
                                      ^ infinite    |
                                      |   loop      |
      The reason why this hangup happens is described as follows:
      1) We write the last extent block of the file whose size is the filesystem 
         maximum size.
      2) "BH_Delay" flag is set on the buffer_head of its block.
      3) - the member, "m_lblk" of struct mpage_da_data is 4294967295 (UINT_MAX)
         - the member, "m_len" of struct mpage_da_data is 1
        mpage_put_bnr_to_bhs() which is called via ext4_da_writepages()
        cannot clear "BH_Delay" flag of the buffer_head because the type of
        m_lblk is ext4_lblk_t and then m_lblk + m_len is overflow.
        Therefore an infinite loop occurs because ext4_da_writepages()
        cannot write the page (which corresponds to the block) since
        "BH_Delay" flag isn't cleared.
      static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd,
      				struct ext4_map_blocks *map)
      	int blocks = map->m_len;
      		do {
      			// cur_logical = 4294967295
      			// map->m_lblk = 4294967295
      			// blocks = 1
      			// *** map->m_lblk + blocks == 0 (OVERFLOW!) ***
      			// (cur_logical >= map->m_lblk + blocks) => true
      			if (cur_logical >= map->m_lblk + blocks)
      NOTE: Mounting with the nodelalloc option will avoid this codepath,
      and thus, avoid this hang
      Signed-off-by: default avatarToshiyuki Okajima <toshi.okajima@jp.fujitsu.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Toshiyuki Okajima's avatar
      ext4: improve llseek error handling for overly large seek offsets · e0d10bfa
      Toshiyuki Okajima authored
      The llseek system call should return EINVAL if passed a seek offset
      which results in a write error.  What this maximum offset should be
      depends on whether or not the huge_file file system feature is set,
      and whether or not the file is extent based or not.
      If the file has no "EXT4_EXTENTS_FL" flag, the maximum size which can be 
      written (write systemcall) is different from the maximum size which can be 
      sought (lseek systemcall).
      For example, the following 2 cases demonstrates the differences
      between the maximum size which can be written, versus the seek offset
      allowed by the llseek system call:
      #1: mkfs.ext3 <dev>; mount -t ext4 <dev>
      #2: mkfs.ext3 <dev>; tune2fs -Oextent,huge_file <dev>; mount -t ext4 <dev>
      Table. the max file size which we can write or seek
             at each filesystem feature tuning and file flag setting
      | \ File flag|                               |                               |
      |      \     |     !EXT4_EXTENTS_FL          |        EXT4_EXTETNS_FL        |
      |case       \|                               |                               |
      | #1         |   write:      2194719883264   | write:       --------------   |
      |            |   seek:       2199023251456   | seek:        --------------   |
      | #2         |   write:      4402345721856   | write:       17592186044415   |
      |            |   seek:      17592186044415   | seek:        17592186044415   |
      The differences exist because ext4 has 2 maxbytes which are sb->s_maxbytes
      (= extent-mapped maxbytes) and EXT4_SB(sb)->s_bitmap_maxbytes (= block-mapped 
      maxbytes).  Although generic_file_llseek uses only extent-mapped maxbytes.
      (llseek of ext4_file_operations is generic_file_llseek which uses
      Therefore we create ext4 llseek function which uses 2 maxbytes.
      The new own function originates from generic_file_llseek().
      If the file flag, "EXT4_EXTENTS_FL" is not set, the function alters 
      inode->i_sb->s_maxbytes into EXT4_SB(inode->i_sb)->s_bitmap_maxbytes.
      Signed-off-by: default avatarToshiyuki Okajima <toshi.okajima@jp.fujitsu.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      Cc: Andreas Dilger <adilger.kernel@dilger.ca>
    • Maciej Żenczykowski's avatar
      ext4: don't update sb journal_devnum when RO dev · c41303ce
      Maciej Żenczykowski authored
      An ext4 filesystem on a read-only device, with an external journal
      which is at a different device number then recorded in the superblock
      will fail to honor the read-only setting of the device and trigger
      a superblock update (write).
      For example:
        - ext4 on a software raid which is in read-only mode
        - external journal on a read-write device which has changed device num
        - attempt to mount with -o journal_dev=<new_number>
        - hits BUG_ON(mddev->ro = 1) in md.c
      Cc: Theodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarMaciej Żenczykowski <zenczykowski@gmail.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: use sb_issue_zeroout in ext4_ext_zeroout · 2407518d
      Lukas Czerner authored
      Change ext4_ext_zeroout to use sb_issue_zeroout instead of its
      own approach to zero out extents.
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: use sb_issue_zeroout in setup_new_group_blocks · a31437b8
      Lukas Czerner authored
      Use sb_issue_zeroout to zero out inode table and descriptor table
      blocks instead of old approach which involves journaling.
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: add interface to advertise ext4 features in sysfs · 857ac889
      Lukas Czerner authored
      User-space should have the opportunity to check what features doest ext4
      support in each particular copy. This adds easy interface by creating new
      "features" directory in sys/fs/ext4/. In that directory files
      advertising feature names can be created.
      Add lazy_itable_init to the feature list.
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: add support for lazy inode table initialization · bfff6873
      Lukas Czerner authored
      When the lazy_itable_init extended option is passed to mke2fs, it
      considerably speeds up filesystem creation because inode tables are
      not zeroed out.  The fact that parts of the inode table are
      uninitialized is not a problem so long as the block group descriptors,
      which contain information regarding how much of the inode table has
      been initialized, has not been corrupted However, if the block group
      checksums are not valid, e2fsck must scan the entire inode table, and
      the the old, uninitialized data could potentially cause e2fsck to
      report false problems.
      Hence, it is important for the inode tables to be initialized as soon
      as possble.  This commit adds this feature so that mke2fs can safely
      use the lazy inode table initialization feature to speed up formatting
      file systems.
      This is done via a new new kernel thread called ext4lazyinit, which is
      created on demand and destroyed, when it is no longer needed.  There
      is only one thread for all ext4 filesystems in the system. When the
      first filesystem with inititable mount option is mounted, ext4lazyinit
      thread is created, then the filesystem can register its request in the
      request list.
      This thread then walks through the list of requests picking up
      scheduled requests and invoking ext4_init_inode_table(). Next schedule
      time for the request is computed by multiplying the time it took to
      zero out last inode table with wait multiplier, which can be set with
      the (init_itable=n) mount option (default is 10).  We are doing
      this so we do not take the whole I/O bandwidth. When the thread is no
      longer necessary (request list is empty) it frees the appropriate
      structures and exits (and can be created later later by another
      We do not disturb regular inode allocations in any way, it just do not
      care whether the inode table is, or is not zeroed. But when zeroing, we
      have to skip used inodes, obviously. Also we should prevent new inode
      allocations from the group, while zeroing is on the way. For that we
      take write alloc_sem lock in ext4_init_inode_table() and read alloc_sem
      in the ext4_claim_inode, so when we are unlucky and allocator hits the
      group which is currently being zeroed, it just has to wait.
      This can be suppresed using the mount option no_init_itable.
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      Add helper function for blkdev_issue_zeroout (sb_issue_discard) · e6fa0be6
      Lukas Czerner authored
      This is done the same way as helper sb_issue_discard for
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Theodore Ts'o's avatar
      jbd2: Add sanity check for attempts to start handle during umount · 5c2178e7
      Theodore Ts'o authored
      An attempt to modify the file system during the call to
      jbd2_destroy_journal() can lead to a system lockup.  So add some
      checking to make it much more obvious when this happens to and to
      determine where the offending code is located.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Sergey Senozhatsky's avatar
      ext4: fix NULL pointer dereference in print_daily_error_info() · a1c6c569
      Sergey Senozhatsky authored
      Fix NULL pointer dereference in print_daily_error_info, when   
      called on unmounted fs (EXT4_SB(sb) returns NULL), by removing error 
      reporting timer in ext4_put_super.
      Google-Bug-Id: 3017663
      Signed-off-by: default avatarSergey Senozhatsky <sergey.senozhatsky@gmail.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: don't hold spinlock while calling ext4_issue_discard() · 53fdcf99
      Lukas Czerner authored
      We can't hold the block group spinlock because we ext4_issue_discard()
      calls wait and hence can get rescheduled.
      Google-Bug-Id: 3017678
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Lukas Czerner's avatar
      ext4: check for negative error code from sb_issue_discard · 58298709
      Lukas Czerner authored
      sb_issue_discard() is returning negative error code, so check for
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Eric Sandeen's avatar
      ext4: don't bump up LONG_MAX nr_to_write by a factor of 8 · b443e733
      Eric Sandeen authored
      I'm uneasy with lots of stuff going on in ext4_da_writepages(),
      but bumping nr_to_write from LLONG_MAX to -8 clearly isn't
      making anything better, so avoid the multiplier in that case.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Eric Sandeen's avatar
      ext4: stop looping in ext4_num_dirty_pages when max_pages reached · 659c6009
      Eric Sandeen authored
      Today we simply break out of the inner loop when we have accumulated
      max_pages; this keeps scanning forwad and doing pagevec_lookup_tag()
      in the while (!done) loop, this does potentially a lot of work
      with no net effect.
      When we have accumulated max_pages, just clean up and return.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Curt Wohlgemuth's avatar
      ext4: use dedicated slab caches for group_info structures · fb1813f4
      Curt Wohlgemuth authored
      ext4_group_info structures are currently allocated with kmalloc().
      With a typical 4K block size, these are 136 bytes each -- meaning
      they'll each consume a 256-byte slab object.  On a system with many
      ext4 large partitions, that's a lot of wasted kernel slab space.
      (E.g., a single 1TB partition will have about 8000 block groups, using
      about 2MB of slab, of which nearly 1MB is wasted.)
      This patch creates an array of slab pointers created as needed --
      depending on the superblock block size -- and uses these slabs to
      allocate the group info objects.
      Google-Bug-Id: 2980809
      Signed-off-by: default avatarCurt Wohlgemuth <curtw@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Wen Congyang's avatar
      ext4: avoid null dereference in trace_ext4_mballoc_discard · b853fd36
      Wen Congyang authored
      ac->inode is set to null in function ext4_mb_release_group_pa(),
      and then trace_ext4_mballoc_discard(ac) is called, the kernel
      will panic.
      BUG: unable to handle kernel NULL pointer dereference at 000000a4
      IP: [<f87e1714>] ftrace_raw_event_ext4__mballoc+0x54/0xc0 [ext4]
      *pdpt = 0000000000abd001 *pde = 0000000000000000
      Oops: 0000 [#1] SMP
      Pid: 550, comm: flush-8:16 Not tainted 2.6.36-rc1 #1 SE7320EP2/Altos G530
      EIP: 0060:[<f87e1714>] EFLAGS: 00010206 CPU: 1
      EIP is at ftrace_raw_event_ext4__mballoc+0x54/0xc0 [ext4]
      EAX: f32ac840 EBX: f3f1cf88 ECX: f32ac840 EDX: 00000000
      ESI: f32ac83c EDI: f880b9d8 EBP: 00000000 ESP: f4b77ae4
       DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
      Process flush-8:16 (pid: 550, ti=f4b76000 task=f613e540 task.ti=f4b76000)
      Call Trace:
       [<f87f5ac1>] ? ext4_mb_release_group_pa+0x121/0x150 [ext4]
       [<f87f8356>] ? ext4_mb_discard_group_preallocations+0x336/0x400 [ext4]
       [<f87fb7f1>] ? ext4_mb_new_blocks+0x3d1/0x4f0 [ext4]
       [<c05a6c5b>] ? __make_request+0x10b/0x440
       [<f87f1fb4>] ? ext4_ext_map_blocks+0x1334/0x1980 [ext4]
       [<c04ac78a>] ? rb_reserve_next_event+0xaa/0x3b0
       [<f87d18d6>] ? ext4_map_blocks+0xd6/0x1d0 [ext4]
       [<f87d2da7>] ? mpage_da_map_blocks+0xc7/0x8a0 [ext4]
       [<c04c8a68>] ? find_get_pages_tag+0x38/0x110
       [<c04d23a5>] ? __pagevec_release+0x15/0x20
       [<f87d3ca5>] ? ext4_da_writepages+0x2b5/0x5d0 [ext4]
       [<c04cfbe0>] ? __writepage+0x0/0x30
       [<c04d0e34>] ? do_writepages+0x14/0x30
       [<c0526600>] ? writeback_single_inode+0xa0/0x240
       [<c0526971>] ? writeback_sb_inodes+0xc1/0x180
       [<c0526ab8>] ? writeback_inodes_wb+0x88/0x140
       [<c0526d7b>] ? wb_writeback+0x20b/0x320
       [<c045aca7>] ? lock_timer_base+0x27/0x50
       [<c0526fe0>] ? wb_do_writeback+0x150/0x190
       [<c05270a8>] ? bdi_writeback_thread+0x88/0x1f0
       [<c043b680>] ? complete+0x40/0x60
       [<c0527020>] ? bdi_writeback_thread+0x0/0x1f0
       [<c0469474>] ? kthread+0x74/0x80
       [<c0469400>] ? kthread+0x0/0x80
       [<c040a23e>] ? kernel_thread_helper+0x6/0x10
      Signed-off-by: default avatarWen Congyang <wency@cn.fujitsu.com>
      Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Brian King's avatar
      jbd2: Fix I/O hang in jbd2_journal_release_jbd_inode · 39e3ac25
      Brian King authored
      This fixes a hang seen in jbd2_journal_release_jbd_inode
      on a lot of Power 6 systems running with ext4. When we get
      in the hung state, all I/O to the disk in question gets blocked
      where we stay indefinitely. Looking at the task list, I can see
      we are stuck in jbd2_journal_release_jbd_inode waiting on a
      wake up. I added some debug code to detect this scenario and
      dump additional data if we were stuck in jbd2_journal_release_jbd_inode
      for longer than 30 minutes. When it hit, I was able to see that
      i_flags was 0, suggesting we missed the wake up.
      This patch changes i_flags to be an unsigned long, uses bit operators
      to access it, and adds barriers around the accesses. Prior to applying
      this patch, we were regularly hitting this hang on numerous systems
      in our test environment. After applying the patch, the hangs no longer
      Signed-off-by: default avatarBrian King <brking@linux.vnet.ibm.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    • Theodore Ts'o's avatar
      ext4: fix EOFBLOCKS_FL handling · 58590b06
      Theodore Ts'o authored
      It turns out we have several problems with how EOFBLOCKS_FL is
      handled.  First of all, there was a fencepost error where we were not
      clearing the EOFBLOCKS_FL when fill in the last uninitialized block,
      but rather when we allocate the next block _after_ the uninitalized
      block.  Secondly we were not testing to see if we needed to clear the
      EOFBLOCKS_FL when writing to the file O_DIRECT or when were converting
      an uninitialized block (which is the most common case).
      Google-Bug-Id: 2928259
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
  2. 28 Sep, 2010 9 commits
  3. 27 Sep, 2010 7 commits