• Mel Gorman's avatar
    mm, page_alloc: reset zonelist iterator after resetting fair zone allocation policy · 0d0bd894
    Mel Gorman authored
    Geert Uytterhoeven reported the following problem that bisected to
    commit c33d6c06 ("mm, page_alloc: avoid looking up the first zone
    in a zonelist twice") on m68k/ARAnyM
    
        BUG: scheduling while atomic: cron/668/0x10c9a0c0
        Modules linked in:
        CPU: 0 PID: 668 Comm: cron Not tainted 4.6.0-atari-05133-gc33d6c06 #364
        Call Trace: [<0003d7d0>] __schedule_bug+0x40/0x54
          __schedule+0x312/0x388
          __schedule+0x0/0x388
          prepare_to_wait+0x0/0x52
          schedule+0x64/0x82
          schedule_timeout+0xda/0x104
          set_next_entity+0x18/0x40
          pick_next_task_fair+0x78/0xda
          io_schedule_timeout+0x36/0x4a
          bit_wait_io+0x0/0x40
          bit_wait_io+0x12/0x40
          __wait_on_bit+0x46/0x76
          wait_on_page_bit_killable+0x64/0x6c
          bit_wait_io+0x0/0x40
          wake_bit_function+0x0/0x4e
          __lock_page_or_retry+0xde/0x124
          do_scan_async+0x114/0x17c
          lookup_swap_cache+0x24/0x4e
          handle_mm_fault+0x626/0x7de
          find_vma+0x0/0x66
          down_read+0x0/0xe
          wait_on_page_bit_killable_timeout+0x77/0x7c
          find_vma+0x16/0x66
          do_page_fault+0xe6/0x23a
          res_func+0xa3c/0x141a
          buserr_c+0x190/0x6d4
          res_func+0xa3c/0x141a
          buserr+0x20/0x28
          res_func+0xa3c/0x141a
          buserr+0x20/0x28
    
    The relationship is not obvious but it's due to a failure to rescan the
    full zonelist after the fair zone allocation policy exhausts the batch
    count.  While this is a functional problem, it's also a performance
    issue.  A page allocator microbenchmark showed the following
    
                                       4.7.0-rc1                  4.7.0-rc1
                                         vanilla                 reset-v1r2
      Min      alloc-odr0-1     327.00 (  0.00%)           326.00 (  0.31%)
      Min      alloc-odr0-2     235.00 (  0.00%)           235.00 (  0.00%)
      Min      alloc-odr0-4     198.00 (  0.00%)           198.00 (  0.00%)
      Min      alloc-odr0-8     170.00 (  0.00%)           170.00 (  0.00%)
      Min      alloc-odr0-16    156.00 (  0.00%)           156.00 (  0.00%)
      Min      alloc-odr0-32    150.00 (  0.00%)           150.00 (  0.00%)
      Min      alloc-odr0-64    146.00 (  0.00%)           146.00 (  0.00%)
      Min      alloc-odr0-128   145.00 (  0.00%)           145.00 (  0.00%)
      Min      alloc-odr0-256   155.00 (  0.00%)           155.00 (  0.00%)
      Min      alloc-odr0-512   168.00 (  0.00%)           165.00 (  1.79%)
      Min      alloc-odr0-1024  175.00 (  0.00%)           174.00 (  0.57%)
      Min      alloc-odr0-2048  180.00 (  0.00%)           180.00 (  0.00%)
      Min      alloc-odr0-4096  187.00 (  0.00%)           186.00 (  0.53%)
      Min      alloc-odr0-8192  190.00 (  0.00%)           190.00 (  0.00%)
      Min      alloc-odr0-16384 191.00 (  0.00%)           191.00 (  0.00%)
      Min      alloc-odr1-1     736.00 (  0.00%)           445.00 ( 39.54%)
      Min      alloc-odr1-2     343.00 (  0.00%)           335.00 (  2.33%)
      Min      alloc-odr1-4     277.00 (  0.00%)           270.00 (  2.53%)
      Min      alloc-odr1-8     238.00 (  0.00%)           233.00 (  2.10%)
      Min      alloc-odr1-16    224.00 (  0.00%)           218.00 (  2.68%)
      Min      alloc-odr1-32    210.00 (  0.00%)           208.00 (  0.95%)
      Min      alloc-odr1-64    207.00 (  0.00%)           203.00 (  1.93%)
      Min      alloc-odr1-128   276.00 (  0.00%)           202.00 ( 26.81%)
      Min      alloc-odr1-256   206.00 (  0.00%)           202.00 (  1.94%)
      Min      alloc-odr1-512   207.00 (  0.00%)           202.00 (  2.42%)
      Min      alloc-odr1-1024  208.00 (  0.00%)           205.00 (  1.44%)
      Min      alloc-odr1-2048  213.00 (  0.00%)           212.00 (  0.47%)
      Min      alloc-odr1-4096  218.00 (  0.00%)           216.00 (  0.92%)
      Min      alloc-odr1-8192  341.00 (  0.00%)           219.00 ( 35.78%)
    
    Note that order-0 allocations are unaffected but higher orders get a
    small boost from this patch and a large reduction in system CPU usage
    overall as can be seen here:
    
                 4.7.0-rc1   4.7.0-rc1
                   vanilla  reset-v1r2
      User           85.32       86.31
      System       2221.39     2053.36
      Elapsed      2368.89     2202.47
    
    Fixes: c33d6c06 ("mm, page_alloc: avoid looking up the first zone in a zonelist twice")
    Link: http://lkml.kernel.org/r/20160531100848.GR2527@techsingularity.netSigned-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Reported-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
    Tested-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
    Tested-by: default avatarMikulas Patocka <mpatocka@redhat.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    0d0bd894
Name
Last commit
Last update
..
kasan Loading commit data...
Kconfig Loading commit data...
Kconfig.debug Loading commit data...
Makefile Loading commit data...
backing-dev.c Loading commit data...
balloon_compaction.c Loading commit data...
bootmem.c Loading commit data...
cleancache.c Loading commit data...
cma.c Loading commit data...
cma.h Loading commit data...
cma_debug.c Loading commit data...
compaction.c Loading commit data...
debug.c Loading commit data...
debug_page_ref.c Loading commit data...
dmapool.c Loading commit data...
early_ioremap.c Loading commit data...
fadvise.c Loading commit data...
failslab.c Loading commit data...
filemap.c Loading commit data...
frame_vector.c Loading commit data...
frontswap.c Loading commit data...
gup.c Loading commit data...
highmem.c Loading commit data...
huge_memory.c Loading commit data...
hugetlb.c Loading commit data...
hugetlb_cgroup.c Loading commit data...
hwpoison-inject.c Loading commit data...
init-mm.c Loading commit data...
internal.h Loading commit data...
interval_tree.c Loading commit data...
kmemcheck.c Loading commit data...
kmemleak-test.c Loading commit data...
kmemleak.c Loading commit data...
ksm.c Loading commit data...
list_lru.c Loading commit data...
maccess.c Loading commit data...
madvise.c Loading commit data...
memblock.c Loading commit data...
memcontrol.c Loading commit data...
memory-failure.c Loading commit data...
memory.c Loading commit data...
memory_hotplug.c Loading commit data...
mempolicy.c Loading commit data...
mempool.c Loading commit data...
memtest.c Loading commit data...
migrate.c Loading commit data...
mincore.c Loading commit data...
mlock.c Loading commit data...
mm_init.c Loading commit data...
mmap.c Loading commit data...
mmu_context.c Loading commit data...
mmu_notifier.c Loading commit data...
mmzone.c Loading commit data...
mprotect.c Loading commit data...
mremap.c Loading commit data...
msync.c Loading commit data...
nobootmem.c Loading commit data...
nommu.c Loading commit data...
oom_kill.c Loading commit data...
page-writeback.c Loading commit data...
page_alloc.c Loading commit data...
page_counter.c Loading commit data...
page_ext.c Loading commit data...
page_idle.c Loading commit data...
page_io.c Loading commit data...
page_isolation.c Loading commit data...
page_owner.c Loading commit data...
page_poison.c Loading commit data...
pagewalk.c Loading commit data...
percpu-km.c Loading commit data...
percpu-vm.c Loading commit data...
percpu.c Loading commit data...
pgtable-generic.c Loading commit data...
process_vm_access.c Loading commit data...
quicklist.c Loading commit data...
readahead.c Loading commit data...
rmap.c Loading commit data...
shmem.c Loading commit data...
slab.c Loading commit data...
slab.h Loading commit data...
slab_common.c Loading commit data...
slob.c Loading commit data...
slub.c Loading commit data...
sparse-vmemmap.c Loading commit data...
sparse.c Loading commit data...
swap.c Loading commit data...
swap_cgroup.c Loading commit data...
swap_state.c Loading commit data...
swapfile.c Loading commit data...
truncate.c Loading commit data...
userfaultfd.c Loading commit data...
util.c Loading commit data...
vmacache.c Loading commit data...
vmalloc.c Loading commit data...
vmpressure.c Loading commit data...
vmscan.c Loading commit data...
vmstat.c Loading commit data...
workingset.c Loading commit data...
z3fold.c Loading commit data...
zbud.c Loading commit data...
zpool.c Loading commit data...
zsmalloc.c Loading commit data...
zswap.c Loading commit data...