• Joonsoo Kim's avatar
    mm/page_ref: add tracepoint to track down page reference manipulation · 95813b8f
    Joonsoo Kim authored
    CMA allocation should be guaranteed to succeed by definition, but,
    unfortunately, it would be failed sometimes.  It is hard to track down
    the problem, because it is related to page reference manipulation and we
    don't have any facility to analyze it.
    This patch adds tracepoints to track down page reference manipulation.
    With it, we can find exact reason of failure and can fix the problem.
    Following is an example of tracepoint output.  (note: this example is
    stale version that printing flags as the number.  Recent version will
    print it as human readable string.)
    <...>-9018  [004]    92.678375: page_ref_set:         pfn=0x17ac9 flags=0x0 count=1 mapcount=0 mapping=(nil) mt=4 val=1
    <...>-9018  [004]    92.678378: kernel_stack:
     => get_page_from_freelist (ffffffff81176659)
     => __alloc_pages_nodemask (ffffffff81176d22)
     => alloc_pages_vma (ffffffff811bf675)
     => handle_mm_fault (ffffffff8119e693)
     => __do_page_fault (ffffffff810631ea)
     => trace_do_page_fault (ffffffff81063543)
     => do_async_page_fault (ffffffff8105c40a)
     => async_page_fault (ffffffff817581d8)
    <...>-9018  [004]    92.678379: page_ref_mod:         pfn=0x17ac9 flags=0x40048 count=2 mapcount=1 mapping=0xffff880015a78dc1 mt=4 val=1
    <...>-9131  [001]    93.174468: test_pages_isolated:  start_pfn=0x17800 end_pfn=0x17c00 fin_pfn=0x17ac9 ret=fail
    <...>-9018  [004]    93.174843: page_ref_mod_and_test: pfn=0x17ac9 flags=0x40068 count=0 mapcount=0 mapping=0xffff880015a78dc1 mt=4 val=-1 ret=1
     => release_pages (ffffffff8117c9e4)
     => free_pages_and_swap_cache (ffffffff811b0697)
     => tlb_flush_mmu_free (ffffffff81199616)
     => tlb_finish_mmu (ffffffff8119a62c)
     => exit_mmap (ffffffff811a53f7)
     => mmput (ffffffff81073f47)
     => do_exit (ffffffff810794e9)
     => do_group_exit (ffffffff81079def)
     => SyS_exit_group (ffffffff81079e74)
     => entry_SYSCALL_64_fastpath (ffffffff817560b6)
    This output shows that problem comes from exit path.  In exit path, to
    improve performance, pages are not freed immediately.  They are gathered
    and processed by batch.  During this process, migration cannot be
    possible and CMA allocation is failed.  This problem is hard to find
    without this page reference tracepoint facility.
    Enabling this feature bloat kernel text 30 KB in my configuration.
       text    data     bss     dec     hex filename
    12127327        2243616 1507328 15878271         f2487f vmlinux_disabled
    12157208        2258880 1507328 15923416         f2f8d8 vmlinux_enabled
    Note that, due to header file dependency problem between mm.h and
    tracepoint.h, this feature has to open code the static key functions for
    tracepoints.  Proposed by Steven Rostedt in following link.
    [arnd@arndb.de: crypto/async_pq: use __free_page() instead of put_page()]
    [iamjoonsoo.kim@lge.com: fix build failure for xtensa]
    [akpm@linux-foundation.org: tweak Kconfig text, per Vlastimil]
    Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Acked-by: default avatarMichal Nazarewicz <mina86@mina86.com>
    Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
    Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
    Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
page_ref.h 2.95 KB