• Lukasz Odzioba's avatar
    mm/swap.c: flush lru pvecs on compound page arrival · 8f182270
    Lukasz Odzioba authored
    Currently we can have compound pages held on per cpu pagevecs, which
    leads to a lot of memory unavailable for reclaim when needed.  In the
    systems with hundreads of processors it can be GBs of memory.
    
    On of the way of reproducing the problem is to not call munmap
    explicitly on all mapped regions (i.e.  after receiving SIGTERM).  After
    that some pages (with THP enabled also huge pages) may end up on
    lru_add_pvec, example below.
    
      void main() {
      #pragma omp parallel
      {
    	size_t size = 55 * 1000 * 1000; // smaller than  MEM/CPUS
    	void *p = mmap(NULL, size, PROT_READ | PROT_WRITE,
    		MAP_PRIVATE | MAP_ANONYMOUS , -1, 0);
    	if (p != MAP_FAILED)
    		memset(p, 0, size);
    	//munmap(p, size); // uncomment to make the problem go away
      }
      }
    
    When we run it with THP enabled it will leave significant amount of
    memory on lru_add_pvec.  This memory will be not reclaimed if we hit
    OOM, so when we run above program in a loop:
    
    	for i in `seq 100`; do ./a.out; done
    
    many processes (95% in my case) will be killed by OOM.
    
    The primary point of the LRU add cache is to save the zone lru_lock
    contention with a hope that more pages will belong to the same zone and
    so their addition can be batched.  The huge page is already a form of
    batched addition (it will add 512 worth of memory in one go) so skipping
    the batching seems like a safer option when compared to a potential
    excess in the caching which can be quite large and much harder to fix
    because lru_add_drain_all is way to expensive and it is not really clear
    what would be a good moment to call it.
    
    Similarly we can reproduce the problem on lru_deactivate_pvec by adding:
    madvise(p, size, MADV_FREE); after memset.
    
    This patch flushes lru pvecs on compound page arrival making the problem
    less severe - after applying it kill rate of above example drops to 0%,
    due to reducing maximum amount of memory held on pvec from 28MB (with
    THP) to 56kB per CPU.
    Suggested-by: default avatarMichal Hocko <mhocko@suse.com>
    Link: http://lkml.kernel.org/r/1466180198-18854-1-git-send-email-lukasz.odzioba@intel.comSigned-off-by: default avatarLukasz Odzioba <lukasz.odzioba@intel.com>
    Acked-by: default avatarMichal Hocko <mhocko@suse.com>
    Cc: Kirill Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Vladimir Davydov <vdavydov@parallels.com>
    Cc: Ming Li <mingli199x@qq.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    8f182270
Name
Last commit
Last update
..
kasan Loading commit data...
Kconfig Loading commit data...
Kconfig.debug Loading commit data...
Makefile Loading commit data...
backing-dev.c Loading commit data...
balloon_compaction.c Loading commit data...
bootmem.c Loading commit data...
cleancache.c Loading commit data...
cma.c Loading commit data...
cma.h Loading commit data...
cma_debug.c Loading commit data...
compaction.c Loading commit data...
debug.c Loading commit data...
debug_page_ref.c Loading commit data...
dmapool.c Loading commit data...
early_ioremap.c Loading commit data...
fadvise.c Loading commit data...
failslab.c Loading commit data...
filemap.c Loading commit data...
frame_vector.c Loading commit data...
frontswap.c Loading commit data...
gup.c Loading commit data...
highmem.c Loading commit data...
huge_memory.c Loading commit data...
hugetlb.c Loading commit data...
hugetlb_cgroup.c Loading commit data...
hwpoison-inject.c Loading commit data...
init-mm.c Loading commit data...
internal.h Loading commit data...
interval_tree.c Loading commit data...
kmemcheck.c Loading commit data...
kmemleak-test.c Loading commit data...
kmemleak.c Loading commit data...
ksm.c Loading commit data...
list_lru.c Loading commit data...
maccess.c Loading commit data...
madvise.c Loading commit data...
memblock.c Loading commit data...
memcontrol.c Loading commit data...
memory-failure.c Loading commit data...
memory.c Loading commit data...
memory_hotplug.c Loading commit data...
mempolicy.c Loading commit data...
mempool.c Loading commit data...
memtest.c Loading commit data...
migrate.c Loading commit data...
mincore.c Loading commit data...
mlock.c Loading commit data...
mm_init.c Loading commit data...
mmap.c Loading commit data...
mmu_context.c Loading commit data...
mmu_notifier.c Loading commit data...
mmzone.c Loading commit data...
mprotect.c Loading commit data...
mremap.c Loading commit data...
msync.c Loading commit data...
nobootmem.c Loading commit data...
nommu.c Loading commit data...
oom_kill.c Loading commit data...
page-writeback.c Loading commit data...
page_alloc.c Loading commit data...
page_counter.c Loading commit data...
page_ext.c Loading commit data...
page_idle.c Loading commit data...
page_io.c Loading commit data...
page_isolation.c Loading commit data...
page_owner.c Loading commit data...
page_poison.c Loading commit data...
pagewalk.c Loading commit data...
percpu-km.c Loading commit data...
percpu-vm.c Loading commit data...
percpu.c Loading commit data...
pgtable-generic.c Loading commit data...
process_vm_access.c Loading commit data...
quicklist.c Loading commit data...
readahead.c Loading commit data...
rmap.c Loading commit data...
shmem.c Loading commit data...
slab.c Loading commit data...
slab.h Loading commit data...
slab_common.c Loading commit data...
slob.c Loading commit data...
slub.c Loading commit data...
sparse-vmemmap.c Loading commit data...
sparse.c Loading commit data...
swap.c Loading commit data...
swap_cgroup.c Loading commit data...
swap_state.c Loading commit data...
swapfile.c Loading commit data...
truncate.c Loading commit data...
userfaultfd.c Loading commit data...
util.c Loading commit data...
vmacache.c Loading commit data...
vmalloc.c Loading commit data...
vmpressure.c Loading commit data...
vmscan.c Loading commit data...
vmstat.c Loading commit data...
workingset.c Loading commit data...
z3fold.c Loading commit data...
zbud.c Loading commit data...
zpool.c Loading commit data...
zsmalloc.c Loading commit data...
zswap.c Loading commit data...