Skip to content
  • Andrew Bresticker's avatar
    memcg: replace ss->id_lock with a rwlock · c1e2ee2d
    Andrew Bresticker authored
    
    
    While back-porting Johannes Weiner's patch "mm: memcg-aware global
    reclaim" for an internal effort, we noticed a significant performance
    regression during page-reclaim heavy workloads due to high contention of
    the ss->id_lock.  This lock protects idr map, and serializes calls to
    idr_get_next() in css_get_next() (which is used during the memcg hierarchy
    walk).
    
    Since idr_get_next() is just doing a look up, we need only serialize it
    with respect to idr_remove()/idr_get_new().  By making the ss->id_lock a
    rwlock, contention is greatly reduced and performance improves.
    
    Tested: cat a 256m file from a ramdisk in a 128m container 50 times on
    each core (one file + container per core) in parallel on a NUMA machine.
    Result is the time for the test to complete in 1 of the containers.
    Both kernels included Johannes' memcg-aware global reclaim patches.
    
    Before rwlock patch: 1710.778s
    After rwlock patch: 152.227s
    
    Signed-off-by: default avatarAndrew Bresticker <abrestic@google.com>
    Cc: Paul Menage <menage@gmail.com>
    Cc: Li Zefan <lizf@cn.fujitsu.com>
    Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
    Cc: Ying Han <yinghan@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    c1e2ee2d