Skip to content
  • Ingo Molnar's avatar
    mm: Optimize the TLB flush of sys_mprotect() and change_protection() users · 1233d588
    Ingo Molnar authored
    
    
    Reuse the NUMA code's 'modified page protections' count that
    change_protection() computes and skip the TLB flush if there's
    no changes to a range that sys_mprotect() modifies.
    
    Given that mprotect() already optimizes the same-flags case
    I expected this optimization to dominantly trigger on
    CONFIG_NUMA_BALANCING=y kernels - but even with that feature
    disabled it triggers rather often.
    
    There's two reasons for that:
    
    1)
    
    While sys_mprotect() already optimizes the same-flag case:
    
            if (newflags == oldflags) {
                    *pprev = vma;
                    return 0;
            }
    
    and this test works in many cases, but it is too sharp in some
    others, where it differentiates between protection values that the
    underlying PTE format makes no distinction about, such as
    PROT_EXEC == PROT_READ on x86.
    
    2)
    
    Even where the pte format over vma flag changes necessiates a
    modification of the pagetables, there might be no pagetables
    yet to modify: they might not be instantiated yet.
    
    During a regular desktop bootup this optimization hits a couple
    of hundred times. During a Java test I measured thousands of
    hits.
    
    So this optimization improves sys_mprotect() in general, not just
    CONFIG_NUMA_BALANCING=y kernels.
    
    [ We could further increase the efficiency of this optimization if
      change_pte_range() and change_huge_pmd() was a bit smarter about
      recognizing exact-same-value protection masks - when the hardware
      can do that safely. This would probably further speed up mprotect(). ]
    
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    1233d588