Skip to content
  • Will Deacon's avatar
    mm: mmu_gather: use tlb->end != 0 only for TLB invalidation · 721c21c1
    Will Deacon authored
    When batching up address ranges for TLB invalidation, we check tlb->end
    != 0 to indicate that some pages have actually been unmapped.
    
    As of commit f045bbb9 ("mmu_gather: fix over-eager
    tlb_flush_mmu_free() calling"), we use the same check for freeing these
    pages in order to avoid a performance regression where we call
    free_pages_and_swap_cache even when no pages are actually queued up.
    
    Unfortunately, the range could have been reset (tlb->end = 0) by
    tlb_end_vma, which has been shown to cause memory leaks on arm64.
    Furthermore, investigation into these leaks revealed that the fullmm
    case on task exit no longer invalidates the TLB, by virtue of tlb->end
     == 0 (in 3.18, need_flush would have been set).
    
    This patch resolves the problem by reverting commit f045bbb9
    
    , using
    instead tlb->local.nr as the predicate for page freeing in
    tlb_flush_mmu_free and ensuring that tlb->end is initialised to a
    non-zero value in the fullmm case.
    
    Tested-by: default avatarMark Langsdorf <mlangsdo@redhat.com>
    Tested-by: default avatarDave Hansen <dave@sr71.net>
    Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    721c21c1