• Catalin Marinas's avatar
    arm64: Invalidate the TLB corresponding to intermediate page table levels · 285994a6
    Catalin Marinas authored
    The ARM architecture allows the caching of intermediate page table
    levels and page table freeing requires a sequence like:
    
    	pmd_clear()
    	TLB invalidation
    	pte page freeing
    
    With commit 5e5f6dc1 (arm64: mm: enable HAVE_RCU_TABLE_FREE logic),
    the page table freeing batching was moved from tlb_remove_page() to
    tlb_remove_table(). The former takes care of TLB invalidation as this is
    also shared with pte clearing and page cache page freeing. The latter,
    however, does not invalidate the TLBs for intermediate page table levels
    as it probably relies on the architecture code to do it if required.
    When the mm->mm_users < 2, tlb_remove_table() does not do any batching
    and page table pages are freed before tlb_finish_mmu() which performs
    the actual TLB invalidation.
    
    This patch introduces __tlb_flush_pgtable() for arm64 and calls it from
    the {pte,pmd,pud}_free_tlb() directly without relying on deferred page
    table freeing.
    
    Fixes: 5e5f6dc1
    
     arm64: mm: enable HAVE_RCU_TABLE_FREE logic
    Reported-by: default avatarJon Masters <jcm@redhat.com>
    Tested-by: default avatarJon Masters <jcm@redhat.com>
    Tested-by: default avatarSteve Capper <steve.capper@linaro.org>
    Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
    285994a6