Commit 3a79d52a authored by Waiman Long's avatar Waiman Long Committed by Linus Torvalds

mm, thp: replace smp_mb after atomic_add by smp_mb__after_atomic

In some architectures like x86, atomic_add() is a full memory barrier.
In that case, an additional smp_mb() is just a waste of time.  This
patch replaces that smp_mb() by smp_mb__after_atomic() which will avoid
the redundant memory barrier in some architectures.

With a 3.16-rc1 based kernel, this patch reduced the execution time of
breaking 1000 transparent huge pages from 38,245us to 30,964us.  A
reduction of 19% which is quite sizeable.  It also reduces the %cpu time
of the __split_huge_page_refcount function in the perf profile from
2.18% to 1.15%.
Signed-off-by: default avatarWaiman Long <>
Acked-by: default avatarKirill A. Shutemov <>
Cc: Andrea Arcangeli <>
Cc: Mel Gorman <>
Cc: Rik van Riel <>
Cc: Scott J Norton <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarLinus Torvalds <>
parent f8303c25
......@@ -1681,7 +1681,7 @@ static void __split_huge_page_refcount(struct page *page,
/* after clearing PageTail the gup refcount can be released */
* retain hwpoison flag of the poisoned tail page:
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment