Commit 75f30861 authored by David Rientjes's avatar David Rientjes Committed by Linus Torvalds
Browse files

mm, thp: avoid excessive compaction latency during fault

Synchronous memory compaction can be very expensive: it can iterate an
enormous amount of memory without aborting, constantly rescheduling,
waiting on page locks and lru_lock, etc, if a pageblock cannot be

Unfortunately, it's too expensive for transparent hugepage page faults and
it's much better to simply fallback to pages.  On 128GB machines, we find
that synchronous memory compaction can take O(seconds) for a single thp

Now that async compaction remembers where it left off without strictly
relying on sync compaction, this makes thp allocations best-effort without
causing egregious latency during fault.  We still need to retry async
compaction after reclaim, but this won't stall for seconds.
Signed-off-by: default avatarDavid Rientjes <>
Acked-by: default avatarMel Gorman <>
Cc: Greg Thelen <>
Cc: Naoya Horiguchi <>
Cc: Vlastimil Babka <>
Cc: Hugh Dickins <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarLinus Torvalds <>
parent e0b9daeb
......@@ -2575,7 +2575,14 @@ rebalance:
if (page)
goto got_pg;
migration_mode = MIGRATE_SYNC_LIGHT;
* It can become very expensive to allocate transparent hugepages at
* fault, so use asynchronous memory compaction for THP unless it is
* khugepaged trying to collapse.
if (!(gfp_mask & __GFP_NO_KSWAPD) || (current->flags & PF_KTHREAD))
migration_mode = MIGRATE_SYNC_LIGHT;
* If compaction is deferred for high-order allocations, it is because
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment