Commit 672917dc authored by Corrado Zoccolo's avatar Corrado Zoccolo Committed by Linus Torvalds

cpuidle: menu governor: reduce latency on exit

Move the state residency accounting and statistics computation off the hot
exit path.

On exit, the need to recompute statistics is recorded, and new statistics
will be computed when menu_select is called again.

The expected effect is to reduce processor wakeup latency from sleep
(C-states).  We are speaking of few hundreds of cycles reduction out of a
several microseconds latency (determined by the hardware transition), so
it is difficult to measure.
Signed-off-by: default avatarCorrado Zoccolo <>
Cc: Venkatesh Pallipadi <>
Cc: Len Brown <>
Cc: Adam Belay <
Acked-by: default avatarArjan van de Ven <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarLinus Torvalds <>
parent 69d25870
......@@ -96,6 +96,7 @@
struct menu_device {
int last_state_idx;
int needs_update;
unsigned int expected_us;
u64 predicted_us;
......@@ -166,6 +167,8 @@ static inline int performance_multiplier(void)
static DEFINE_PER_CPU(struct menu_device, menu_devices);
static void menu_update(struct cpuidle_device *dev);
* menu_select - selects the next idle state to enter
* @dev: the CPU
......@@ -180,6 +183,11 @@ static int menu_select(struct cpuidle_device *dev)
data->last_state_idx = 0;
data->exit_us = 0;
if (data->needs_update) {
data->needs_update = 0;
/* Special case when user has set very strict latency requirement */
if (unlikely(latency_req == 0))
return 0;
......@@ -231,13 +239,23 @@ static int menu_select(struct cpuidle_device *dev)
* menu_reflect - attempts to guess what happened after entry
* menu_reflect - records that data structures need update
* @dev: the CPU
* NOTE: it's important to be fast here because this operation will add to
* the overall exit latency.
static void menu_reflect(struct cpuidle_device *dev)
struct menu_device *data = &__get_cpu_var(menu_devices);
data->needs_update = 1;
* menu_update - attempts to guess what happened after entry
* @dev: the CPU
static void menu_update(struct cpuidle_device *dev)
struct menu_device *data = &__get_cpu_var(menu_devices);
int last_idx = data->last_state_idx;
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment