mirrors
/
linux-stable
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
1
0
Fork 0
Code Issues Releases Wiki Activity
linux-stable/drivers/powercap/idle_inject.c

423 lines
13 KiB
C
Raw Normal View History

powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright 2018 Linaro Limited
*
* Author: Daniel Lezcano <daniel.lezcano@linaro.org>
*
* The idle injection framework provides a way to force CPUs to enter idle
* states for a specified fraction of time over a specified period.
*
* It relies on the smpboot kthreads feature providing common code for CPU
* hotplug and thread [un]parking.
*
* All of the kthreads used for idle injection are created at init time.
*
powercap: fix typo in a comment in idle_inject.c Drop superfluous "the" from the comment in line 15. Signed-off-by: Jason Wang <wangborong@cdjrlc.com> [ rjw: Subject edit, new changelog ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2021-12-12 03:06:40 +00:00
* Next, the users of the idle injection framework provide a cpumask via
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
* its register function. The kthreads will be synchronized with respect to
* this cpumask.
*
* The idle + run duration is specified via separate helpers and that allows
* idle injection to be started.
*
powercap: idle_inject: Replace play_idle() with play_idle_precise() in comments After commit 333cff6c963fbc ("powercap/drivers/idle_inject: Specify idle state max latency"), we convert to use play_idle_precise() with max allowed latency to specify the idle state. Some function comments still use play_idle(), let's update it to play_idle_precise(). Signed-off-by: Yangtao Li <tiny.windzz@gmail.com> Signed-off-by: Frank Lee <frank@allwinnertech.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-06-21 08:04:12 +00:00
* The idle injection kthreads will call play_idle_precise() with the idle
* duration and max allowed latency specified as per the above.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*
* After all of them have been woken up, a timer is set to start the next idle
* injection cycle.
*
* The timer interrupt handler will wake up the idle injection kthreads for
* all of the CPUs in the cpumask provided by the user.
*
* Idle injection is stopped synchronously and no leftover idle injection
* kthread activity after its completion is guaranteed.
*
* It is up to the user of this framework to provide a lock for higher-level
* synchronization to prevent race conditions like starting idle injection
* while unregistering from the framework.
*/
#define pr_fmt(fmt) "ii_dev: " fmt
#include <linux/cpu.h>
#include <linux/hrtimer.h>
#include <linux/kthread.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/smpboot.h>
powercap: include header to fix -Wmissing-prototypes Include the linux/idle_inject.h header to fix W=1 build warning: drivers/powercap/idle_inject.c:152:6: warning: no previous prototype for ‘idle_inject_set_duration’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:167:6: warning: no previous prototype for ‘idle_inject_get_duration’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:179:6: warning: no previous prototype for ‘idle_inject_set_latency’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:195:5: warning: no previous prototype for ‘idle_inject_start’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:227:6: warning: no previous prototype for ‘idle_inject_stop’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:299:28: warning: no previous prototype for ‘idle_inject_register’ [-Wmissing-prototypes] drivers/powercap/idle_inject.c:345:6: warning: no previous prototype for ‘idle_inject_unregister’ [-Wmissing-prototypes] Signed-off-by: Pujin Shi <shipj@lemote.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-09-22 04:46:52 +00:00
#include <linux/idle_inject.h>
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
#include <uapi/linux/sched/types.h>
/**
* struct idle_inject_thread - task on/off switch structure
* @tsk: task injecting the idle cycles
* @should_run: whether or not to run the task (for the smpboot kthread API)
*/
struct idle_inject_thread {
struct task_struct *tsk;
int should_run;
};
/**
* struct idle_inject_device - idle injection data
* @timer: idle injection period timer
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
* @idle_duration_us: duration of CPU idle time to inject
* @run_duration_us: duration of CPU run time to allow
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
* @latency_us: max allowed latency
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
* @update: Optional callback deciding whether or not to skip idle
* injection in the given cycle.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
* @cpumask: mask of CPUs affected by idle injection
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
*
* This structure is used to define per instance idle inject device data. Each
* instance has an idle duration, a run duration and mask of CPUs to inject
* idle.
*
* Actual CPU idle time is injected by calling kernel scheduler interface
* play_idle_precise(). There is one optional callback that can be registered
* by calling idle_inject_register_full():
*
* update() - This callback is invoked just before waking up CPUs to inject
* idle. If it returns false, CPUs are not woken up to inject idle in the given
* cycle. It also allows the caller to readjust the idle and run duration by
* calling idle_inject_set_duration() for the next cycle.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*/
struct idle_inject_device {
struct hrtimer timer;
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
unsigned int idle_duration_us;
unsigned int run_duration_us;
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
unsigned int latency_us;
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
bool (*update)(void);
powercap: idle_inject: Replace zero-length array with flexible-array member The current codebase makes use of the zero-length array language extension to the C90 standard, but the preferred mechanism to declare variable-length types such as these ones is a flexible array member[1][2], introduced in C99: struct foo { int stuff; struct boo array[]; }; By making use of the mechanism above, we will get a compiler warning in case the flexible array does not occur last in the structure, which will help us prevent some kind of undefined behavior bugs from being inadvertently introduced[3] to the codebase from now on. Also, notice that, dynamic memory allocations won't be affected by this change: "Flexible array members have incomplete type, and so the sizeof operator may not be applied. As a quirk of the original implementation of zero-length arrays, sizeof evaluates to zero."[1] Lastly, fix the following checkpatch warning: WARNING: Prefer 'unsigned long' over 'unsigned long int' as the int is unnecessary + unsigned long int cpumask[]; This issue was found with the help of Coccinelle. [1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html [2] https://github.com/KSPP/linux/issues/21 [3] commit 76497732932f ("cxgb3/l2t: Fix undefined behaviour") Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-02-27 19:07:21 +00:00
unsigned long cpumask[];
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
};
static DEFINE_PER_CPU(struct idle_inject_thread, idle_inject_thread);
static DEFINE_PER_CPU(struct idle_inject_device *, idle_inject_device);
/**
* idle_inject_wakeup - Wake up idle injection threads
* @ii_dev: target idle injection device
*
* Every idle injection task associated with the given idle injection device
* and running on an online CPU will be woken up.
*/
static void idle_inject_wakeup(struct idle_inject_device *ii_dev)
{
struct idle_inject_thread *iit;
unsigned int cpu;
for_each_cpu_and(cpu, to_cpumask(ii_dev->cpumask), cpu_online_mask) {
iit = per_cpu_ptr(&idle_inject_thread, cpu);
iit->should_run = 1;
wake_up_process(iit->tsk);
}
}
/**
* idle_inject_timer_fn - idle injection timer function
* @timer: idle injection hrtimer
*
* This function is called when the idle injection timer expires. It wakes up
* idle injection tasks associated with the timer and they, in turn, invoke
powercap: idle_inject: Replace play_idle() with play_idle_precise() in comments After commit 333cff6c963fbc ("powercap/drivers/idle_inject: Specify idle state max latency"), we convert to use play_idle_precise() with max allowed latency to specify the idle state. Some function comments still use play_idle(), let's update it to play_idle_precise(). Signed-off-by: Yangtao Li <tiny.windzz@gmail.com> Signed-off-by: Frank Lee <frank@allwinnertech.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-06-21 08:04:12 +00:00
* play_idle_precise() to inject a specified amount of CPU idle time.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*
* Return: HRTIMER_RESTART.
*/
static enum hrtimer_restart idle_inject_timer_fn(struct hrtimer *timer)
{
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
unsigned int duration_us;
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
struct idle_inject_device *ii_dev =
container_of(timer, struct idle_inject_device, timer);
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
if (!ii_dev->update || (ii_dev->update && ii_dev->update()))
idle_inject_wakeup(ii_dev);
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
duration_us = READ_ONCE(ii_dev->run_duration_us);
duration_us += READ_ONCE(ii_dev->idle_duration_us);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
hrtimer_forward_now(timer, ns_to_ktime(duration_us * NSEC_PER_USEC));
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
return HRTIMER_RESTART;
}
/**
* idle_inject_fn - idle injection work function
* @cpu: the CPU owning the task
*
powercap: idle_inject: Replace play_idle() with play_idle_precise() in comments After commit 333cff6c963fbc ("powercap/drivers/idle_inject: Specify idle state max latency"), we convert to use play_idle_precise() with max allowed latency to specify the idle state. Some function comments still use play_idle(), let's update it to play_idle_precise(). Signed-off-by: Yangtao Li <tiny.windzz@gmail.com> Signed-off-by: Frank Lee <frank@allwinnertech.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-06-21 08:04:12 +00:00
* This function calls play_idle_precise() to inject a specified amount of CPU
* idle time.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*/
static void idle_inject_fn(unsigned int cpu)
{
struct idle_inject_device *ii_dev;
struct idle_inject_thread *iit;
ii_dev = per_cpu(idle_inject_device, cpu);
iit = per_cpu_ptr(&idle_inject_thread, cpu);
/*
* Let the smpboot main loop know that the task should not run again.
*/
iit->should_run = 0;
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
play_idle_precise(READ_ONCE(ii_dev->idle_duration_us) * NSEC_PER_USEC,
READ_ONCE(ii_dev->latency_us) * NSEC_PER_USEC);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
/**
* idle_inject_set_duration - idle and run duration update helper
powercap: idle_inject: Fix warnings with make W=1 Fix following warning at three places: Function parameter or member 'ii_dev' not described. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-11-29 23:08:21 +00:00
* @ii_dev: idle injection control device structure
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
* @run_duration_us: CPU run time to allow in microseconds
* @idle_duration_us: CPU idle time to inject in microseconds
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*/
void idle_inject_set_duration(struct idle_inject_device *ii_dev,
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
unsigned int run_duration_us,
unsigned int idle_duration_us)
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
{
powercap: idle_inject: Support 100% idle injection The users of the idle injection framework allow 100% idle injection. For example: thermal/cpuidle_cooling.c driver. When the ratio is set to 100%, the runtime_duration becomes zero. However, idle_inject_set_duration() in the idle injection framework silently ignores run_duration_us == 0 without any error (it is a void function). The caller will then assume that everything is fine and 100% idle is effective, but in reality the idle duration will not change. There are two options: - The caller may change their max state to 99% instead of 100% and document that 100% is not supported by the idle inject framework. - Add 100% idle support to the idle inject framework. Since there are other protections via RT throttling, this framework can allow 100% idle. The RT throttling will be activated at 95% idle by default. The caller disabling RT throttling and injecting 100% idle, should be aware that CPU can't be used at all. The idle inject timer is started for (run_duration_us + idle_duration_us) duration. Hence replace (run_duration_us && idle_duration_us) with (run_duration_us + idle_duration_us) in the function idle_inject_set_duration(). Also check for !(run_duration_us + idle_duration_us) to return -EINVAL in idle_inject_start(). Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> [ rjw: Changelog edits ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-01-17 18:22:40 +00:00
if (run_duration_us + idle_duration_us) {
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
WRITE_ONCE(ii_dev->run_duration_us, run_duration_us);
WRITE_ONCE(ii_dev->idle_duration_us, idle_duration_us);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
powercap: idle_inject: Support 100% idle injection The users of the idle injection framework allow 100% idle injection. For example: thermal/cpuidle_cooling.c driver. When the ratio is set to 100%, the runtime_duration becomes zero. However, idle_inject_set_duration() in the idle injection framework silently ignores run_duration_us == 0 without any error (it is a void function). The caller will then assume that everything is fine and 100% idle is effective, but in reality the idle duration will not change. There are two options: - The caller may change their max state to 99% instead of 100% and document that 100% is not supported by the idle inject framework. - Add 100% idle support to the idle inject framework. Since there are other protections via RT throttling, this framework can allow 100% idle. The RT throttling will be activated at 95% idle by default. The caller disabling RT throttling and injecting 100% idle, should be aware that CPU can't be used at all. The idle inject timer is started for (run_duration_us + idle_duration_us) duration. Hence replace (run_duration_us && idle_duration_us) with (run_duration_us + idle_duration_us) in the function idle_inject_set_duration(). Also check for !(run_duration_us + idle_duration_us) to return -EINVAL in idle_inject_start(). Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> [ rjw: Changelog edits ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-01-17 18:22:40 +00:00
if (!run_duration_us)
pr_debug("CPU is forced to 100 percent idle\n");
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_set_duration, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
/**
* idle_inject_get_duration - idle and run duration retrieval helper
powercap: idle_inject: Fix warnings with make W=1 Fix following warning at three places: Function parameter or member 'ii_dev' not described. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-11-29 23:08:21 +00:00
* @ii_dev: idle injection control device structure
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
* @run_duration_us: memory location to store the current CPU run time
* @idle_duration_us: memory location to store the current CPU idle time
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*/
void idle_inject_get_duration(struct idle_inject_device *ii_dev,
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
unsigned int *run_duration_us,
unsigned int *idle_duration_us)
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
{
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
*run_duration_us = READ_ONCE(ii_dev->run_duration_us);
*idle_duration_us = READ_ONCE(ii_dev->idle_duration_us);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_get_duration, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
/**
* idle_inject_set_latency - set the maximum latency allowed
powercap: idle_inject: Fix warnings with make W=1 Fix following warning at three places: Function parameter or member 'ii_dev' not described. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-11-29 23:08:21 +00:00
* @ii_dev: idle injection control device structure
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
* @latency_us: set the latency requirement for the idle state
*/
void idle_inject_set_latency(struct idle_inject_device *ii_dev,
unsigned int latency_us)
{
WRITE_ONCE(ii_dev->latency_us, latency_us);
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_set_latency, IDLE_INJECT);
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
/**
* idle_inject_start - start idle injections
* @ii_dev: idle injection control device structure
*
* The function starts idle injection by first waking up all of the idle
* injection kthreads associated with @ii_dev to let them inject CPU idle time
* sets up a timer to start the next idle injection period.
*
* Return: -EINVAL if the CPU idle or CPU run time is not set or 0 on success.
*/
int idle_inject_start(struct idle_inject_device *ii_dev)
{
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
unsigned int idle_duration_us = READ_ONCE(ii_dev->idle_duration_us);
unsigned int run_duration_us = READ_ONCE(ii_dev->run_duration_us);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
powercap: idle_inject: Support 100% idle injection The users of the idle injection framework allow 100% idle injection. For example: thermal/cpuidle_cooling.c driver. When the ratio is set to 100%, the runtime_duration becomes zero. However, idle_inject_set_duration() in the idle injection framework silently ignores run_duration_us == 0 without any error (it is a void function). The caller will then assume that everything is fine and 100% idle is effective, but in reality the idle duration will not change. There are two options: - The caller may change their max state to 99% instead of 100% and document that 100% is not supported by the idle inject framework. - Add 100% idle support to the idle inject framework. Since there are other protections via RT throttling, this framework can allow 100% idle. The RT throttling will be activated at 95% idle by default. The caller disabling RT throttling and injecting 100% idle, should be aware that CPU can't be used at all. The idle inject timer is started for (run_duration_us + idle_duration_us) duration. Hence replace (run_duration_us && idle_duration_us) with (run_duration_us + idle_duration_us) in the function idle_inject_set_duration(). Also check for !(run_duration_us + idle_duration_us) to return -EINVAL in idle_inject_start(). Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> [ rjw: Changelog edits ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-01-17 18:22:40 +00:00
if (!(idle_duration_us + run_duration_us))
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
return -EINVAL;
pr_debug("Starting injecting idle cycles on CPUs '%*pbl'\n",
cpumask_pr_args(to_cpumask(ii_dev->cpumask)));
idle_inject_wakeup(ii_dev);
hrtimer_start(&ii_dev->timer,
powercap: idle_inject: Use higher resolution for idle injection The resolution of the idle injection is limited to 1ms. If there is a need for an injection of 1.2 ms, it is not possible. The idle injection API is not yet used, so it is safe to convert the existing API to the new time unit instead of adding more functions. Convert to microsecond in order to use a finer grain time unit when injecting idle cycles. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2019-08-02 17:34:24 +00:00
ns_to_ktime((idle_duration_us + run_duration_us) *
NSEC_PER_USEC),
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
HRTIMER_MODE_REL);
return 0;
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_start, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
/**
* idle_inject_stop - stops idle injections
* @ii_dev: idle injection control device structure
*
* The function stops idle injection and waits for the threads to finish work.
* If CPU idle time is being injected when this function runs, then it will
* wait until the end of the cycle.
*
* When it returns, there is no more idle injection kthread activity. The
* kthreads are scheduled out and the periodic timer is off.
*/
void idle_inject_stop(struct idle_inject_device *ii_dev)
{
struct idle_inject_thread *iit;
unsigned int cpu;
pr_debug("Stopping idle injection on CPUs '%*pbl'\n",
cpumask_pr_args(to_cpumask(ii_dev->cpumask)));
hrtimer_cancel(&ii_dev->timer);
/*
* Stopping idle injection requires all of the idle injection kthreads
* associated with the given cpumask to be parked and stay that way, so
* prevent CPUs from going online at this point. Any CPUs going online
* after the loop below will be covered by clearing the should_run flag
* that will cause the smpboot main loop to schedule them out.
*/
cpu_hotplug_disable();
/*
* Iterate over all (online + offline) CPUs here in case one of them
* goes offline with the should_run flag set so as to prevent its idle
* injection kthread from running when the CPU goes online again after
* the ii_dev has been freed.
*/
for_each_cpu(cpu, to_cpumask(ii_dev->cpumask)) {
iit = per_cpu_ptr(&idle_inject_thread, cpu);
iit->should_run = 0;
sched: Add TASK_ANY for wait_task_inactive() Now that wait_task_inactive()'s @match_state argument is a mask (like ttwu()) it is possible to replace the special !match_state case with an 'all-states' value such that any blocked state will match. Suggested-by: Ingo Molnar (mingo@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/YxhkzfuFTvRnpUaH@hirez.programming.kicks-ass.net
2022-09-06 10:39:55 +00:00
wait_task_inactive(iit->tsk, TASK_ANY);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
cpu_hotplug_enable();
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_stop, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
/**
* idle_inject_setup - prepare the current task for idle injection
* @cpu: not used
*
* Called once, this function is in charge of setting the current task's
* scheduler parameters to make it an RT task.
*/
static void idle_inject_setup(unsigned int cpu)
{
sched,powercap: Convert to sched_set_fifo*() Because SCHED_FIFO is a broken scheduler model (see previous patches) take away the priority field, the kernel can't possibly make an informed decision. Effectively no change. Cc: daniel.lezcano@linaro.org Cc: rafael.j.wysocki@intel.com Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Ingo Molnar <mingo@kernel.org>
2020-04-21 10:09:13 +00:00
sched_set_fifo(current);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
}
/**
* idle_inject_should_run - function helper for the smpboot API
* @cpu: CPU the kthread is running on
*
* Return: whether or not the thread can run.
*/
static int idle_inject_should_run(unsigned int cpu)
{
struct idle_inject_thread *iit =
per_cpu_ptr(&idle_inject_thread, cpu);
return iit->should_run;
}
/**
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
* idle_inject_register_full - initialize idle injection on a set of CPUs
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
* @cpumask: CPUs to be affected by idle injection
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
* @update: This callback is called just before waking up CPUs to inject
* idle
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*
* This function creates an idle injection control device structure for the
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
* given set of CPUs and initializes the timer associated with it. This
* function also allows to register update()callback.
* It does not start any injection cycles.
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
*
* Return: NULL if memory allocation fails, idle injection control device
* pointer on success.
*/
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
struct idle_inject_device *idle_inject_register_full(struct cpumask *cpumask,
bool (*update)(void))
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
{
struct idle_inject_device *ii_dev;
int cpu, cpu_rb;
ii_dev = kzalloc(sizeof(*ii_dev) + cpumask_size(), GFP_KERNEL);
if (!ii_dev)
return NULL;
cpumask_copy(to_cpumask(ii_dev->cpumask), cpumask);
hrtimer_init(&ii_dev->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
ii_dev->timer.function = idle_inject_timer_fn;
powercap/drivers/idle_inject: Specify idle state max latency Currently the idle injection framework uses the play_idle() function which puts the current CPU in an idle state. The idle state is the deepest one, as specified by the latency constraint when calling the subsequent play_idle_precise() function with the INT_MAX. The idle_injection is used by the cpuidle_cooling device which computes the idle / run duration to mitigate the temperature by injecting idle cycles. The cooling device has no control on the depth of the idle state. Allow finer control of the idle injection mechanism by allowing to specify the latency for the idle state. Thus the cooling device has the ability to have a guarantee on the exit latency of the idle states it is injecting. Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org> Link: https://lore.kernel.org/r/20200429103644.5492-1-daniel.lezcano@linaro.org
2020-04-29 10:36:39 +00:00
ii_dev->latency_us = UINT_MAX;
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
ii_dev->update = update;
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
for_each_cpu(cpu, to_cpumask(ii_dev->cpumask)) {
if (per_cpu(idle_inject_device, cpu)) {
pr_err("cpu%d is already registered\n", cpu);
goto out_rollback;
}
per_cpu(idle_inject_device, cpu) = ii_dev;
}
return ii_dev;
out_rollback:
for_each_cpu(cpu_rb, to_cpumask(ii_dev->cpumask)) {
if (cpu == cpu_rb)
break;
per_cpu(idle_inject_device, cpu_rb) = NULL;
}
kfree(ii_dev);
return NULL;
}
powercap: idle_inject: Add update callback The powercap/idle_inject core uses play_idle_precise() to inject idle time. But play_idle_precise() can't ensure that the CPU is fully idle for the specified duration because of wakeups due to interrupts. To compensate for the reduced idle time due to these wakes, the caller can adjust requested idle time for the next cycle. The goal of idle injection is to keep system at some idle percent on average, so this is fine to overshoot or undershoot instantaneous idle times. The idle inject core provides an interface idle_inject_set_duration() to set idle and runtime duration. Some architectures provide interface to get actual idle time observed by the hardware. So, the effective idle percent can be adjusted using the hardware feedback. For example, Intel CPUs provides package idle counters, which is currently used by Intel powerclamp driver to readjust runtime duration. When the caller's desired idle time over a period is less or greater than the actual CPU idle time observed by the hardware, caller can readjust idle and runtime duration for the next cycle. The only way this can be done currently is by monitoring hardware idle time from a different software thread and readjust idle and runtime duration using idle_inject_set_duration(). This can be avoided by adding a callback which callers can register and readjust from this callback function. Add a capability to register an optional update() callback, which can be called from the idle inject core before waking up CPUs for idle injection. This callback can be registered via a new interface: idle_inject_register_full(). During this process of constantly adjusting idle and runtime duration there can be some cases where actual idle time is more than the desired. In this case idle inject can be skipped for a cycle. If update() callback returns false, then the idle inject core skips waking up CPUs for the idle injection. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:52 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_register_full, IDLE_INJECT);
/**
* idle_inject_register - initialize idle injection on a set of CPUs
* @cpumask: CPUs to be affected by idle injection
*
* This function creates an idle injection control device structure for the
* given set of CPUs and initializes the timer associated with it. It does not
* start any injection cycles.
*
* Return: NULL if memory allocation fails, idle injection control device
* pointer on success.
*/
struct idle_inject_device *idle_inject_register(struct cpumask *cpumask)
{
return idle_inject_register_full(cpumask, NULL);
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_register, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
/**
* idle_inject_unregister - unregister idle injection control device
* @ii_dev: idle injection control device to unregister
*
* The function stops idle injection for the given control device,
* unregisters its kthreads and frees memory allocated when that device was
* created.
*/
void idle_inject_unregister(struct idle_inject_device *ii_dev)
{
unsigned int cpu;
idle_inject_stop(ii_dev);
for_each_cpu(cpu, to_cpumask(ii_dev->cpumask))
per_cpu(idle_inject_device, cpu) = NULL;
kfree(ii_dev);
}
powercap: idle_inject: Export symbols Export symbols for external interfaces, so that they can be used in other loadable modules. Export is done under name space IDLE_INJECT. Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-02-01 18:28:51 +00:00
EXPORT_SYMBOL_NS_GPL(idle_inject_unregister, IDLE_INJECT);
powercap / idle_inject: Add an idle injection framework Initially, the cpu_cooling device for ARM was changed by adding a new policy inserting idle cycles. The intel_powerclamp driver does a similar action. Instead of implementing idle injections privately in the cpu_cooling device, move the idle injection code in a dedicated framework and give the opportunity to other frameworks to make use of it. The framework relies on the smpboot kthreads which handles via its main loop the common code for hotplugging and [un]parking. This code was previously tested with the cpu cooling device and went through several iterations. It results now in split code and API exported in the header file. It was tested with the cpu cooling device with success. Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org> [ rjw: Rewrite of all comments ] Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2018-06-26 10:53:29 +00:00
static struct smp_hotplug_thread idle_inject_threads = {
.store = &idle_inject_thread.tsk,
.setup = idle_inject_setup,
.thread_fn = idle_inject_fn,
.thread_comm = "idle_inject/%u",
.thread_should_run = idle_inject_should_run,
};
static int __init idle_inject_init(void)
{
return smpboot_register_percpu_thread(&idle_inject_threads);
}
early_initcall(idle_inject_init);