mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-10-28 23:24:50 +00:00
e12f31d3e5
Both avg_overlap and avg_wakeup had an inherent problem in that their accuracy was detrimentally affected by cross-cpu wakeups, this because we are missing the necessary call to update_curr(). This can't be fixed without increasing overhead in our already too fat fastpath. Additionally, with recent load balancing changes making us prefer to place tasks in an idle cache domain (which is good for compute bound loads), communicating tasks suffer when a sync wakeup, which would enable affine placement, is turned into a non-sync wakeup by SYNC_LESS. With one task on the runqueue, wake_affine() rejects the affine wakeup request, leaving the unfortunate where placed, taking frequent cache misses. Remove it, and recover some fastpath cycles. Signed-off-by: Mike Galbraith <efault@gmx.de> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> LKML-Reference: <1268301121.6785.30.camel@marge.simson.net> Signed-off-by: Ingo Molnar <mingo@elte.hu>
96 lines
2.6 KiB
C
96 lines
2.6 KiB
C
/*
|
|
* Disregards a certain amount of sleep time (sched_latency_ns) and
|
|
* considers the task to be running during that period. This gives it
|
|
* a service deficit on wakeup, allowing it to run sooner.
|
|
*/
|
|
SCHED_FEAT(FAIR_SLEEPERS, 1)
|
|
|
|
/*
|
|
* Only give sleepers 50% of their service deficit. This allows
|
|
* them to run sooner, but does not allow tons of sleepers to
|
|
* rip the spread apart.
|
|
*/
|
|
SCHED_FEAT(GENTLE_FAIR_SLEEPERS, 1)
|
|
|
|
/*
|
|
* By not normalizing the sleep time, heavy tasks get an effective
|
|
* longer period, and lighter task an effective shorter period they
|
|
* are considered running.
|
|
*/
|
|
SCHED_FEAT(NORMALIZED_SLEEPER, 0)
|
|
|
|
/*
|
|
* Place new tasks ahead so that they do not starve already running
|
|
* tasks
|
|
*/
|
|
SCHED_FEAT(START_DEBIT, 1)
|
|
|
|
/*
|
|
* Should wakeups try to preempt running tasks.
|
|
*/
|
|
SCHED_FEAT(WAKEUP_PREEMPT, 1)
|
|
|
|
/*
|
|
* When converting the wakeup granularity to virtual time, do it such
|
|
* that heavier tasks preempting a lighter task have an edge.
|
|
*/
|
|
SCHED_FEAT(ASYM_GRAN, 1)
|
|
|
|
/*
|
|
* Always wakeup-preempt SYNC wakeups, see SYNC_WAKEUPS.
|
|
*/
|
|
SCHED_FEAT(WAKEUP_SYNC, 0)
|
|
|
|
/*
|
|
* Use the SYNC wakeup hint, pipes and the likes use this to indicate
|
|
* the remote end is likely to consume the data we just wrote, and
|
|
* therefore has cache benefit from being placed on the same cpu, see
|
|
* also AFFINE_WAKEUPS.
|
|
*/
|
|
SCHED_FEAT(SYNC_WAKEUPS, 1)
|
|
|
|
/*
|
|
* Based on load and program behaviour, see if it makes sense to place
|
|
* a newly woken task on the same cpu as the task that woke it --
|
|
* improve cache locality. Typically used with SYNC wakeups as
|
|
* generated by pipes and the like, see also SYNC_WAKEUPS.
|
|
*/
|
|
SCHED_FEAT(AFFINE_WAKEUPS, 1)
|
|
|
|
/*
|
|
* Prefer to schedule the task we woke last (assuming it failed
|
|
* wakeup-preemption), since its likely going to consume data we
|
|
* touched, increases cache locality.
|
|
*/
|
|
SCHED_FEAT(NEXT_BUDDY, 0)
|
|
|
|
/*
|
|
* Prefer to schedule the task that ran last (when we did
|
|
* wake-preempt) as that likely will touch the same data, increases
|
|
* cache locality.
|
|
*/
|
|
SCHED_FEAT(LAST_BUDDY, 1)
|
|
|
|
/*
|
|
* Consider buddies to be cache hot, decreases the likelyness of a
|
|
* cache buddy being migrated away, increases cache locality.
|
|
*/
|
|
SCHED_FEAT(CACHE_HOT_BUDDY, 1)
|
|
|
|
/*
|
|
* Use arch dependent cpu power functions
|
|
*/
|
|
SCHED_FEAT(ARCH_POWER, 0)
|
|
|
|
SCHED_FEAT(HRTICK, 0)
|
|
SCHED_FEAT(DOUBLE_TICK, 0)
|
|
SCHED_FEAT(LB_BIAS, 1)
|
|
SCHED_FEAT(LB_SHARES_UPDATE, 1)
|
|
SCHED_FEAT(ASYM_EFF_LOAD, 1)
|
|
|
|
/*
|
|
* Spin-wait on mutex acquisition when the mutex owner is running on
|
|
* another cpu -- assumes that when the owner is running, it will soon
|
|
* release the lock. Decreases scheduling overhead.
|
|
*/
|
|
SCHED_FEAT(OWNER_SPIN, 1)
|