linux-stable/kernel
Waiman Long 668802c257 tick/broadcast: Reduce lock cacheline contention
It was observed that on an Intel x86 system without the ARAT (Always
running APIC timer) feature and with fairly large number of CPUs as
well as CPUs coming in and out of intel_idle frequently, the lock
contention on the tick_broadcast_lock can become significant.

To reduce contention, the lock is put into its own cacheline and all
the cpumask_var_t variables are put into the __read_mostly section.

Running the SP benchmark of the NAS Parallel Benchmarks on a 4-socket
16-core 32-thread Nehalam system, the performance number improved
from 3353.94 Mop/s to 3469.31 Mop/s when this patch was applied on
a 4.9.6 kernel.  This is a 3.4% improvement.

Signed-off-by: Waiman Long <longman@redhat.com>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/1485799063-20857-1-git-send-email-longman@redhat.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2017-02-04 08:54:46 +01:00
..
bpf bpf: do not use KMALLOC_SHIFT_MAX 2017-01-10 18:31:54 -08:00
configs config: android: enable CONFIG_SECCOMP 2016-10-11 15:06:32 -07:00
debug kdb: call vkdb_printf() from vprintk_default() only when wanted 2016-12-14 16:04:08 -08:00
events Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2016-12-23 16:49:12 -08:00
gcov
irq genirq/affinity: Fix node generation from cpumask 2016-12-15 12:32:35 +01:00
livepatch livepatch/module: make TAINT_LIVEPATCH module-specific 2016-08-26 14:42:08 +02:00
locking Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
power Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
printk Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
rcu rcu: Don't kick unless grace period or request 2016-11-14 10:46:31 -08:00
sched timers: Omit POSIX timer stuff from task_struct when disabled 2017-01-27 13:05:26 -08:00
time tick/broadcast: Reduce lock cacheline contention 2017-02-04 08:54:46 +01:00
trace clocksource: Use a plain u64 instead of cycle_t 2016-12-25 11:04:12 +01:00
.gitignore
acct.c
async.c
audit.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-12-17 18:44:00 -08:00
audit.h audit_log_{name,link_denied}: constify struct path 2016-12-05 19:00:38 -05:00
audit_fsnotify.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-12-17 18:44:00 -08:00
audit_tree.c Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit 2017-01-05 23:06:06 -08:00
audit_watch.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-12-17 18:44:00 -08:00
auditfilter.c audit: add support for session ID user filter 2016-11-29 15:10:12 -05:00
auditsc.c Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit 2016-12-14 14:06:40 -08:00
backtracetest.c
bounds.c
capability.c capability: export has_capability 2017-01-12 07:01:56 -07:00
cgroup.c cgroup: add support for eBPF programs 2016-11-25 16:25:52 -05:00
cgroup_freezer.c
cgroup_pids.c
compat.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
configs.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
context_tracking.c
cpu.c smp/hotplug: Undo tglxs brainfart 2016-12-26 17:30:24 -08:00
cpu_pm.c
cpuset.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
extable.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
fork.c timers: Omit POSIX timer stuff from task_struct when disabled 2017-01-27 13:05:26 -08:00
freezer.c
futex.c ktime: Get rid of the union 2016-12-25 17:21:22 +01:00
futex_compat.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
groups.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
hung_task.c hung_task: decrement sysctl_hung_task_warnings only if it is positive 2016-12-12 18:55:09 -08:00
irq_work.c
jump_label.c jump_labels: API for flushing deferred jump label updates 2017-01-12 14:33:16 +01:00
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES 2016-10-25 11:31:51 +02:00
Kconfig.preempt
kcov.c kcov: make kcov work properly with KASLR enabled 2016-12-20 09:48:47 -08:00
kexec.c
kexec_core.c kexec: add cond_resched into kimage_alloc_crash_control_pages 2016-12-14 16:04:07 -08:00
kexec_file.c ima: on soft reboot, save the measurement list 2016-12-20 09:48:44 -08:00
kexec_internal.h kexec_file: Allow arch-specific memory walking for kexec_add_buffer 2016-11-30 23:14:57 +11:00
kmod.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
kprobes.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
ksysfs.c
kthread.c kthread: add __printf attributes 2016-12-12 18:55:06 -08:00
latencytop.c
Makefile Merge branch 'akpm' (patches from Andrew) 2016-12-14 17:25:18 -08:00
membarrier.c
memremap.c mm: fix devm_memremap_pages crash, use mem_hotplug_{begin, done} 2017-01-10 18:31:54 -08:00
module-internal.h
module.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
module_signing.c
notifier.c
nsproxy.c
padata.c padata: Remove unused but set variables 2016-10-25 11:08:10 +08:00
panic.c taint/module: Clean up global and module taint flags handling 2016-11-26 11:18:01 -08:00
params.c
pid.c
pid_namespace.c Merge branch 'nsfs-ioctls' into HEAD 2016-09-22 20:00:36 -05:00
profile.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
ptrace.c ptrace: Don't allow accessing an undumpable mm 2016-11-22 12:57:38 -06:00
range.c
reboot.c
relay.c relay: check array offset before using it 2016-12-14 16:04:08 -08:00
resource.c
seccomp.c Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2016-12-14 13:57:44 -08:00
signal.c signal: protect SIGNAL_UNKILLABLE from unintentional clearing. 2017-01-10 18:31:55 -08:00
smp.c kernel/smp: Tell the user we're bringing up secondary CPUs 2016-10-26 12:02:35 +02:00
smpboot.c kthread/smpboot: do not park in kthread_create_on_cpu() 2016-10-11 15:06:33 -07:00
smpboot.h
softirq.c softirq: Display IRQ_POLL for irq-poll statistics 2016-10-21 15:45:47 -06:00
stacktrace.c
stop_machine.c locking/core, stop_machine: Yield the CPU during stop machine() 2016-11-16 10:15:09 +01:00
sys.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
sys_ni.c move aio compat to fs/aio.c 2016-12-22 22:58:37 -05:00
sysctl.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
sysctl_binary.c sysctl: add KERN_CONT to deprecated_sysctl_warning() 2016-12-14 16:04:07 -08:00
task_work.c
taskstats.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-11-15 10:54:36 -05:00
test_kprobes.c
torture.c torture: Convert torture_shutdown() to hrtimer 2016-08-22 10:01:49 -07:00
tracepoint.c tracing: Have the reg function allow to fail 2016-12-09 09:13:30 -05:00
tsacct.c
ucount.c mntns: Add a limit on the number of mount namespaces. 2016-08-31 07:28:35 -05:00
uid16.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
up.c smp: Add function to execute a function synchronously on a CPU 2016-09-05 13:52:39 +02:00
user-return-notifier.c
user.c
user_namespace.c Merge branch 'nsfs-ioctls' into HEAD 2016-09-22 20:00:36 -05:00
utsname.c Merge branch 'nsfs-ioctls' into HEAD 2016-09-22 20:00:36 -05:00
utsname_sysctl.c
watchdog.c kernel/watchdog.c: move hardlockup detector to separate file 2016-12-14 16:04:08 -08:00
watchdog_hld.c kernel/watchdog.c: move hardlockup detector to separate file 2016-12-14 16:04:08 -08:00
workqueue.c Merge branch 'for-4.9' into for-4.10 2016-10-19 12:12:40 -04:00
workqueue_internal.h