Jason Low 37e9562453 locking/rwsem: Allow conservative optimistic spinning when readers have lock ... Commit 4fc828e24c ("locking/rwsem: Support optimistic spinning") introduced a major performance regression for workloads such as xfs_repair which mix read and write locking of the mmap_sem across many threads. The result was xfs_repair ran 5x slower on 3.16-rc2 than on 3.15 and using 20x more system CPU time. Perf profiles indicate in some workloads that significant time can be spent spinning on !owner. This is because we don't set the lock owner when readers(s) obtain the rwsem. In this patch, we'll modify rwsem_can_spin_on_owner() such that we'll return false if there is no lock owner. The rationale is that if we just entered the slowpath, yet there is no lock owner, then there is a possibility that a reader has the lock. To be conservative, we'll avoid spinning in these situations. This patch reduced the total run time of the xfs_repair workload from about 4 minutes 24 seconds down to approximately 1 minute 26 seconds, back to close to the same performance as on 3.15. Retesting of AIM7, which were some of the workloads used to test the original optimistic spinning code, confirmed that we still get big performance gains with optimistic spinning, even with this additional regression fix. Davidlohr found that while the 'custom' workload took a performance hit of ~-14% to throughput for >300 users with this additional patch, the overall gain with optimistic spinning is still ~+45%. The 'disk' workload even improved by ~+15% at >1000 users. Tested-by: Dave Chinner <dchinner@redhat.com> Acked-by: Davidlohr Bueso <davidlohr@hp.com> Signed-off-by: Jason Low <jason.low2@hp.com> Signed-off-by: Peter Zijlstra <peterz@infradead.org> Cc: Tim Chen <tim.c.chen@linux.intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/1404532172.2572.30.camel@j-VirtualBox Signed-off-by: Ingo Molnar <mingo@kernel.org>		2014-07-16 13:28:02 +02:00
..
debug	kernel/printk: use symbolic defines for console loglevels	2014-06-04 16:54:17 -07:00
events	Oleg Nesterov found and fixed a bug in the perf/ftrace/uprobes code where	2014-07-03 18:37:25 -07:00
gcov	gcov: add support for GCC 4.9	2014-06-10 15:34:46 -07:00
irq	genirq: Fix memory leak when calling irq_free_hwirqs()	2014-07-05 21:42:08 +02:00
locking	locking/rwsem: Allow conservative optimistic spinning when readers have lock	2014-07-16 13:28:02 +02:00
power	x86, kaslr: boot-time selectable with hibernation	2014-06-16 23:30:44 +02:00
printk	kernel/printk/printk.c: revert "printk: enable interrupts before calling console_trylock_for_printk()"	2014-07-03 09:21:54 -07:00
rcu	Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next	2014-06-03 12:57:53 -07:00
sched	Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2014-06-12 19:42:15 -07:00
time	Merge branch 'akpm' (patchbomb from Andrew) into next	2014-06-04 16:55:13 -07:00
trace	tracing: Remove ftrace_stop/start() from reading the trace file	2014-07-01 12:45:54 -04:00
.gitignore	Ignore generated file kernel/x509_certificate_list	2013-12-10 18:21:34 +00:00
acct.c	ipc, kernel: clear whitespace	2014-06-06 16:08:14 -07:00
async.c
audit.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next	2014-06-12 14:27:40 -07:00
audit.h	audit: Use struct net not pid_t to remember the network namespce to reply in	2014-03-20 10:10:53 -04:00
audit_tree.c	inotify: Fix reporting of cookies for inotify events	2014-02-18 11:17:17 +01:00
audit_watch.c	inotify: Fix reporting of cookies for inotify events	2014-02-18 11:17:17 +01:00
auditfilter.c	Merge git://git.infradead.org/users/eparis/audit	2014-04-12 12:38:53 -07:00
auditsc.c	auditsc: audit_krule mask accesses need bounds checking	2014-06-10 08:44:40 -07:00
backtracetest.c	kernel/backtracetest.c: replace no level printk by pr_info()	2014-06-04 16:54:14 -07:00
bounds.c	mm: do not allocate page->ptl dynamically, if spinlock_t fits to long	2013-12-20 12:25:45 -08:00
capability.c	fs,userns: Change inode_capable to capable_wrt_inode_uidgid	2014-06-10 13:57:22 -07:00
cgroup.c	cgroup: fix a race between cgroup_mount() and cgroup_kill_sb()	2014-06-30 10:16:26 -04:00
cgroup_freezer.c	cgroup: remove css_parent()	2014-05-16 13:22:48 -04:00
compat.c	kernel/compat.c: use sizeof() instead of sizeof	2014-06-04 16:54:19 -07:00
configs.c
context_tracking.c	x86/kprobes: Fix build errors and blacklist context_track_user	2014-06-14 09:07:44 +02:00
cpu.c	More ACPI and power management updates for 3.16-rc1	2014-06-12 13:14:19 -07:00
cpu_pm.c
cpuset.c	cpuset: break kernfs active protection in cpuset_write_resmask()	2014-07-01 16:42:28 -04:00
crash_dump.c
cred.c
delayacct.c	kernel/delayacct.c: remove redundant checking in __delayacct_add_tsk()	2013-11-13 12:09:12 +09:00
dma.c
elfcore.c	switch elf_core_write_extra_phdrs() to dump_emit()	2013-11-09 00:16:23 -05:00
exec_domain.c	kernel/exec_domain.c: code clean-up	2014-06-04 16:54:15 -07:00
exit.c	signals: mv {dis,}allow_signal() from sched.h/exit.c to signal.[ch]	2014-06-06 16:08:11 -07:00
extable.c	asmlinkage: Make main_extable_sort_needed visible	2014-02-13 18:13:22 -08:00
fork.c	tracing: Fix syscall_*regfunc() vs copy_process() race	2014-06-21 00:15:12 -04:00
freezer.c	libata, freezer: avoid block device removal while system is frozen	2013-12-19 13:50:32 -05:00
futex.c	Merge branch 'next' (accumulated 3.16 merge window patches) into master	2014-06-08 11:31:16 -07:00
futex_compat.c	compat: Get rid of (get|put)_compat_time(val|spec)	2014-02-02 14:09:12 -08:00
groups.c	kernel/groups.c: remove return value of set_groups	2014-04-03 16:21:05 -07:00
hrtimer.c	Merge branch 'perf/urgent' into perf/core, to resolve conflict and to prepare for new patches	2014-06-06 07:55:06 +02:00
hung_task.c	kernel/hung_task.c: convert simple_strtoul to kstrtouint	2014-06-04 16:54:15 -07:00
irq_work.c	perf/x86: Warn to early_printk() in case irq_work is too slow	2014-02-21 21:49:07 +01:00
itimer.c
jump_label.c	static_key: WARN on usage before jump_label_init was called	2013-10-19 19:45:35 -04:00
kallsyms.c	kernel: use macros from compiler.h instead of __attribute__((...))	2014-04-07 16:36:11 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz	kernel: remove CONFIG_USE_GENERIC_SMP_HELPERS	2013-11-15 09:32:22 +09:00
Kconfig.locks	locking/rwlocks: Introduce 'qrwlocks' - fair, queued rwlocks	2014-06-06 07:58:28 +02:00
Kconfig.preempt
kexec.c	kexec: save PG_head_mask in VMCOREINFO	2014-06-23 16:47:43 -07:00
kmod.c	signals: change wait_for_helper() to use kernel_sigaction()	2014-06-06 16:08:12 -07:00
kprobes.c	kprobes: Show blacklist entries via debugfs	2014-04-24 10:26:41 +02:00
ksysfs.c	kobject: Make support for uevent_helper optional.	2014-04-25 12:00:49 -07:00
kthread.c	kthread: fix return value of kthread_create() upon SIGKILL.	2014-06-04 16:53:51 -07:00
latencytop.c	kernel/latencytop.c: convert seq_printf to seq_puts	2014-06-04 16:54:15 -07:00
Makefile	Merge branch 'x86-asmlinkage-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2014-03-31 14:13:25 -07:00
module-internal.h	KEYS: Separate the kernel signature checking keyring from module signing	2013-09-25 17:17:01 +01:00
module.c	Most of this is cleaning up various driver sysfs permissions so we can	2014-06-11 16:09:14 -07:00
module_signing.c	keys: change asymmetric keys to use common hash definitions	2013-10-25 17:15:18 -04:00
notifier.c	kprobes, notifier: Use NOKPROBE_SYMBOL macro in notifier	2014-04-24 10:26:39 +02:00
nsproxy.c
padata.c	padata: Fix wrong usage of rcu_dereference()	2013-12-05 21:28:42 +08:00
panic.c	kernel/panic.c: add "crash_kexec_post_notifiers" option for kdump after panic_notifers	2014-06-06 16:08:12 -07:00
params.c	param: hand arguments after -- straight to init	2014-04-28 11:48:34 +09:30
pid.c	pidns: fix free_pid() to handle the first fork failure	2013-09-30 14:31:03 -07:00
pid_namespace.c	pid_namespace: pidns_get() should check task_active_pid_ns() != NULL	2014-04-02 16:20:21 -07:00
posix-cpu-timers.c	posix-timers: Convert abuses of BUG_ON to WARN_ON	2013-12-09 16:56:29 +01:00
posix-timers.c
profile.c	kernel/profile.c: use static const char instead of static char	2014-06-06 16:08:13 -07:00
ptrace.c	kernel/compat: convert to COMPAT_SYSCALL_DEFINE	2014-03-06 15:35:10 +01:00
range.c
reboot.c	kernel/reboot.c: convert simple_strtoul to kstrtoint	2014-06-04 16:54:15 -07:00
relay.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2014-04-12 14:49:50 -07:00
res_counter.c	kernel/res_counter.c: replace simple_strtoull by kstrtoull	2014-06-04 16:54:15 -07:00
resource.c	resources: Clarify sanity check message	2014-05-23 10:47:21 -06:00
seccomp.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next	2014-06-12 14:27:40 -07:00
signal.c	signals: introduce kernel_sigaction()	2014-06-06 16:08:12 -07:00
smp.c	CPU hotplug, smp: flush any pending IPI callbacks before CPU offline	2014-06-23 16:47:43 -07:00
smpboot.c
smpboot.h
softirq.c	Merge branch 'rcu/next' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu	2014-05-22 11:36:10 +02:00
stacktrace.c
stop_machine.c	kernel/stop_machine.c: kernel-doc warning fix	2014-06-04 16:54:15 -07:00
sys.c	sched: Consolidate open coded implementations of nice level frobbing into nice_to_rlimit() and rlimit_to_nice()	2014-05-22 11:16:36 +02:00
sys_ni.c	sys_sgetmask/sys_ssetmask: add CONFIG_SGETMASK_SYSCALL	2014-06-04 16:54:14 -07:00
sysctl.c	kernel/watchdog.c: print traces for all cpus on lockup detection	2014-06-23 16:47:44 -07:00
sysctl_binary.c	kernel/sysctl_binary.c: use scnprintf() instead of snprintf()	2013-11-13 12:09:33 +09:00
system_certificates.S	KEYS: correct alignment of system_certificate_list content in assembly file	2013-12-10 18:25:28 +00:00
system_keyring.c	KEYS: correct alignment of system_certificate_list content in assembly file	2013-12-10 18:25:28 +00:00
task_work.c
taskstats.c	genetlink: only pass array to genl_register_family_with_ops()	2013-11-19 16:39:05 -05:00
test_kprobes.c
time.c
timeconst.bc
timer.c	timer: Prevent overflow in apply_slack	2014-04-30 13:46:17 +02:00
torture.c	torture: Remove __init from torture_init_begin/end	2014-05-14 09:46:30 -07:00
tracepoint.c	tracing: syscall_regfunc() should not skip kernel threads	2014-06-21 00:15:26 -04:00
tsacct.c
uid16.c
up.c	smp: Rename __smp_call_function_single() to smp_call_function_single_async()	2014-02-24 14:47:15 -08:00
user-return-notifier.c
user.c	kernel/user.c: drop unused field 'files' from user_struct	2014-06-04 16:54:16 -07:00
user_namespace.c	kernel/user_namespace.c: kernel-doc/checkpatch fixes	2014-06-06 16:08:13 -07:00
utsname.c
utsname_sysctl.c	sysctl: convert use of typedef ctl_table to struct ctl_table	2014-06-06 16:08:16 -07:00
watchdog.c	kernel/watchdog.c: print traces for all cpus on lockup detection	2014-06-23 16:47:44 -07:00
workqueue.c	workqueue: zero cpumask of wq_numa_possible_cpumask on init	2014-07-07 09:56:48 -04:00
workqueue_internal.h	workqueue: rename manager_mutex to attach_mutex	2014-05-20 10:59:32 -04:00