Waiman Long d10e819d13 locking/rwsem: Allow slowpath writer to ignore handoff bit if not set by first waiter ... commit 6eebd5fb20 upstream. With commit d257cc8cb8 ("locking/rwsem: Make handoff bit handling more consistent"), the writer that sets the handoff bit can be interrupted out without clearing the bit if the wait queue isn't empty. This disables reader and writer optimistic lock spinning and stealing. Now if a non-first writer in the queue is somehow woken up or a new waiter enters the slowpath, it can't acquire the lock. This is not the case before commit d257cc8cb8 as the writer that set the handoff bit will clear it when exiting out via the out_nolock path. This is less efficient as the busy rwsem stays in an unlock state for a longer time. In some cases, this new behavior may cause lockups as shown in [1] and [2]. This patch allows a non-first writer to ignore the handoff bit if it is not originally set or initiated by the first waiter. This patch is shown to be effective in fixing the lockup problem reported in [1]. [1] https://lore.kernel.org/lkml/20220617134325.GC30825@techsingularity.net/ [2] https://lore.kernel.org/lkml/3f02975c-1a9d-be20-32cf-f1d8e3dfafcc@oracle.com/ Fixes: d257cc8cb8 ("locking/rwsem: Make handoff bit handling more consistent") Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: John Donnelly <john.p.donnelly@oracle.com> Tested-by: Mel Gorman <mgorman@techsingularity.net> Link: https://lore.kernel.org/r/20220622200419.778799-1-longman@redhat.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>		2022-08-03 12:03:56 +02:00
..
bpf	bpf: Make sure mac_header was set before using it	2022-07-29 17:25:24 +02:00
cgroup	cgroup: Use separate src/dst nodes when preloading css_sets for migration	2022-07-21 21:24:13 +02:00
configs
debug	lockdown: also lock down previous kgdb use	2022-05-25 09:57:37 +02:00
dma	dma-direct: use the correct size for dma_set_encrypted()	2022-06-29 09:03:31 +02:00
entry	signal: Replace force_fatal_sig with force_exit_sig when in doubt	2021-11-25 09:49:07 +01:00
events	perf/core: Fix data race between perf_event_set_output() and perf_mmap_close()	2022-07-29 17:25:11 +02:00
gcov	Kconfig: Introduce ARCH_WANTS_NO_INSTR and CC_HAS_NO_PROFILE_FN_ATTR	2021-06-22 11:07:18 -07:00
irq	random: remove unused irq_flags argument from add_interrupt_randomness()	2022-05-30 09:29:00 +02:00
kcsan	LKMM updates:	2021-09-02 13:00:15 -07:00
livepatch	livepatch: Fix build failure on 32 bits processors	2022-04-08 14:23:29 +02:00
locking	locking/rwsem: Allow slowpath writer to ignore handoff bit if not set by first waiter	2022-08-03 12:03:56 +02:00
power	PM: suspend: fix return value of __setup handler	2022-04-08 14:23:07 +02:00
printk	printk: wake waiters for safe and NMI contexts	2022-06-09 10:22:49 +02:00
rcu	rcu: Make TASKS_RUDE_RCU select IRQ_WORK	2022-06-09 10:22:32 +02:00
sched	sched/deadline: Fix BUG_ON condition for deboosted tasks	2022-07-29 17:25:24 +02:00
time	fix race between exit_itimers() and /proc/pid/timers	2022-07-21 21:24:11 +02:00
trace	tracing: Fix return value of trace_pid_write()	2022-07-29 17:25:29 +02:00
.gitignore
acct.c	kernel/acct.c: use dedicated helper to access rlimit values	2021-09-08 11:50:26 -07:00
async.c	Revert "module, async: async_synchronize_full() on module init iff async is used"	2022-02-23 12:03:07 +01:00
audit.c	audit: improve audit queue handling when "audit=1" on cmdline	2022-02-08 18:34:03 +01:00
audit.h	audit: log AUDIT_TIME_* records only from rules	2022-04-08 14:23:06 +02:00
audit_fsnotify.c
audit_tree.c	audit: move put_tree() to avoid trim_trees refcount underflow and UAF	2021-08-24 18:52:36 -04:00
audit_watch.c
auditfilter.c
auditsc.c	audit: log AUDIT_TIME_* records only from rules	2022-04-08 14:23:06 +02:00
backtracetest.c
bounds.c
capability.c
cfi.c	cfi: Fix __cfi_slowpath_diag RCU usage with cpuidle	2022-06-22 14:22:04 +02:00
compat.c	arch: remove compat_alloc_user_space	2021-09-08 15:32:35 -07:00
configs.c
context_tracking.c
cpu.c	random: clear fast pool, crng, and batches in cpuhp bring up	2022-05-30 09:29:09 +02:00
cpu_pm.c	PM: cpu: Make notifier chain use a raw_spinlock_t	2021-08-16 18:55:32 +02:00
crash_core.c	kernel/crash_core: suppress unknown crashkernel parameter warning	2021-12-29 12:28:49 +01:00
crash_dump.c
cred.c	ucounts: Base set_cred_ucounts changes on the real user	2022-02-23 12:03:20 +01:00
delayacct.c
dma.c
exec_domain.c
exit.c	fix race between exit_itimers() and /proc/pid/timers	2022-07-21 21:24:11 +02:00
extable.c
fail_function.c
fork.c	sched: Fix yet more sched_fork() races	2022-03-08 19:12:49 +01:00
freezer.c	sched: Add get_current_state()	2021-06-18 11:43:08 +02:00
futex.c	futex: Remove unused variable 'vpid' in futex_proxy_trylock_atomic()	2021-09-03 23:00:22 +02:00
gen_kheaders.sh	kbuild: clean up ${quiet} checks in shell scripts	2021-05-27 04:01:50 +09:00
groups.c
hung_task.c	Merge branch 'akpm' (patches from Andrew)	2021-07-02 12:08:10 -07:00
iomem.c
irq_work.c	irq_work: Make irq_work_queue() NMI-safe again	2021-06-10 10:00:08 +02:00
jump_label.c	jump_label: Fix jump_label_text_reserved() vs __init	2021-07-05 10:46:20 +02:00
kallsyms.c	module: add printk formats to add module build ID to stacktraces	2021-07-08 11:48:22 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks	locking/rwlock: Provide RT variant	2021-08-17 17:50:51 +02:00
Kconfig.preempt	sched/core: Disable CONFIG_SCHED_CORE by default	2021-06-28 22:43:05 +02:00
kcov.c
kexec.c	kexec: avoid compat_alloc_user_space	2021-09-08 15:32:34 -07:00
kexec_core.c	Merge branch 'rework/printk_safe-removal' into for-linus	2021-08-30 16:36:10 +02:00
kexec_elf.c
kexec_file.c	ima: force signature verification when CONFIG_KEXEC_SIG is configured	2022-07-21 21:24:29 +02:00
kexec_internal.h
kheaders.c
kmod.c
kprobes.c	kprobes: Limit max data_size of the kretprobe instances	2021-12-08 09:04:41 +01:00
ksysfs.c
kthread.c	Merge branch 'akpm' (patches from Andrew)	2021-06-29 17:29:11 -07:00
latencytop.c
Makefile	static_call: Don't make __static_call_return0 static	2022-04-13 20:59:28 +02:00
module-internal.h
module.c	module: fix [e_shstrndx].sh_size=0 OOB access	2022-07-12 16:35:09 +02:00
module_signature.c
module_signing.c
notifier.c	notifier: Remove atomic_notifier_call_chain_robust()	2021-08-16 18:55:32 +02:00
nsproxy.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
padata.c	padata: Remove repeated verbose license text	2021-08-27 16:30:18 +08:00
panic.c	Merge branch 'rework/printk_safe-removal' into for-linus	2021-08-30 16:36:10 +02:00
params.c	params: lift param_set_uint_minmax to common code	2021-08-16 14:42:22 +02:00
pid.c	kernel/pid.c: implement additional checks upon pidfd_create() parameters	2021-08-10 12:53:07 +02:00
pid_namespace.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
profile.c	profiling: fix shift-out-of-bounds bugs	2021-09-08 11:50:26 -07:00
ptrace.c	ptrace: Reimplement PTRACE_KILL by always sending SIGKILL	2022-06-09 10:22:29 +02:00
range.c
reboot.c	reboot: Add hardware protection power-off	2021-06-21 13:08:36 +01:00
regset.c
relay.c
resource.c	kernel/resource: fix kfree() of bootmem memory again	2022-04-08 14:23:43 +02:00
resource_kunit.c
rseq.c	rseq: Remove broken uapi field layout on 32-bit little endian	2022-04-08 14:23:10 +02:00
scftorture.c	scftorture: Fix distribution of short handler delays	2022-06-09 10:22:46 +02:00
scs.c	scs: Release kasan vmalloc poison in scs_free process	2021-11-18 19:16:29 +01:00
seccomp.c	seccomp: Invalidate seccomp mode to catch death failures	2022-02-16 12:56:38 +01:00
signal.c	signal handling: don't use BUG_ON() for debugging	2022-07-21 21:24:42 +02:00
smp.c	smp: Fix offline cpu check in flush_smp_call_function_queue()	2022-04-20 09:34:21 +02:00
smpboot.c	smpboot: Replace deprecated CPU-hotplug functions.	2021-08-10 14:57:42 +02:00
smpboot.h
softirq.c	genirq: Change force_irqthreads to a static key	2021-08-10 22:50:07 +02:00
stackleak.c	gcc-plugins/stackleak: Use noinstr in favor of notrace	2022-02-23 12:03:07 +01:00
stacktrace.c	stacktrace: move filter_irq_stacks() to kernel/stacktrace.c	2022-04-13 20:59:28 +02:00
static_call.c	static_call: Don't make __static_call_return0 static	2022-04-13 20:59:28 +02:00
static_call_inline.c	static_call: Don't make __static_call_return0 static	2022-04-13 20:59:28 +02:00
stop_machine.c
sys.c	ucounts: Move RLIMIT_NPROC handling after set_user	2022-02-23 12:03:20 +01:00
sys_ni.c	compat: remove some compat entry points	2021-09-08 15:32:35 -07:00
sysctl-test.c	kernel/sysctl-test: Remove some casts which are no-longer required	2021-06-23 16:41:24 -06:00
sysctl.c	sysctl: move some boundary constants from sysctl.c to sysctl_vals	2022-07-29 17:25:11 +02:00
task_work.c
taskstats.c
test_kprobes.c
torture.c	torture: Replace deprecated CPU-hotplug functions.	2021-08-10 10:48:07 -07:00
tracepoint.c	tracepoint: Fix kerneldoc comments	2021-08-16 11:39:51 -04:00
tsacct.c	taskstats: Cleanup the use of task->exit_code	2022-01-27 11:05:35 +01:00
ucount.c	ucounts: Handle wrapping in is_ucounts_overlimit	2022-02-23 12:03:20 +01:00
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c	fs/epoll: use a per-cpu counter for user's watches count	2021-09-08 11:50:27 -07:00
user_namespace.c	ucounts: Fix systemd LimitNPROC with private users regression	2022-03-08 19:12:42 +01:00
usermode_driver.c	Merge branch 'work.namei' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2021-07-03 11:41:14 -07:00
utsname.c
utsname_sysctl.c
watch_queue.c	watch_queue: Fix missing locking in add_watch_to_object()	2022-08-03 12:03:43 +02:00
watchdog.c	kernel: watchdog: modify the explanation related to watchdog thread	2021-06-29 10:53:46 -07:00
watchdog_hld.c
workqueue.c	workqueue: Fix unbind_workers() VS wq_worker_running() race	2022-01-16 09:12:41 +01:00
workqueue_internal.h	workqueue: Assign a color to barrier work items	2021-08-17 07:49:10 -10:00