mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-09-12 21:57:43 +00:00
4a8e65b0c3
SRCU callbacks acceleration might fail if the preceding callbacks
advance also fails. This can happen when the following steps are met:
1) The RCU_WAIT_TAIL segment has callbacks (say for gp_num 8) and the
RCU_NEXT_READY_TAIL also has callbacks (say for gp_num 12).
2) The grace period for RCU_WAIT_TAIL is observed as started but not yet
completed so rcu_seq_current() returns 4 + SRCU_STATE_SCAN1 = 5.
3) This value is passed to rcu_segcblist_advance() which can't move
any segment forward and fails.
4) srcu_gp_start_if_needed() still proceeds with callback acceleration.
But then the call to rcu_seq_snap() observes the grace period for the
RCU_WAIT_TAIL segment (gp_num 8) as completed and the subsequent one
for the RCU_NEXT_READY_TAIL segment as started
(ie: 8 + SRCU_STATE_SCAN1 = 9) so it returns a snapshot of the
next grace period, which is 16.
5) The value of 16 is passed to rcu_segcblist_accelerate() but the
freshly enqueued callback in RCU_NEXT_TAIL can't move to
RCU_NEXT_READY_TAIL which already has callbacks for a previous grace
period (gp_num = 12). So acceleration fails.
6) Note in all these steps, srcu_invoke_callbacks() hadn't had a chance
to run srcu_invoke_callbacks().
Then some very bad outcome may happen if the following happens:
7) Some other CPU races and starts the grace period number 16 before the
CPU handling previous steps had a chance. Therefore srcu_gp_start()
isn't called on the latter sdp to fix the acceleration leak from
previous steps with a new pair of call to advance/accelerate.
8) The grace period 16 completes and srcu_invoke_callbacks() is finally
called. All the callbacks from previous grace periods (8 and 12) are
correctly advanced and executed but callbacks in RCU_NEXT_READY_TAIL
still remain. Then rcu_segcblist_accelerate() is called with a
snaphot of 20.
9) Since nothing started the grace period number 20, callbacks stay
unhandled.
This has been reported in real load:
[3144162.608392] INFO: task kworker/136:12:252684 blocked for more
than 122 seconds.
[3144162.615986] Tainted: G O K 5.4.203-1-tlinux4-0011.1 #1
[3144162.623053] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[3144162.631162] kworker/136:12 D 0 252684 2 0x90004000
[3144162.631189] Workqueue: kvm-irqfd-cleanup irqfd_shutdown [kvm]
[3144162.631192] Call Trace:
[3144162.631202] __schedule+0x2ee/0x660
[3144162.631206] schedule+0x33/0xa0
[3144162.631209] schedule_timeout+0x1c4/0x340
[3144162.631214] ? update_load_avg+0x82/0x660
[3144162.631217] ? raw_spin_rq_lock_nested+0x1f/0x30
[3144162.631218] wait_for_completion+0x119/0x180
[3144162.631220] ? wake_up_q+0x80/0x80
[3144162.631224] __synchronize_srcu.part.19+0x81/0xb0
[3144162.631226] ? __bpf_trace_rcu_utilization+0x10/0x10
[3144162.631227] synchronize_srcu+0x5f/0xc0
[3144162.631236] irqfd_shutdown+0x3c/0xb0 [kvm]
[3144162.631239] ? __schedule+0x2f6/0x660
[3144162.631243] process_one_work+0x19a/0x3a0
[3144162.631244] worker_thread+0x37/0x3a0
[3144162.631247] kthread+0x117/0x140
[3144162.631247] ? process_one_work+0x3a0/0x3a0
[3144162.631248] ? __kthread_cancel_work+0x40/0x40
[3144162.631250] ret_from_fork+0x1f/0x30
Fix this with taking the snapshot for acceleration _before_ the read
of the current grace period number.
The only side effect of this solution is that callbacks advancing happen
then _after_ the full barrier in rcu_seq_snap(). This is not a problem
because that barrier only cares about:
1) Ordering accesses of the update side before call_srcu() so they don't
bleed.
2) See all the accesses prior to the grace period of the current gp_num
The only things callbacks advancing need to be ordered against are
carried by snp locking.
Reported-by: Yong He <alexyonghe@tencent.com>
Co-developed-by:: Yong He <alexyonghe@tencent.com>
Signed-off-by: Yong He <alexyonghe@tencent.com>
Co-developed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Co-developed-by: Neeraj upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj upadhyay <Neeraj.Upadhyay@amd.com>
Link: http://lore.kernel.org/CANZk6aR+CqZaqmMWrC2eRRPY12qAZnDZLwLnHZbNi=xXMB401g@mail.gmail.com
Fixes:
|
||
---|---|---|
.. | ||
bpf | ||
cgroup | ||
configs | ||
debug | ||
dma | ||
entry | ||
events | ||
futex | ||
gcov | ||
irq | ||
kcsan | ||
livepatch | ||
locking | ||
module | ||
power | ||
printk | ||
rcu | ||
sched | ||
time | ||
trace | ||
.gitignore | ||
acct.c | ||
async.c | ||
audit.c | ||
audit.h | ||
audit_fsnotify.c | ||
audit_tree.c | ||
audit_watch.c | ||
auditfilter.c | ||
auditsc.c | ||
backtracetest.c | ||
bounds.c | ||
capability.c | ||
cfi.c | ||
compat.c | ||
configs.c | ||
context_tracking.c | ||
cpu.c | ||
cpu_pm.c | ||
crash_core.c | ||
crash_dump.c | ||
cred.c | ||
delayacct.c | ||
dma.c | ||
exec_domain.c | ||
exit.c | ||
extable.c | ||
fail_function.c | ||
fork.c | ||
freezer.c | ||
gen_kheaders.sh | ||
groups.c | ||
hung_task.c | ||
iomem.c | ||
irq_work.c | ||
jump_label.c | ||
kallsyms.c | ||
kallsyms_internal.h | ||
kallsyms_selftest.c | ||
kallsyms_selftest.h | ||
kcmp.c | ||
Kconfig.freezer | ||
Kconfig.hz | ||
Kconfig.kexec | ||
Kconfig.locks | ||
Kconfig.preempt | ||
kcov.c | ||
kexec.c | ||
kexec_core.c | ||
kexec_elf.c | ||
kexec_file.c | ||
kexec_internal.h | ||
kheaders.c | ||
kprobes.c | ||
ksyms_common.c | ||
ksysfs.c | ||
kthread.c | ||
latencytop.c | ||
Makefile | ||
module_signature.c | ||
notifier.c | ||
nsproxy.c | ||
padata.c | ||
panic.c | ||
params.c | ||
pid.c | ||
pid_namespace.c | ||
pid_sysctl.h | ||
profile.c | ||
ptrace.c | ||
range.c | ||
reboot.c | ||
regset.c | ||
relay.c | ||
resource.c | ||
resource_kunit.c | ||
rseq.c | ||
scftorture.c | ||
scs.c | ||
seccomp.c | ||
signal.c | ||
smp.c | ||
smpboot.c | ||
smpboot.h | ||
softirq.c | ||
stackleak.c | ||
stacktrace.c | ||
static_call.c | ||
static_call_inline.c | ||
stop_machine.c | ||
sys.c | ||
sys_ni.c | ||
sysctl-test.c | ||
sysctl.c | ||
task_work.c | ||
taskstats.c | ||
torture.c | ||
tracepoint.c | ||
tsacct.c | ||
ucount.c | ||
uid16.c | ||
uid16.h | ||
umh.c | ||
up.c | ||
user-return-notifier.c | ||
user.c | ||
user_namespace.c | ||
usermode_driver.c | ||
utsname.c | ||
utsname_sysctl.c | ||
vhost_task.c | ||
watch_queue.c | ||
watchdog.c | ||
watchdog_buddy.c | ||
watchdog_perf.c | ||
workqueue.c | ||
workqueue_internal.h |