linux-stable/kernel/rcu
Frederic Weisbecker 7625926086 rcu/nocb: Fix shrinker race against callback enqueuer
The shrinker resets the lazy callbacks counter in order to trigger the
pending lazy queue flush though the rcuog kthread. The counter reset is
protected by the ->nocb_lock against concurrent accesses...except
for one of them. Here is a list of existing synchronized readers/writer:

1) The first lazy enqueuer (incrementing ->lazy_len to 1) does so under
   ->nocb_lock and ->nocb_bypass_lock.

2) The further lazy enqueuers (incrementing ->lazy_len above 1) do so
   under ->nocb_bypass_lock _only_.

3) The lazy flush checks and resets to 0 under ->nocb_lock and
	->nocb_bypass_lock.

The shrinker protects its ->lazy_len reset against cases 1) and 3) but
not against 2). As such, setting ->lazy_len to 0 under the ->nocb_lock
may be cancelled right away by an overwrite from an enqueuer, leading
rcuog to ignore the flush.

To avoid that, use the proper bypass flush API which takes care of all
those details.

Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2023-05-09 17:26:58 -07:00
..
Kconfig rcu: Remove CONFIG_SRCU 2023-04-05 13:47:41 +00:00
Kconfig.debug rcu: Allow up to five minutes expedited RCU CPU stall-warning timeouts 2023-01-09 12:09:52 -08:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu.h Merge branches 'rcu/staging-core', 'rcu/staging-docs' and 'rcu/staging-kfree', remote-tracking branches 'paul/srcu-cf.2023.04.04a', 'fbq/rcu/lockdep.2023.03.27a' and 'fbq/rcu/rcutorture.2023.03.20a' into rcu/staging 2023-04-05 13:50:37 +00:00
rcu_segcblist.c rcu: Throttle callback invocation based on number of ready callbacks 2023-01-03 17:28:34 -08:00
rcu_segcblist.h rcu: Throttle callback invocation based on number of ready callbacks 2023-01-03 17:28:34 -08:00
rcuscale.c Merge branches 'rcu/staging-core', 'rcu/staging-docs' and 'rcu/staging-kfree', remote-tracking branches 'paul/srcu-cf.2023.04.04a', 'fbq/rcu/lockdep.2023.03.27a' and 'fbq/rcu/rcutorture.2023.03.20a' into rcu/staging 2023-04-05 13:50:37 +00:00
rcutorture.c Merge branches 'rcu/staging-core', 'rcu/staging-docs' and 'rcu/staging-kfree', remote-tracking branches 'paul/srcu-cf.2023.04.04a', 'fbq/rcu/lockdep.2023.03.27a' and 'fbq/rcu/rcutorture.2023.03.20a' into rcu/staging 2023-04-05 13:50:37 +00:00
refscale.c refscale: Move shutdown from wait_event() to wait_event_idle() 2023-03-20 11:12:16 -07:00
srcutiny.c rcu: Annotate SRCU's update-side lockdep dependencies 2023-03-27 11:15:59 -07:00
srcutree.c Merge branches 'rcu/staging-core', 'rcu/staging-docs' and 'rcu/staging-kfree', remote-tracking branches 'paul/srcu-cf.2023.04.04a', 'fbq/rcu/lockdep.2023.03.27a' and 'fbq/rcu/rcutorture.2023.03.20a' into rcu/staging 2023-04-05 13:50:37 +00:00
sync.c rcu/sync: Use call_rcu_hurry() instead of call_rcu 2022-11-29 14:04:33 -08:00
tasks.h Merge branches 'rcu/staging-core', 'rcu/staging-docs' and 'rcu/staging-kfree', remote-tracking branches 'paul/srcu-cf.2023.04.04a', 'fbq/rcu/lockdep.2023.03.27a' and 'fbq/rcu/rcutorture.2023.03.20a' into rcu/staging 2023-04-05 13:50:37 +00:00
tiny.c rcu: Refactor kvfree_call_rcu() and high-level helpers 2023-01-03 17:48:40 -08:00
tree.c RCU Changes for 6.4: 2023-04-24 12:16:14 -07:00
tree.h rcu: Add RCU stall diagnosis information 2023-01-05 12:21:11 -08:00
tree_exp.h rcu: Protect rcu_print_task_exp_stall() ->exp_tasks access 2023-04-05 13:47:44 +00:00
tree_nocb.h rcu/nocb: Fix shrinker race against callback enqueuer 2023-05-09 17:26:58 -07:00
tree_plugin.h rcu: Synchronize ->qsmaskinitnext in rcu_boost_kthread_setaffinity() 2022-10-18 14:59:57 -07:00
tree_stall.h rcu: Allow up to five minutes expedited RCU CPU stall-warning timeouts 2023-01-09 12:09:52 -08:00
update.c Merge branch 'stall.2023.01.09a' into HEAD 2023-02-02 16:40:07 -08:00