linux-stable/kernel/rcu
Paul E. McKenney 10c5357874 rcu: Don't deboost before reporting expedited quiescent state
Currently rcu_preempt_deferred_qs_irqrestore() releases rnp->boost_mtx
before reporting the expedited quiescent state.  Under heavy real-time
load, this can result in this function being preempted before the
quiescent state is reported, which can in turn prevent the expedited grace
period from completing.  Tim Murray reports that the resulting expedited
grace periods can take hundreds of milliseconds and even more than one
second, when they should normally complete in less than a millisecond.

This was fine given that there were no particular response-time
constraints for synchronize_rcu_expedited(), as it was designed
for throughput rather than latency.  However, some users now need
sub-100-millisecond response-time constratints.

This patch therefore follows Neeraj's suggestion (seconded by Tim and
by Uladzislau Rezki) of simply reversing the two operations.

Reported-by: Tim Murray <timmurray@google.com>
Reported-by: Joel Fernandes <joelaf@google.com>
Reported-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Reviewed-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Tested-by: Tim Murray <timmurray@google.com>
Cc: Todd Kjos <tkjos@google.com>
Cc: Sandeep Patil <sspatil@google.com>
Cc: <stable@vger.kernel.org> # 5.4.x
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2022-02-01 17:19:41 -08:00
..
Kconfig Merge branches 'doc.2021.11.30c', 'exp.2021.12.07a', 'fastnohz.2021.11.30c', 'fixes.2021.11.30c', 'nocb.2021.12.09a', 'nolibc.2021.11.30c', 'tasks.2021.12.09a', 'torture.2021.12.07a' and 'torturescript.2021.11.30c' into HEAD 2021-12-09 11:38:09 -08:00
Kconfig.debug rcu: Restrict RCU_STRICT_GRACE_PERIOD to at most four CPUs 2021-05-10 16:22:54 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu.h Merge branches 'bitmaprange.2021.05.10c', 'doc.2021.05.10c', 'fixes.2021.05.13a', 'kvfree_rcu.2021.05.10c', 'mmdumpobj.2021.05.10c', 'nocb.2021.05.12a', 'srcu.2021.05.12a', 'tasks.2021.05.18a' and 'torture.2021.05.10c' into HEAD 2021-05-18 10:56:19 -07:00
rcu_segcblist.c rcu/nocb: Invoke rcu_core() at the start of deoffloading 2021-12-07 16:24:44 -08:00
rcu_segcblist.h rcu/nocb: Prepare state machine for a new step 2021-12-07 16:24:44 -08:00
rcuscale.c rcuscale: Always log error message 2021-12-07 16:36:17 -08:00
rcutorture.c rcu: Mark accesses to boost_starttime 2022-02-01 17:19:02 -08:00
refscale.c refscale: Add missing '\n' to flush message 2021-12-07 16:36:12 -08:00
srcutiny.c srcu: Prevent redundant __srcu_read_unlock() wakeup 2021-11-30 17:28:16 -08:00
srcutree.c Merge branches 'bitmaprange.2021.05.10c', 'doc.2021.05.10c', 'fixes.2021.05.13a', 'kvfree_rcu.2021.05.10c', 'mmdumpobj.2021.05.10c', 'nocb.2021.05.12a', 'srcu.2021.05.12a', 'tasks.2021.05.18a' and 'torture.2021.05.10c' into HEAD 2021-05-18 10:56:19 -07:00
sync.c rcu: Fix various typos in comments 2021-05-12 12:11:05 -07:00
tasks.h rcu-tasks: Fix computation of CPU-to-list shift counts 2022-01-26 13:04:05 -08:00
tiny.c srcu: Initialize SRCU after timers 2021-05-10 16:03:35 -07:00
tree.c rcu: Elevate priority of offloaded callback threads 2022-02-01 17:19:25 -08:00
tree.h rcu: Remove unused rcu_state.boost 2022-02-01 17:19:02 -08:00
tree_exp.h rcu/exp: Mark current CPU as exp-QS in IPI loop second pass 2021-12-07 16:22:22 -08:00
tree_nocb.h rcu: Elevate priority of offloaded callback threads 2022-02-01 17:19:25 -08:00
tree_plugin.h rcu: Don't deboost before reporting expedited quiescent state 2022-02-01 17:19:41 -08:00
tree_stall.h rcu: Remove the RCU_FAST_NO_HZ Kconfig option 2021-11-30 17:24:47 -08:00
update.c RCU pull request for v5.16 2021-11-01 20:25:38 -07:00