Waiman Long d37192627f blk-cgroup: Properly propagate the iostat update up the hierarchy ... [ Upstream commit 9d230c0996 ] During a cgroup_rstat_flush() call, the lowest level of nodes are flushed first before their parents. Since commit 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()"), iostat propagation was still done to the parent. Grandparent, however, may not get the iostat update if the parent has no blkg_iostat_set queued in its lhead lockless list. Fix this iostat propagation problem by queuing the parent's global blkg->iostat into one of its percpu lockless lists to make sure that the delta will always be propagated up to the grandparent and so on toward the root blkcg. Note that successive calls to __blkcg_rstat_flush() are serialized by the cgroup_rstat_lock. So no special barrier is used in the reading and writing of blkg->iostat.lqueued. Fixes: 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()") Reported-by: Dan Schatzberg <schatzberg.dan@gmail.com> Closes: https://lore.kernel.org/lkml/ZkO6l%2FODzadSgdhC@dschatzberg-fedora-PF3DHTBV/ Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Acked-by: Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/r/20240515143059.276677-1-longman@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>		2024-06-12 11:39:37 +02:00
..
partitions	block: fix and simplify blkdevparts= cmdline parsing	2024-05-30 09:44:10 +02:00
badblocks.c	badblocks: avoid checking invalid range in badblocks_check()	2023-12-23 18:38:08 -07:00
bdev.c	vfs-6.9-rc6.fixes	2024-04-26 11:01:28 -07:00
bfq-cgroup.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
bfq-iosched.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
bfq-iosched.h	block, bfq: remove BFQ_WEIGHT_LEGACY_DFL	2023-04-06 16:17:32 -06:00
bfq-wf2q.c	block, bfq: inject I/O to underutilized actuators	2023-01-29 15:18:33 -07:00
bio-integrity.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00
bio.c	for-6.9/block-20240310	2024-03-11 11:43:44 -07:00
blk-cgroup-fc-appid.c	block: Replace all non-returning strlcpy with strscpy	2023-06-01 09:13:31 -06:00
blk-cgroup-rwstat.c	Revert "blk-cgroup: pin the gendisk in struct blkcg_gq"	2023-02-14 14:24:09 -07:00
blk-cgroup-rwstat.h	block: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
blk-cgroup.c	blk-cgroup: Properly propagate the iostat update up the hierarchy	2024-06-12 11:39:37 +02:00
blk-cgroup.h	block: fix q->blkg_list corruption during disk rebind	2024-04-07 15:50:13 -06:00
blk-core.c	block: support to account io_ticks precisely	2024-05-30 09:44:10 +02:00
blk-crypto-fallback.c	block, fs: Restore the per-bio/request data lifetime fields	2024-02-06 14:31:05 +01:00
blk-crypto-internal.h	blk-crypto: remove blk_crypto_insert_cloned_request()	2023-03-16 09:35:09 -06:00
blk-crypto-profile.c	blk-crypto: use dynamic lock class for blk_crypto_profile::lock	2023-07-05 16:36:12 -06:00
blk-crypto-sysfs.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-crypto.c	blk-crypto: make blk_crypto_evict_key() more robust	2023-03-16 09:35:09 -06:00
blk-flush.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
blk-ia-ranges.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-integrity.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00
blk-ioc.c	blk-ioc: fix recursive spin_lock/unlock_irq() in ioc_clear_queue()	2023-06-07 07:51:00 -06:00
blk-iocost.c	blk-iocost: do not WARN if iocg was already offlined	2024-04-19 08:06:24 -06:00
blk-iolatency.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
blk-ioprio.c	blk-ioprio: Introduce promote-to-rt policy	2023-06-06 22:26:26 -06:00
blk-ioprio.h	blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit	2022-09-26 19:09:31 -06:00
blk-lib.c	Revert "blk-lib: check for kill signal"	2024-03-13 20:35:48 -06:00
blk-map.c	block: Fix WARNING in _copy_from_iter	2024-01-23 08:56:55 -07:00
blk-merge.c	block: support to account io_ticks precisely	2024-05-30 09:44:10 +02:00
blk-mq-cpumap.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-debugfs-zoned.c	block: move zone related fields to struct gendisk	2022-07-06 06:46:26 -06:00
blk-mq-debugfs.c	blk-mq: Remove the hctx 'run' debugfs attribute	2024-01-17 14:16:34 -07:00
blk-mq-debugfs.h	block: remove per-disk debugfs files in blk_unregister_queue	2022-06-17 07:31:05 -06:00
blk-mq-pci.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-sched.c	blk-mq: Remove the hctx 'run' debugfs attribute	2024-01-17 14:16:34 -07:00
blk-mq-sched.h	blk-mq: make sure elevator callbacks aren't called for passthrough request	2023-05-18 19:42:54 -06:00
blk-mq-sysfs.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-tag.c	for-6.5/block-2023-06-23	2023-06-26 12:47:20 -07:00
blk-mq-virtio.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq.c	block: support to account io_ticks precisely	2024-05-30 09:44:10 +02:00
blk-mq.h	blk-mq: update driver tags request table when start request	2023-09-22 08:52:13 -06:00
blk-pm.c	block: Remove blk_set_runtime_active()	2023-11-20 10:22:40 -07:00
blk-pm.h
blk-rq-qos.c	block: correct stale comment in rq_qos_wait	2023-09-18 14:15:28 -06:00
blk-rq-qos.h	block: skip QUEUE_FLAG_STATS and rq-qos for passthrough io	2023-12-01 18:29:18 -07:00
blk-settings.c	block: set default max segment size in case of virt_boundary	2024-05-06 20:27:51 -06:00
blk-stat.c	block: prevent division by zero in blk_rq_stat_sum()	2024-03-06 08:31:54 -07:00
blk-stat.h	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-sysfs.c	block: use queue_limits_commit_update in queue_discard_max_store	2024-02-13 08:56:59 -07:00
blk-throttle.c	blk-throttle: Eliminate redundant checks for data direction	2024-02-05 10:16:12 -07:00
blk-throttle.h	blk-throttle: print signed value 'carryover_bytes/ios' for user	2023-08-30 10:15:01 -06:00
blk-timeout.c
blk-wbt.c	for-6.9/block-20240310	2024-03-11 11:43:44 -07:00
blk-wbt.h	blk-wbt: remove the separate write cache tracking	2023-12-26 09:28:10 -07:00
blk-zoned.c	block: Do not include rbtree.h in blk-zoned.c	2024-02-22 10:35:18 -07:00
blk.h	block: support to account io_ticks precisely	2024-05-30 09:44:10 +02:00
bounce.c	block, fs: Restore the per-bio/request data lifetime fields	2024-02-06 14:31:05 +01:00
bsg-lib.c	block: pass a queue_limits argument to blk_mq_init_queue	2024-02-13 08:56:59 -07:00
bsg.c	SCSI misc on 20230629	2023-06-30 11:57:07 -07:00
disk-events.c	block: move bdev_mark_dead out of disk_check_media_change	2023-10-28 13:29:23 +02:00
early-lookup.c	block: don't return -EINVAL for not found names in devt_from_devname	2023-06-22 09:09:33 -06:00
elevator.c	blk-mq: release scheduler resource when request completes	2023-08-19 07:47:17 -06:00
elevator.h	blk-mq: pass a flags argument to elevator_type->insert_requests	2023-04-13 06:52:30 -06:00
fops.c	block: refine the EOF check in blkdev_iomap_begin	2024-05-30 09:44:10 +02:00
genhd.c	block: support to account io_ticks precisely	2024-05-30 09:44:10 +02:00
holder.c	block: fix deadlock between bd_link_disk_holder and partition scan	2024-02-23 07:44:19 -07:00
ioctl.c	block: propagate partition scanning errors to the BLKRRPART ioctl	2024-04-18 09:34:34 -06:00
ioprio.c	block: move __get_task_ioprio() into header file	2024-01-08 12:27:39 -07:00
Kconfig	block: Add config option to not allow writing to mounted devices	2023-11-18 14:59:25 +01:00
Kconfig.iosched	block: Default to use cgroup support for BFQ	2023-01-30 09:42:42 -07:00
kyber-iosched.c	blk-mq: pass a flags argument to elevator_type->insert_requests	2023-04-13 06:52:30 -06:00
Makefile	block: move the code to do early boot lookup of block devices to block/	2023-06-05 10:57:40 -06:00
mq-deadline.c	Revert "block/mq-deadline: use correct way to throttling write requests"	2024-03-13 15:56:14 -06:00
opal_proto.h	block: sed-opal: handle empty atoms when parsing response	2024-02-16 15:52:45 -07:00
sed-opal.c	for-6.9/block-20240310	2024-03-11 11:43:44 -07:00
t10-pi.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00