Ming Lei 9f5ede3c01 block: throttle split bio in case of iops limit ... Commit 111be88398 ("block-throttle: avoid double charge") marks bio as BIO_THROTTLED unconditionally if __blk_throtl_bio() is called on this bio, then this bio won't be called into __blk_throtl_bio() any more. This way is to avoid double charge in case of bio splitting. It is reasonable for read/write throughput limit, but not reasonable for IOPS limit because block layer provides io accounting against split bio. Chunguang Xu has already observed this issue and fixed it in commit 4f1e9630af ("blk-throtl: optimize IOPS throttle for large IO scenarios"). However, that patch only covers bio splitting in __blk_queue_split(), and we have other kind of bio splitting, such as bio_split() & submit_bio_noacct() and other ways. This patch tries to fix the issue in one generic way by always charging the bio for iops limit in blk_throtl_bio(). This way is reasonable: re-submission & fast-cloned bio is charged if it is submitted to same disk/queue, and BIO_THROTTLED will be cleared if bio->bi_bdev is changed. This new approach can get much more smooth/stable iops limit compared with commit 4f1e9630af ("blk-throtl: optimize IOPS throttle for large IO scenarios") since that commit can't throttle current split bios actually. Also this way won't cause new double bio iops charge in blk_throtl_dispatch_work_fn() in which blk_throtl_bio() won't be called any more. Reported-by: Ning Li <lining2020x@163.com> Acked-by: Tejun Heo <tj@kernel.org> Cc: Chunguang Xu <brookxu@tencent.com> Signed-off-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/20220216044514.2903784-7-ming.lei@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk>		2022-02-16 19:42:28 -07:00
..
partitions	block: remove genhd.h	2022-02-02 07:49:59 -07:00
badblocks.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
bdev.c	block: deprecate autoloading based on dev_t	2022-02-02 07:49:58 -07:00
bfq-cgroup.c	block, bfq: fix UAF problem in bfqg_stats_init()	2021-10-19 15:18:30 -06:00
bfq-iosched.c	block: simplify ioc_lookup_icq	2021-11-29 06:41:29 -07:00
bfq-iosched.h	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
bfq-wf2q.c	block: Introduce IOPRIO_NR_LEVELS	2021-08-18 07:21:12 -06:00
bio-integrity.c	block: clone crypto and integrity data in __bio_clone_fast	2022-02-04 07:43:18 -07:00
bio.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-cgroup-rwstat.c	blk-cgroup: Fix the recursive blkg rwstat	2021-03-05 11:32:15 -07:00
blk-cgroup-rwstat.h	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-cgroup.c	blk-cgroup: set blkg iostat after percpu stat aggregation	2022-02-15 14:13:12 -07:00
blk-cgroup.h	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-core.c	block: merge submit_bio_checks() into submit_bio_noacct	2022-02-16 19:42:28 -07:00
blk-crypto-fallback.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-crypto-internal.h	block: move struct request to blk-mq.h	2021-10-18 06:17:02 -06:00
blk-crypto-profile.c	blk-crypto: remove blk_crypto_unregister()	2021-11-29 06:38:51 -07:00
blk-crypto.c	block: clone crypto and integrity data in __bio_clone_fast	2022-02-04 07:43:18 -07:00
blk-flush.c	block: pass a block_device and opf to bio_init	2022-02-02 07:49:59 -07:00
blk-ia-ranges.c	block: fix memory leak in disk_register_independent_access_ranges	2022-01-23 09:13:09 -07:00
blk-integrity.c	blk-crypto: remove blk_crypto_unregister()	2021-11-29 06:38:51 -07:00
blk-ioc.c	block: drop needless assignment in set_task_ioprio()	2021-12-23 07:10:07 -07:00
blk-iocost.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-iolatency.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-ioprio.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-ioprio.h	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-lib.c	blk-lib: don't check bdev_get_queue() NULL check	2022-02-15 07:51:46 -07:00
blk-map.c	Merge branch 'akpm' (patches from Andrew)	2021-09-03 10:08:28 -07:00
blk-merge.c	block: throttle split bio in case of iops limit	2022-02-16 19:42:28 -07:00
blk-mq-cpumap.c	blk-mq: remove the calling of local_memory_node()	2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c
blk-mq-debugfs.c	blk-mq: check q->poll_stat in queue_poll_stat_show	2021-12-02 08:20:49 -07:00
blk-mq-debugfs.h	blk-mq: no need to check return value of debugfs_create functions	2019-06-13 03:00:30 -06:00
blk-mq-pci.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-rdma.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-sched.c	blk-mq: run dispatch lock once in case of issuing from list	2021-12-03 14:51:29 -07:00
blk-mq-sched.h	block: move blk_mq_sched_assign_ioc to blk-ioc.c	2021-11-29 06:41:29 -07:00
blk-mq-sysfs.c	blk-mq: move srcu from blk_mq_hw_ctx to request_queue	2021-12-03 14:51:29 -07:00
blk-mq-tag.c	sbitmap: Delete old sbitmap_queue_get_shallow()	2022-02-08 06:55:49 -07:00
blk-mq-tag.h	blk-mq: Delete busy_iter_fn	2021-12-06 13:18:47 -07:00
blk-mq-virtio.c	blk-mq: Fix typo in comment	2020-03-17 20:55:21 +01:00
blk-mq.c	block: move blk_crypto_bio_prep() out of blk-mq.c	2022-02-16 19:42:27 -07:00
blk-mq.h	blk-mq: don't run might_sleep() if the operation needn't blocking	2021-12-06 09:40:42 -07:00
blk-pm.c	scsi: block: pm: Always set request queue runtime active in blk_post_runtime_resume()	2021-12-22 23:38:29 -05:00
blk-pm.h	block: Remove unused blk_pm_*() function definitions	2021-02-22 06:33:48 -07:00
blk-rq-qos.c	rq-qos: fix missed wake-ups in rq_qos_throttle try two	2021-06-08 15:12:57 -06:00
blk-rq-qos.h	block: only mark bio as tracked if it really is tracked	2021-10-18 08:50:47 -06:00
blk-settings.c	block: Fix partition check for host-aware zoned block devices	2021-10-27 06:58:01 -06:00
blk-stat.c	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-stat.h	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-sysfs.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
blk-throttle.c	block: throttle split bio in case of iops limit	2022-02-16 19:42:28 -07:00
blk-throttle.h	block: throttle split bio in case of iops limit	2022-02-16 19:42:28 -07:00
blk-timeout.c	block: blk-timeout: delete duplicated word	2020-07-31 16:29:47 -06:00
blk-wbt.c	blk-wbt: prevent NULL pointer dereference in wb_timer_fn	2021-10-19 06:13:41 -06:00
blk-wbt.h	blk-wbt: introduce a new disable state to prevent false positive by rwb_enabled()	2021-06-21 15:03:41 -06:00
blk-zoned.c	block: pass a block_device and opf to bio_init	2022-02-02 07:49:59 -07:00
blk.h	block: don't check bio in blk_throtl_dispatch_work_fn	2022-02-16 19:42:28 -07:00
bounce.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
bsg-lib.c	block: remove the gendisk argument to blk_execute_rq	2021-11-29 06:41:29 -07:00
bsg.c	scsi: bsg: Fix device unregistration	2021-09-14 00:22:15 -04:00
disk-events.c	block: remove genhd.h	2022-02-02 07:49:59 -07:00
elevator.c	block: partition include/linux/blk-cgroup.h	2022-02-11 10:02:41 -07:00
elevator.h	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
fops.c	block: pass a block_device and opf to bio_init	2022-02-02 07:49:59 -07:00
genhd.c	block: remove genhd.h	2022-02-02 07:49:59 -07:00
holder.c	block: remove genhd.h	2022-02-02 07:49:59 -07:00
ioctl.c	block: merge disk_scan_partitions and blkdev_reread_part	2021-11-29 06:35:21 -07:00
ioprio.c	for-5.17/block-2022-01-11	2022-01-12 10:26:52 -08:00
Kconfig	blk-mq: make the blk-mq stacking code optional	2022-02-16 19:39:09 -07:00
Kconfig.iosched	block: only build the icq tracking code when needed	2021-12-16 10:59:02 -07:00
kyber-iosched.c	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
Makefile	block: remove blk-exec.c	2021-11-29 06:34:50 -07:00
mq-deadline.c	block: fix async_depth sysfs interface for mq-deadline	2022-01-20 10:54:02 -07:00
opal_proto.h	block: sed-opal: Change the check condition for regular session validity	2020-03-12 08:00:10 -06:00
sed-opal.c	block: remove genhd.h	2022-02-02 07:49:59 -07:00
t10-pi.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00