linux-stable/block
Jan Kara 221f655b57 block: do not reverse request order when flushing plug list
[ Upstream commit 34e0a279a9 ]

Commit 26fed4ac4e ("block: flush plug based on hardware and software
queue order") changed flushing of plug list to submit requests one
device at a time. However while doing that it also started using
list_add_tail() instead of list_add() used previously thus effectively
submitting requests in reverse order. Also when forming a rq_list with
remaining requests (in case two or more devices are used), we
effectively reverse the ordering of the plug list for each device we
process. Submitting requests in reverse order has negative impact on
performance for rotational disks (when BFQ is not in use). We observe
10-25% regression in random 4k write throughput, as well as ~20%
regression in MariaDB OLTP benchmark on rotational storage on btrfs
filesystem.

Fix the problem by preserving ordering of the plug list when inserting
requests into the queuelist as well as by appending to requeue_list
instead of prepending to it.

Fixes: 26fed4ac4e ("block: flush plug based on hardware and software queue order")
Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20230313093002.11756-1-jack@suse.cz
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2023-03-22 13:33:46 +01:00
..
partitions block: don't add partitions if GD_SUPPRESS_PART_SCAN is set 2022-09-03 11:29:03 -06:00
badblocks.c block/badblocks: Remove redundant assignments 2022-04-23 07:15:26 -06:00
bdev.c vfs: support STATX_DIOALIGN on block devices 2022-09-11 19:47:12 -05:00
bfq-cgroup.c block, bfq: fix uaf for bfqq in bic_set_bfqq() 2023-02-09 11:28:06 +01:00
bfq-iosched.c block, bfq: fix uaf for bfqq in bic_set_bfqq() 2023-02-09 11:28:06 +01:00
bfq-iosched.h block, bfq: remove unused variable for bfq_queue 2022-10-20 05:46:49 -07:00
bfq-wf2q.c block, bfq: remove useless parameter for bfq_add/del_bfqq_busy() 2022-08-22 10:07:56 -06:00
bio-integrity.c block: bio-integrity: Copy flags when bio_integrity_payload is cloned 2023-03-10 09:32:46 +01:00
bio.c block: clear bio->bi_bdev when putting a bio back in the cache 2023-03-10 09:34:02 +01:00
blk-cgroup-fc-appid.c cgroup: Homogenize cgroup_get_from_id() return value 2022-08-26 10:57:41 -10:00
blk-cgroup-rwstat.c
blk-cgroup-rwstat.h block: Use the new blk_opf_t type 2022-07-14 12:14:30 -06:00
blk-cgroup.c Revert "blk-cgroup: dropping parent refcount after pd_free_fn() is done" 2023-03-11 11:03:45 +01:00
blk-cgroup.h blk-cgroup: pass a gendisk to blkcg_init_queue and blkcg_exit_queue 2022-09-26 19:09:31 -06:00
blk-core.c block: be a bit more careful in checking for NULL bdev while polling 2023-03-10 09:34:02 +01:00
blk-crypto-fallback.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
blk-crypto-internal.h blk-crypto: show crypto capabilities in sysfs 2022-02-28 06:40:23 -07:00
blk-crypto-profile.c blk-crypto: remove blk_crypto_unregister() 2021-11-29 06:38:51 -07:00
blk-crypto-sysfs.c blk-crypto: show crypto capabilities in sysfs 2022-02-28 06:40:23 -07:00
blk-crypto.c blk-crypto: show crypto capabilities in sysfs 2022-02-28 06:40:23 -07:00
blk-flush.c block: change request end_io handler to pass back a return value 2022-09-30 07:49:09 -06:00
blk-ia-ranges.c block: simplify disk_set_independent_access_ranges 2022-06-29 08:36:46 -06:00
blk-integrity.c blk-crypto: remove blk_crypto_unregister() 2021-11-29 06:38:51 -07:00
blk-ioc.c block: fix default IO priority handling again 2022-06-27 06:29:12 -06:00
blk-iocost.c blk-iocost: fix divide by 0 error in calc_lcoefs() 2023-03-10 09:33:46 +01:00
blk-iolatency.c blk-cgroup: pass a gendisk to blkcg_schedule_throttle 2022-09-26 19:17:28 -06:00
blk-ioprio.c blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit 2022-09-26 19:09:31 -06:00
blk-ioprio.h blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit 2022-09-26 19:09:31 -06:00
blk-lib.c blk-lib: fix blkdev_issue_secure_erase 2022-09-15 00:25:17 -06:00
blk-map.c block: extend functionality to map bvec iterator 2022-09-30 07:51:13 -06:00
blk-merge.c block: use proper return value from bio_failfast() 2023-03-10 09:32:47 +01:00
blk-mq-cpumap.c block: Change the return type of blk_mq_map_queues() into void 2022-08-22 10:07:53 -06:00
blk-mq-debugfs-zoned.c block: move zone related fields to struct gendisk 2022-07-06 06:46:26 -06:00
blk-mq-debugfs.c for-6.1/block-2022-10-03 2022-10-07 09:19:14 -07:00
blk-mq-debugfs.h block: remove per-disk debugfs files in blk_unregister_queue 2022-06-17 07:31:05 -06:00
blk-mq-pci.c block: Change the return type of blk_mq_map_queues() into void 2022-08-22 10:07:53 -06:00
blk-mq-rdma.c block: Change the return type of blk_mq_map_queues() into void 2022-08-22 10:07:53 -06:00
blk-mq-sched.c blk-mq: correct stale comment of .get_budget 2023-03-10 09:32:44 +01:00
blk-mq-sched.h block: move blk_mq_sched_assign_ioc to blk-ioc.c 2021-11-29 06:41:29 -07:00
blk-mq-sysfs.c blk-mq: fix possible memleak when register 'hctx' failed 2022-12-31 13:33:03 +01:00
blk-mq-tag.c sbitmap: fix batched wait_cnt accounting 2022-09-12 00:10:34 -06:00
blk-mq-tag.h blk-mq: blk_mq_tag_busy is no need to return a value 2022-06-27 06:29:12 -06:00
blk-mq-virtio.c block: Change the return type of blk_mq_map_queues() into void 2022-08-22 10:07:53 -06:00
blk-mq.c block: do not reverse request order when flushing plug list 2023-03-22 13:33:46 +01:00
blk-mq.h block: adapt blk_mq_plug() to not plug for writes that require a zone lock 2022-09-29 07:45:47 -06:00
blk-pm.c scsi: block: pm: Always set request queue runtime active in blk_post_runtime_resume() 2021-12-22 23:38:29 -05:00
blk-pm.h
blk-rq-qos.c block/rq_qos: Use atomic_try_cmpxchg in atomic_inc_below 2022-07-12 14:38:52 -06:00
blk-rq-qos.h block/blk-rq-qos: delete useless enmu RQ_QOS_IOPRIO 2022-09-21 19:50:53 -06:00
blk-settings.c block: make blk_set_default_limits() private 2022-11-16 15:58:11 -07:00
blk-stat.c block: make queue stat accounting a reference 2021-12-14 17:23:05 -07:00
blk-stat.h block: make queue stat accounting a reference 2021-12-14 17:23:05 -07:00
blk-sysfs.c blk-throttle: pass a gendisk to blk_throtl_register_queue 2022-09-26 19:17:27 -06:00
blk-throttle.c blk-throttle: pass a gendisk to blk_throtl_cancel_bios 2022-09-26 19:17:28 -06:00
blk-throttle.h blk-throttle: pass a gendisk to blk_throtl_cancel_bios 2022-09-26 19:17:28 -06:00
blk-timeout.c
blk-wbt.c blk-wbt: fix that 'rwb->wc' is always set to 1 in wbt_init() 2022-10-09 07:48:16 -06:00
blk-wbt.h blk-wbt: remove wbt_track stub 2022-03-31 12:58:38 -06:00
blk-zoned.c block: adapt blk_mq_plug() to not plug for writes that require a zone lock 2022-09-29 07:45:47 -06:00
blk.h block: Revert "block: Do not reread partition table on exclusively open device" 2023-03-17 08:50:20 +01:00
bounce.c block: change the blk_queue_bounce calling convention 2022-08-02 17:22:54 -06:00
bsg-lib.c blk-mq: Drop blk_mq_ops.timeout 'reserved' arg 2022-07-06 06:33:53 -06:00
bsg.c scsi: core: bsg: Remove usage of the deprecated ida_simple_xxx() API 2022-06-21 21:22:51 -04:00
disk-events.c block: remove genhd.h 2022-02-02 07:49:59 -07:00
elevator.c blk-mq: use quiesced elevator switch when reinitializing queues 2022-09-27 09:58:56 -06:00
elevator.h block: Use the new blk_opf_t type 2022-07-14 12:14:30 -06:00
fops.c block: don't allow multiple bios for IOCB_NOWAIT issue 2023-03-10 09:34:02 +01:00
genhd.c block: fix wrong mode for blkdev_put() from disk_scan_partitions() 2023-03-17 08:50:27 +01:00
holder.c block: remove WARN_ON() from bd_link_disk_holder 2022-06-23 07:48:05 -06:00
ioctl.c block: fix scan partition for exclusively open device again 2023-03-17 08:50:21 +01:00
ioprio.c block: Fix handling of tasks without ioprio in ioprio_get(2) 2022-06-27 06:29:12 -06:00
Kconfig block: remove "select BLK_RQ_IO_DATA_LEN" from BLK_CGROUP_IOCOST dependency 2022-06-29 08:35:57 -06:00
Kconfig.iosched block: only build the icq tracking code when needed 2021-12-16 10:59:02 -07:00
kyber-iosched.c block/kyber: Use the new blk_opf_t type 2022-07-14 12:14:30 -06:00
Makefile blk-cgroup: move blkcg_{get,set}_fc_appid out of line 2022-05-02 14:06:20 -06:00
mq-deadline.c block: mq-deadline: Rename deadline_is_seq_writes() 2023-01-24 07:24:44 +01:00
opal_proto.h block: sed-opal: Add ioctl to return device status 2022-08-22 07:52:51 -06:00
sed-opal.c block: sed-opal: kmalloc the cmd/resp buffers 2022-11-08 07:14:35 -07:00
t10-pi.c block: add pi for extended integrity 2022-03-07 12:48:35 -07:00