linux-stable/block
Paolo Valente 89f4d27c1b blok, bfq: do not plug I/O if all queues are weight-raised
[ Upstream commit c8765de0ad ]

To reduce latency for interactive and soft real-time applications, bfq
privileges the bfq_queues containing the I/O of these
applications. These privileged queues, referred-to as weight-raised
queues, get a much higher share of the device throughput
w.r.t. non-privileged queues. To preserve this higher share, the I/O
of any non-weight-raised queue must be plugged whenever a sync
weight-raised queue, while being served, remains temporarily empty. To
attain this goal, bfq simply plugs any I/O (from any queue), if a sync
weight-raised queue remains empty while in service.

Unfortunately, this plugging typically lowers throughput with random
I/O, on devices with internal queueing (because it reduces the filling
level of the internal queues of the device).

This commit addresses this issue by restricting the cases where
plugging is performed: if a sync weight-raised queue remains empty
while in service, then I/O plugging is performed only if some of the
active bfq_queues are *not* weight-raised (which is actually the only
circumstance where plugging is needed to preserve the higher share of
the throughput of weight-raised queues). This restriction proved able
to boost throughput in really many use cases needing only maximum
throughput.

Signed-off-by: Paolo Valente <paolo.valente@linaro.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-11-20 18:46:44 +01:00
..
partitions partitions/aix: append null character to print data from disk 2018-07-27 09:17:41 -06:00
badblocks.c badblocks: fix wrong return value in badblocks_set if badblocks are disabled 2017-11-03 11:29:50 -07:00
bfq-cgroup.c block: bfq: swap puts in bfqg_and_blkg_put 2018-09-06 11:32:58 -06:00
bfq-iosched.c blok, bfq: do not plug I/O if all queues are weight-raised 2019-11-20 18:46:44 +01:00
bfq-iosched.h block, bfq: inject other-queue I/O into seeky idle queues on NCQ flash 2019-11-20 18:46:44 +01:00
bfq-wf2q.c block, bfq: correctly charge and reset entity service in all cases 2018-11-13 11:08:28 -08:00
bio-integrity.c block/bio-integrity: fix a memory leak bug 2019-07-31 07:27:08 +02:00
bio.c block: do not leak memory in bio_copy_user_iov() 2019-04-17 08:38:51 +02:00
blk-cgroup.c blkcg: make blkcg_print_stat() print stats only for online blkgs 2019-11-12 19:21:19 +01:00
blk-core.c blk-mq: change gfp flags to GFP_NOIO in blk_mq_realloc_hw_ctxs 2019-10-01 08:26:10 +02:00
blk-exec.c blk-mq-sched: remove unused 'can_block' arg from blk_mq_sched_insert_request 2018-01-17 09:49:21 -07:00
blk-flush.c block: fix null pointer dereference in blk_mq_rq_timed_out() 2019-10-05 13:10:08 +02:00
blk-integrity.c block drivers/block: Use octal not symbolic permissions 2018-05-24 13:38:59 -06:00
blk-ioc.c block, mm: remove unnecessary __GFP_HIGH flag 2018-07-09 09:07:54 -06:00
blk-iolatency.c blk-iolatency: fix STS_AGAIN handling 2019-09-16 08:21:41 +02:00
blk-lib.c block: make sure writesame bio is aligned with logical block size 2018-11-13 11:08:16 -08:00
blk-map.c Merge branch 'for-4.16/block' of git://git.kernel.dk/linux-block 2018-01-29 11:51:49 -08:00
blk-merge.c block: make sure discard bio is aligned with logical block size 2018-11-13 11:08:16 -08:00
blk-mq-cpumap.c blk-mq: don't keep offline CPUs mapped to hctx 0 2018-04-10 08:38:46 -06:00
blk-mq-debugfs-zoned.c block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n 2018-07-09 09:07:52 -06:00
blk-mq-debugfs.c block, scsi: Change the preempt-only flag into a counter 2019-08-04 09:30:57 +02:00
blk-mq-debugfs.h block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n 2018-07-09 09:07:52 -06:00
blk-mq-pci.c blk-mq: code clean-up by adding an API to clear set->mq_map 2018-07-09 09:07:53 -06:00
blk-mq-rdma.c block: Add rdma affinity based queue mapping helper 2017-08-08 14:58:03 -04:00
blk-mq-sched.c block: mq-deadline: Fix write completion handling 2019-01-13 09:51:07 +01:00
blk-mq-sched.h block: mq-deadline: Fix write completion handling 2019-01-13 09:51:07 +01:00
blk-mq-sysfs.c blk-mq: free hw queue's resource in hctx's release handler 2019-09-16 08:22:13 +02:00
blk-mq-tag.c blk-mq: Allow blocking queue tag iter callbacks 2018-09-25 20:17:59 -06:00
blk-mq-tag.h Merge branch 'for-4.15/block' of git://git.kernel.dk/linux-block 2017-11-14 15:32:19 -08:00
blk-mq-virtio.c blk-mq: provide a default queue mapping for virtio device 2017-02-27 20:54:05 +02:00
blk-mq.c block: fix null pointer dereference in blk_mq_rq_timed_out() 2019-10-05 13:10:08 +02:00
blk-mq.h blk-mq: free hw queue's resource in hctx's release handler 2019-09-16 08:22:13 +02:00
blk-rq-qos.c blk-wbt: fix performance regression in wbt scale_up/scale_down 2019-10-17 13:45:16 -07:00
blk-rq-qos.h blk-rq-qos: fix first node deletion of rq_qos_del() 2019-10-29 09:20:09 +01:00
blk-settings.c block: allow max_discard_segments to be stacked 2018-07-24 14:46:39 -06:00
blk-softirq.c block: fix timeout changes for legacy request drivers 2018-06-19 11:27:18 -06:00
blk-stat.c blk-stat: export helpers for modifying blk_rq_stat 2018-07-09 09:07:54 -06:00
blk-stat.h block: deactivate blk_stat timer in wbt_disable_default() 2019-01-13 09:51:06 +01:00
blk-sysfs.c blk-mq: move cancel of requeue_work to the front of blk_exit_queue 2019-10-01 08:26:10 +02:00
blk-tag.c for-linus-20180616 2018-06-17 05:37:55 +09:00
blk-throttle.c blk-throttle: fix zero wait time for iops throttled group 2019-07-26 09:14:30 +02:00
blk-timeout.c blk-mq: Fix timeout handling in case the timeout handler returns BLK_EH_DONE 2018-06-23 10:25:45 -06:00
blk-wbt.c blk-wbt: fix performance regression in wbt scale_up/scale_down 2019-10-17 13:45:16 -07:00
blk-wbt.h block: remove external dependency on wbt_flags 2018-07-09 09:07:54 -06:00
blk-zoned.c block: Remove a superfluous cast from blkdev_report_zones() 2018-07-09 09:07:52 -06:00
blk.h block: fix null pointer dereference in blk_mq_rq_timed_out() 2019-10-05 13:10:08 +02:00
bounce.c block: copy ioprio in __bio_clone_fast() and bounce 2018-12-01 09:37:32 +01:00
bsg-lib.c block/bsg-lib: use PTR_ERR_OR_ZERO to simplify the flow path 2018-08-01 09:13:03 -06:00
bsg.c block: bsg: move atomic_t ref_count variable to refcount API 2018-08-27 19:17:02 -06:00
cfq-iosched.c cfq: Suppress compiler warnings about comparisons 2018-08-07 17:57:13 -06:00
cmdline-parser.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compat_ioctl.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
deadline-iosched.c block drivers/block: Use octal not symbolic permissions 2018-05-24 13:38:59 -06:00
elevator.c block: fix deadline elevator drain for zoned block devices 2018-09-26 19:57:24 -06:00
genhd.c block: fix use-after-free on gendisk 2019-05-31 06:46:18 -07:00
ioctl.c block: pass inclusive 'lend' parameter to truncate_inode_pages_range 2018-02-23 15:20:19 -07:00
ioprio.c block: add ioprio_check_cap function 2018-05-31 10:50:54 -04:00
Kconfig block: introduce blk-iolatency io controller 2018-07-09 09:07:54 -06:00
Kconfig.iosched License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kyber-iosched.c block: kyber: make kyber more friendly with merging 2018-05-30 10:47:40 -06:00
Makefile block: introduce blk-iolatency io controller 2018-07-09 09:07:54 -06:00
mq-deadline.c block: mq-deadline: Fix queue restart handling 2019-10-07 18:57:19 +02:00
noop-iosched.c block: move existing elevator ops to union 2017-01-17 10:03:33 -07:00
opal_proto.h block: sed-opal: Set MBRDone on S3 resume path if TPER is MBREnabled 2017-09-11 09:45:52 -06:00
partition-generic.c block: fix use-after-free on gendisk 2019-05-31 06:46:18 -07:00
scsi_ioctl.c block: consistently use GFP_NOIO instead of __GFP_NORECLAIM 2018-05-14 08:55:18 -06:00
sed-opal.c block: sed-opal: fix IOC_OPAL_ENABLE_DISABLE_MBR 2019-05-31 06:46:24 -07:00
t10-pi.c block: move dif_prepare/dif_complete functions to block layer 2018-07-30 08:27:02 -06:00