Jack Wang 21bd9a68fe md/raid1: Avoid lock contention from wake_up() ... wake_up is called unconditionally in a few paths such as make_request(), which cause lock contention under high concurrency workload like below raid1_end_write_request wake_up __wake_up_common_lock spin_lock_irqsave Improve performance by only call wake_up() if waitqueue is not empty Fio test script: [global] name=random reads and writes ioengine=libaio direct=1 readwrite=randrw rwmixread=70 iodepth=64 buffered=0 filename=/dev/md0 size=1G runtime=30 time_based randrepeat=0 norandommap refill_buffers ramp_time=10 bs=4k numjobs=400 group_reporting=1 [job1] Test result with 2 ramdisk in raid1 on a Intel Broadwell 56 cores server. Before this patch With this patch READ BW=4621MB/s BW=7337MB/s WRITE BW=1980MB/s BW=3144MB/s The patch is inspired by Yu Kuai's change for raid10: https://lore.kernel.org/r/20230621105728.1268542-1-yukuai1@huaweicloud.com Cc: Yu Kuai <yukuai3@huawei.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reviewed-by: Yu Kuai <yukuai3@huawei.com> Link: https://lore.kernel.org/r/20230705113227.148494-1-jinpu.wang@ionos.com Signed-off-by: Song Liu <song@kernel.org>		2023-07-27 00:13:30 -07:00
..
bcache	Merge branch 'for-6.5/block-late' into block-6.5	2023-06-28 16:08:19 -06:00
persistent-data	dm thin metadata: Fix ABBA deadlock by resetting dm_bufio_client	2023-06-16 18:24:13 -04:00
dm-audit.c	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-audit.h	dm: introduce audit event module for device mapper	2021-10-27 16:53:47 -04:00
dm-bio-prison-v1.c	dm: improve hash_locks sizing and hash function	2023-03-30 15:57:51 -04:00
dm-bio-prison-v1.h	dm bio prison v1: add dm_cell_key_has_valid_range	2023-03-30 15:57:51 -04:00
dm-bio-prison-v2.c	dm: address space issues relative to switch/while/for/...	2023-02-14 14:23:06 -05:00
dm-bio-prison-v2.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-bio-record.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-bufio.c	dm: get rid of GFP_NOIO workarounds for __vmalloc and kvmalloc	2023-06-27 16:06:54 -04:00
dm-builtin.c	dm: adjust EXPORT_SYMBOL() to follow functions immediately	2023-02-14 14:23:07 -05:00
dm-cache-background-tracker.c	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-cache-background-tracker.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-cache-block-types.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-cache-metadata.c	Revert "mm: shrinkers: convert shrinker_rwsem to mutex"	2023-06-19 13:19:33 -07:00
dm-cache-metadata.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-cache-policy-internal.h	dm: add missing empty lines	2023-02-14 14:23:06 -05:00
dm-cache-policy-smq.c	dm: add missing empty lines	2023-02-14 14:23:06 -05:00
dm-cache-policy.c	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-cache-policy.h	dm: address indent/space issues	2023-02-14 14:23:06 -05:00
dm-cache-target.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-clone-metadata.c	dm clone metadata: remove unused function	2021-04-19 13:20:31 -04:00
dm-clone-metadata.h
dm-clone-target.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-core.h	dm: support turning off block-core's io stats accounting	2023-06-16 18:24:13 -04:00
dm-crypt.c	This update includes the following changes:	2023-06-30 21:27:13 -07:00
dm-delay.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-dust.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-ebs-target.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-era-target.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-exception-store.c	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-exception-store.h	dm: avoid spaces before function arguments or in favour of tabs	2023-02-14 14:23:06 -05:00
dm-flakey.c	dm flakey: introduce random_read_corrupt and random_write_corrupt options	2023-06-16 18:24:13 -04:00
dm-ima.c	dm: avoid inline filenames	2023-02-14 14:23:07 -05:00
dm-ima.h	dm: avoid inline filenames	2023-02-14 14:23:07 -05:00
dm-init.c	dm: open code dm_get_dev_t in dm_init_init	2023-06-05 10:57:40 -06:00
dm-integrity.c	- Update DM crypt to allocate compound pages if possible.	2023-06-30 12:16:00 -07:00
dm-io-rewind.c	dm: avoid void function return statements	2023-02-14 14:23:07 -05:00
dm-io-tracker.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-io.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-ioctl.c	- Update DM crypt to allocate compound pages if possible.	2023-06-30 12:16:00 -07:00
dm-kcopyd.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-linear.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-log-userspace-base.c	dm: avoid void function return statements	2023-02-14 14:23:07 -05:00
dm-log-userspace-transfer.c	dm: avoid split of quoted strings where possible	2023-02-14 14:23:07 -05:00
dm-log-userspace-transfer.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-log-writes.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-log.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-mpath.c	dm: push error reporting down to dm_register_target()	2023-04-11 12:01:01 -04:00
dm-mpath.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-path-selector.c	dm: adjust EXPORT_SYMBOL() to follow functions immediately	2023-02-14 14:23:07 -05:00
dm-path-selector.h	dm: avoid spaces before function arguments or in favour of tabs	2023-02-14 14:23:06 -05:00
dm-ps-historical-service-time.c	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-ps-io-affinity.c	dm: address space issues relative to switch/while/for/...	2023-02-14 14:23:06 -05:00
dm-ps-queue-length.c	dm: avoid spaces before function arguments or in favour of tabs	2023-02-14 14:23:06 -05:00
dm-ps-round-robin.c	dm: correct block comments format.	2023-02-14 14:23:06 -05:00
dm-ps-service-time.c	dm: avoid spaces before function arguments or in favour of tabs	2023-02-14 14:23:06 -05:00
dm-raid.c	Revert "md: unlock mddev before reap sync_thread in action_store"	2023-07-27 00:13:28 -07:00
dm-raid1.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-region-hash.c	dm: correct block comments format.	2023-02-14 14:23:06 -05:00
dm-rq.c	dm: avoid using symbolic permissions	2023-02-14 14:23:07 -05:00
dm-rq.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-snap-persistent.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-snap-transient.c	dm: avoid split of quoted strings where possible	2023-02-14 14:23:07 -05:00
dm-snap.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-stats.c	dm stats: check for and propagate alloc_percpu failure	2023-03-16 13:37:06 -04:00
dm-stats.h	dm stats: check for and propagate alloc_percpu failure	2023-03-16 13:37:06 -04:00
dm-stripe.c	dm: remove unnecessary (void*) conversions	2023-04-11 12:01:01 -04:00
dm-switch.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-sysfs.c	dm sysfs: make kobj_type structure constant	2023-02-14 14:23:08 -05:00
dm-table.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-target.c	dm: push error reporting down to dm_register_target()	2023-04-11 12:01:01 -04:00
dm-thin-metadata.c	- Update DM crypt to allocate compound pages if possible.	2023-06-30 12:16:00 -07:00
dm-thin-metadata.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-thin.c	- Update DM crypt to allocate compound pages if possible.	2023-06-30 12:16:00 -07:00
dm-uevent.c	dm: avoid spaces before function arguments or in favour of tabs	2023-02-14 14:23:06 -05:00
dm-uevent.h	dm: fix undue/missing spaces	2023-02-14 14:23:06 -05:00
dm-unstripe.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-verity-fec.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-verity-fec.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-verity-loadpin.c	dm: verity-loadpin: Add NULL pointer check for 'bdev' parameter	2023-06-28 10:43:04 -07:00
dm-verity-target.c	block: replace fmode_t with a block-specific type for block open flags	2023-06-12 08:04:05 -06:00
dm-verity-verify-sig.c	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-verity-verify-sig.h	dm: add missing SPDX-License-Indentifiers	2023-02-14 14:23:06 -05:00
dm-verity.h	dm: change "unsigned" to "unsigned int"	2023-02-14 14:23:06 -05:00
dm-writecache.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-zero.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-zone.c	dm zone: Use the bitmap API to allocate bitmaps	2023-06-16 18:24:13 -04:00
dm-zoned-metadata.c	dm: dm-zoned: use __bio_add_page for adding single metadata page	2023-05-31 09:50:02 -06:00
dm-zoned-reclaim.c	dm kcopyd: avoid useless atomic operations	2021-06-04 12:07:24 -04:00
dm-zoned-target.c	dm: add helper macro for simple DM target module init and exit	2023-04-11 12:09:08 -04:00
dm-zoned.h	dm/dm-zoned: Use the enum req_op type	2022-07-14 12:14:31 -06:00
dm.c	- Update DM crypt to allocate compound pages if possible.	2023-06-30 12:16:00 -07:00
dm.h	dm: remove stale/redundant dm_internal_{suspend,resume} prototypes in dm.h	2023-06-16 18:24:14 -04:00
Kconfig	md: deprecate bitmap file support	2023-07-27 00:13:29 -07:00
Makefile	hardening updates for v5.20-rc1	2022-08-02 14:38:59 -07:00
md-autodetect.c	init: improve the name_to_dev_t interface	2023-06-05 10:56:46 -06:00
md-bitmap.c	md: make bitmap file support optional	2023-07-27 00:13:29 -07:00
md-bitmap.h	md-bitmap: don't use ->index for pages backing the bitmap file	2023-07-27 00:13:29 -07:00
md-cluster.c	md: protect md_thread with rcu	2023-06-13 15:25:39 -07:00
md-cluster.h
md-faulty.c	md/md-faulty: enable io accounting	2023-07-27 00:13:30 -07:00
md-linear.c	md/md-linear: enable io accounting	2023-07-27 00:13:30 -07:00
md-linear.h
md-multipath.c	md/md-multipath: enable io accounting	2023-07-27 00:13:29 -07:00
md-multipath.h
md.c	md: restore 'noio_flag' for the last mddev_resume()	2023-07-27 00:13:30 -07:00
md.h	md: also clone new io if io accounting is disabled	2023-07-27 00:13:29 -07:00
raid0.c	md: move initialization and destruction of 'io_acct_set' to md.c	2023-07-27 00:13:29 -07:00
raid0.h	md/raid0: add discard support for the 'original' layout	2023-06-30 15:43:50 -07:00
raid1-10.c	md/raid1-10: fix casting from randomized structure in raid1_submit_write()	2023-06-23 09:33:16 -07:00
raid1.c	md/raid1: Avoid lock contention from wake_up()	2023-07-27 00:13:30 -07:00
raid1.h	md/raid1: switch to use md_account_bio() for io accounting	2023-07-27 00:13:29 -07:00
raid5-cache.c	md: protect md_thread with rcu	2023-06-13 15:25:39 -07:00
raid5-log.h	md/raid5-ppl: Drop unused argument from ppl_handle_flush_request()	2022-08-02 17:14:31 -06:00
raid5-ppl.c	md: raid5: use __bio_add_page to add single page to new bio	2023-05-31 09:50:02 -06:00
raid5.c	raid5: fix missing io accounting in raid5_align_endio()	2023-07-27 00:13:29 -07:00
raid5.h	hardening updates for v6.5-rc1	2023-06-27 21:24:18 -07:00
raid10.c	md: remove redundant check in fix_read_error()	2023-07-27 00:13:30 -07:00
raid10.h	md/raid10: switch to use md_account_bio() for io accounting	2023-07-27 00:13:29 -07:00