mirrors/linux-stable
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git synced 2024-09-13 14:14:37 +00:00
Code Issues Releases Wiki Activity
linux-stable/fs/xfs
History
Dave Chinner 977ec4ddf0 xfs: don't take a spinlock unconditionally in the DIO fastpath 
Because this happens at high thread counts on high IOPS devices
doing mixed read/write AIO-DIO to a single file at about a million
iops:

   64.09%     0.21%  [kernel]            [k] io_submit_one
   - 63.87% io_submit_one
      - 44.33% aio_write
         - 42.70% xfs_file_write_iter
            - 41.32% xfs_file_dio_write_aligned
               - 25.51% xfs_file_write_checks
                  - 21.60% _raw_spin_lock
                     - 21.59% do_raw_spin_lock
                        - 19.70% __pv_queued_spin_lock_slowpath

This also happens of the IO completion IO path:

   22.89%     0.69%  [kernel]            [k] xfs_dio_write_end_io
   - 22.49% xfs_dio_write_end_io
      - 21.79% _raw_spin_lock
         - 20.97% do_raw_spin_lock
            - 20.10% __pv_queued_spin_lock_slowpath

IOWs, fio is burning ~14 whole CPUs on this spin lock.

So, do an unlocked check against inode size first, then if we are
at/beyond EOF, take the spinlock and recheck. This makes the
spinlock disappear from the overwrite fastpath.

I'd like to report that fixing this makes things go faster. It
doesn't - it just exposes the the XFS_ILOCK as the next severe
contention point doing extent mapping lookups, and that now burns
all the 14 CPUs this spinlock was burning.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
2021-06-02 15:00:38 -07:00
..
libxfs xfs: mark xfs_bmap_set_attrforkoff static 2021-06-02 14:58:59 -07:00
scrub xfs: fix deadlock retry tracepoint arguments 2021-05-20 08:31:22 -07:00
Kconfig xfs: fix Kconfig asking about XFS_SUPPORT_V4 when XFS_FS=n 2020-10-16 15:34:28 -07:00
kmem.c xfs: remove kmem_realloc() 2020-09-06 18:05:51 -07:00
kmem.h xfs: Remove kmem_zalloc_large() 2020-09-15 20:52:41 -07:00
Makefile
mrlock.h
xfs.h
xfs_acl.c xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_acl.h fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
xfs_aops.c iomap: remove unused private field from ioend 2021-05-04 08:54:29 -07:00
xfs_aops.h
xfs_attr_inactive.c
xfs_attr_list.c xfs: rename and simplify xfs_bmap_one_block 2021-04-15 09:35:50 -07:00
xfs_bio_io.c block: Add bio_max_segs 2021-02-26 15:49:51 -07:00
xfs_bmap_item.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_bmap_item.h
xfs_bmap_util.c xfs: remove unnecessary shifts 2021-06-01 12:53:59 -07:00
xfs_bmap_util.h
xfs_buf.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_buf.h xfs: don't drain buffer lru on freeze and read-only remount 2021-01-22 16:54:50 -08:00
xfs_buf_item.c xfs: optimise xfs_buf_item_size/format for contiguous regions 2021-03-25 16:47:51 -07:00
xfs_buf_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_buf_item_recover.c xfs: fix finobt btree block recovery ordering 2020-09-30 07:28:52 -07:00
xfs_dir2_readdir.c xfs: remove XFS_IFINLINE 2021-04-15 09:35:51 -07:00
xfs_discard.c
xfs_discard.h
xfs_dquot.c xfs: move the XFS_IFEXTENTS check into xfs_iread_extents 2021-04-15 09:35:50 -07:00
xfs_dquot.h xfs: refactor default quota grace period setting code 2020-09-15 20:52:40 -07:00
xfs_dquot_item.c xfs: stop using q_core.d_id in the quota code 2020-07-28 20:24:14 -07:00
xfs_dquot_item.h
xfs_dquot_item_recover.c xfs: rename the ondisk dquot d_flags to d_type 2020-07-28 20:24:14 -07:00
xfs_error.c xfs: add error injection for per-AG resv failure 2021-03-25 16:47:53 -07:00
xfs_error.h
xfs_export.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_export.h
xfs_extent_busy.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_extent_busy.h treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_extfree_item.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_extfree_item.h
xfs_file.c xfs: don't take a spinlock unconditionally in the DIO fastpath 2021-06-02 15:00:38 -07:00
xfs_filestream.c xfs: drop the obsolete comment on filestream locking 2020-09-25 11:34:08 -07:00
xfs_filestream.h xfs: move the di_flags field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_fsmap.c xfs: drop freeze protection when running GETFSMAP 2021-03-24 10:36:05 -07:00
xfs_fsmap.h xfs: fix deadlock and streamline xfs_getfsmap performance 2020-10-07 08:40:29 -07:00
xfs_fsops.c xfs: remove obsolete AGF counter debugging 2021-04-29 07:44:18 -07:00
xfs_fsops.h xfs: get rid of xfs_growfs_{data,log}_t 2021-02-03 09:18:50 -08:00
xfs_globals.c xfs: consolidate the eofblocks and cowblocks workers 2021-02-03 09:18:49 -08:00
xfs_health.c
xfs_icache.c xfs: move the xfs_can_free_eofblocks call under the IOLOCK 2021-04-07 14:38:16 -07:00
xfs_icache.h xfs: rename block gc start and stop functions 2021-02-03 09:18:50 -08:00
xfs_icreate_item.c xfs: Remove kmem_zone_zalloc() usage 2020-07-28 20:24:14 -07:00
xfs_icreate_item.h
xfs_inode.c xfs: validate extsz hints against rt extent size when rtinherit is set 2021-05-24 18:01:04 -07:00
xfs_inode.h xfs: move the di_crtime field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_inode_item.c xfs: rename struct xfs_legacy_ictimestamp 2021-04-22 18:29:25 -07:00
xfs_inode_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_inode_item_recover.c xfs: rename struct xfs_legacy_ictimestamp 2021-04-22 18:29:25 -07:00
xfs_ioctl.c xfs: validate extsz hints against rt extent size when rtinherit is set 2021-05-24 18:01:04 -07:00
xfs_ioctl.h xfs: convert to fileattr 2021-04-12 15:04:29 +02:00
xfs_ioctl32.c xfs: convert to fileattr 2021-04-12 15:04:29 +02:00
xfs_ioctl32.h xfs: convert to fileattr 2021-04-12 15:04:29 +02:00
xfs_iomap.c xfs: remove XFS_IFEXTENTS 2021-04-15 09:35:51 -07:00
xfs_iomap.h
xfs_iops.c xfs: clean up open-coded fs block unit conversions 2021-06-01 12:53:59 -07:00
xfs_iops.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_itable.c xfs: move the di_crtime field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_itable.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_iwalk.c xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_iwalk.h
xfs_linux.h xfs: move the di_flags field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_log.c xfs: don't allow log writes if the data device is readonly 2021-05-04 08:43:27 -07:00
xfs_log.h xfs: cover the log during log quiesce 2021-01-22 16:54:51 -08:00
xfs_log_cil.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_log_priv.h xfs: Modify xlog_ticket_alloc() to use kernel's MM API 2020-07-28 20:24:14 -07:00
xfs_log_recover.c xfs: remove the di_dmevmask and di_dmstate fields from struct xfs_icdinode 2021-04-07 14:37:03 -07:00
xfs_message.c
xfs_message.h xfs: validate extsz hints against rt extent size when rtinherit is set 2021-05-24 18:01:04 -07:00
xfs_mount.c xfs: set aside allocation btree blocks from block reservation 2021-04-29 07:45:44 -07:00
xfs_mount.h xfs: introduce in-core global counter of allocbt blocks 2021-04-29 07:45:44 -07:00
xfs_mru_cache.c xfs: set WQ_SYSFS on all workqueues in debug mode 2021-02-03 09:18:49 -08:00
xfs_mru_cache.h
xfs_ondisk.h xfs: rename struct xfs_legacy_ictimestamp 2021-04-22 18:29:25 -07:00
xfs_pnfs.c xfs: move the di_size field to struct xfs_inode 2021-04-07 14:37:03 -07:00
xfs_pnfs.h
xfs_pwork.c xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_pwork.h xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_qm.c xfs: move the XFS_IFEXTENTS check into xfs_iread_extents 2021-04-15 09:35:50 -07:00
xfs_qm.h xfs: refactor quota expiration timer modification 2020-09-15 20:52:40 -07:00
xfs_qm_bhv.c xfs: move the di_projid field to struct xfs_inode 2021-04-07 14:37:03 -07:00
xfs_qm_syscalls.c xfs: move the di_size field to struct xfs_inode 2021-04-07 14:37:03 -07:00
xfs_quota.h xfs: remove xfs_qm_vop_chown_reserve 2021-02-03 09:18:49 -08:00
xfs_quotaops.c xfs: move the di_nblocks field to struct xfs_inode 2021-04-07 14:37:03 -07:00
xfs_refcount_item.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_refcount_item.h
xfs_reflink.c xfs: fix xfs_reflink_unshare usage of filemap_write_and_wait_range 2021-04-29 07:45:44 -07:00
xfs_reflink.h xfs: move helpers that lock and unlock two inodes against userspace IO 2020-07-06 10:46:57 -07:00
xfs_rmap_item.c treewide: Change list_sort to use const pointers 2021-04-08 16:04:22 -07:00
xfs_rmap_item.h
xfs_rtalloc.c xfs: move the di_flags field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_rtalloc.h xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_stats.c xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_stats.h xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_super.c xfs: move the di_flags field to struct xfs_inode 2021-04-07 14:37:05 -07:00
xfs_super.h xfs: remove xfs_quiesce_attr declaration 2021-04-16 08:28:36 -07:00
xfs_symlink.c New code for 5.13: 2021-04-29 10:43:51 -07:00
xfs_symlink.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_sysctl.c xfs: restore speculative_cow_prealloc_lifetime sysctl 2021-02-24 10:16:08 -08:00
xfs_sysctl.h xfs: consolidate the eofblocks and cowblocks workers 2021-02-03 09:18:49 -08:00
xfs_sysfs.c
xfs_sysfs.h xfs: Fix UBSAN null-ptr-deref in xfs_sysfs_init 2020-08-07 11:50:17 -07:00
xfs_trace.c xfs: add a tracepoint for blockgc scans 2021-02-03 09:18:49 -08:00
xfs_trace.h xfs: move the di_size field to struct xfs_inode 2021-04-07 14:37:03 -07:00
xfs_trans.c xfs: update superblock counters correctly for !lazysbcount 2021-04-29 07:44:18 -07:00
xfs_trans.h xfs: remove obsolete AGF counter debugging 2021-04-29 07:44:18 -07:00
xfs_trans_ail.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_trans_buf.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_trans_dquot.c xfs: shut down the filesystem if we screw up quota reservation 2021-02-03 09:18:49 -08:00
xfs_trans_priv.h
xfs_xattr.c xfs: prevent metadata files from being inactivated 2021-03-25 16:47:50 -07:00