linux-stable/fs/btrfs
Filipe Manana 3ee56a58ad btrfs: reserve space for delayed refs on a per ref basis
Currently when reserving space for delayed refs we do it on a per ref head
basis. This is generally enough because most back refs for an extent end
up being inlined in the extent item - with the default leaf size of 16K we
can have at most 33 inline back refs (this is calculated by the macro
BTRFS_MAX_EXTENT_ITEM_SIZE()). The amount of bytes reserved for each ref
head is given by btrfs_calc_delayed_ref_bytes(), which basically
corresponds to a single path for insertion into the extent tree plus
another path for insertion into the free space tree if it's enabled.

However if we have reached the limit of inline refs or we have a mix of
inline and non-inline refs, then we will need to insert a non-inline ref
and update the existing extent item to update the total number of
references for the extent. This implies we need reserved space for two
insertion paths in the extent tree, but we only reserved for one path.
The extent item and the non-inline ref item may be located in different
leaves, or even if they are located in the same leaf, after updating the
extent item and before inserting the non-inline ref item, the extent
buffers in the btree path may have been written (due to memory pressure
for e.g.), in which case we need to COW the entire path again. In this
case since we have not reserved enough space for the delayed refs block
reserve, we will use the global block reserve.

If we are in a situation where the fs has no more unallocated space enough
to allocate a new metadata block group and available space in the existing
metadata block groups is close to the maximum size of the global block
reserve (512M), we may end up consuming too much of the free metadata
space to the point where we can't commit any future transaction because it
will fail, with -ENOSPC, during its commit when trying to allocate an
extent for some COW operation (running delayed refs generated by running
delayed refs or COWing the root tree's root node at commit_cowonly_roots()
for example). Such dramatic scenario can happen if we have many delayed
refs that require the insertion of non-inline ref items, due to too many
reflinks or snapshots. We also have situations where we use the global
block reserve because we could not in advance know that we will need
space to update some trees (block group creation for example), so this
all adds up to increase the chances of exhausting the global block reserve
and making any future transaction commit to fail with -ENOSPC and turn
the fs into RO mode, or fail the mount operation in case the mount needs
to start and commit a transaction, such as when we have orphans to cleanup
for example - such case was reported and hit by someone running a SLE
(SUSE Linux Enterprise) distribution for example - where the fs had no
more unallocated space that could be used to allocate a new metadata block
group, and the available metadata space was about 1.5M, not enough to
commit a transaction to cleanup an orphan inode (or do relocation of data
block groups that were far from being full).

So reserve space for delayed refs by individual refs and not by ref heads,
as we may need to COW multiple extent tree paths due to non-inline ref
items.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-12 16:44:06 +02:00
..
tests btrfs: tests: test invalid splitting when skipping pinned drop extent_map 2023-08-21 14:54:49 +02:00
accessors.c
accessors.h btrfs: include asm/unaligned.h in accessors.h 2023-10-12 16:44:02 +02:00
acl.c
acl.h
async-thread.c btrfs: include trace header in where necessary 2023-10-12 16:44:03 +02:00
async-thread.h btrfs: use alloc_ordered_workqueue() to create ordered workqueues 2023-06-19 13:59:30 +02:00
backref.c btrfs: remove v0 extent handling 2023-08-21 14:54:48 +02:00
backref.h btrfs: reduce size of prelim_ref::level 2023-10-12 16:44:04 +02:00
bio.c btrfs: remove the need_raid_map parameter from btrfs_map_block() 2023-10-12 16:44:05 +02:00
bio.h btrfs: add an ordered_extent pointer to struct btrfs_bio 2023-06-19 13:59:36 +02:00
block-group.c btrfs: allow to run delayed refs by bytes to be released instead of count 2023-10-12 16:44:06 +02:00
block-group.h btrfs: rename add_new_free_space() to btrfs_add_new_free_space() 2023-08-21 14:52:12 +02:00
block-rsv.c btrfs: pass a space_info argument to btrfs_reserve_metadata_bytes() 2023-10-12 16:44:05 +02:00
block-rsv.h btrfs: move btrfs_check_trunc_cache_free_space into block-rsv.c 2023-06-19 13:59:24 +02:00
btrfs_inode.h btrfs: reduce size and reorder compression members in struct btrfs_inode 2023-10-12 16:44:04 +02:00
compression.c btrfs: make btrfs_compressed_bioset static 2023-06-19 17:01:44 +02:00
compression.h btrfs: pass an ordered_extent to btrfs_submit_compressed_write 2023-06-19 13:59:36 +02:00
ctree.c btrfs: reduce arguments of helpers space accounting root item 2023-10-12 16:44:04 +02:00
ctree.h btrfs: remove extraneous includes from ctree.h 2023-10-12 16:44:03 +02:00
defrag.c btrfs: drop gfp from parameter extent state helpers 2023-06-19 13:59:30 +02:00
defrag.h
delalloc-space.c btrfs: pass a space_info argument to btrfs_reserve_metadata_bytes() 2023-10-12 16:44:05 +02:00
delalloc-space.h
delayed-inode.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
delayed-inode.h btrfs: add __counted_by for struct btrfs_delayed_item and use struct_size() 2023-10-11 11:37:19 +02:00
delayed-ref.c btrfs: reserve space for delayed refs on a per ref basis 2023-10-12 16:44:06 +02:00
delayed-ref.h btrfs: remove the refcount warning/check at btrfs_put_delayed_ref() 2023-10-12 16:44:05 +02:00
dev-replace.c btrfs: check-integrity: remove btrfsic_unmount() function 2023-10-12 16:44:05 +02:00
dev-replace.h
dir-item.c
dir-item.h btrfs: add fscrypt related dependencies to respective headers 2023-10-12 16:44:02 +02:00
discard.c btrfs: unexport btrfs_run_discard_work and make it static 2023-06-19 13:59:25 +02:00
discard.h btrfs: unexport btrfs_run_discard_work and make it static 2023-06-19 13:59:25 +02:00
disk-io.c btrfs: reserve space for delayed refs on a per ref basis 2023-10-12 16:44:06 +02:00
disk-io.h btrfs: make btrfs_cleanup_fs_roots() static 2023-08-21 14:52:18 +02:00
export.c
export.h
extent-io-tree.c btrfs: make find_first_extent_bit() return a boolean 2023-08-21 14:52:12 +02:00
extent-io-tree.h btrfs: make find_first_extent_bit() return a boolean 2023-08-21 14:52:12 +02:00
extent-tree.c btrfs: reserve space for delayed refs on a per ref basis 2023-10-12 16:44:06 +02:00
extent-tree.h btrfs: allow to run delayed refs by bytes to be released instead of count 2023-10-12 16:44:06 +02:00
extent_io.c btrfs: check-integrity: remove btrfsic_unmount() function 2023-10-12 16:44:05 +02:00
extent_io.h btrfs: move extent_buffer::lock_owner to debug section 2023-10-12 16:44:05 +02:00
extent_map.c btrfs: fix incorrect splitting in btrfs_drop_extent_map_range 2023-08-18 14:38:10 +02:00
extent_map.h btrfs: pass the new logical address to split_extent_map 2023-06-19 13:59:33 +02:00
file-item.c btrfs: scrub: avoid unnecessary csum tree search preparing stripes 2023-08-21 14:54:48 +02:00
file-item.h btrfs: scrub: avoid unnecessary csum tree search preparing stripes 2023-08-21 14:54:48 +02:00
file.c btrfs: include linux/iomap.h in file.c 2023-10-12 16:44:02 +02:00
file.h
free-space-cache.c btrfs: remove btrfs_crc32c wrapper 2023-10-12 16:44:02 +02:00
free-space-cache.h btrfs: move btrfs_check_trunc_cache_free_space into block-rsv.c 2023-06-19 13:59:24 +02:00
free-space-tree.c btrfs: rename add_new_free_space() to btrfs_add_new_free_space() 2023-08-21 14:52:12 +02:00
free-space-tree.h btrfs: make clear_cache mount option to rebuild FST without disabling it 2023-05-10 14:51:27 +02:00
fs.c
fs.h btrfs: check-integrity: remove CONFIG_BTRFS_FS_CHECK_INTEGRITY option 2023-10-12 16:44:05 +02:00
inode-item.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
inode-item.h btrfs: add fscrypt related dependencies to respective headers 2023-10-12 16:44:02 +02:00
inode.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
ioctl.c for-6.6-rc4-tag 2023-10-06 08:07:47 -07:00
ioctl.h
Kconfig btrfs: check-integrity: remove CONFIG_BTRFS_FS_CHECK_INTEGRITY option 2023-10-12 16:44:05 +02:00
locking.c btrfs: move extent_buffer::lock_owner to debug section 2023-10-12 16:44:05 +02:00
locking.h btrfs: do not block starts waiting on previous transaction commit 2023-09-08 14:10:49 +02:00
lru_cache.c
lru_cache.h btrfs: remove btrfs_lru_cache_is_full() inline function 2023-04-17 18:01:18 +02:00
lzo.c btrfs: disable allocation warnings for compression workspaces 2023-06-19 13:59:34 +02:00
Makefile btrfs: check-integrity: remove btrfsic_unmount() function 2023-10-12 16:44:05 +02:00
messages.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
messages.h btrfs: remove v0 extent handling 2023-08-21 14:54:48 +02:00
misc.h minmax: add in_range() macro 2023-08-24 16:20:18 -07:00
ordered-data.c btrfs: check for BTRFS_FS_ERROR in pending ordered assert 2023-09-08 14:10:59 +02:00
ordered-data.h btrfs: add a btrfs_finish_ordered_extent helper 2023-06-19 13:59:37 +02:00
orphan.c
orphan.h
print-tree.c btrfs: remove v0 extent handling 2023-08-21 14:54:48 +02:00
print-tree.h btrfs: print-tree: pass const extent buffer pointer 2023-06-19 13:59:22 +02:00
props.c btrfs: move btrfs_name_hash to dir-item.h 2023-10-12 16:44:02 +02:00
props.h
qgroup.c btrfs: move functions comments from qgroup.h to qgroup.c 2023-10-12 16:44:04 +02:00
qgroup.h btrfs: move functions comments from qgroup.h to qgroup.c 2023-10-12 16:44:04 +02:00
raid56.c btrfs: scrub: avoid unnecessary csum tree search preparing stripes 2023-08-21 14:54:48 +02:00
raid56.h btrfs: raid56: remove unused BTRFS_RBIO_REBUILD_MISSING 2023-08-21 14:52:12 +02:00
rcu-string.h
ref-verify.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
ref-verify.h
reflink.c btrfs: convert to ctime accessor functions 2023-07-13 10:28:04 +02:00
reflink.h
relocation.c btrfs: drop __must_check annotations 2023-10-12 16:44:04 +02:00
relocation.h btrfs: pass an ordered_extent to btrfs_reloc_clone_csums 2023-06-19 13:59:36 +02:00
root-tree.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
root-tree.h btrfs: drop __must_check annotations 2023-10-12 16:44:04 +02:00
scrub.c btrfs: remove the need_raid_map parameter from btrfs_map_block() 2023-10-12 16:44:05 +02:00
scrub.h btrfs: scrub: remove scrub_bio structure 2023-04-17 18:01:24 +02:00
send.c btrfs: remove btrfs_crc32c wrapper 2023-10-12 16:44:02 +02:00
send.h
space-info.c btrfs: allow to run delayed refs by bytes to be released instead of count 2023-10-12 16:44:06 +02:00
space-info.h btrfs: pass a space_info argument to btrfs_reserve_metadata_bytes() 2023-10-12 16:44:05 +02:00
subpage.c btrfs: stop setting PageError in the data I/O path 2023-06-19 13:59:35 +02:00
subpage.h btrfs: stop setting PageError in the data I/O path 2023-06-19 13:59:35 +02:00
super.c btrfs: check-integrity: remove CONFIG_BTRFS_FS_CHECK_INTEGRITY option 2023-10-12 16:44:05 +02:00
super.h
sysfs.c btrfs: sysfs: show if ACL support has been compiled in 2023-08-21 14:52:12 +02:00
sysfs.h
transaction.c btrfs: allow to run delayed refs by bytes to be released instead of count 2023-10-12 16:44:06 +02:00
transaction.h btrfs: always print transaction aborted messages with an error level 2023-10-04 01:03:59 +02:00
tree-checker.c btrfs: move btrfs_name_hash to dir-item.h 2023-10-12 16:44:02 +02:00
tree-checker.h btrfs: move btrfs_verify_level_key into tree-checker.c 2023-06-19 13:59:25 +02:00
tree-log.c btrfs: reduce parameters of btrfs_pin_extent_for_log_replay 2023-10-12 16:44:04 +02:00
tree-log.h btrfs: change for_rename argument of btrfs_record_unlink_dir() to bool 2023-06-19 13:59:26 +02:00
tree-mod-log.c btrfs: avoid tree mod log ENOMEM failures when we don't need to log 2023-06-19 13:59:38 +02:00
tree-mod-log.h
ulist.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00
ulist.h
uuid-tree.c
uuid-tree.h
verity.c btrfs: convert btrfs_read_merkle_tree_page() to use a folio 2023-09-13 18:40:54 +02:00
verity.h
volumes.c btrfs: remove the need_raid_map parameter from btrfs_map_block() 2023-10-12 16:44:05 +02:00
volumes.h btrfs: remove the need_raid_map parameter from btrfs_map_block() 2023-10-12 16:44:05 +02:00
xattr.c btrfs: convert to ctime accessor functions 2023-07-13 10:28:04 +02:00
xattr.h
zlib.c btrfs: disable allocation warnings for compression workspaces 2023-06-19 13:59:34 +02:00
zoned.c btrfs: remove the need_raid_map parameter from btrfs_map_block() 2023-10-12 16:44:05 +02:00
zoned.h btrfs: zoned: reserve zones for an active metadata/system block group 2023-08-21 14:52:19 +02:00
zstd.c btrfs: reformat remaining kdoc style comments 2023-10-12 16:44:04 +02:00