mirrors
/
linux-stable
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
1
0
Fork 0
Code Issues Releases Wiki Activity

New code for 6.9: 

* Online Repair;
   ** New ondisk structures being repaired.
      - Inode's mode field by trying to obtain file type value from the a
        directory entry.
      - Quota counters.
      - Link counts of inodes.
      - FS summary counters.
      - rmap btrees.
        Support for in-memory btrees has been added to support repair of rmap
        btrees.
   ** Misc changes
      - Report corruption of metadata to the health tracking subsystem.
      - Enable indirect health reporting when resources are scarce.
      - Reduce memory usage while reparing refcount btree.
      - Extend "Bmap update" intent item to support atomic extent swapping on
        the realtime device.
      - Extend "Bmap update" intent item to support extended attribute fork and
        unwritten extents.
   ** Code cleanups
      - Bmap log intent.
      - Btree block pointer checking.
      - Btree readahead.
      - Buffer target.
      - Symbolic link code.
   * Remove mrlock wrapper around the rwsem.
   * Convert all the GFP_NOFS flag usages to use the scoped
     memalloc_nofs_save() API instead of direct calls with the GFP_NOFS.
   * Refactor and simplify xfile abstraction. Lower level APIs in
     shmem.c are required to be exported in order to achieve this.
   * Skip checking alignment constraints for inode chunk allocations when block
     size is larger than inode chunk size.
   * Do not submit delwri buffers collected during log recovery when an error
     has been encountered.
   * Fix SEEK_HOLE/DATA for file regions which have active COW extents.
   * Fix lock order inversion when executing error handling path during
     shrinking a filesystem.
   * Remove duplicate ifdefs.
 
 Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQQjMC4mbgVeU7MxEIYH7y4RirJu9AUCZemMkgAKCRAH7y4RirJu
 9ON5AP0Vda6sMn/ZUYoLo9ZUrUvlUb8L0dhEN5JL0XfyWW5ogAD/bH4G6pKSNyTw
 cSEjryuDakirdHLt5g0c+QHd2a/fzw0=
 =ymKk
 -----END PGP SIGNATURE-----

Merge tag 'xfs-6.9-merge-8' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

Pull xfs updates from Chandan Babu:

 - Online repair updates:
    - More ondisk structures being repaired:
        - Inode's mode field by trying to obtain file type value from
          the a directory entry
        - Quota counters
        - Link counts of inodes
        - FS summary counters
        - Support for in-memory btrees has been added to support repair
          of rmap btrees
    - Misc changes:
        - Report corruption of metadata to the health tracking subsystem
        - Enable indirect health reporting when resources are scarce
        - Reduce memory usage while repairing refcount btree
        - Extend "Bmap update" intent item to support atomic extent
          swapping on the realtime device
        - Extend "Bmap update" intent item to support extended attribute
          fork and unwritten extents
    - Code cleanups:
        - Bmap log intent
        - Btree block pointer checking
        - Btree readahead
        - Buffer target
        - Symbolic link code

 - Remove mrlock wrapper around the rwsem

 - Convert all the GFP_NOFS flag usages to use the scoped
   memalloc_nofs_save() API instead of direct calls with the GFP_NOFS

 - Refactor and simplify xfile abstraction. Lower level APIs in shmem.c
   are required to be exported in order to achieve this

 - Skip checking alignment constraints for inode chunk allocations when
   block size is larger than inode chunk size

 - Do not submit delwri buffers collected during log recovery when an
   error has been encountered

 - Fix SEEK_HOLE/DATA for file regions which have active COW extents

 - Fix lock order inversion when executing error handling path during
   shrinking a filesystem

 - Remove duplicate ifdefs

* tag 'xfs-6.9-merge-8' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (183 commits)
  xfs: shrink failure needs to hold AGI buffer
  mm/shmem.c: Use new form of *@param in kernel-doc
  kernel-doc: Add unary operator * to $type_param_ref
  xfs: use kvfree() in xlog_cil_free_logvec()
  xfs: xfs_btree_bload_prep_block() should use __GFP_NOFAIL
  xfs: fix scrub stats file permissions
  xfs: fix log recovery erroring out on refcount recovery failure
  xfs: move symlink target write function to libxfs
  xfs: move remote symlink target read function to libxfs
  xfs: move xfs_symlink_remote.c declarations to xfs_symlink_remote.h
  xfs: xfs_bmap_finish_one should map unwritten extents properly
  xfs: support deferred bmap updates on the attr fork
  xfs: support recovering bmap intent items targetting realtime extents
  xfs: add a realtime flag to the bmap update log redo items
  xfs: add a xattr_entry helper
  xfs: fix xfs_bunmapi to allow unmapping of partial rt extents
  xfs: move xfs_bmap_defer_add to xfs_bmap_item.c
  xfs: reuse xfs_bmap_update_cancel_item
  xfs: add a bi_entry helper
  xfs: remove xfs_trans_set_bmap_flags
  ...
This commit is contained in:
Linus Torvalds 2024-03-13 13:52:24 -07:00
parent 279d44ceb8 75bcffbb9e
commit babbcc0232
186 changed files with 13274 additions and 3585 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
Documentation/filesystems/xfs/xfs-online-fsck-design.rst
View File

@ -1915,19 +1915,13 @@ four of those five higher level data structures.
The fifth use case is discussed in the :ref:`realtime summary <rtsummary>` case
study.
The most general storage interface supported by the xfile enables the reading
and writing of arbitrary quantities of data at arbitrary offsets in the xfile.
This capability is provided by ``xfile_pread`` and ``xfile_pwrite`` functions,
which behave similarly to their userspace counterparts.
XFS is very record-based, which suggests that the ability to load and store
complete records is important.
To support these cases, a pair of ``xfile_obj_load`` and ``xfile_obj_store``
functions are provided to read and persist objects into an xfile.
They are internally the same as pread and pwrite, except that they treat any
error as an out of memory error.
For online repair, squashing error conditions in this manner is an acceptable
behavior because the only reaction is to abort the operation back to userspace.
All five xfile usecases can be serviced by these four functions.
To support these cases, a pair of ``xfile_load`` and ``xfile_store``
functions are provided to read and persist objects into an xfile that treat any
error as an out of memory error. For online repair, squashing error conditions
in this manner is an acceptable behavior because the only reaction is to abort
the operation back to userspace.
However, no discussion of file access idioms is complete without answering the
question, "But what about mmap?"
@ -1939,15 +1933,14 @@ tmpfs can only push a pagecache folio to the swap cache if the folio is neither
pinned nor locked, which means the xfile must not pin too many folios.
Short term direct access to xfile contents is done by locking the pagecache
folio and mapping it into kernel address space.
Programmatic access (e.g. pread and pwrite) uses this mechanism.
Folio locks are not supposed to be held for long periods of time, so long
term direct access to xfile contents is done by bumping the folio refcount,
folio and mapping it into kernel address space. Object load and store uses this
mechanism. Folio locks are not supposed to be held for long periods of time, so
long term direct access to xfile contents is done by bumping the folio refcount,
mapping it into kernel address space, and dropping the folio lock.
These long term users *must* be responsive to memory reclaim by hooking into
the shrinker infrastructure to know when to release folios.
The ``xfile_get_page`` and ``xfile_put_page`` functions are provided to
The ``xfile_get_folio`` and ``xfile_put_folio`` functions are provided to
retrieve the (locked) folio that backs part of an xfile and to release it.
The only code to use these folio lease functions are the xfarray
:ref:`sorting<xfarray_sort>` algorithms and the :ref:`in-memory
@ -2277,13 +2270,12 @@ follows:
pointing to the xfile.
3. Pass the buffer cache target, buffer ops, and other information to
``xfbtree_create`` to write an initial tree header and root block to the
xfile.
``xfbtree_init`` to initialize the passed in ``struct xfbtree`` and write an
initial root block to the xfile.
Each btree type should define a wrapper that passes necessary arguments to
the creation function.
For example, rmap btrees define ``xfs_rmapbt_mem_create`` to take care of
all the necessary details for callers.
A ``struct xfbtree`` object will be returned.
4. Pass the xfbtree object to the btree cursor creation function for the
btree type.

13
fs/xfs/Kconfig
View File

@ -124,12 +124,24 @@ config XFS_DRAIN_INTENTS
bool
select JUMP_LABEL if HAVE_ARCH_JUMP_LABEL
config XFS_LIVE_HOOKS
bool
select JUMP_LABEL if HAVE_ARCH_JUMP_LABEL
config XFS_MEMORY_BUFS
bool
config XFS_BTREE_IN_MEM
bool
config XFS_ONLINE_SCRUB
bool "XFS online metadata check support"
default n
depends on XFS_FS
depends on TMPFS && SHMEM
select XFS_LIVE_HOOKS
select XFS_DRAIN_INTENTS
select XFS_MEMORY_BUFS
help
If you say Y here you will be able to check metadata on a
mounted XFS filesystem. This feature is intended to reduce
@ -164,6 +176,7 @@ config XFS_ONLINE_REPAIR
bool "XFS online metadata repair support"
default n
depends on XFS_FS && XFS_ONLINE_SCRUB
select XFS_BTREE_IN_MEM
help
If you say Y here you will be able to repair metadata on a
mounted XFS filesystem. This feature is intended to reduce

15
fs/xfs/Makefile
View File

@ -92,8 +92,7 @@ xfs-y += xfs_aops.o \
xfs_symlink.o \
xfs_sysfs.o \
xfs_trans.o \
xfs_xattr.o \
kmem.o
xfs_xattr.o
# low-level transaction/log code
xfs-y += xfs_log.o \
@ -137,6 +136,9 @@ xfs-$(CONFIG_FS_DAX) += xfs_notify_failure.o
endif
xfs-$(CONFIG_XFS_DRAIN_INTENTS) += xfs_drain.o
xfs-$(CONFIG_XFS_LIVE_HOOKS) += xfs_hooks.o
xfs-$(CONFIG_XFS_MEMORY_BUFS) += xfs_buf_mem.o
xfs-$(CONFIG_XFS_BTREE_IN_MEM) += libxfs/xfs_btree_mem.o
# online scrub/repair
ifeq ($(CONFIG_XFS_ONLINE_SCRUB),y)
@ -159,6 +161,8 @@ xfs-y += $(addprefix scrub/, \
health.o \
ialloc.o \
inode.o \
iscan.o \
nlinks.o \
parent.o \
readdir.o \
refcount.o \
@ -179,6 +183,7 @@ xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \
xfs-$(CONFIG_XFS_QUOTA) += $(addprefix scrub/, \
dqiterate.o \
quota.o \
quotacheck.o \
)
# online repair
@ -188,12 +193,17 @@ xfs-y += $(addprefix scrub/, \
alloc_repair.o \
bmap_repair.o \
cow_repair.o \
fscounters_repair.o \
ialloc_repair.o \
inode_repair.o \
newbt.o \
nlinks_repair.o \
rcbag_btree.o \
rcbag.o \
reap.o \
refcount_repair.o \
repair.o \
rmap_repair.o \
)
xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \
@ -202,6 +212,7 @@ xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \
xfs-$(CONFIG_XFS_QUOTA) += $(addprefix scrub/, \
quota_repair.o \
quotacheck_repair.o \
)
endif
endif

30
fs/xfs/kmem.c
View File

@ -1,30 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (c) 2000-2005 Silicon Graphics, Inc.
* All Rights Reserved.
*/
#include "xfs.h"
#include "xfs_message.h"
#include "xfs_trace.h"
void *
kmem_alloc(size_t size, xfs_km_flags_t flags)
{
int retries = 0;
gfp_t lflags = kmem_flags_convert(flags);
void *ptr;
trace_kmem_alloc(size, flags, _RET_IP_);
do {
ptr = kmalloc(size, lflags);
if (ptr || (flags & KM_MAYFAIL))
return ptr;
if (!(++retries % 100))
xfs_err(NULL,
"%s(%u) possible memory allocation deadlock size %u in %s (mode:0x%x)",
current->comm, current->pid,
(unsigned int)size, __func__, lflags);
memalloc_retry_wait(lflags);
} while (1);
}

83
fs/xfs/kmem.h
View File

@ -1,83 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2000-2005 Silicon Graphics, Inc.
* All Rights Reserved.
*/
#ifndef __XFS_SUPPORT_KMEM_H__
#define __XFS_SUPPORT_KMEM_H__
#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/mm.h>
#include <linux/vmalloc.h>
/*
* General memory allocation interfaces
*/
typedef unsigned __bitwise xfs_km_flags_t;
#define KM_NOFS ((__force xfs_km_flags_t)0x0004u)
#define KM_MAYFAIL ((__force xfs_km_flags_t)0x0008u)
#define KM_ZERO ((__force xfs_km_flags_t)0x0010u)
#define KM_NOLOCKDEP ((__force xfs_km_flags_t)0x0020u)
/*
* We use a special process flag to avoid recursive callbacks into
* the filesystem during transactions. We will also issue our own
* warnings, so we explicitly skip any generic ones (silly of us).
*/
static inline gfp_t
kmem_flags_convert(xfs_km_flags_t flags)
{
gfp_t lflags;
BUG_ON(flags & ~(KM_NOFS | KM_MAYFAIL | KM_ZERO | KM_NOLOCKDEP));
lflags = GFP_KERNEL | __GFP_NOWARN;
if (flags & KM_NOFS)
lflags &= ~__GFP_FS;
/*
* Default page/slab allocator behavior is to retry for ever
* for small allocations. We can override this behavior by using
* __GFP_RETRY_MAYFAIL which will tell the allocator to retry as long
* as it is feasible but rather fail than retry forever for all
* request sizes.
*/
if (flags & KM_MAYFAIL)
lflags |= __GFP_RETRY_MAYFAIL;
if (flags & KM_ZERO)
lflags |= __GFP_ZERO;
if (flags & KM_NOLOCKDEP)
lflags |= __GFP_NOLOCKDEP;
return lflags;
}
extern void *kmem_alloc(size_t, xfs_km_flags_t);
static inline void kmem_free(const void *ptr)
{
kvfree(ptr);
}
static inline void *
kmem_zalloc(size_t size, xfs_km_flags_t flags)
{
return kmem_alloc(size, flags | KM_ZERO);
}
/*
* Zone interfaces
*/
static inline struct page *
kmem_to_page(void *addr)
{
if (is_vmalloc_addr(addr))
return vmalloc_to_page(addr);
return virt_to_page(addr);
}
#endif /* __XFS_SUPPORT_KMEM_H__ */

79
fs/xfs/libxfs/xfs_ag.c
View File

@ -217,6 +217,7 @@ xfs_initialize_perag_data(
*/
if (fdblocks > sbp->sb_dblocks || ifree > ialloc) {
xfs_alert(mp, "AGF corruption. Please run xfs_repair.");
xfs_fs_mark_sick(mp, XFS_SICK_FS_COUNTERS);
error = -EFSCORRUPTED;
goto out;
}
@ -241,7 +242,7 @@ __xfs_free_perag(
struct xfs_perag *pag = container_of(head, struct xfs_perag, rcu_head);
ASSERT(!delayed_work_pending(&pag->pag_blockgc_work));
kmem_free(pag);
kfree(pag);
}
/*
@ -263,7 +264,7 @@ xfs_free_perag(
xfs_defer_drain_free(&pag->pag_intents_drain);
cancel_delayed_work_sync(&pag->pag_blockgc_work);
xfs_buf_hash_destroy(pag);
xfs_buf_cache_destroy(&pag->pag_bcache);
/* drop the mount's active reference */
xfs_perag_rele(pag);
@ -351,9 +352,9 @@ xfs_free_unused_perag_range(
spin_unlock(&mp->m_perag_lock);
if (!pag)
break;
xfs_buf_hash_destroy(pag);
xfs_buf_cache_destroy(&pag->pag_bcache);
xfs_defer_drain_free(&pag->pag_intents_drain);
kmem_free(pag);
kfree(pag);
}
}
@ -381,7 +382,7 @@ xfs_initialize_perag(
continue;
}
pag = kmem_zalloc(sizeof(*pag), KM_MAYFAIL);
pag = kzalloc(sizeof(*pag), GFP_KERNEL | __GFP_RETRY_MAYFAIL);
if (!pag) {
error = -ENOMEM;
goto out_unwind_new_pags;
@ -389,7 +390,7 @@ xfs_initialize_perag(
pag->pag_agno = index;
pag->pag_mount = mp;
error = radix_tree_preload(GFP_NOFS);
error = radix_tree_preload(GFP_KERNEL | __GFP_RETRY_MAYFAIL);
if (error)
goto out_free_pag;
@ -416,9 +417,10 @@ xfs_initialize_perag(
init_waitqueue_head(&pag->pag_active_wq);
pag->pagb_count = 0;
pag->pagb_tree = RB_ROOT;
xfs_hooks_init(&pag->pag_rmap_update_hooks);
#endif /* __KERNEL__ */
error = xfs_buf_hash_init(pag);
error = xfs_buf_cache_init(&pag->pag_bcache);
if (error)
goto out_remove_pag;
@ -453,7 +455,7 @@ out_remove_pag:
radix_tree_delete(&mp->m_perag_tree, index);
spin_unlock(&mp->m_perag_lock);
out_free_pag:
kmem_free(pag);
kfree(pag);
out_unwind_new_pags:
/* unwind any prior newly initialized pags */
xfs_free_unused_perag_range(mp, first_initialised, agcount);
@ -491,7 +493,7 @@ xfs_btroot_init(
struct xfs_buf *bp,
struct aghdr_init_data *id)
{
xfs_btree_init_block(mp, bp, id->type, 0, 0, id->agno);
xfs_btree_init_buf(mp, bp, id->bc_ops, 0, 0, id->agno);
}
/* Finish initializing a free space btree. */
@ -549,7 +551,7 @@ xfs_freesp_init_recs(
}
/*
* Alloc btree root block init functions
* bnobt/cntbt btree root block init functions
*/
static void
xfs_bnoroot_init(
@ -557,17 +559,7 @@ xfs_bnoroot_init(
struct xfs_buf *bp,
struct aghdr_init_data *id)
{
xfs_btree_init_block(mp, bp, XFS_BTNUM_BNO, 0, 0, id->agno);
xfs_freesp_init_recs(mp, bp, id);
}
static void
xfs_cntroot_init(
struct xfs_mount *mp,
struct xfs_buf *bp,
struct aghdr_init_data *id)
{
xfs_btree_init_block(mp, bp, XFS_BTNUM_CNT, 0, 0, id->agno);
xfs_btree_init_buf(mp, bp, id->bc_ops, 0, 0, id->agno);
xfs_freesp_init_recs(mp, bp, id);
}
@ -583,7 +575,7 @@ xfs_rmaproot_init(
struct xfs_btree_block *block = XFS_BUF_TO_BLOCK(bp);
struct xfs_rmap_rec *rrec;
xfs_btree_init_block(mp, bp, XFS_BTNUM_RMAP, 0, 4, id->agno);
xfs_btree_init_buf(mp, bp, id->bc_ops, 0, 4, id->agno);
/*
* mark the AG header regions as static metadata The BNO
@ -678,14 +670,13 @@ xfs_agfblock_init(
agf->agf_versionnum = cpu_to_be32(XFS_AGF_VERSION);
agf->agf_seqno = cpu_to_be32(id->agno);
agf->agf_length = cpu_to_be32(id->agsize);
agf->agf_roots[XFS_BTNUM_BNOi] = cpu_to_be32(XFS_BNO_BLOCK(mp));
agf->agf_roots[XFS_BTNUM_CNTi] = cpu_to_be32(XFS_CNT_BLOCK(mp));
agf->agf_levels[XFS_BTNUM_BNOi] = cpu_to_be32(1);
agf->agf_levels[XFS_BTNUM_CNTi] = cpu_to_be32(1);
agf->agf_bno_root = cpu_to_be32(XFS_BNO_BLOCK(mp));
agf->agf_cnt_root = cpu_to_be32(XFS_CNT_BLOCK(mp));
agf->agf_bno_level = cpu_to_be32(1);
agf->agf_cnt_level = cpu_to_be32(1);
if (xfs_has_rmapbt(mp)) {
agf->agf_roots[XFS_BTNUM_RMAPi] =
cpu_to_be32(XFS_RMAP_BLOCK(mp));
agf->agf_levels[XFS_BTNUM_RMAPi] = cpu_to_be32(1);
agf->agf_rmap_root = cpu_to_be32(XFS_RMAP_BLOCK(mp));
agf->agf_rmap_level = cpu_to_be32(1);
agf->agf_rmap_blocks = cpu_to_be32(1);
}
@ -796,7 +787,7 @@ struct xfs_aghdr_grow_data {
size_t numblks;
const struct xfs_buf_ops *ops;
aghdr_init_work_f work;
xfs_btnum_t type;
const struct xfs_btree_ops *bc_ops;
bool need_init;
};
@ -850,13 +841,15 @@ xfs_ag_init_headers(
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_bnobt_buf_ops,
.work = &xfs_bnoroot_init,
.bc_ops = &xfs_bnobt_ops,
.need_init = true
},
{ /* CNT root block */
.daddr = XFS_AGB_TO_DADDR(mp, id->agno, XFS_CNT_BLOCK(mp)),
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_cntbt_buf_ops,
.work = &xfs_cntroot_init,
.work = &xfs_bnoroot_init,
.bc_ops = &xfs_cntbt_ops,
.need_init = true
},
{ /* INO root block */
@ -864,7 +857,7 @@ xfs_ag_init_headers(
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_inobt_buf_ops,
.work = &xfs_btroot_init,
.type = XFS_BTNUM_INO,
.bc_ops = &xfs_inobt_ops,
.need_init = true
},
{ /* FINO root block */
@ -872,7 +865,7 @@ xfs_ag_init_headers(
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_finobt_buf_ops,
.work = &xfs_btroot_init,
.type = XFS_BTNUM_FINO,
.bc_ops = &xfs_finobt_ops,
.need_init = xfs_has_finobt(mp)
},
{ /* RMAP root block */
@ -880,6 +873,7 @@ xfs_ag_init_headers(
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_rmapbt_buf_ops,
.work = &xfs_rmaproot_init,
.bc_ops = &xfs_rmapbt_ops,
.need_init = xfs_has_rmapbt(mp)
},
{ /* REFC root block */
@ -887,7 +881,7 @@ xfs_ag_init_headers(
.numblks = BTOBB(mp->m_sb.sb_blocksize),
.ops = &xfs_refcountbt_buf_ops,
.work = &xfs_btroot_init,
.type = XFS_BTNUM_REFC,
.bc_ops = &xfs_refcountbt_ops,
.need_init = xfs_has_reflink(mp)
},
{ /* NULL terminating block */
@ -905,7 +899,7 @@ xfs_ag_init_headers(
id->daddr = dp->daddr;
id->numblks = dp->numblks;
id->type = dp->type;
id->bc_ops = dp->bc_ops;
error = xfs_ag_init_hdr(mp, id, dp->work, dp->ops);
if (error)
break;
@ -950,8 +944,10 @@ xfs_ag_shrink_space(
agf = agfbp->b_addr;
aglen = be32_to_cpu(agi->agi_length);
/* some extra paranoid checks before we shrink the ag */
if (XFS_IS_CORRUPT(mp, agf->agf_length != agi->agi_length))
if (XFS_IS_CORRUPT(mp, agf->agf_length != agi->agi_length)) {
xfs_ag_mark_sick(pag, XFS_SICK_AG_AGF);
return -EFSCORRUPTED;
}
if (delta >= aglen)
return -EINVAL;
@ -979,14 +975,23 @@ xfs_ag_shrink_space(
if (error) {
/*
* if extent allocation fails, need to roll the transaction to
* If extent allocation fails, need to roll the transaction to
* ensure that the AGFL fixup has been committed anyway.
*
* We need to hold the AGF across the roll to ensure nothing can
* access the AG for allocation until the shrink is fully
* cleaned up. And due to the resetting of the AG block
* reservation space needing to lock the AGI, we also have to
* hold that so we don't get AGI/AGF lock order inversions in
* the error handling path.
*/
xfs_trans_bhold(*tpp, agfbp);
xfs_trans_bhold(*tpp, agibp);
err2 = xfs_trans_roll(tpp);
if (err2)
return err2;
xfs_trans_bjoin(*tpp, agfbp);
xfs_trans_bjoin(*tpp, agibp);
goto resv_init_out;
}

18
fs/xfs/libxfs/xfs_ag.h
View File

@ -36,8 +36,9 @@ struct xfs_perag {
atomic_t pag_active_ref; /* active reference count */
wait_queue_head_t pag_active_wq;/* woken active_ref falls to zero */
unsigned long pag_opstate;
uint8_t pagf_levels[XFS_BTNUM_AGF];
/* # of levels in bno & cnt btree */
uint8_t pagf_bno_level; /* # of levels in bno btree */
uint8_t pagf_cnt_level; /* # of levels in cnt btree */
uint8_t pagf_rmap_level;/* # of levels in rmap btree */
uint32_t pagf_flcount; /* count of blocks in freelist */
xfs_extlen_t pagf_freeblks; /* total free blocks */
xfs_extlen_t pagf_longest; /* longest free space */
@ -86,8 +87,10 @@ struct xfs_perag {
* Alternate btree heights so that online repair won't trip the write
* verifiers while rebuilding the AG btrees.
*/
uint8_t pagf_repair_levels[XFS_BTNUM_AGF];
uint8_t pagf_repair_bno_level;
uint8_t pagf_repair_cnt_level;
uint8_t pagf_repair_refcount_level;
uint8_t pagf_repair_rmap_level;
#endif
spinlock_t pag_state_lock;
@ -104,9 +107,7 @@ struct xfs_perag {
int pag_ici_reclaimable; /* reclaimable inodes */
unsigned long pag_ici_reclaim_cursor; /* reclaim restart point */
/* buffer cache index */
spinlock_t pag_buf_lock; /* lock for pag_buf_hash */
struct rhashtable pag_buf_hash;
struct xfs_buf_cache pag_bcache;
/* background prealloc block trimming */
struct delayed_work pag_blockgc_work;
@ -119,6 +120,9 @@ struct xfs_perag {
* inconsistencies.
*/
struct xfs_defer_drain pag_intents_drain;
/* Hook to feed rmapbt updates to an active online repair. */
struct xfs_hooks pag_rmap_update_hooks;
#endif /* __KERNEL__ */
};
@ -331,7 +335,7 @@ struct aghdr_init_data {
/* per header data */
xfs_daddr_t daddr; /* header location */
size_t numblks; /* size of header */
xfs_btnum_t type; /* type of btree root block */
const struct xfs_btree_ops *bc_ops; /* btree ops */
};
int xfs_ag_init_headers(struct xfs_mount *mp, struct aghdr_init_data *id);

258
fs/xfs/libxfs/xfs_alloc.c
View File

@ -26,6 +26,7 @@
#include "xfs_ag.h"
#include "xfs_ag_resv.h"
#include "xfs_bmap.h"
#include "xfs_health.h"
struct kmem_cache *xfs_extfree_item_cache;
@ -150,23 +151,38 @@ xfs_alloc_ag_max_usable(
return mp->m_sb.sb_agblocks - blocks;
}
static int
xfs_alloc_lookup(
struct xfs_btree_cur *cur,
xfs_lookup_t dir,
xfs_agblock_t bno,
xfs_extlen_t len,
int *stat)
{
int error;
cur->bc_rec.a.ar_startblock = bno;
cur->bc_rec.a.ar_blockcount = len;
error = xfs_btree_lookup(cur, dir, stat);
if (*stat == 1)
cur->bc_flags |= XFS_BTREE_ALLOCBT_ACTIVE;
else
cur->bc_flags &= ~XFS_BTREE_ALLOCBT_ACTIVE;
return error;
}
/*
* Lookup the record equal to [bno, len] in the btree given by cur.
*/
STATIC int /* error */
static inline int /* error */
xfs_alloc_lookup_eq(
struct xfs_btree_cur *cur, /* btree cursor */
xfs_agblock_t bno, /* starting block of extent */
xfs_extlen_t len, /* length of extent */
int *stat) /* success/failure */
{
int error;
cur->bc_rec.a.ar_startblock = bno;
cur->bc_rec.a.ar_blockcount = len;
error = xfs_btree_lookup(cur, XFS_LOOKUP_EQ, stat);
cur->bc_ag.abt.active = (*stat == 1);
return error;
return xfs_alloc_lookup(cur, XFS_LOOKUP_EQ, bno, len, stat);
}
/*
@ -180,13 +196,7 @@ xfs_alloc_lookup_ge(
xfs_extlen_t len, /* length of extent */
int *stat) /* success/failure */
{
int error;
cur->bc_rec.a.ar_startblock = bno;
cur->bc_rec.a.ar_blockcount = len;
error = xfs_btree_lookup(cur, XFS_LOOKUP_GE, stat);
cur->bc_ag.abt.active = (*stat == 1);
return error;
return xfs_alloc_lookup(cur, XFS_LOOKUP_GE, bno, len, stat);
}
/*
@ -200,19 +210,14 @@ xfs_alloc_lookup_le(
xfs_extlen_t len, /* length of extent */
int *stat) /* success/failure */
{
int error;
cur->bc_rec.a.ar_startblock = bno;
cur->bc_rec.a.ar_blockcount = len;
error = xfs_btree_lookup(cur, XFS_LOOKUP_LE, stat);
cur->bc_ag.abt.active = (*stat == 1);
return error;
return xfs_alloc_lookup(cur, XFS_LOOKUP_LE, bno, len, stat);
}
static inline bool
xfs_alloc_cur_active(
struct xfs_btree_cur *cur)
{
return cur && cur->bc_ag.abt.active;
return cur && (cur->bc_flags & XFS_BTREE_ALLOCBT_ACTIVE);
}
/*
@ -268,12 +273,12 @@ xfs_alloc_complain_bad_rec(
struct xfs_mount *mp = cur->bc_mp;
xfs_warn(mp,
"%s Freespace BTree record corruption in AG %d detected at %pS!",
cur->bc_btnum == XFS_BTNUM_BNO ? "Block" : "Size",
cur->bc_ag.pag->pag_agno, fa);
"%sbt record corruption in AG %d detected at %pS!",
cur->bc_ops->name, cur->bc_ag.pag->pag_agno, fa);
xfs_warn(mp,
"start block 0x%x block count 0x%x", irec->ar_startblock,
irec->ar_blockcount);
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
@ -497,14 +502,18 @@ xfs_alloc_fixup_trees(
if (XFS_IS_CORRUPT(mp,
i != 1 ||
nfbno1 != fbno ||
nflen1 != flen))
nflen1 != flen)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
#endif
} else {
if ((error = xfs_alloc_lookup_eq(cnt_cur, fbno, flen, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
}
/*
* Look up the record in the by-block tree if necessary.
@ -516,14 +525,18 @@ xfs_alloc_fixup_trees(
if (XFS_IS_CORRUPT(mp,
i != 1 ||
nfbno1 != fbno ||
nflen1 != flen))
nflen1 != flen)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
#endif
} else {
if ((error = xfs_alloc_lookup_eq(bno_cur, fbno, flen, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
}
#ifdef DEBUG
@ -536,8 +549,10 @@ xfs_alloc_fixup_trees(
if (XFS_IS_CORRUPT(mp,
bnoblock->bb_numrecs !=
cntblock->bb_numrecs))
cntblock->bb_numrecs)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
}
#endif
@ -567,30 +582,40 @@ xfs_alloc_fixup_trees(
*/
if ((error = xfs_btree_delete(cnt_cur, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
/*
* Add new by-size btree entry(s).
*/
if (nfbno1 != NULLAGBLOCK) {
if ((error = xfs_alloc_lookup_eq(cnt_cur, nfbno1, nflen1, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 0))
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
if ((error = xfs_btree_insert(cnt_cur, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
}
if (nfbno2 != NULLAGBLOCK) {
if ((error = xfs_alloc_lookup_eq(cnt_cur, nfbno2, nflen2, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 0))
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
if ((error = xfs_btree_insert(cnt_cur, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
return -EFSCORRUPTED;
}
}
/*
* Fix up the by-block btree entry(s).
@ -601,8 +626,10 @@ xfs_alloc_fixup_trees(
*/
if ((error = xfs_btree_delete(bno_cur, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
} else {
/*
* Update the by-block entry to start later|be shorter.
@ -616,12 +643,16 @@ xfs_alloc_fixup_trees(
*/
if ((error = xfs_alloc_lookup_eq(bno_cur, nfbno2, nflen2, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 0))
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
if ((error = xfs_btree_insert(bno_cur, &i)))
return error;
if (XFS_IS_CORRUPT(mp, i != 1))
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
return -EFSCORRUPTED;
}
}
return 0;
}
@ -755,6 +786,8 @@ xfs_alloc_read_agfl(
mp, tp, mp->m_ddev_targp,
XFS_AG_DADDR(mp, pag->pag_agno, XFS_AGFL_DADDR(mp)),
XFS_FSS_TO_BB(mp, 1), 0, &bp, &xfs_agfl_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_ag_mark_sick(pag, XFS_SICK_AG_AGFL);
if (error)
return error;
xfs_buf_set_ref(bp, XFS_AGFL_REF);
@ -776,6 +809,7 @@ xfs_alloc_update_counters(
if (unlikely(be32_to_cpu(agf->agf_freeblks) >
be32_to_cpu(agf->agf_length))) {
xfs_buf_mark_corrupt(agbp);
xfs_ag_mark_sick(agbp->b_pag, XFS_SICK_AG_AGF);
return -EFSCORRUPTED;
}
@ -828,8 +862,8 @@ xfs_alloc_cur_setup(
* attempt a small allocation.
*/
if (!acur->cnt)
acur->cnt = xfs_allocbt_init_cursor(args->mp, args->tp,
args->agbp, args->pag, XFS_BTNUM_CNT);
acur->cnt = xfs_cntbt_init_cursor(args->mp, args->tp,
args->agbp, args->pag);
error = xfs_alloc_lookup_ge(acur->cnt, 0, args->maxlen, &i);
if (error)
return error;
@ -838,11 +872,11 @@ xfs_alloc_cur_setup(
* Allocate the bnobt left and right search cursors.
*/
if (!acur->bnolt)
acur->bnolt = xfs_allocbt_init_cursor(args->mp, args->tp,
args->agbp, args->pag, XFS_BTNUM_BNO);
acur->bnolt = xfs_bnobt_init_cursor(args->mp, args->tp,
args->agbp, args->pag);
if (!acur->bnogt)
acur->bnogt = xfs_allocbt_init_cursor(args->mp, args->tp,
args->agbp, args->pag, XFS_BTNUM_BNO);
acur->bnogt = xfs_bnobt_init_cursor(args->mp, args->tp,
args->agbp, args->pag);
return i == 1 ? 0 : -ENOSPC;
}
@ -884,15 +918,17 @@ xfs_alloc_cur_check(
bool busy;
unsigned busy_gen = 0;
bool deactivate = false;
bool isbnobt = cur->bc_btnum == XFS_BTNUM_BNO;
bool isbnobt = xfs_btree_is_bno(cur->bc_ops);
*new = 0;
error = xfs_alloc_get_rec(cur, &bno, &len, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(args->mp, i != 1))
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
/*
* Check minlen and deactivate a cntbt cursor if out of acceptable size
@ -958,9 +994,8 @@ xfs_alloc_cur_check(
deactivate = true;
out:
if (deactivate)
cur->bc_ag.abt.active = false;
trace_xfs_alloc_cur_check(args->mp, cur->bc_btnum, bno, len, diff,
*new);
cur->bc_flags &= ~XFS_BTREE_ALLOCBT_ACTIVE;
trace_xfs_alloc_cur_check(cur, bno, len, diff, *new);
return 0;
}
@ -1098,6 +1133,7 @@ xfs_alloc_ag_vextent_small(
if (error)
goto error;
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(ccur);
error = -EFSCORRUPTED;
goto error;
}
@ -1132,6 +1168,7 @@ xfs_alloc_ag_vextent_small(
*fbnop = args->agbno = fbno;
*flenp = args->len = 1;
if (XFS_IS_CORRUPT(args->mp, fbno >= be32_to_cpu(agf->agf_length))) {
xfs_btree_mark_sick(ccur);
error = -EFSCORRUPTED;
goto error;
}
@ -1197,8 +1234,8 @@ xfs_alloc_ag_vextent_exact(
/*
* Allocate/initialize a cursor for the by-number freespace btree.
*/
bno_cur = xfs_allocbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag, XFS_BTNUM_BNO);
bno_cur = xfs_bnobt_init_cursor(args->mp, args->tp, args->agbp,
args->pag);
/*
* Lookup bno and minlen in the btree (minlen is irrelevant, really).
@ -1218,6 +1255,7 @@ xfs_alloc_ag_vextent_exact(
if (error)
goto error0;
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1257,8 +1295,8 @@ xfs_alloc_ag_vextent_exact(
* We are allocating agbno for args->len
* Allocate/initialize a cursor for the by-size btree.
*/
cnt_cur = xfs_allocbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag, XFS_BTNUM_CNT);
cnt_cur = xfs_cntbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag);
ASSERT(args->agbno + args->len <= be32_to_cpu(agf->agf_length));
error = xfs_alloc_fixup_trees(cnt_cur, bno_cur, fbno, flen, args->agbno,
args->len, XFSA_FIXUP_BNO_OK);
@ -1330,7 +1368,7 @@ xfs_alloc_walk_iter(
if (error)
return error;
if (i == 0)
cur->bc_ag.abt.active = false;
cur->bc_flags &= ~XFS_BTREE_ALLOCBT_ACTIVE;
if (count > 0)
count--;
@ -1444,7 +1482,7 @@ xfs_alloc_ag_vextent_locality(
if (error)
return error;
if (i) {
acur->cnt->bc_ag.abt.active = true;
acur->cnt->bc_flags |= XFS_BTREE_ALLOCBT_ACTIVE;
fbcur = acur->cnt;
fbinc = false;
}
@ -1497,8 +1535,10 @@ xfs_alloc_ag_vextent_lastblock(
error = xfs_alloc_get_rec(acur->cnt, bno, len, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(args->mp, i != 1))
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(acur->cnt);
return -EFSCORRUPTED;
}
if (*len >= args->minlen)
break;
error = xfs_btree_increment(acur->cnt, 0, &i);
@ -1670,8 +1710,8 @@ restart:
/*
* Allocate and initialize a cursor for the by-size btree.
*/
cnt_cur = xfs_allocbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag, XFS_BTNUM_CNT);
cnt_cur = xfs_cntbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag);
bno_cur = NULL;
/*
@ -1710,6 +1750,7 @@ restart:
if (error)
goto error0;
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1756,6 +1797,7 @@ restart:
rlen != 0 &&
(rlen > flen ||
rbno + rlen > fbno + flen))) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1778,6 +1820,7 @@ restart:
&i)))
goto error0;
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1790,6 +1833,7 @@ restart:
rlen != 0 &&
(rlen > flen ||
rbno + rlen > fbno + flen))) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1806,6 +1850,7 @@ restart:
&i)))
goto error0;
if (XFS_IS_CORRUPT(args->mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1844,14 +1889,15 @@ restart:
rlen = args->len;
if (XFS_IS_CORRUPT(args->mp, rlen > flen)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
/*
* Allocate and initialize a cursor for the by-block tree.
*/
bno_cur = xfs_allocbt_init_cursor(args->mp, args->tp, args->agbp,
args->pag, XFS_BTNUM_BNO);
bno_cur = xfs_bnobt_init_cursor(args->mp, args->tp, args->agbp,
args->pag);
if ((error = xfs_alloc_fixup_trees(cnt_cur, bno_cur, fbno, flen,
rbno, rlen, XFSA_FIXUP_CNT_OK)))
goto error0;
@ -1863,6 +1909,7 @@ restart:
if (XFS_IS_CORRUPT(args->mp,
args->agbno + args->len >
be32_to_cpu(agf->agf_length))) {
xfs_ag_mark_sick(args->pag, XFS_SICK_AG_BNOBT);
error = -EFSCORRUPTED;
goto error0;
}
@ -1924,7 +1971,7 @@ xfs_free_ag_extent(
/*
* Allocate and initialize a cursor for the by-block btree.
*/
bno_cur = xfs_allocbt_init_cursor(mp, tp, agbp, pag, XFS_BTNUM_BNO);
bno_cur = xfs_bnobt_init_cursor(mp, tp, agbp, pag);
/*
* Look for a neighboring block on the left (lower block numbers)
* that is contiguous with this space.
@ -1938,6 +1985,7 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_get_rec(bno_cur, &ltbno, &ltlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1953,6 +2001,7 @@ xfs_free_ag_extent(
* Very bad.
*/
if (XFS_IS_CORRUPT(mp, ltbno + ltlen > bno)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1971,6 +2020,7 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_get_rec(bno_cur, &gtbno, &gtlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1986,6 +2036,7 @@ xfs_free_ag_extent(
* Very bad.
*/
if (XFS_IS_CORRUPT(mp, bno + len > gtbno)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1994,7 +2045,7 @@ xfs_free_ag_extent(
/*
* Now allocate and initialize a cursor for the by-size tree.
*/
cnt_cur = xfs_allocbt_init_cursor(mp, tp, agbp, pag, XFS_BTNUM_CNT);
cnt_cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
/*
* Have both left and right contiguous neighbors.
* Merge all three into a single free block.
@ -2006,12 +2057,14 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_lookup_eq(cnt_cur, ltbno, ltlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
if ((error = xfs_btree_delete(cnt_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2021,12 +2074,14 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_lookup_eq(cnt_cur, gtbno, gtlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
if ((error = xfs_btree_delete(cnt_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2036,6 +2091,7 @@ xfs_free_ag_extent(
if ((error = xfs_btree_delete(bno_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2045,6 +2101,7 @@ xfs_free_ag_extent(
if ((error = xfs_btree_decrement(bno_cur, 0, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2064,6 +2121,7 @@ xfs_free_ag_extent(
i != 1 ||
xxbno != ltbno ||
xxlen != ltlen)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2088,12 +2146,14 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_lookup_eq(cnt_cur, ltbno, ltlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
if ((error = xfs_btree_delete(cnt_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2104,6 +2164,7 @@ xfs_free_ag_extent(
if ((error = xfs_btree_decrement(bno_cur, 0, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2123,12 +2184,14 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_lookup_eq(cnt_cur, gtbno, gtlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
if ((error = xfs_btree_delete(cnt_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2151,6 +2214,7 @@ xfs_free_ag_extent(
if ((error = xfs_btree_insert(bno_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(bno_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2163,12 +2227,14 @@ xfs_free_ag_extent(
if ((error = xfs_alloc_lookup_eq(cnt_cur, nbno, nlen, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
if ((error = xfs_btree_insert(cnt_cur, &i)))
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2267,8 +2333,9 @@ xfs_alloc_min_freelist(
struct xfs_perag *pag)
{
/* AG btrees have at least 1 level. */
static const uint8_t fake_levels[XFS_BTNUM_AGF] = {1, 1, 1};
const uint8_t *levels = pag ? pag->pagf_levels : fake_levels;
const unsigned int bno_level = pag ? pag->pagf_bno_level : 1;
const unsigned int cnt_level = pag ? pag->pagf_cnt_level : 1;
const unsigned int rmap_level = pag ? pag->pagf_rmap_level : 1;
unsigned int min_free;
ASSERT(mp->m_alloc_maxlevels > 0);
@ -2295,16 +2362,12 @@ xfs_alloc_min_freelist(
*/
/* space needed by-bno freespace btree */
min_free = min_t(unsigned int, levels[XFS_BTNUM_BNOi] + 1,
mp->m_alloc_maxlevels) * 2 - 2;
min_free = min(bno_level + 1, mp->m_alloc_maxlevels) * 2 - 2;
/* space needed by-size freespace btree */
min_free += min_t(unsigned int, levels[XFS_BTNUM_CNTi] + 1,
mp->m_alloc_maxlevels) * 2 - 2;
min_free += min(cnt_level + 1, mp->m_alloc_maxlevels) * 2 - 2;
/* space needed reverse mapping used space btree */
if (xfs_has_rmapbt(mp))
min_free += min_t(unsigned int, levels[XFS_BTNUM_RMAPi] + 1,
mp->m_rmap_maxlevels) * 2 - 2;
min_free += min(rmap_level + 1, mp->m_rmap_maxlevels) * 2 - 2;
return min_free;
}
@ -2691,13 +2754,14 @@ xfs_exact_minlen_extent_available(
xfs_extlen_t flen;
int error = 0;
cnt_cur = xfs_allocbt_init_cursor(args->mp, args->tp, agbp,
args->pag, XFS_BTNUM_CNT);
cnt_cur = xfs_cntbt_init_cursor(args->mp, args->tp, agbp,
args->pag);
error = xfs_alloc_lookup_ge(cnt_cur, 0, args->minlen, stat);
if (error)
goto out;
if (*stat == 0) {
xfs_btree_mark_sick(cnt_cur);
error = -EFSCORRUPTED;
goto out;
}
@ -2987,8 +3051,8 @@ xfs_alloc_log_agf(
offsetof(xfs_agf_t, agf_versionnum),
offsetof(xfs_agf_t, agf_seqno),
offsetof(xfs_agf_t, agf_length),
offsetof(xfs_agf_t, agf_roots[0]),
offsetof(xfs_agf_t, agf_levels[0]),
offsetof(xfs_agf_t, agf_bno_root), /* also cnt/rmap root */
offsetof(xfs_agf_t, agf_bno_level), /* also cnt/rmap levels */
offsetof(xfs_agf_t, agf_flfirst),
offsetof(xfs_agf_t, agf_fllast),
offsetof(xfs_agf_t, agf_flcount),
@ -3167,12 +3231,10 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_freeblks) > agf_length)
return __this_address;
if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) >
mp->m_alloc_maxlevels ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) >
mp->m_alloc_maxlevels)
if (be32_to_cpu(agf->agf_bno_level) < 1 ||
be32_to_cpu(agf->agf_cnt_level) < 1 ||
be32_to_cpu(agf->agf_bno_level) > mp->m_alloc_maxlevels ||
be32_to_cpu(agf->agf_cnt_level) > mp->m_alloc_maxlevels)
return __this_address;
if (xfs_has_lazysbcount(mp) &&
@ -3183,9 +3245,8 @@ xfs_agf_verify(
if (be32_to_cpu(agf->agf_rmap_blocks) > agf_length)
return __this_address;
if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]) >
mp->m_rmap_maxlevels)
if (be32_to_cpu(agf->agf_rmap_level) < 1 ||
be32_to_cpu(agf->agf_rmap_level) > mp->m_rmap_maxlevels)
return __this_address;
}
@ -3268,6 +3329,8 @@ xfs_read_agf(
error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp,
XFS_AG_DADDR(mp, pag->pag_agno, XFS_AGF_DADDR(mp)),
XFS_FSS_TO_BB(mp, 1), flags, agfbpp, &xfs_agf_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_ag_mark_sick(pag, XFS_SICK_AG_AGF);
if (error)
return error;
@ -3309,12 +3372,9 @@ xfs_alloc_read_agf(
pag->pagf_btreeblks = be32_to_cpu(agf->agf_btreeblks);
pag->pagf_flcount = be32_to_cpu(agf->agf_flcount);
pag->pagf_longest = be32_to_cpu(agf->agf_longest);
pag->pagf_levels[XFS_BTNUM_BNOi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNOi]);
pag->pagf_levels[XFS_BTNUM_CNTi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNTi]);
pag->pagf_levels[XFS_BTNUM_RMAPi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAPi]);
pag->pagf_bno_level = be32_to_cpu(agf->agf_bno_level);
pag->pagf_cnt_level = be32_to_cpu(agf->agf_cnt_level);
pag->pagf_rmap_level = be32_to_cpu(agf->agf_rmap_level);
pag->pagf_refcount_level = be32_to_cpu(agf->agf_refcount_level);
if (xfs_agfl_needs_reset(pag->pag_mount, agf))
set_bit(XFS_AGSTATE_AGFL_NEEDS_RESET, &pag->pag_opstate);
@ -3343,10 +3403,8 @@ xfs_alloc_read_agf(
ASSERT(pag->pagf_btreeblks == be32_to_cpu(agf->agf_btreeblks));
ASSERT(pag->pagf_flcount == be32_to_cpu(agf->agf_flcount));
ASSERT(pag->pagf_longest == be32_to_cpu(agf->agf_longest));
ASSERT(pag->pagf_levels[XFS_BTNUM_BNOi] ==
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNOi]));
ASSERT(pag->pagf_levels[XFS_BTNUM_CNTi] ==
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNTi]));
ASSERT(pag->pagf_bno_level == be32_to_cpu(agf->agf_bno_level));
ASSERT(pag->pagf_cnt_level == be32_to_cpu(agf->agf_cnt_level));
}
#endif
if (agfbpp)
@ -3895,17 +3953,23 @@ __xfs_free_extent(
return -EIO;
error = xfs_free_extent_fix_freelist(tp, pag, &agbp);
if (error)
if (error) {
if (xfs_metadata_is_sick(error))
xfs_ag_mark_sick(pag, XFS_SICK_AG_BNOBT);
return error;
}
agf = agbp->b_addr;
if (XFS_IS_CORRUPT(mp, agbno >= mp->m_sb.sb_agblocks)) {
xfs_ag_mark_sick(pag, XFS_SICK_AG_BNOBT);
error = -EFSCORRUPTED;
goto err_release;
}
/* validate the extent size is legal now we have the agf locked */
if (XFS_IS_CORRUPT(mp, agbno + len > be32_to_cpu(agf->agf_length))) {
xfs_ag_mark_sick(pag, XFS_SICK_AG_BNOBT);
error = -EFSCORRUPTED;
goto err_release;
}
@ -3962,7 +4026,7 @@ xfs_alloc_query_range(
union xfs_btree_irec high_brec = { .a = *high_rec };
struct xfs_alloc_query_range_info query = { .priv = priv, .fn = fn };
ASSERT(cur->bc_btnum == XFS_BTNUM_BNO);
ASSERT(xfs_btree_is_bno(cur->bc_ops));
return xfs_btree_query_range(cur, &low_brec, &high_brec,
xfs_alloc_query_range_helper, &query);
}
@ -3976,7 +4040,7 @@ xfs_alloc_query_all(
{
struct xfs_alloc_query_range_info query;
ASSERT(cur->bc_btnum == XFS_BTNUM_BNO);
ASSERT(xfs_btree_is_bno(cur->bc_ops));
query.priv = priv;
query.fn = fn;
return xfs_btree_query_all(cur, xfs_alloc_query_range_helper, &query);

191
fs/xfs/libxfs/xfs_alloc_btree.c
View File

@ -16,6 +16,7 @@
#include "xfs_alloc.h"
#include "xfs_extent_busy.h"
#include "xfs_error.h"
#include "xfs_health.h"
#include "xfs_trace.h"
#include "xfs_trans.h"
#include "xfs_ag.h"
@ -23,13 +24,22 @@
static struct kmem_cache *xfs_allocbt_cur_cache;
STATIC struct xfs_btree_cur *
xfs_allocbt_dup_cursor(
xfs_bnobt_dup_cursor(
struct xfs_btree_cur *cur)
{
return xfs_allocbt_init_cursor(cur->bc_mp, cur->bc_tp,
cur->bc_ag.agbp, cur->bc_ag.pag, cur->bc_btnum);
return xfs_bnobt_init_cursor(cur->bc_mp, cur->bc_tp, cur->bc_ag.agbp,
cur->bc_ag.pag);
}
STATIC struct xfs_btree_cur *
xfs_cntbt_dup_cursor(
struct xfs_btree_cur *cur)
{
return xfs_cntbt_init_cursor(cur->bc_mp, cur->bc_tp, cur->bc_ag.agbp,
cur->bc_ag.pag);
}
STATIC void
xfs_allocbt_set_root(
struct xfs_btree_cur *cur,
@ -38,13 +48,18 @@ xfs_allocbt_set_root(
{
struct xfs_buf *agbp = cur->bc_ag.agbp;
struct xfs_agf *agf = agbp->b_addr;
int btnum = cur->bc_btnum;
ASSERT(ptr->s != 0);
agf->agf_roots[btnum] = ptr->s;
be32_add_cpu(&agf->agf_levels[btnum], inc);
cur->bc_ag.pag->pagf_levels[btnum] += inc;
if (xfs_btree_is_bno(cur->bc_ops)) {
agf->agf_bno_root = ptr->s;
be32_add_cpu(&agf->agf_bno_level, inc);
cur->bc_ag.pag->pagf_bno_level += inc;
} else {
agf->agf_cnt_root = ptr->s;
be32_add_cpu(&agf->agf_cnt_level, inc);
cur->bc_ag.pag->pagf_cnt_level += inc;
}
xfs_alloc_log_agf(cur->bc_tp, agbp, XFS_AGF_ROOTS | XFS_AGF_LEVELS);
}
@ -116,7 +131,7 @@ xfs_allocbt_update_lastrec(
__be32 len;
int numrecs;
ASSERT(cur->bc_btnum == XFS_BTNUM_CNT);
ASSERT(!xfs_btree_is_bno(cur->bc_ops));
switch (reason) {
case LASTREC_UPDATE:
@ -226,7 +241,10 @@ xfs_allocbt_init_ptr_from_cur(
ASSERT(cur->bc_ag.pag->pag_agno == be32_to_cpu(agf->agf_seqno));
ptr->s = agf->agf_roots[cur->bc_btnum];
if (xfs_btree_is_bno(cur->bc_ops))
ptr->s = agf->agf_bno_root;
else
ptr->s = agf->agf_cnt_root;
}
STATIC int64_t
@ -299,13 +317,12 @@ xfs_allocbt_verify(
struct xfs_perag *pag = bp->b_pag;
xfs_failaddr_t fa;
unsigned int level;
xfs_btnum_t btnum = XFS_BTNUM_BNOi;
if (!xfs_verify_magic(bp, block->bb_magic))
return __this_address;
if (xfs_has_crc(mp)) {
fa = xfs_btree_sblock_v5hdr_verify(bp);
fa = xfs_btree_agblock_v5hdr_verify(bp);
if (fa)
return fa;
}
@ -320,26 +337,32 @@ xfs_allocbt_verify(
* against.
*/
level = be16_to_cpu(block->bb_level);
if (bp->b_ops->magic[0] == cpu_to_be32(XFS_ABTC_MAGIC))
btnum = XFS_BTNUM_CNTi;
if (pag && xfs_perag_initialised_agf(pag)) {
unsigned int maxlevel = pag->pagf_levels[btnum];
unsigned int maxlevel, repair_maxlevel = 0;
#ifdef CONFIG_XFS_ONLINE_REPAIR
/*
* Online repair could be rewriting the free space btrees, so
* we'll validate against the larger of either tree while this
* is going on.
*/
maxlevel = max_t(unsigned int, maxlevel,
pag->pagf_repair_levels[btnum]);
if (bp->b_ops->magic[0] == cpu_to_be32(XFS_ABTC_MAGIC)) {
maxlevel = pag->pagf_cnt_level;
#ifdef CONFIG_XFS_ONLINE_REPAIR
repair_maxlevel = pag->pagf_repair_cnt_level;
#endif
if (level >= maxlevel)
} else {
maxlevel = pag->pagf_bno_level;
#ifdef CONFIG_XFS_ONLINE_REPAIR
repair_maxlevel = pag->pagf_repair_bno_level;
#endif
}
if (level >= max(maxlevel, repair_maxlevel))
return __this_address;
} else if (level >= mp->m_alloc_maxlevels)
return __this_address;
return xfs_btree_sblock_verify(bp, mp->m_alloc_mxr[level != 0]);
return xfs_btree_agblock_verify(bp, mp->m_alloc_mxr[level != 0]);
}
static void
@ -348,7 +371,7 @@ xfs_allocbt_read_verify(
{
xfs_failaddr_t fa;
if (!xfs_btree_sblock_verify_crc(bp))
if (!xfs_btree_agblock_verify_crc(bp))
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_allocbt_verify(bp);
@ -372,7 +395,7 @@ xfs_allocbt_write_verify(
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
return;
}
xfs_btree_sblock_calc_crc(bp);
xfs_btree_agblock_calc_crc(bp);
}
@ -454,11 +477,19 @@ xfs_allocbt_keys_contiguous(
be32_to_cpu(key2->alloc.ar_startblock));
}
static const struct xfs_btree_ops xfs_bnobt_ops = {
const struct xfs_btree_ops xfs_bnobt_ops = {
.name = "bno",
.type = XFS_BTREE_TYPE_AG,
.rec_len = sizeof(xfs_alloc_rec_t),
.key_len = sizeof(xfs_alloc_key_t),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.dup_cursor = xfs_allocbt_dup_cursor,
.lru_refs = XFS_ALLOC_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_abtb_2),
.sick_mask = XFS_SICK_AG_BNOBT,
.dup_cursor = xfs_bnobt_dup_cursor,
.set_root = xfs_allocbt_set_root,
.alloc_block = xfs_allocbt_alloc_block,
.free_block = xfs_allocbt_free_block,
@ -477,11 +508,20 @@ static const struct xfs_btree_ops xfs_bnobt_ops = {
.keys_contiguous = xfs_allocbt_keys_contiguous,
};
static const struct xfs_btree_ops xfs_cntbt_ops = {
const struct xfs_btree_ops xfs_cntbt_ops = {
.name = "cnt",
.type = XFS_BTREE_TYPE_AG,
.geom_flags = XFS_BTGEO_LASTREC_UPDATE,
.rec_len = sizeof(xfs_alloc_rec_t),
.key_len = sizeof(xfs_alloc_key_t),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.dup_cursor = xfs_allocbt_dup_cursor,
.lru_refs = XFS_ALLOC_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_abtc_2),
.sick_mask = XFS_SICK_AG_CNTBT,
.dup_cursor = xfs_cntbt_dup_cursor,
.set_root = xfs_allocbt_set_root,
.alloc_block = xfs_allocbt_alloc_block,
.free_block = xfs_allocbt_free_block,
@ -500,76 +540,55 @@ static const struct xfs_btree_ops xfs_cntbt_ops = {
.keys_contiguous = NULL, /* not needed right now */
};
/* Allocate most of a new allocation btree cursor. */
STATIC struct xfs_btree_cur *
xfs_allocbt_init_common(
/*
* Allocate a new bnobt cursor.
*
* For staging cursors tp and agbp are NULL.
*/
struct xfs_btree_cur *
xfs_bnobt_init_cursor(
struct xfs_mount *mp,
struct xfs_trans *tp,
struct xfs_perag *pag,
xfs_btnum_t btnum)
struct xfs_buf *agbp,
struct xfs_perag *pag)
{
struct xfs_btree_cur *cur;
ASSERT(btnum == XFS_BTNUM_BNO || btnum == XFS_BTNUM_CNT);
cur = xfs_btree_alloc_cursor(mp, tp, btnum, mp->m_alloc_maxlevels,
xfs_allocbt_cur_cache);
cur->bc_ag.abt.active = false;
if (btnum == XFS_BTNUM_CNT) {
cur->bc_ops = &xfs_cntbt_ops;
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_abtc_2);
cur->bc_flags = XFS_BTREE_LASTREC_UPDATE;
} else {
cur->bc_ops = &xfs_bnobt_ops;
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_abtb_2);
}
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_bnobt_ops,
mp->m_alloc_maxlevels, xfs_allocbt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.agbp = agbp;
if (agbp) {
struct xfs_agf *agf = agbp->b_addr;
if (xfs_has_crc(mp))
cur->bc_flags |= XFS_BTREE_CRC_BLOCKS;
cur->bc_nlevels = be32_to_cpu(agf->agf_bno_level);
}
return cur;
}
/*
* Allocate a new allocation btree cursor.
* Allocate a new cntbt cursor.
*
* For staging cursors tp and agbp are NULL.
*/
struct xfs_btree_cur * /* new alloc btree cursor */
xfs_allocbt_init_cursor(
struct xfs_mount *mp, /* file system mount point */
struct xfs_trans *tp, /* transaction pointer */
struct xfs_buf *agbp, /* buffer for agf structure */
struct xfs_perag *pag,
xfs_btnum_t btnum) /* btree identifier */
{
struct xfs_agf *agf = agbp->b_addr;
struct xfs_btree_cur *cur;
cur = xfs_allocbt_init_common(mp, tp, pag, btnum);
if (btnum == XFS_BTNUM_CNT)
cur->bc_nlevels = be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]);
else
cur->bc_nlevels = be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]);
cur->bc_ag.agbp = agbp;
return cur;
}
/* Create a free space btree cursor with a fake root for staging. */
struct xfs_btree_cur *
xfs_allocbt_stage_cursor(
xfs_cntbt_init_cursor(
struct xfs_mount *mp,
struct xbtree_afakeroot *afake,
struct xfs_perag *pag,
xfs_btnum_t btnum)
struct xfs_trans *tp,
struct xfs_buf *agbp,
struct xfs_perag *pag)
{
struct xfs_btree_cur *cur;
cur = xfs_allocbt_init_common(mp, NULL, pag, btnum);
xfs_btree_stage_afakeroot(cur, afake);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_cntbt_ops,
mp->m_alloc_maxlevels, xfs_allocbt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.agbp = agbp;
if (agbp) {
struct xfs_agf *agf = agbp->b_addr;
cur->bc_nlevels = be32_to_cpu(agf->agf_cnt_level);
}
return cur;
}
@ -588,16 +607,16 @@ xfs_allocbt_commit_staged_btree(
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
agf->agf_roots[cur->bc_btnum] = cpu_to_be32(afake->af_root);
agf->agf_levels[cur->bc_btnum] = cpu_to_be32(afake->af_levels);
if (xfs_btree_is_bno(cur->bc_ops)) {
agf->agf_bno_root = cpu_to_be32(afake->af_root);
agf->agf_bno_level = cpu_to_be32(afake->af_levels);
} else {
agf->agf_cnt_root = cpu_to_be32(afake->af_root);
agf->agf_cnt_level = cpu_to_be32(afake->af_levels);
}
xfs_alloc_log_agf(tp, agbp, XFS_AGF_ROOTS | XFS_AGF_LEVELS);
if (cur->bc_btnum == XFS_BTNUM_BNO) {
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_bnobt_ops);
} else {
cur->bc_flags |= XFS_BTREE_LASTREC_UPDATE;
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_cntbt_ops);
}
xfs_btree_commit_afakeroot(cur, tp, agbp);
}
/* Calculate number of records in an alloc btree block. */

10
fs/xfs/libxfs/xfs_alloc_btree.h
View File

@ -47,12 +47,12 @@ struct xbtree_afakeroot;
(maxrecs) * sizeof(xfs_alloc_key_t) + \
((index) - 1) * sizeof(xfs_alloc_ptr_t)))
extern struct xfs_btree_cur *xfs_allocbt_init_cursor(struct xfs_mount *mp,
struct xfs_btree_cur *xfs_bnobt_init_cursor(struct xfs_mount *mp,
struct xfs_trans *tp, struct xfs_buf *bp,
struct xfs_perag *pag, xfs_btnum_t btnum);
struct xfs_btree_cur *xfs_allocbt_stage_cursor(struct xfs_mount *mp,
struct xbtree_afakeroot *afake, struct xfs_perag *pag,
xfs_btnum_t btnum);
struct xfs_perag *pag);
struct xfs_btree_cur *xfs_cntbt_init_cursor(struct xfs_mount *mp,
struct xfs_trans *tp, struct xfs_buf *bp,
struct xfs_perag *pag);
extern int xfs_allocbt_maxrecs(struct xfs_mount *, int, int);
extern xfs_extlen_t xfs_allocbt_calc_size(struct xfs_mount *mp,
unsigned long long len);

5
fs/xfs/libxfs/xfs_attr.c
View File

@ -224,7 +224,7 @@ int
xfs_attr_get_ilocked(
struct xfs_da_args *args)
{
ASSERT(xfs_isilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL));
xfs_assert_ilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL);
if (!xfs_inode_hasattr(args->dp))
return -ENOATTR;
@ -891,7 +891,8 @@ xfs_attr_defer_add(
struct xfs_attr_intent *new;
new = kmem_cache_zalloc(xfs_attr_intent_cache, GFP_NOFS | __GFP_NOFAIL);
new = kmem_cache_zalloc(xfs_attr_intent_cache,
GFP_KERNEL | __GFP_NOFAIL);
new->xattri_op_flags = op_flags;
new->xattri_da_args = args;

22
fs/xfs/libxfs/xfs_attr_leaf.c
View File

@ -29,6 +29,7 @@
#include "xfs_log.h"
#include "xfs_ag.h"
#include "xfs_errortag.h"
#include "xfs_health.h"
/*
@ -879,8 +880,7 @@ xfs_attr_shortform_to_leaf(
trace_xfs_attr_sf_to_leaf(args);
tmpbuffer = kmem_alloc(size, 0);
ASSERT(tmpbuffer != NULL);
tmpbuffer = kmalloc(size, GFP_KERNEL | __GFP_NOFAIL);
memcpy(tmpbuffer, ifp->if_data, size);
sf = (struct xfs_attr_sf_hdr *)tmpbuffer;
@ -924,7 +924,7 @@ xfs_attr_shortform_to_leaf(
}
error = 0;
out:
kmem_free(tmpbuffer);
kfree(tmpbuffer);
return error;
}
@ -1059,7 +1059,7 @@ xfs_attr3_leaf_to_shortform(
trace_xfs_attr_leaf_to_sf(args);
tmpbuffer = kmem_alloc(args->geo->blksize, 0);
tmpbuffer = kmalloc(args->geo->blksize, GFP_KERNEL | __GFP_NOFAIL);
if (!tmpbuffer)
return -ENOMEM;
@ -1125,7 +1125,7 @@ xfs_attr3_leaf_to_shortform(
error = 0;
out:
kmem_free(tmpbuffer);
kfree(tmpbuffer);
return error;
}
@ -1533,7 +1533,7 @@ xfs_attr3_leaf_compact(
trace_xfs_attr_leaf_compact(args);
tmpbuffer = kmem_alloc(args->geo->blksize, 0);
tmpbuffer = kmalloc(args->geo->blksize, GFP_KERNEL | __GFP_NOFAIL);
memcpy(tmpbuffer, bp->b_addr, args->geo->blksize);
memset(bp->b_addr, 0, args->geo->blksize);
leaf_src = (xfs_attr_leafblock_t *)tmpbuffer;
@ -1571,7 +1571,7 @@ xfs_attr3_leaf_compact(
*/
xfs_trans_log_buf(trans, bp, 0, args->geo->blksize - 1);
kmem_free(tmpbuffer);
kfree(tmpbuffer);
}
/*
@ -2250,7 +2250,8 @@ xfs_attr3_leaf_unbalance(
struct xfs_attr_leafblock *tmp_leaf;
struct xfs_attr3_icleaf_hdr tmphdr;
tmp_leaf = kmem_zalloc(state->args->geo->blksize, 0);
tmp_leaf = kzalloc(state->args->geo->blksize,
GFP_KERNEL | __GFP_NOFAIL);
/*
* Copy the header into the temp leaf so that all the stuff
@ -2290,7 +2291,7 @@ xfs_attr3_leaf_unbalance(
}
memcpy(save_leaf, tmp_leaf, state->args->geo->blksize);
savehdr = tmphdr; /* struct copy */
kmem_free(tmp_leaf);
kfree(tmp_leaf);
}
xfs_attr3_leaf_hdr_to_disk(state->args->geo, save_leaf, &savehdr);
@ -2343,6 +2344,7 @@ xfs_attr3_leaf_lookup_int(
entries = xfs_attr3_leaf_entryp(leaf);
if (ichdr.count >= args->geo->blksize / 8) {
xfs_buf_mark_corrupt(bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -2362,10 +2364,12 @@ xfs_attr3_leaf_lookup_int(
}
if (!(probe >= 0 && (!ichdr.count || probe < ichdr.count))) {
xfs_buf_mark_corrupt(bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
if (!(span <= 4 || be32_to_cpu(entry->hashval) == hashval)) {
xfs_buf_mark_corrupt(bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}

37
fs/xfs/libxfs/xfs_attr_remote.c
View File

@ -22,6 +22,7 @@
#include "xfs_attr_remote.h"
#include "xfs_trace.h"
#include "xfs_error.h"
#include "xfs_health.h"
#define ATTR_RMTVALUE_MAPSIZE 1 /* # of map entries at once */
@ -276,17 +277,18 @@ xfs_attr3_rmt_hdr_set(
*/
STATIC int
xfs_attr_rmtval_copyout(
struct xfs_mount *mp,
struct xfs_buf *bp,
xfs_ino_t ino,
int *offset,
int *valuelen,
uint8_t **dst)
struct xfs_mount *mp,
struct xfs_buf *bp,
struct xfs_inode *dp,
int *offset,
int *valuelen,
uint8_t **dst)
{
char *src = bp->b_addr;
xfs_daddr_t bno = xfs_buf_daddr(bp);
int len = BBTOB(bp->b_length);
int blksize = mp->m_attr_geo->blksize;
char *src = bp->b_addr;
xfs_ino_t ino = dp->i_ino;
xfs_daddr_t bno = xfs_buf_daddr(bp);
int len = BBTOB(bp->b_length);
int blksize = mp->m_attr_geo->blksize;
ASSERT(len >= blksize);
@ -302,6 +304,7 @@ xfs_attr_rmtval_copyout(
xfs_alert(mp,
"remote attribute header mismatch bno/off/len/owner (0x%llx/0x%x/Ox%x/0x%llx)",
bno, *offset, byte_cnt, ino);
xfs_dirattr_mark_sick(dp, XFS_ATTR_FORK);
return -EFSCORRUPTED;
}
hdr_size = sizeof(struct xfs_attr3_rmt_hdr);
@ -418,10 +421,12 @@ xfs_attr_rmtval_get(
dblkcnt = XFS_FSB_TO_BB(mp, map[i].br_blockcount);
error = xfs_buf_read(mp->m_ddev_targp, dblkno, dblkcnt,
0, &bp, &xfs_attr3_rmt_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_dirattr_mark_sick(args->dp, XFS_ATTR_FORK);
if (error)
return error;
error = xfs_attr_rmtval_copyout(mp, bp, args->dp->i_ino,
error = xfs_attr_rmtval_copyout(mp, bp, args->dp,
&offset, &valuelen,
&dst);
xfs_buf_relse(bp);
@ -545,11 +550,13 @@ xfs_attr_rmtval_stale(
struct xfs_buf *bp;
int error;
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL);
if (XFS_IS_CORRUPT(mp, map->br_startblock == DELAYSTARTBLOCK) ||
XFS_IS_CORRUPT(mp, map->br_startblock == HOLESTARTBLOCK))
XFS_IS_CORRUPT(mp, map->br_startblock == HOLESTARTBLOCK)) {
xfs_bmap_mark_sick(ip, XFS_ATTR_FORK);
return -EFSCORRUPTED;
}
error = xfs_buf_incore(mp->m_ddev_targp,
XFS_FSB_TO_DADDR(mp, map->br_startblock),
@ -659,8 +666,10 @@ xfs_attr_rmtval_invalidate(
blkcnt, &map, &nmap, XFS_BMAPI_ATTRFORK);
if (error)
return error;
if (XFS_IS_CORRUPT(args->dp->i_mount, nmap != 1))
if (XFS_IS_CORRUPT(args->dp->i_mount, nmap != 1)) {
xfs_bmap_mark_sick(args->dp, XFS_ATTR_FORK);
return -EFSCORRUPTED;
}
error = xfs_attr_rmtval_stale(args->dp, &map, XBF_TRYLOCK);
if (error)
return error;

365
fs/xfs/libxfs/xfs_bmap.c
View File

File diff suppressed because it is too large Load Diff

19
fs/xfs/libxfs/xfs_bmap.h
View File

@ -232,6 +232,10 @@ enum xfs_bmap_intent_type {
XFS_BMAP_UNMAP,
};
#define XFS_BMAP_INTENT_STRINGS \
{ XFS_BMAP_MAP, "map" }, \
{ XFS_BMAP_UNMAP, "unmap" }
struct xfs_bmap_intent {
struct list_head bi_list;
enum xfs_bmap_intent_type bi_type;
@ -241,14 +245,11 @@ struct xfs_bmap_intent {
struct xfs_bmbt_irec bi_bmap;
};
void xfs_bmap_update_get_group(struct xfs_mount *mp,
struct xfs_bmap_intent *bi);
int xfs_bmap_finish_one(struct xfs_trans *tp, struct xfs_bmap_intent *bi);
void xfs_bmap_map_extent(struct xfs_trans *tp, struct xfs_inode *ip,
struct xfs_bmbt_irec *imap);
int whichfork, struct xfs_bmbt_irec *imap);
void xfs_bmap_unmap_extent(struct xfs_trans *tp, struct xfs_inode *ip,
struct xfs_bmbt_irec *imap);
int whichfork, struct xfs_bmbt_irec *imap);
static inline uint32_t xfs_bmap_fork_to_state(int whichfork)
{
@ -280,4 +281,12 @@ extern struct kmem_cache *xfs_bmap_intent_cache;
int __init xfs_bmap_intent_init_cache(void);
void xfs_bmap_intent_destroy_cache(void);
typedef int (*xfs_bmap_query_range_fn)(
struct xfs_btree_cur *cur,
struct xfs_bmbt_irec *rec,
void *priv);
int xfs_bmap_query_all(struct xfs_btree_cur *cur, xfs_bmap_query_range_fn fn,
void *priv);
#endif /* __XFS_BMAP_H__ */

152
fs/xfs/libxfs/xfs_bmap_btree.c
View File

@ -26,6 +26,22 @@
static struct kmem_cache *xfs_bmbt_cur_cache;
void
xfs_bmbt_init_block(
struct xfs_inode *ip,
struct xfs_btree_block *buf,
struct xfs_buf *bp,
__u16 level,
__u16 numrecs)
{
if (bp)
xfs_btree_init_buf(ip->i_mount, bp, &xfs_bmbt_ops, level,
numrecs, ip->i_ino);
else
xfs_btree_init_block(ip->i_mount, buf, &xfs_bmbt_ops, level,
numrecs, ip->i_ino);
}
/*
* Convert on-disk form of btree root to in-memory form.
*/
@ -44,9 +60,7 @@ xfs_bmdr_to_bmbt(
xfs_bmbt_key_t *tkp;
__be64 *tpp;
xfs_btree_init_block_int(mp, rblock, XFS_BUF_DADDR_NULL,
XFS_BTNUM_BMAP, 0, 0, ip->i_ino,
XFS_BTREE_LONG_PTRS);
xfs_bmbt_init_block(ip, rblock, NULL, 0, 0);
rblock->bb_level = dblock->bb_level;
ASSERT(be16_to_cpu(rblock->bb_level) > 0);
rblock->bb_numrecs = dblock->bb_numrecs;
@ -171,13 +185,8 @@ xfs_bmbt_dup_cursor(
new = xfs_bmbt_init_cursor(cur->bc_mp, cur->bc_tp,
cur->bc_ino.ip, cur->bc_ino.whichfork);
/*
* Copy the firstblock, dfops, and flags values,
* since init cursor doesn't get them.
*/
new->bc_ino.flags = cur->bc_ino.flags;
new->bc_flags |= (cur->bc_flags &
(XFS_BTREE_BMBT_INVALID_OWNER | XFS_BTREE_BMBT_WASDEL));
return new;
}
@ -189,10 +198,10 @@ xfs_bmbt_update_cursor(
ASSERT((dst->bc_tp->t_highest_agno != NULLAGNUMBER) ||
(dst->bc_ino.ip->i_diflags & XFS_DIFLAG_REALTIME));
dst->bc_ino.allocated += src->bc_ino.allocated;
dst->bc_bmap.allocated += src->bc_bmap.allocated;
dst->bc_tp->t_highest_agno = src->bc_tp->t_highest_agno;
src->bc_ino.allocated = 0;
src->bc_bmap.allocated = 0;
}
STATIC int
@ -211,7 +220,7 @@ xfs_bmbt_alloc_block(
xfs_rmap_ino_bmbt_owner(&args.oinfo, cur->bc_ino.ip->i_ino,
cur->bc_ino.whichfork);
args.minlen = args.maxlen = args.prod = 1;
args.wasdel = cur->bc_ino.flags & XFS_BTCUR_BMBT_WASDEL;
args.wasdel = cur->bc_flags & XFS_BTREE_BMBT_WASDEL;
if (!args.wasdel && args.tp->t_blk_res == 0)
return -ENOSPC;
@ -247,7 +256,7 @@ xfs_bmbt_alloc_block(
}
ASSERT(args.len == 1);
cur->bc_ino.allocated++;
cur->bc_bmap.allocated++;
cur->bc_ino.ip->i_nblocks++;
xfs_trans_log_inode(args.tp, cur->bc_ino.ip, XFS_ILOG_CORE);
xfs_trans_mod_dquot_byino(args.tp, cur->bc_ino.ip,
@ -360,14 +369,6 @@ xfs_bmbt_init_rec_from_cur(
xfs_bmbt_disk_set_all(&rec->bmbt, &cur->bc_rec.b);
}
STATIC void
xfs_bmbt_init_ptr_from_cur(
struct xfs_btree_cur *cur,
union xfs_btree_ptr *ptr)
{
ptr->l = 0;
}
STATIC int64_t
xfs_bmbt_key_diff(
struct xfs_btree_cur *cur,
@ -419,7 +420,7 @@ xfs_bmbt_verify(
* XXX: need a better way of verifying the owner here. Right now
* just make sure there has been one set.
*/
fa = xfs_btree_lblock_v5hdr_verify(bp, XFS_RMAP_OWN_UNKNOWN);
fa = xfs_btree_fsblock_v5hdr_verify(bp, XFS_RMAP_OWN_UNKNOWN);
if (fa)
return fa;
}
@ -435,7 +436,7 @@ xfs_bmbt_verify(
if (level > max(mp->m_bm_maxlevels[0], mp->m_bm_maxlevels[1]))
return __this_address;
return xfs_btree_lblock_verify(bp, mp->m_bmap_dmxr[level != 0]);
return xfs_btree_fsblock_verify(bp, mp->m_bmap_dmxr[level != 0]);
}
static void
@ -444,7 +445,7 @@ xfs_bmbt_read_verify(
{
xfs_failaddr_t fa;
if (!xfs_btree_lblock_verify_crc(bp))
if (!xfs_btree_fsblock_verify_crc(bp))
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_bmbt_verify(bp);
@ -468,7 +469,7 @@ xfs_bmbt_write_verify(
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
return;
}
xfs_btree_lblock_calc_crc(bp);
xfs_btree_fsblock_calc_crc(bp);
}
const struct xfs_buf_ops xfs_bmbt_buf_ops = {
@ -515,9 +516,16 @@ xfs_bmbt_keys_contiguous(
be64_to_cpu(key2->bmbt.br_startoff));
}
static const struct xfs_btree_ops xfs_bmbt_ops = {
const struct xfs_btree_ops xfs_bmbt_ops = {
.name = "bmap",
.type = XFS_BTREE_TYPE_INODE,
.rec_len = sizeof(xfs_bmbt_rec_t),
.key_len = sizeof(xfs_bmbt_key_t),
.ptr_len = XFS_BTREE_LONG_PTR_LEN,
.lru_refs = XFS_BMAP_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_bmbt_2),
.dup_cursor = xfs_bmbt_dup_cursor,
.update_cursor = xfs_bmbt_update_cursor,
@ -529,7 +537,6 @@ static const struct xfs_btree_ops xfs_bmbt_ops = {
.init_key_from_rec = xfs_bmbt_init_key_from_rec,
.init_high_key_from_rec = xfs_bmbt_init_high_key_from_rec,
.init_rec_from_cur = xfs_bmbt_init_rec_from_cur,
.init_ptr_from_cur = xfs_bmbt_init_ptr_from_cur,
.key_diff = xfs_bmbt_key_diff,
.diff_two_keys = xfs_bmbt_diff_two_keys,
.buf_ops = &xfs_bmbt_buf_ops,
@ -538,35 +545,10 @@ static const struct xfs_btree_ops xfs_bmbt_ops = {
.keys_contiguous = xfs_bmbt_keys_contiguous,
};
static struct xfs_btree_cur *
xfs_bmbt_init_common(
struct xfs_mount *mp,
struct xfs_trans *tp,
struct xfs_inode *ip,
int whichfork)
{
struct xfs_btree_cur *cur;
ASSERT(whichfork != XFS_COW_FORK);
cur = xfs_btree_alloc_cursor(mp, tp, XFS_BTNUM_BMAP,
mp->m_bm_maxlevels[whichfork], xfs_bmbt_cur_cache);
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_bmbt_2);
cur->bc_ops = &xfs_bmbt_ops;
cur->bc_flags = XFS_BTREE_LONG_PTRS | XFS_BTREE_ROOT_IN_INODE;
if (xfs_has_crc(mp))
cur->bc_flags |= XFS_BTREE_CRC_BLOCKS;
cur->bc_ino.ip = ip;
cur->bc_ino.allocated = 0;
cur->bc_ino.flags = 0;
return cur;
}
/*
* Allocate a new bmap btree cursor.
* Create a new bmap btree cursor.
*
* For staging cursors -1 in passed in whichfork.
*/
struct xfs_btree_cur *
xfs_bmbt_init_cursor(
@ -575,15 +557,34 @@ xfs_bmbt_init_cursor(
struct xfs_inode *ip,
int whichfork)
{
struct xfs_ifork *ifp = xfs_ifork_ptr(ip, whichfork);
struct xfs_btree_cur *cur;
unsigned int maxlevels;
cur = xfs_bmbt_init_common(mp, tp, ip, whichfork);
ASSERT(whichfork != XFS_COW_FORK);
cur->bc_nlevels = be16_to_cpu(ifp->if_broot->bb_level) + 1;
cur->bc_ino.forksize = xfs_inode_fork_size(ip, whichfork);
/*
* The Data fork always has larger maxlevel, so use that for staging
* cursors.
*/
switch (whichfork) {
case XFS_STAGING_FORK:
maxlevels = mp->m_bm_maxlevels[XFS_DATA_FORK];
break;
default:
maxlevels = mp->m_bm_maxlevels[whichfork];
break;
}
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_bmbt_ops, maxlevels,
xfs_bmbt_cur_cache);
cur->bc_ino.ip = ip;
cur->bc_ino.whichfork = whichfork;
cur->bc_bmap.allocated = 0;
if (whichfork != XFS_STAGING_FORK) {
struct xfs_ifork *ifp = xfs_ifork_ptr(ip, whichfork);
cur->bc_nlevels = be16_to_cpu(ifp->if_broot->bb_level) + 1;
cur->bc_ino.forksize = xfs_inode_fork_size(ip, whichfork);
}
return cur;
}
@ -598,33 +599,6 @@ xfs_bmbt_block_maxrecs(
return blocklen / (sizeof(xfs_bmbt_key_t) + sizeof(xfs_bmbt_ptr_t));
}
/*
* Allocate a new bmap btree cursor for reloading an inode block mapping data
* structure. Note that callers can use the staged cursor to reload extents
* format inode forks if they rebuild the iext tree and commit the staged
* cursor immediately.
*/
struct xfs_btree_cur *
xfs_bmbt_stage_cursor(
struct xfs_mount *mp,
struct xfs_inode *ip,
struct xbtree_ifakeroot *ifake)
{
struct xfs_btree_cur *cur;
struct xfs_btree_ops *ops;
/* data fork always has larger maxheight */
cur = xfs_bmbt_init_common(mp, NULL, ip, XFS_DATA_FORK);
cur->bc_nlevels = ifake->if_levels;
cur->bc_ino.forksize = ifake->if_fork_size;
/* Don't let anyone think we're attached to the real fork yet. */
cur->bc_ino.whichfork = -1;
xfs_btree_stage_ifakeroot(cur, ifake, &ops);
ops->update_cursor = NULL;
return cur;
}
/*
* Swap in the new inode fork root. Once we pass this point the newly rebuilt
* mappings are in place and we have to kill off any old btree blocks.
@ -665,7 +639,7 @@ xfs_bmbt_commit_staged_btree(
break;
}
xfs_trans_log_inode(tp, cur->bc_ino.ip, flags);
xfs_btree_commit_ifakeroot(cur, tp, whichfork, &xfs_bmbt_ops);
xfs_btree_commit_ifakeroot(cur, tp, whichfork);
}
/*
@ -751,7 +725,7 @@ xfs_bmbt_change_owner(
ASSERT(xfs_ifork_ptr(ip, whichfork)->if_format == XFS_DINODE_FMT_BTREE);
cur = xfs_bmbt_init_cursor(ip->i_mount, tp, ip, whichfork);
cur->bc_ino.flags |= XFS_BTCUR_BMBT_INVALID_OWNER;
cur->bc_flags |= XFS_BTREE_BMBT_INVALID_OWNER;
error = xfs_btree_change_owner(cur, new_owner, buffer_list);
xfs_btree_del_cursor(cur, error);

5
fs/xfs/libxfs/xfs_bmap_btree.h
View File

@ -107,8 +107,6 @@ extern int xfs_bmbt_change_owner(struct xfs_trans *tp, struct xfs_inode *ip,
extern struct xfs_btree_cur *xfs_bmbt_init_cursor(struct xfs_mount *,
struct xfs_trans *, struct xfs_inode *, int);
struct xfs_btree_cur *xfs_bmbt_stage_cursor(struct xfs_mount *mp,
struct xfs_inode *ip, struct xbtree_ifakeroot *ifake);
void xfs_bmbt_commit_staged_btree(struct xfs_btree_cur *cur,
struct xfs_trans *tp, int whichfork);
@ -120,4 +118,7 @@ unsigned int xfs_bmbt_maxlevels_ondisk(void);
int __init xfs_bmbt_init_cur_cache(void);
void xfs_bmbt_destroy_cur_cache(void);
void xfs_bmbt_init_block(struct xfs_inode *ip, struct xfs_btree_block *buf,
struct xfs_buf *bp, __u16 level, __u16 numrecs);
#endif /* __XFS_BMAP_BTREE_H__ */

1098
fs/xfs/libxfs/xfs_btree.c
View File

File diff suppressed because it is too large Load Diff

274
fs/xfs/libxfs/xfs_btree.h
View File

@ -55,15 +55,8 @@ union xfs_btree_rec {
#define XFS_LOOKUP_LE ((xfs_lookup_t)XFS_LOOKUP_LEi)
#define XFS_LOOKUP_GE ((xfs_lookup_t)XFS_LOOKUP_GEi)
#define XFS_BTNUM_BNO ((xfs_btnum_t)XFS_BTNUM_BNOi)
#define XFS_BTNUM_CNT ((xfs_btnum_t)XFS_BTNUM_CNTi)
#define XFS_BTNUM_BMAP ((xfs_btnum_t)XFS_BTNUM_BMAPi)
#define XFS_BTNUM_INO ((xfs_btnum_t)XFS_BTNUM_INOi)
#define XFS_BTNUM_FINO ((xfs_btnum_t)XFS_BTNUM_FINOi)
#define XFS_BTNUM_RMAP ((xfs_btnum_t)XFS_BTNUM_RMAPi)
#define XFS_BTNUM_REFC ((xfs_btnum_t)XFS_BTNUM_REFCi)
uint32_t xfs_btree_magic(int crc, xfs_btnum_t btnum);
struct xfs_btree_ops;
uint32_t xfs_btree_magic(struct xfs_mount *mp, const struct xfs_btree_ops *ops);
/*
* For logging record fields.
@ -86,9 +79,11 @@ uint32_t xfs_btree_magic(int crc, xfs_btnum_t btnum);
* Generic stats interface
*/
#define XFS_BTREE_STATS_INC(cur, stat) \
XFS_STATS_INC_OFF((cur)->bc_mp, (cur)->bc_statoff + __XBTS_ ## stat)
XFS_STATS_INC_OFF((cur)->bc_mp, \
(cur)->bc_ops->statoff + __XBTS_ ## stat)
#define XFS_BTREE_STATS_ADD(cur, stat, val) \
XFS_STATS_ADD_OFF((cur)->bc_mp, (cur)->bc_statoff + __XBTS_ ## stat, val)
XFS_STATS_ADD_OFF((cur)->bc_mp, \
(cur)->bc_ops->statoff + __XBTS_ ## stat, val)
enum xbtree_key_contig {
XBTREE_KEY_GAP = 0,
@ -111,10 +106,37 @@ static inline enum xbtree_key_contig xbtree_key_contig(uint64_t x, uint64_t y)
return XBTREE_KEY_OVERLAP;
}
#define XFS_BTREE_LONG_PTR_LEN (sizeof(__be64))
#define XFS_BTREE_SHORT_PTR_LEN (sizeof(__be32))
enum xfs_btree_type {
XFS_BTREE_TYPE_AG,
XFS_BTREE_TYPE_INODE,
XFS_BTREE_TYPE_MEM,
};
struct xfs_btree_ops {
/* size of the key and record structures */
size_t key_len;
size_t rec_len;
const char *name;
/* Type of btree - AG-rooted or inode-rooted */
enum xfs_btree_type type;
/* XFS_BTGEO_* flags that determine the geometry of the btree */
unsigned int geom_flags;
/* size of the key, pointer, and record structures */
size_t key_len;
size_t ptr_len;
size_t rec_len;
/* LRU refcount to set on each btree buffer created */
unsigned int lru_refs;
/* offset of btree stats array */
unsigned int statoff;
/* sick mask for health reporting (only for XFS_BTREE_TYPE_AG) */
unsigned int sick_mask;
/* cursor operations */
struct xfs_btree_cur *(*dup_cursor)(struct xfs_btree_cur *);
@ -199,6 +221,10 @@ struct xfs_btree_ops {
const union xfs_btree_key *mask);
};
/* btree geometry flags */
#define XFS_BTGEO_LASTREC_UPDATE (1U << 0) /* track last rec externally */
#define XFS_BTGEO_OVERLAPPING (1U << 1) /* overlapping intervals */
/*
* Reasons for the update_lastrec method to be called.
*/
@ -215,39 +241,6 @@ union xfs_btree_irec {
struct xfs_refcount_irec rc;
};
/* Per-AG btree information. */
struct xfs_btree_cur_ag {
struct xfs_perag *pag;
union {
struct xfs_buf *agbp;
struct xbtree_afakeroot *afake; /* for staging cursor */
};
union {
struct {
unsigned int nr_ops; /* # record updates */
unsigned int shape_changes; /* # of extent splits */
} refc;
struct {
bool active; /* allocation cursor state */
} abt;
};
};
/* Btree-in-inode cursor information */
struct xfs_btree_cur_ino {
struct xfs_inode *ip;
struct xbtree_ifakeroot *ifake; /* for staging cursor */
int allocated;
short forksize;
char whichfork;
char flags;
/* We are converting a delalloc reservation */
#define XFS_BTCUR_BMBT_WASDEL (1 << 0)
/* For extent swap, ignore owner check in verifier */
#define XFS_BTCUR_BMBT_INVALID_OWNER (1 << 1)
};
struct xfs_btree_level {
/* buffer pointer */
struct xfs_buf *bp;
@ -272,21 +265,38 @@ struct xfs_btree_cur
const struct xfs_btree_ops *bc_ops;
struct kmem_cache *bc_cache; /* cursor cache */
unsigned int bc_flags; /* btree features - below */
xfs_btnum_t bc_btnum; /* identifies which btree type */
union xfs_btree_irec bc_rec; /* current insert/search record value */
uint8_t bc_nlevels; /* number of levels in the tree */
uint8_t bc_maxlevels; /* maximum levels for this btree type */
int bc_statoff; /* offset of btree stats array */
/*
* Short btree pointers need an agno to be able to turn the pointers
* into physical addresses for IO, so the btree cursor switches between
* bc_ino and bc_ag based on whether XFS_BTREE_LONG_PTRS is set for the
* cursor.
*/
/* per-type information */
union {
struct xfs_btree_cur_ag bc_ag;
struct xfs_btree_cur_ino bc_ino;
struct {
struct xfs_inode *ip;
short forksize;
char whichfork;
struct xbtree_ifakeroot *ifake; /* for staging cursor */
} bc_ino;
struct {
struct xfs_perag *pag;
struct xfs_buf *agbp;
struct xbtree_afakeroot *afake; /* for staging cursor */
} bc_ag;
struct {
struct xfbtree *xfbtree;
struct xfs_perag *pag;
} bc_mem;
};
/* per-format private data */
union {
struct {
int allocated;
} bc_bmap; /* bmapbt */
struct {
unsigned int nr_ops; /* # record updates */
unsigned int shape_changes; /* # of extent splits */
} bc_refc; /* refcountbt */
};
/* Must be at the end of the struct! */
@ -304,18 +314,22 @@ xfs_btree_cur_sizeof(unsigned int nlevels)
return struct_size_t(struct xfs_btree_cur, bc_levels, nlevels);
}
/* cursor flags */
#define XFS_BTREE_LONG_PTRS (1<<0) /* pointers are 64bits long */
#define XFS_BTREE_ROOT_IN_INODE (1<<1) /* root may be variable size */
#define XFS_BTREE_LASTREC_UPDATE (1<<2) /* track last rec externally */
#define XFS_BTREE_CRC_BLOCKS (1<<3) /* uses extended btree blocks */
#define XFS_BTREE_OVERLAPPING (1<<4) /* overlapping intervals */
/* cursor state flags */
/*
* The root of this btree is a fakeroot structure so that we can stage a btree
* rebuild without leaving it accessible via primary metadata. The ops struct
* is dynamically allocated and must be freed when the cursor is deleted.
*/
#define XFS_BTREE_STAGING (1<<5)
#define XFS_BTREE_STAGING (1U << 0)
/* We are converting a delalloc reservation (only for bmbt btrees) */
#define XFS_BTREE_BMBT_WASDEL (1U << 1)
/* For extent swap, ignore owner check in verifier (only for bmbt btrees) */
#define XFS_BTREE_BMBT_INVALID_OWNER (1U << 2)
/* Cursor is active (only for allocbt btrees) */
#define XFS_BTREE_ALLOCBT_ACTIVE (1U << 3)
#define XFS_BTREE_NOERROR 0
#define XFS_BTREE_ERROR 1
@ -325,14 +339,10 @@ xfs_btree_cur_sizeof(unsigned int nlevels)
*/
#define XFS_BUF_TO_BLOCK(bp) ((struct xfs_btree_block *)((bp)->b_addr))
/*
* Internal long and short btree block checks. They return NULL if the
* block is ok or the address of the failed check otherwise.
*/
xfs_failaddr_t __xfs_btree_check_lblock(struct xfs_btree_cur *cur,
struct xfs_btree_block *block, int level, struct xfs_buf *bp);
xfs_failaddr_t __xfs_btree_check_sblock(struct xfs_btree_cur *cur,
xfs_failaddr_t __xfs_btree_check_block(struct xfs_btree_cur *cur,
struct xfs_btree_block *block, int level, struct xfs_buf *bp);
int __xfs_btree_check_ptr(struct xfs_btree_cur *cur,
const union xfs_btree_ptr *ptr, int index, int level);
/*
* Check that block header is ok.
@ -344,24 +354,6 @@ xfs_btree_check_block(
int level, /* level of the btree block */
struct xfs_buf *bp); /* buffer containing block, if any */
/*
* Check that (long) pointer is ok.
*/
bool /* error (0 or EFSCORRUPTED) */
xfs_btree_check_lptr(
struct xfs_btree_cur *cur, /* btree cursor */
xfs_fsblock_t fsbno, /* btree block disk address */
int level); /* btree block level */
/*
* Check that (short) pointer is ok.
*/
bool /* error (0 or EFSCORRUPTED) */
xfs_btree_check_sptr(
struct xfs_btree_cur *cur, /* btree cursor */
xfs_agblock_t agbno, /* btree block disk address */
int level); /* btree block level */
/*
* Delete the btree cursor.
*/
@ -391,64 +383,15 @@ xfs_btree_offsets(
int *first, /* output: first byte offset */
int *last); /* output: last byte offset */
/*
* Get a buffer for the block, return it read in.
* Long-form addressing.
*/
int /* error */
xfs_btree_read_bufl(
struct xfs_mount *mp, /* file system mount point */
struct xfs_trans *tp, /* transaction pointer */
xfs_fsblock_t fsbno, /* file system block number */
struct xfs_buf **bpp, /* buffer for fsbno */
int refval, /* ref count value for buffer */
const struct xfs_buf_ops *ops);
/*
* Read-ahead the block, don't wait for it, don't return a buffer.
* Long-form addressing.
*/
void /* error */
xfs_btree_reada_bufl(
struct xfs_mount *mp, /* file system mount point */
xfs_fsblock_t fsbno, /* file system block number */
xfs_extlen_t count, /* count of filesystem blocks */
const struct xfs_buf_ops *ops);
/*
* Read-ahead the block, don't wait for it, don't return a buffer.
* Short-form addressing.
*/
void /* error */
xfs_btree_reada_bufs(
struct xfs_mount *mp, /* file system mount point */
xfs_agnumber_t agno, /* allocation group number */
xfs_agblock_t agbno, /* allocation group block number */
xfs_extlen_t count, /* count of filesystem blocks */
const struct xfs_buf_ops *ops);
/*
* Initialise a new btree block header
*/
void
xfs_btree_init_block(
struct xfs_mount *mp,
struct xfs_buf *bp,
xfs_btnum_t btnum,
__u16 level,
__u16 numrecs,
__u64 owner);
void
xfs_btree_init_block_int(
struct xfs_mount *mp,
struct xfs_btree_block *buf,
xfs_daddr_t blkno,
xfs_btnum_t btnum,
__u16 level,
__u16 numrecs,
__u64 owner,
unsigned int flags);
void xfs_btree_init_buf(struct xfs_mount *mp, struct xfs_buf *bp,
const struct xfs_btree_ops *ops, __u16 level, __u16 numrecs,
__u64 owner);
void xfs_btree_init_block(struct xfs_mount *mp,
struct xfs_btree_block *buf, const struct xfs_btree_ops *ops,
__u16 level, __u16 numrecs, __u64 owner);
/*
* Common btree core entry points.
@ -467,10 +410,10 @@ int xfs_btree_change_owner(struct xfs_btree_cur *cur, uint64_t new_owner,
/*
* btree block CRC helpers
*/
void xfs_btree_lblock_calc_crc(struct xfs_buf *);
bool xfs_btree_lblock_verify_crc(struct xfs_buf *);
void xfs_btree_sblock_calc_crc(struct xfs_buf *);
bool xfs_btree_sblock_verify_crc(struct xfs_buf *);
void xfs_btree_fsblock_calc_crc(struct xfs_buf *);
bool xfs_btree_fsblock_verify_crc(struct xfs_buf *);
void xfs_btree_agblock_calc_crc(struct xfs_buf *);
bool xfs_btree_agblock_verify_crc(struct xfs_buf *);
/*
* Internal btree helpers also used by xfs_bmap.c.
@ -510,12 +453,14 @@ static inline int xfs_btree_get_level(const struct xfs_btree_block *block)
#define XFS_FILBLKS_MIN(a,b) min_t(xfs_filblks_t, (a), (b))
#define XFS_FILBLKS_MAX(a,b) max_t(xfs_filblks_t, (a), (b))
xfs_failaddr_t xfs_btree_sblock_v5hdr_verify(struct xfs_buf *bp);
xfs_failaddr_t xfs_btree_sblock_verify(struct xfs_buf *bp,
xfs_failaddr_t xfs_btree_agblock_v5hdr_verify(struct xfs_buf *bp);
xfs_failaddr_t xfs_btree_agblock_verify(struct xfs_buf *bp,
unsigned int max_recs);
xfs_failaddr_t xfs_btree_lblock_v5hdr_verify(struct xfs_buf *bp,
xfs_failaddr_t xfs_btree_fsblock_v5hdr_verify(struct xfs_buf *bp,
uint64_t owner);
xfs_failaddr_t xfs_btree_lblock_verify(struct xfs_buf *bp,
xfs_failaddr_t xfs_btree_fsblock_verify(struct xfs_buf *bp,
unsigned int max_recs);
xfs_failaddr_t xfs_btree_memblock_verify(struct xfs_buf *bp,
unsigned int max_recs);
unsigned int xfs_btree_compute_maxlevels(const unsigned int *limits,
@ -690,7 +635,7 @@ xfs_btree_islastblock(
block = xfs_btree_get_block(cur, level, &bp);
if (cur->bc_flags & XFS_BTREE_LONG_PTRS)
if (cur->bc_ops->ptr_len == XFS_BTREE_LONG_PTR_LEN)
return block->bb_u.l.bb_rightsib == cpu_to_be64(NULLFSBLOCK);
return block->bb_u.s.bb_rightsib == cpu_to_be32(NULLAGBLOCK);
}
@ -714,21 +659,28 @@ void xfs_btree_copy_ptrs(struct xfs_btree_cur *cur,
void xfs_btree_copy_keys(struct xfs_btree_cur *cur,
union xfs_btree_key *dst_key,
const union xfs_btree_key *src_key, int numkeys);
void xfs_btree_init_ptr_from_cur(struct xfs_btree_cur *cur,
union xfs_btree_ptr *ptr);
static inline struct xfs_btree_cur *
xfs_btree_alloc_cursor(
struct xfs_mount *mp,
struct xfs_trans *tp,
xfs_btnum_t btnum,
const struct xfs_btree_ops *ops,
uint8_t maxlevels,
struct kmem_cache *cache)
{
struct xfs_btree_cur *cur;
cur = kmem_cache_zalloc(cache, GFP_NOFS | __GFP_NOFAIL);
ASSERT(ops->ptr_len == XFS_BTREE_LONG_PTR_LEN ||
ops->ptr_len == XFS_BTREE_SHORT_PTR_LEN);
/* BMBT allocations can come through from non-transactional context. */
cur = kmem_cache_zalloc(cache,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
cur->bc_ops = ops;
cur->bc_tp = tp;
cur->bc_mp = mp;
cur->bc_btnum = btnum;
cur->bc_maxlevels = maxlevels;
cur->bc_cache = cache;
@ -740,4 +692,14 @@ void xfs_btree_destroy_cur_caches(void);
int xfs_btree_goto_left_edge(struct xfs_btree_cur *cur);
/* Does this level of the cursor point to the inode root (and not a block)? */
static inline bool
xfs_btree_at_iroot(
const struct xfs_btree_cur *cur,
int level)
{
return cur->bc_ops->type == XFS_BTREE_TYPE_INODE &&
level == cur->bc_nlevels - 1;
}
#endif /* __XFS_BTREE_H__ */

347
fs/xfs/libxfs/xfs_btree_mem.c Normal file
View File

@ -0,0 +1,347 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_log_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_trans.h"
#include "xfs_btree.h"
#include "xfs_error.h"
#include "xfs_buf_mem.h"
#include "xfs_btree_mem.h"
#include "xfs_ag.h"
#include "xfs_buf_item.h"
#include "xfs_trace.h"
/* Set the root of an in-memory btree. */
void
xfbtree_set_root(
struct xfs_btree_cur *cur,
const union xfs_btree_ptr *ptr,
int inc)
{
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_MEM);
cur->bc_mem.xfbtree->root = *ptr;
cur->bc_mem.xfbtree->nlevels += inc;
}
/* Initialize a pointer from the in-memory btree header. */
void
xfbtree_init_ptr_from_cur(
struct xfs_btree_cur *cur,
union xfs_btree_ptr *ptr)
{
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_MEM);
*ptr = cur->bc_mem.xfbtree->root;
}
/* Duplicate an in-memory btree cursor. */
struct xfs_btree_cur *
xfbtree_dup_cursor(
struct xfs_btree_cur *cur)
{
struct xfs_btree_cur *ncur;
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_MEM);
ncur = xfs_btree_alloc_cursor(cur->bc_mp, cur->bc_tp, cur->bc_ops,
cur->bc_maxlevels, cur->bc_cache);
ncur->bc_flags = cur->bc_flags;
ncur->bc_nlevels = cur->bc_nlevels;
ncur->bc_mem.xfbtree = cur->bc_mem.xfbtree;
if (cur->bc_mem.pag)
ncur->bc_mem.pag = xfs_perag_hold(cur->bc_mem.pag);
return ncur;
}
/* Close the btree xfile and release all resources. */
void
xfbtree_destroy(
struct xfbtree *xfbt)
{
xfs_buftarg_drain(xfbt->target);
}
/* Compute the number of bytes available for records. */
static inline unsigned int
xfbtree_rec_bytes(
struct xfs_mount *mp,
const struct xfs_btree_ops *ops)
{
return XMBUF_BLOCKSIZE - XFS_BTREE_LBLOCK_CRC_LEN;
}
/* Initialize an empty leaf block as the btree root. */
STATIC int
xfbtree_init_leaf_block(
struct xfs_mount *mp,
struct xfbtree *xfbt,
const struct xfs_btree_ops *ops)
{
struct xfs_buf *bp;
xfbno_t bno = xfbt->highest_bno++;
int error;
error = xfs_buf_get(xfbt->target, xfbno_to_daddr(bno), XFBNO_BBSIZE,
&bp);
if (error)
return error;
trace_xfbtree_create_root_buf(xfbt, bp);
bp->b_ops = ops->buf_ops;
xfs_btree_init_buf(mp, bp, ops, 0, 0, xfbt->owner);
xfs_buf_relse(bp);
xfbt->root.l = cpu_to_be64(bno);
return 0;
}
/*
* Create an in-memory btree root that can be used with the given xmbuf.
* Callers must set xfbt->owner.
*/
int
xfbtree_init(
struct xfs_mount *mp,
struct xfbtree *xfbt,
struct xfs_buftarg *btp,
const struct xfs_btree_ops *ops)
{
unsigned int blocklen = xfbtree_rec_bytes(mp, ops);
unsigned int keyptr_len;
int error;
/* Requires a long-format CRC-format btree */
if (!xfs_has_crc(mp)) {
ASSERT(xfs_has_crc(mp));
return -EINVAL;
}
if (ops->ptr_len != XFS_BTREE_LONG_PTR_LEN) {
ASSERT(ops->ptr_len == XFS_BTREE_LONG_PTR_LEN);
return -EINVAL;
}
memset(xfbt, 0, sizeof(*xfbt));
xfbt->target = btp;
/* Set up min/maxrecs for this btree. */
keyptr_len = ops->key_len + sizeof(__be64);
xfbt->maxrecs[0] = blocklen / ops->rec_len;
xfbt->maxrecs[1] = blocklen / keyptr_len;
xfbt->minrecs[0] = xfbt->maxrecs[0] / 2;
xfbt->minrecs[1] = xfbt->maxrecs[1] / 2;
xfbt->highest_bno = 0;
xfbt->nlevels = 1;
/* Initialize the empty btree. */
error = xfbtree_init_leaf_block(mp, xfbt, ops);
if (error)
goto err_freesp;
trace_xfbtree_init(mp, xfbt, ops);
return 0;
err_freesp:
xfs_buftarg_drain(xfbt->target);
return error;
}
/* Allocate a block to our in-memory btree. */
int
xfbtree_alloc_block(
struct xfs_btree_cur *cur,
const union xfs_btree_ptr *start,
union xfs_btree_ptr *new,
int *stat)
{
struct xfbtree *xfbt = cur->bc_mem.xfbtree;
xfbno_t bno = xfbt->highest_bno++;
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_MEM);
trace_xfbtree_alloc_block(xfbt, cur, bno);
/* Fail if the block address exceeds the maximum for the buftarg. */
if (!xfbtree_verify_bno(xfbt, bno)) {
ASSERT(xfbtree_verify_bno(xfbt, bno));
*stat = 0;
return 0;
}
new->l = cpu_to_be64(bno);
*stat = 1;
return 0;
}
/* Free a block from our in-memory btree. */
int
xfbtree_free_block(
struct xfs_btree_cur *cur,
struct xfs_buf *bp)
{
struct xfbtree *xfbt = cur->bc_mem.xfbtree;
xfs_daddr_t daddr = xfs_buf_daddr(bp);
xfbno_t bno = xfs_daddr_to_xfbno(daddr);
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_MEM);
trace_xfbtree_free_block(xfbt, cur, bno);
if (bno + 1 == xfbt->highest_bno)
xfbt->highest_bno--;
return 0;
}
/* Return the minimum number of records for a btree block. */
int
xfbtree_get_minrecs(
struct xfs_btree_cur *cur,
int level)
{
struct xfbtree *xfbt = cur->bc_mem.xfbtree;
return xfbt->minrecs[level != 0];
}
/* Return the maximum number of records for a btree block. */
int
xfbtree_get_maxrecs(
struct xfs_btree_cur *cur,
int level)
{
struct xfbtree *xfbt = cur->bc_mem.xfbtree;
return xfbt->maxrecs[level != 0];
}
/* If this log item is a buffer item that came from the xfbtree, return it. */
static inline struct xfs_buf *
xfbtree_buf_match(
struct xfbtree *xfbt,
const struct xfs_log_item *lip)
{
const struct xfs_buf_log_item *bli;
struct xfs_buf *bp;
if (lip->li_type != XFS_LI_BUF)
return NULL;
bli = container_of(lip, struct xfs_buf_log_item, bli_item);
bp = bli->bli_buf;
if (bp->b_target != xfbt->target)
return NULL;
return bp;
}
/*
* Commit changes to the incore btree immediately by writing all dirty xfbtree
* buffers to the backing xfile. This detaches all xfbtree buffers from the
* transaction, even on failure. The buffer locks are dropped between the
* delwri queue and submit, so the caller must synchronize btree access.
*
* Normally we'd let the buffers commit with the transaction and get written to
* the xfile via the log, but online repair stages ephemeral btrees in memory
* and uses the btree_staging functions to write new btrees to disk atomically.
* The in-memory btree (and its backing store) are discarded at the end of the
* repair phase, which means that xfbtree buffers cannot commit with the rest
* of a transaction.
*
* In other words, online repair only needs the transaction to collect buffer
* pointers and to avoid buffer deadlocks, not to guarantee consistency of
* updates.
*/
int
xfbtree_trans_commit(
struct xfbtree *xfbt,
struct xfs_trans *tp)
{
struct xfs_log_item *lip, *n;
bool tp_dirty = false;
int error = 0;
/*
* For each xfbtree buffer attached to the transaction, write the dirty
* buffers to the xfile and release them.
*/
list_for_each_entry_safe(lip, n, &tp->t_items, li_trans) {
struct xfs_buf *bp = xfbtree_buf_match(xfbt, lip);
if (!bp) {
if (test_bit(XFS_LI_DIRTY, &lip->li_flags))
tp_dirty |= true;
continue;
}
trace_xfbtree_trans_commit_buf(xfbt, bp);
xmbuf_trans_bdetach(tp, bp);
/*
* If the buffer fails verification, note the failure but
* continue walking the transaction items so that we remove all
* ephemeral btree buffers.
*/
if (!error)
error = xmbuf_finalize(bp);
xfs_buf_relse(bp);
}
/*
* Reset the transaction's dirty flag to reflect the dirty state of the
* log items that are still attached.
*/
tp->t_flags = (tp->t_flags & ~XFS_TRANS_DIRTY) |
(tp_dirty ? XFS_TRANS_DIRTY : 0);
return error;
}
/*
* Cancel changes to the incore btree by detaching all the xfbtree buffers.
* Changes are not undone, so callers must not access the btree ever again.
*/
void
xfbtree_trans_cancel(
struct xfbtree *xfbt,
struct xfs_trans *tp)
{
struct xfs_log_item *lip, *n;
bool tp_dirty = false;
list_for_each_entry_safe(lip, n, &tp->t_items, li_trans) {
struct xfs_buf *bp = xfbtree_buf_match(xfbt, lip);
if (!bp) {
if (test_bit(XFS_LI_DIRTY, &lip->li_flags))
tp_dirty |= true;
continue;
}
trace_xfbtree_trans_cancel_buf(xfbt, bp);
xmbuf_trans_bdetach(tp, bp);
xfs_buf_relse(bp);
}
/*
* Reset the transaction's dirty flag to reflect the dirty state of the
* log items that are still attached.
*/
tp->t_flags = (tp->t_flags & ~XFS_TRANS_DIRTY) |
(tp_dirty ? XFS_TRANS_DIRTY : 0);
}

75
fs/xfs/libxfs/xfs_btree_mem.h Normal file
View File

@ -0,0 +1,75 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_BTREE_MEM_H__
#define __XFS_BTREE_MEM_H__
typedef uint64_t xfbno_t;
#define XFBNO_BLOCKSIZE (XMBUF_BLOCKSIZE)
#define XFBNO_BBSHIFT (XMBUF_BLOCKSHIFT - BBSHIFT)
#define XFBNO_BBSIZE (XFBNO_BLOCKSIZE >> BBSHIFT)
static inline xfs_daddr_t xfbno_to_daddr(xfbno_t blkno)
{
return blkno << XFBNO_BBSHIFT;
}
static inline xfbno_t xfs_daddr_to_xfbno(xfs_daddr_t daddr)
{
return daddr >> XFBNO_BBSHIFT;
}
struct xfbtree {
/* buffer cache target for this in-memory btree */
struct xfs_buftarg *target;
/* Highest block number that has been written to. */
xfbno_t highest_bno;
/* Owner of this btree. */
unsigned long long owner;
/* Btree header */
union xfs_btree_ptr root;
unsigned int nlevels;
/* Minimum and maximum records per block. */
unsigned int maxrecs[2];
unsigned int minrecs[2];
};
#ifdef CONFIG_XFS_BTREE_IN_MEM
static inline bool xfbtree_verify_bno(struct xfbtree *xfbt, xfbno_t bno)
{
return xmbuf_verify_daddr(xfbt->target, xfbno_to_daddr(bno));
}
void xfbtree_set_root(struct xfs_btree_cur *cur,
const union xfs_btree_ptr *ptr, int inc);
void xfbtree_init_ptr_from_cur(struct xfs_btree_cur *cur,
union xfs_btree_ptr *ptr);
struct xfs_btree_cur *xfbtree_dup_cursor(struct xfs_btree_cur *cur);
int xfbtree_get_minrecs(struct xfs_btree_cur *cur, int level);
int xfbtree_get_maxrecs(struct xfs_btree_cur *cur, int level);
int xfbtree_alloc_block(struct xfs_btree_cur *cur,
const union xfs_btree_ptr *start, union xfs_btree_ptr *ptr,
int *stat);
int xfbtree_free_block(struct xfs_btree_cur *cur, struct xfs_buf *bp);
/* Callers must set xfbt->target and xfbt->owner before calling this */
int xfbtree_init(struct xfs_mount *mp, struct xfbtree *xfbt,
struct xfs_buftarg *btp, const struct xfs_btree_ops *ops);
void xfbtree_destroy(struct xfbtree *xfbt);
int xfbtree_trans_commit(struct xfbtree *xfbt, struct xfs_trans *tp);
void xfbtree_trans_cancel(struct xfbtree *xfbt, struct xfs_trans *tp);
#else
# define xfbtree_verify_bno(...) (false)
#endif /* CONFIG_XFS_BTREE_IN_MEM */
#endif /* __XFS_BTREE_MEM_H__ */

133
fs/xfs/libxfs/xfs_btree_staging.c
View File

@ -38,63 +38,6 @@
* specific btree type to commit the new btree into the filesystem.
*/
/*
* Don't allow staging cursors to be duplicated because they're supposed to be
* kept private to a single thread.
*/
STATIC struct xfs_btree_cur *
xfs_btree_fakeroot_dup_cursor(
struct xfs_btree_cur *cur)
{
ASSERT(0);
return NULL;
}
/*
* Don't allow block allocation for a staging cursor, because staging cursors
* do not support regular btree modifications.
*
* Bulk loading uses a separate callback to obtain new blocks from a
* preallocated list, which prevents ENOSPC failures during loading.
*/
STATIC int
xfs_btree_fakeroot_alloc_block(
struct xfs_btree_cur *cur,
const union xfs_btree_ptr *start_bno,
union xfs_btree_ptr *new_bno,
int *stat)
{
ASSERT(0);
return -EFSCORRUPTED;
}
/*
* Don't allow block freeing for a staging cursor, because staging cursors
* do not support regular btree modifications.
*/
STATIC int
xfs_btree_fakeroot_free_block(
struct xfs_btree_cur *cur,
struct xfs_buf *bp)
{
ASSERT(0);
return -EFSCORRUPTED;
}
/* Initialize a pointer to the root block from the fakeroot. */
STATIC void
xfs_btree_fakeroot_init_ptr_from_cur(
struct xfs_btree_cur *cur,
union xfs_btree_ptr *ptr)
{
struct xbtree_afakeroot *afake;
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
afake = cur->bc_ag.afake;
ptr->s = cpu_to_be32(afake->af_root);
}
/*
* Bulk Loading for AG Btrees
* ==========================
@ -109,47 +52,20 @@ xfs_btree_fakeroot_init_ptr_from_cur(
* cursor into a regular btree cursor.
*/
/* Update the btree root information for a per-AG fake root. */
STATIC void
xfs_btree_afakeroot_set_root(
struct xfs_btree_cur *cur,
const union xfs_btree_ptr *ptr,
int inc)
{
struct xbtree_afakeroot *afake = cur->bc_ag.afake;
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
afake->af_root = be32_to_cpu(ptr->s);
afake->af_levels += inc;
}
/*
* Initialize a AG-rooted btree cursor with the given AG btree fake root.
* The btree cursor's bc_ops will be overridden as needed to make the staging
* functionality work.
*/
void
xfs_btree_stage_afakeroot(
struct xfs_btree_cur *cur,
struct xbtree_afakeroot *afake)
{
struct xfs_btree_ops *nops;
ASSERT(!(cur->bc_flags & XFS_BTREE_STAGING));
ASSERT(!(cur->bc_flags & XFS_BTREE_ROOT_IN_INODE));
ASSERT(cur->bc_ops->type != XFS_BTREE_TYPE_INODE);
ASSERT(cur->bc_tp == NULL);
nops = kmem_alloc(sizeof(struct xfs_btree_ops), KM_NOFS);
memcpy(nops, cur->bc_ops, sizeof(struct xfs_btree_ops));
nops->alloc_block = xfs_btree_fakeroot_alloc_block;
nops->free_block = xfs_btree_fakeroot_free_block;
nops->init_ptr_from_cur = xfs_btree_fakeroot_init_ptr_from_cur;
nops->set_root = xfs_btree_afakeroot_set_root;
nops->dup_cursor = xfs_btree_fakeroot_dup_cursor;
cur->bc_ag.afake = afake;
cur->bc_nlevels = afake->af_levels;
cur->bc_ops = nops;
cur->bc_flags |= XFS_BTREE_STAGING;
}
@ -163,17 +79,15 @@ void
xfs_btree_commit_afakeroot(
struct xfs_btree_cur *cur,
struct xfs_trans *tp,
struct xfs_buf *agbp,
const struct xfs_btree_ops *ops)
struct xfs_buf *agbp)
{
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
ASSERT(cur->bc_tp == NULL);
trace_xfs_btree_commit_afakeroot(cur);
kmem_free((void *)cur->bc_ops);
cur->bc_ag.afake = NULL;
cur->bc_ag.agbp = agbp;
cur->bc_ops = ops;
cur->bc_flags &= ~XFS_BTREE_STAGING;
cur->bc_tp = tp;
}
@ -211,29 +125,16 @@ xfs_btree_commit_afakeroot(
void
xfs_btree_stage_ifakeroot(
struct xfs_btree_cur *cur,
struct xbtree_ifakeroot *ifake,
struct xfs_btree_ops **new_ops)
struct xbtree_ifakeroot *ifake)
{
struct xfs_btree_ops *nops;
ASSERT(!(cur->bc_flags & XFS_BTREE_STAGING));
ASSERT(cur->bc_flags & XFS_BTREE_ROOT_IN_INODE);
ASSERT(cur->bc_ops->type == XFS_BTREE_TYPE_INODE);
ASSERT(cur->bc_tp == NULL);
nops = kmem_alloc(sizeof(struct xfs_btree_ops), KM_NOFS);
memcpy(nops, cur->bc_ops, sizeof(struct xfs_btree_ops));
nops->alloc_block = xfs_btree_fakeroot_alloc_block;
nops->free_block = xfs_btree_fakeroot_free_block;
nops->init_ptr_from_cur = xfs_btree_fakeroot_init_ptr_from_cur;
nops->dup_cursor = xfs_btree_fakeroot_dup_cursor;
cur->bc_ino.ifake = ifake;
cur->bc_nlevels = ifake->if_levels;
cur->bc_ops = nops;
cur->bc_ino.forksize = ifake->if_fork_size;
cur->bc_flags |= XFS_BTREE_STAGING;
if (new_ops)
*new_ops = nops;
}
/*
@ -246,18 +147,15 @@ void
xfs_btree_commit_ifakeroot(
struct xfs_btree_cur *cur,
struct xfs_trans *tp,
int whichfork,
const struct xfs_btree_ops *ops)
int whichfork)
{
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
ASSERT(cur->bc_tp == NULL);
trace_xfs_btree_commit_ifakeroot(cur);
kmem_free((void *)cur->bc_ops);
cur->bc_ino.ifake = NULL;
cur->bc_ino.whichfork = whichfork;
cur->bc_ops = ops;
cur->bc_flags &= ~XFS_BTREE_STAGING;
cur->bc_tp = tp;
}
@ -397,8 +295,7 @@ xfs_btree_bload_prep_block(
struct xfs_btree_block *new_block;
int ret;
if ((cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) &&
level == cur->bc_nlevels - 1) {
if (xfs_btree_at_iroot(cur, level)) {
struct xfs_ifork *ifp = xfs_btree_ifork_ptr(cur);
size_t new_size;
@ -406,14 +303,12 @@ xfs_btree_bload_prep_block(
/* Allocate a new incore btree root block. */
new_size = bbl->iroot_size(cur, level, nr_this_block, priv);
ifp->if_broot = kmem_zalloc(new_size, 0);
ifp->if_broot = kzalloc(new_size, GFP_KERNEL | __GFP_NOFAIL);
ifp->if_broot_bytes = (int)new_size;
/* Initialize it and send it out. */
xfs_btree_init_block_int(cur->bc_mp, ifp->if_broot,
XFS_BUF_DADDR_NULL, cur->bc_btnum, level,
nr_this_block, cur->bc_ino.ip->i_ino,
cur->bc_flags);
xfs_btree_init_block(cur->bc_mp, ifp->if_broot, cur->bc_ops,
level, nr_this_block, cur->bc_ino.ip->i_ino);
*bpp = NULL;
*blockp = ifp->if_broot;
@ -704,7 +599,7 @@ xfs_btree_bload_compute_geometry(
xfs_btree_bload_level_geometry(cur, bbl, level, nr_this_level,
&avg_per_block, &level_blocks, &dontcare64);
if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) {
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE) {
/*
* If all the items we want to store at this level
* would fit in the inode root block, then we have our
@ -763,7 +658,7 @@ xfs_btree_bload_compute_geometry(
return -EOVERFLOW;
bbl->btree_height = cur->bc_nlevels;
if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE)
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE)
bbl->nr_blocks = nr_blocks - 1;
else
bbl->nr_blocks = nr_blocks;
@ -890,7 +785,7 @@ xfs_btree_bload(
}
/* Initialize the new root. */
if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) {
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE) {
ASSERT(xfs_btree_ptr_is_null(cur, &ptr));
cur->bc_ino.ifake->if_levels = cur->bc_nlevels;
cur->bc_ino.ifake->if_blocks = total_blocks - 1;

10
fs/xfs/libxfs/xfs_btree_staging.h
View File

@ -22,7 +22,7 @@ struct xbtree_afakeroot {
void xfs_btree_stage_afakeroot(struct xfs_btree_cur *cur,
struct xbtree_afakeroot *afake);
void xfs_btree_commit_afakeroot(struct xfs_btree_cur *cur, struct xfs_trans *tp,
struct xfs_buf *agbp, const struct xfs_btree_ops *ops);
struct xfs_buf *agbp);
/* Fake root for an inode-rooted btree. */
struct xbtree_ifakeroot {
@ -41,10 +41,9 @@ struct xbtree_ifakeroot {
/* Cursor interactions with fake roots for inode-rooted btrees. */
void xfs_btree_stage_ifakeroot(struct xfs_btree_cur *cur,
struct xbtree_ifakeroot *ifake,
struct xfs_btree_ops **new_ops);
struct xbtree_ifakeroot *ifake);
void xfs_btree_commit_ifakeroot(struct xfs_btree_cur *cur, struct xfs_trans *tp,
int whichfork, const struct xfs_btree_ops *ops);
int whichfork);
/* Bulk loading of staged btrees. */
typedef int (*xfs_btree_bload_get_records_fn)(struct xfs_btree_cur *cur,
@ -76,8 +75,7 @@ struct xfs_btree_bload {
/*
* This function should return the size of the in-core btree root
* block. It is only necessary for XFS_BTREE_ROOT_IN_INODE btree
* types.
* block. It is only necessary for XFS_BTREE_TYPE_INODE btrees.
*/
xfs_btree_bload_iroot_size_fn iroot_size;

59
fs/xfs/libxfs/xfs_da_btree.c
View File

@ -23,6 +23,7 @@
#include "xfs_buf_item.h"
#include "xfs_log.h"
#include "xfs_errortag.h"
#include "xfs_health.h"
/*
* xfs_da_btree.c
@ -85,7 +86,8 @@ xfs_da_state_alloc(
{
struct xfs_da_state *state;
state = kmem_cache_zalloc(xfs_da_state_cache, GFP_NOFS | __GFP_NOFAIL);
state = kmem_cache_zalloc(xfs_da_state_cache,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
state->args = args;
state->mp = args->dp->i_mount;
return state;
@ -352,6 +354,8 @@ const struct xfs_buf_ops xfs_da3_node_buf_ops = {
static int
xfs_da3_node_set_type(
struct xfs_trans *tp,
struct xfs_inode *dp,
int whichfork,
struct xfs_buf *bp)
{
struct xfs_da_blkinfo *info = bp->b_addr;
@ -373,6 +377,7 @@ xfs_da3_node_set_type(
XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, tp->t_mountp,
info, sizeof(*info));
xfs_trans_brelse(tp, bp);
xfs_dirattr_mark_sick(dp, whichfork);
return -EFSCORRUPTED;
}
}
@ -391,7 +396,7 @@ xfs_da3_node_read(
&xfs_da3_node_buf_ops);
if (error || !*bpp || !tp)
return error;
return xfs_da3_node_set_type(tp, *bpp);
return xfs_da3_node_set_type(tp, dp, whichfork, *bpp);
}
int
@ -408,6 +413,8 @@ xfs_da3_node_read_mapped(
error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp, mappedbno,
XFS_FSB_TO_BB(mp, xfs_dabuf_nfsb(mp, whichfork)), 0,
bpp, &xfs_da3_node_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_dirattr_mark_sick(dp, whichfork);
if (error || !*bpp)
return error;
@ -418,7 +425,7 @@ xfs_da3_node_read_mapped(
if (!tp)
return 0;
return xfs_da3_node_set_type(tp, *bpp);
return xfs_da3_node_set_type(tp, dp, whichfork, *bpp);
}
/*
@ -631,6 +638,7 @@ xfs_da3_split(
if (node->hdr.info.forw) {
if (be32_to_cpu(node->hdr.info.forw) != addblk->blkno) {
xfs_buf_mark_corrupt(oldblk->bp);
xfs_da_mark_sick(state->args);
error = -EFSCORRUPTED;
goto out;
}
@ -644,6 +652,7 @@ xfs_da3_split(
if (node->hdr.info.back) {
if (be32_to_cpu(node->hdr.info.back) != addblk->blkno) {
xfs_buf_mark_corrupt(oldblk->bp);
xfs_da_mark_sick(state->args);
error = -EFSCORRUPTED;
goto out;
}
@ -1635,6 +1644,7 @@ xfs_da3_node_lookup_int(
if (magic != XFS_DA_NODE_MAGIC && magic != XFS_DA3_NODE_MAGIC) {
xfs_buf_mark_corrupt(blk->bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -1650,6 +1660,7 @@ xfs_da3_node_lookup_int(
/* Tree taller than we can handle; bail out! */
if (nodehdr.level >= XFS_DA_NODE_MAXDEPTH) {
xfs_buf_mark_corrupt(blk->bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -1658,6 +1669,7 @@ xfs_da3_node_lookup_int(
expected_level = nodehdr.level - 1;
else if (expected_level != nodehdr.level) {
xfs_buf_mark_corrupt(blk->bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
} else
expected_level--;
@ -1709,12 +1721,16 @@ xfs_da3_node_lookup_int(
}
/* We can't point back to the root. */
if (XFS_IS_CORRUPT(dp->i_mount, blkno == args->geo->leafblk))
if (XFS_IS_CORRUPT(dp->i_mount, blkno == args->geo->leafblk)) {
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
}
if (XFS_IS_CORRUPT(dp->i_mount, expected_level != 0))
if (XFS_IS_CORRUPT(dp->i_mount, expected_level != 0)) {
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
/*
* A leaf block that ends in the hashval that we are interested in
@ -1732,6 +1748,7 @@ xfs_da3_node_lookup_int(
args->blkno = blk->blkno;
} else {
ASSERT(0);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
if (((retval == -ENOENT) || (retval == -ENOATTR)) &&
@ -2182,7 +2199,8 @@ xfs_da_grow_inode_int(
* If we didn't get it and the block might work if fragmented,
* try without the CONTIG flag. Loop until we get it all.
*/
mapp = kmem_alloc(sizeof(*mapp) * count, 0);
mapp = kmalloc(sizeof(*mapp) * count,
GFP_KERNEL | __GFP_NOFAIL);
for (b = *bno, mapi = 0; b < *bno + count; ) {
c = (int)(*bno + count - b);
nmap = min(XFS_BMAP_MAX_NMAP, c);
@ -2219,7 +2237,7 @@ xfs_da_grow_inode_int(
out_free_map:
if (mapp != &map)
kmem_free(mapp);
kfree(mapp);
return error;
}
@ -2297,8 +2315,10 @@ xfs_da3_swap_lastblock(
error = xfs_bmap_last_before(tp, dp, &lastoff, w);
if (error)
return error;
if (XFS_IS_CORRUPT(mp, lastoff == 0))
if (XFS_IS_CORRUPT(mp, lastoff == 0)) {
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
/*
* Read the last block in the btree space.
*/
@ -2348,6 +2368,7 @@ xfs_da3_swap_lastblock(
if (XFS_IS_CORRUPT(mp,
be32_to_cpu(sib_info->forw) != last_blkno ||
sib_info->magic != dead_info->magic)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2368,6 +2389,7 @@ xfs_da3_swap_lastblock(
if (XFS_IS_CORRUPT(mp,
be32_to_cpu(sib_info->back) != last_blkno ||
sib_info->magic != dead_info->magic)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2390,6 +2412,7 @@ xfs_da3_swap_lastblock(
xfs_da3_node_hdr_from_disk(dp->i_mount, &par_hdr, par_node);
if (XFS_IS_CORRUPT(mp,
level >= 0 && level != par_hdr.level + 1)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2401,6 +2424,7 @@ xfs_da3_swap_lastblock(
entno++)
continue;
if (XFS_IS_CORRUPT(mp, entno == par_hdr.count)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2426,6 +2450,7 @@ xfs_da3_swap_lastblock(
xfs_trans_brelse(tp, par_buf);
par_buf = NULL;
if (XFS_IS_CORRUPT(mp, par_blkno == 0)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2435,6 +2460,7 @@ xfs_da3_swap_lastblock(
par_node = par_buf->b_addr;
xfs_da3_node_hdr_from_disk(dp->i_mount, &par_hdr, par_node);
if (XFS_IS_CORRUPT(mp, par_hdr.level != level)) {
xfs_da_mark_sick(args);
error = -EFSCORRUPTED;
goto done;
}
@ -2518,7 +2544,8 @@ xfs_dabuf_map(
int error = 0, nirecs, i;
if (nfsb > 1)
irecs = kmem_zalloc(sizeof(irec) * nfsb, KM_NOFS);
irecs = kzalloc(sizeof(irec) * nfsb,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
nirecs = nfsb;
error = xfs_bmapi_read(dp, bno, nfsb, irecs, &nirecs,
@ -2531,7 +2558,8 @@ xfs_dabuf_map(
* larger one that needs to be free by the caller.
*/
if (nirecs > 1) {
map = kmem_zalloc(nirecs * sizeof(struct xfs_buf_map), KM_NOFS);
map = kzalloc(nirecs * sizeof(struct xfs_buf_map),
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
if (!map) {
error = -ENOMEM;
goto out_free_irecs;
@ -2557,12 +2585,13 @@ xfs_dabuf_map(
*nmaps = nirecs;
out_free_irecs:
if (irecs != &irec)
kmem_free(irecs);
kfree(irecs);
return error;
invalid_mapping:
/* Caller ok with no mapping. */
if (XFS_IS_CORRUPT(mp, !(flags & XFS_DABUF_MAP_HOLE_OK))) {
xfs_dirattr_mark_sick(dp, whichfork);
error = -EFSCORRUPTED;
if (xfs_error_level >= XFS_ERRLEVEL_LOW) {
xfs_alert(mp, "%s: bno %u inode %llu",
@ -2613,7 +2642,7 @@ xfs_da_get_buf(
out_free:
if (mapp != &map)
kmem_free(mapp);
kfree(mapp);
return error;
}
@ -2644,6 +2673,8 @@ xfs_da_read_buf(
error = xfs_trans_read_buf_map(mp, tp, mp->m_ddev_targp, mapp, nmap, 0,
&bp, ops);
if (xfs_metadata_is_sick(error))
xfs_dirattr_mark_sick(dp, whichfork);
if (error)
goto out_free;
@ -2654,7 +2685,7 @@ xfs_da_read_buf(
*bpp = bp;
out_free:
if (mapp != &map)
kmem_free(mapp);
kfree(mapp);
return error;
}
@ -2685,7 +2716,7 @@ xfs_da_reada_buf(
out_free:
if (mapp != &map)
kmem_free(mapp);
kfree(mapp);
return error;
}

11
fs/xfs/libxfs/xfs_da_format.h
View File

@ -159,6 +159,17 @@ struct xfs_da3_intnode {
#define XFS_DIR3_FT_MAX 9
#define XFS_DIR3_FTYPE_STR \
{ XFS_DIR3_FT_UNKNOWN, "unknown" }, \
{ XFS_DIR3_FT_REG_FILE, "file" }, \
{ XFS_DIR3_FT_DIR, "directory" }, \
{ XFS_DIR3_FT_CHRDEV, "char" }, \
{ XFS_DIR3_FT_BLKDEV, "block" }, \
{ XFS_DIR3_FT_FIFO, "fifo" }, \
{ XFS_DIR3_FT_SOCK, "sock" }, \
{ XFS_DIR3_FT_SYMLINK, "symlink" }, \
{ XFS_DIR3_FT_WHT, "whiteout" }
/*
* Byte offset in data block and shortform entry.
*/

25
fs/xfs/libxfs/xfs_defer.c
View File

@ -819,16 +819,16 @@ xfs_defer_can_append(
/* Create a new pending item at the end of the transaction list. */
static inline struct xfs_defer_pending *
xfs_defer_alloc(
struct xfs_trans *tp,
struct list_head *dfops,
const struct xfs_defer_op_type *ops)
{
struct xfs_defer_pending *dfp;
dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
GFP_NOFS | __GFP_NOFAIL);
GFP_KERNEL | __GFP_NOFAIL);
dfp->dfp_ops = ops;
INIT_LIST_HEAD(&dfp->dfp_work);
list_add_tail(&dfp->dfp_list, &tp->t_dfops);
list_add_tail(&dfp->dfp_list, dfops);
return dfp;
}
@ -846,7 +846,7 @@ xfs_defer_add(
dfp = xfs_defer_find_last(tp, ops);
if (!dfp || !xfs_defer_can_append(dfp, ops))
dfp = xfs_defer_alloc(tp, ops);
dfp = xfs_defer_alloc(&tp->t_dfops, ops);
xfs_defer_add_item(dfp, li);
trace_xfs_defer_add_item(tp->t_mountp, dfp, li);
@ -870,7 +870,7 @@ xfs_defer_add_barrier(
if (dfp)
return;
xfs_defer_alloc(tp, &xfs_barrier_defer_type);
xfs_defer_alloc(&tp->t_dfops, &xfs_barrier_defer_type);
trace_xfs_defer_add_item(tp->t_mountp, dfp, NULL);
}
@ -885,14 +885,9 @@ xfs_defer_start_recovery(
struct list_head *r_dfops,
const struct xfs_defer_op_type *ops)
{
struct xfs_defer_pending *dfp;
struct xfs_defer_pending *dfp = xfs_defer_alloc(r_dfops, ops);
dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
GFP_NOFS | __GFP_NOFAIL);
dfp->dfp_ops = ops;
dfp->dfp_intent = lip;
INIT_LIST_HEAD(&dfp->dfp_work);
list_add_tail(&dfp->dfp_list, r_dfops);
}
/*
@ -979,7 +974,7 @@ xfs_defer_ops_capture(
return ERR_PTR(error);
/* Create an object to capture the defer ops. */
dfc = kmem_zalloc(sizeof(*dfc), KM_NOFS);
dfc = kzalloc(sizeof(*dfc), GFP_KERNEL | __GFP_NOFAIL);
INIT_LIST_HEAD(&dfc->dfc_list);
INIT_LIST_HEAD(&dfc->dfc_dfops);
@ -1011,7 +1006,7 @@ xfs_defer_ops_capture(
* transaction.
*/
for (i = 0; i < dfc->dfc_held.dr_inos; i++) {
ASSERT(xfs_isilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL));
xfs_assert_ilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL);
ihold(VFS_I(dfc->dfc_held.dr_ip[i]));
}
@ -1038,7 +1033,7 @@ xfs_defer_ops_capture_abort(
for (i = 0; i < dfc->dfc_held.dr_inos; i++)
xfs_irele(dfc->dfc_held.dr_ip[i]);
kmem_free(dfc);
kfree(dfc);
}
/*
@ -1114,7 +1109,7 @@ xfs_defer_ops_continue(
list_splice_init(&dfc->dfc_dfops, &tp->t_dfops);
tp->t_flags |= dfc->dfc_tpflags;
kmem_free(dfc);
kfree(dfc);
}
/* Release the resources captured and continued during recovery. */

59
fs/xfs/libxfs/xfs_dir2.c
View File

@ -18,6 +18,7 @@
#include "xfs_errortag.h"
#include "xfs_error.h"
#include "xfs_trace.h"
#include "xfs_health.h"
const struct xfs_name xfs_name_dotdot = {
.name = (const unsigned char *)"..",
@ -25,6 +26,12 @@ const struct xfs_name xfs_name_dotdot = {
.type = XFS_DIR3_FT_DIR,
};
const struct xfs_name xfs_name_dot = {
.name = (const unsigned char *)".",
.len = 1,
.type = XFS_DIR3_FT_DIR,
};
/*
* Convert inode mode to directory entry filetype
*/
@ -104,13 +111,13 @@ xfs_da_mount(
ASSERT(mp->m_sb.sb_versionnum & XFS_SB_VERSION_DIRV2BIT);
ASSERT(xfs_dir2_dirblock_bytes(&mp->m_sb) <= XFS_MAX_BLOCKSIZE);
mp->m_dir_geo = kmem_zalloc(sizeof(struct xfs_da_geometry),
KM_MAYFAIL);
mp->m_attr_geo = kmem_zalloc(sizeof(struct xfs_da_geometry),
KM_MAYFAIL);
mp->m_dir_geo = kzalloc(sizeof(struct xfs_da_geometry),
GFP_KERNEL | __GFP_RETRY_MAYFAIL);
mp->m_attr_geo = kzalloc(sizeof(struct xfs_da_geometry),
GFP_KERNEL | __GFP_RETRY_MAYFAIL);
if (!mp->m_dir_geo || !mp->m_attr_geo) {
kmem_free(mp->m_dir_geo);
kmem_free(mp->m_attr_geo);
kfree(mp->m_dir_geo);
kfree(mp->m_attr_geo);
return -ENOMEM;
}
@ -178,8 +185,8 @@ void
xfs_da_unmount(
struct xfs_mount *mp)
{
kmem_free(mp->m_dir_geo);
kmem_free(mp->m_attr_geo);
kfree(mp->m_dir_geo);
kfree(mp->m_attr_geo);
}
/*
@ -236,7 +243,7 @@ xfs_dir_init(
if (error)
return error;
args = kmem_zalloc(sizeof(*args), KM_NOFS);
args = kzalloc(sizeof(*args), GFP_KERNEL | __GFP_NOFAIL);
if (!args)
return -ENOMEM;
@ -244,7 +251,7 @@ xfs_dir_init(
args->dp = dp;
args->trans = tp;
error = xfs_dir2_sf_create(args, pdp->i_ino);
kmem_free(args);
kfree(args);
return error;
}
@ -273,7 +280,7 @@ xfs_dir_createname(
XFS_STATS_INC(dp->i_mount, xs_dir_create);
}
args = kmem_zalloc(sizeof(*args), KM_NOFS);
args = kzalloc(sizeof(*args), GFP_KERNEL | __GFP_NOFAIL);
if (!args)
return -ENOMEM;
@ -313,7 +320,7 @@ xfs_dir_createname(
rval = xfs_dir2_node_addname(args);
out_free:
kmem_free(args);
kfree(args);
return rval;
}
@ -333,7 +340,8 @@ xfs_dir_cilookup_result(
!(args->op_flags & XFS_DA_OP_CILOOKUP))
return -EEXIST;
args->value = kmem_alloc(len, KM_NOFS | KM_MAYFAIL);
args->value = kmalloc(len,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_RETRY_MAYFAIL);
if (!args->value)
return -ENOMEM;
@ -364,15 +372,8 @@ xfs_dir_lookup(
ASSERT(S_ISDIR(VFS_I(dp)->i_mode));
XFS_STATS_INC(dp->i_mount, xs_dir_lookup);
/*
* We need to use KM_NOFS here so that lockdep will not throw false
* positive deadlock warnings on a non-transactional lookup path. It is
* safe to recurse into inode recalim in that case, but lockdep can't
* easily be taught about it. Hence KM_NOFS avoids having to add more
* lockdep Doing this avoids having to add a bunch of lockdep class
* annotations into the reclaim path for the ilock.
*/
args = kmem_zalloc(sizeof(*args), KM_NOFS);
args = kzalloc(sizeof(*args),
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
args->geo = dp->i_mount->m_dir_geo;
args->name = name->name;
args->namelen = name->len;
@ -419,7 +420,7 @@ out_check_rval:
}
out_free:
xfs_iunlock(dp, lock_mode);
kmem_free(args);
kfree(args);
return rval;
}
@ -441,7 +442,7 @@ xfs_dir_removename(
ASSERT(S_ISDIR(VFS_I(dp)->i_mode));
XFS_STATS_INC(dp->i_mount, xs_dir_remove);
args = kmem_zalloc(sizeof(*args), KM_NOFS);
args = kzalloc(sizeof(*args), GFP_KERNEL | __GFP_NOFAIL);
if (!args)
return -ENOMEM;
@ -477,7 +478,7 @@ xfs_dir_removename(
else
rval = xfs_dir2_node_removename(args);
out_free:
kmem_free(args);
kfree(args);
return rval;
}
@ -502,7 +503,7 @@ xfs_dir_replace(
if (rval)
return rval;
args = kmem_zalloc(sizeof(*args), KM_NOFS);
args = kzalloc(sizeof(*args), GFP_KERNEL | __GFP_NOFAIL);
if (!args)
return -ENOMEM;
@ -538,7 +539,7 @@ xfs_dir_replace(
else
rval = xfs_dir2_node_replace(args);
out_free:
kmem_free(args);
kfree(args);
return rval;
}
@ -626,8 +627,10 @@ xfs_dir2_isblock(
return 0;
*isblock = true;
if (XFS_IS_CORRUPT(mp, args->dp->i_disk_size != args->geo->blksize))
if (XFS_IS_CORRUPT(mp, args->dp->i_disk_size != args->geo->blksize)) {
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
return 0;
}

13
fs/xfs/libxfs/xfs_dir2.h
View File

@ -22,6 +22,19 @@ struct xfs_dir3_icfree_hdr;
struct xfs_dir3_icleaf_hdr;
extern const struct xfs_name xfs_name_dotdot;
extern const struct xfs_name xfs_name_dot;
static inline bool
xfs_dir2_samename(
const struct xfs_name *n1,
const struct xfs_name *n2)
{
if (n1 == n2)
return true;
if (n1->len != n2->len)
return false;
return !memcmp(n1->name, n2->name, n1->len);
}
/*
* Convert inode mode to directory entry filetype

8
fs/xfs/libxfs/xfs_dir2_block.c
View File

@ -20,6 +20,7 @@
#include "xfs_error.h"
#include "xfs_trace.h"
#include "xfs_log.h"
#include "xfs_health.h"
/*
* Local function prototypes.
@ -152,6 +153,7 @@ xfs_dir3_block_read(
__xfs_buf_mark_corrupt(*bpp, fa);
xfs_trans_brelse(tp, *bpp);
*bpp = NULL;
xfs_dirattr_mark_sick(dp, XFS_DATA_FORK);
return -EFSCORRUPTED;
}
@ -1108,7 +1110,7 @@ xfs_dir2_sf_to_block(
* Copy the directory into a temporary buffer.
* Then pitch the incore inode data so we can make extents.
*/
sfp = kmem_alloc(ifp->if_bytes, 0);
sfp = kmalloc(ifp->if_bytes, GFP_KERNEL | __GFP_NOFAIL);
memcpy(sfp, oldsfp, ifp->if_bytes);
xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
@ -1253,7 +1255,7 @@ xfs_dir2_sf_to_block(
sfep = xfs_dir2_sf_nextentry(mp, sfp, sfep);
}
/* Done with the temporary buffer */
kmem_free(sfp);
kfree(sfp);
/*
* Sort the leaf entries by hash value.
*/
@ -1268,6 +1270,6 @@ xfs_dir2_sf_to_block(
xfs_dir3_data_check(dp, bp);
return 0;
out_free:
kmem_free(sfp);
kfree(sfp);
return error;
}

3
fs/xfs/libxfs/xfs_dir2_data.c
View File

@ -18,6 +18,7 @@
#include "xfs_trans.h"
#include "xfs_buf_item.h"
#include "xfs_log.h"
#include "xfs_health.h"
static xfs_failaddr_t xfs_dir2_data_freefind_verify(
struct xfs_dir2_data_hdr *hdr, struct xfs_dir2_data_free *bf,
@ -433,6 +434,7 @@ xfs_dir3_data_read(
__xfs_buf_mark_corrupt(*bpp, fa);
xfs_trans_brelse(tp, *bpp);
*bpp = NULL;
xfs_dirattr_mark_sick(dp, XFS_DATA_FORK);
return -EFSCORRUPTED;
}
@ -1198,6 +1200,7 @@ xfs_dir2_data_use_free(
corrupt:
xfs_corruption_error(__func__, XFS_ERRLEVEL_LOW, args->dp->i_mount,
hdr, sizeof(*hdr), __FILE__, __LINE__, fa);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}

3
fs/xfs/libxfs/xfs_dir2_leaf.c
View File

@ -19,6 +19,7 @@
#include "xfs_trace.h"
#include "xfs_trans.h"
#include "xfs_buf_item.h"
#include "xfs_health.h"
/*
* Local function declarations.
@ -1393,8 +1394,10 @@ xfs_dir2_leaf_removename(
bestsp = xfs_dir2_leaf_bests_p(ltp);
if (be16_to_cpu(bestsp[db]) != oldbest) {
xfs_buf_mark_corrupt(lbp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
/*
* Mark the former data entry unused.
*/

7
fs/xfs/libxfs/xfs_dir2_node.c
View File

@ -20,6 +20,7 @@
#include "xfs_trans.h"
#include "xfs_buf_item.h"
#include "xfs_log.h"
#include "xfs_health.h"
/*
* Function declarations.
@ -231,6 +232,7 @@ __xfs_dir3_free_read(
__xfs_buf_mark_corrupt(*bpp, fa);
xfs_trans_brelse(tp, *bpp);
*bpp = NULL;
xfs_dirattr_mark_sick(dp, XFS_DATA_FORK);
return -EFSCORRUPTED;
}
@ -443,6 +445,7 @@ xfs_dir2_leaf_to_node(
if (be32_to_cpu(ltp->bestcount) >
(uint)dp->i_disk_size / args->geo->blksize) {
xfs_buf_mark_corrupt(lbp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -517,6 +520,7 @@ xfs_dir2_leafn_add(
*/
if (index < 0) {
xfs_buf_mark_corrupt(bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -736,6 +740,7 @@ xfs_dir2_leafn_lookup_for_addname(
cpu_to_be16(NULLDATAOFF))) {
if (curfdb != newfdb)
xfs_trans_brelse(tp, curbp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
curfdb = newfdb;
@ -804,6 +809,7 @@ xfs_dir2_leafn_lookup_for_entry(
xfs_dir3_leaf_check(dp, bp);
if (leafhdr.count <= 0) {
xfs_buf_mark_corrupt(bp);
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}
@ -1739,6 +1745,7 @@ xfs_dir2_node_add_datablk(
} else {
xfs_alert(mp, " ... fblk is NULL");
}
xfs_da_mark_sick(args);
return -EFSCORRUPTED;
}

16
fs/xfs/libxfs/xfs_dir2_sf.c
View File

@ -276,7 +276,7 @@ xfs_dir2_block_to_sf(
* format the data into. Once we have formatted the data, we can free
* the block and copy the formatted data into the inode literal area.
*/
sfp = kmem_alloc(mp->m_sb.sb_inodesize, 0);
sfp = kmalloc(mp->m_sb.sb_inodesize, GFP_KERNEL | __GFP_NOFAIL);
memcpy(sfp, sfhp, xfs_dir2_sf_hdr_size(sfhp->i8count));
/*
@ -350,7 +350,7 @@ xfs_dir2_block_to_sf(
xfs_dir2_sf_check(args);
out:
xfs_trans_log_inode(args->trans, dp, logflags);
kmem_free(sfp);
kfree(sfp);
return error;
}
@ -524,7 +524,7 @@ xfs_dir2_sf_addname_hard(
* Copy the old directory to the stack buffer.
*/
old_isize = (int)dp->i_disk_size;
buf = kmem_alloc(old_isize, 0);
buf = kmalloc(old_isize, GFP_KERNEL | __GFP_NOFAIL);
oldsfp = (xfs_dir2_sf_hdr_t *)buf;
memcpy(oldsfp, dp->i_df.if_data, old_isize);
/*
@ -576,7 +576,7 @@ xfs_dir2_sf_addname_hard(
sfep = xfs_dir2_sf_nextentry(mp, sfp, sfep);
memcpy(sfep, oldsfep, old_isize - nbytes);
}
kmem_free(buf);
kfree(buf);
dp->i_disk_size = new_isize;
xfs_dir2_sf_check(args);
}
@ -1151,7 +1151,7 @@ xfs_dir2_sf_toino4(
* Don't want xfs_idata_realloc copying the data here.
*/
oldsize = dp->i_df.if_bytes;
buf = kmem_alloc(oldsize, 0);
buf = kmalloc(oldsize, GFP_KERNEL | __GFP_NOFAIL);
ASSERT(oldsfp->i8count == 1);
memcpy(buf, oldsfp, oldsize);
/*
@ -1190,7 +1190,7 @@ xfs_dir2_sf_toino4(
/*
* Clean up the inode.
*/
kmem_free(buf);
kfree(buf);
dp->i_disk_size = newsize;
xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE | XFS_ILOG_DDATA);
}
@ -1223,7 +1223,7 @@ xfs_dir2_sf_toino8(
* Don't want xfs_idata_realloc copying the data here.
*/
oldsize = dp->i_df.if_bytes;
buf = kmem_alloc(oldsize, 0);
buf = kmalloc(oldsize, GFP_KERNEL | __GFP_NOFAIL);
ASSERT(oldsfp->i8count == 0);
memcpy(buf, oldsfp, oldsize);
/*
@ -1262,7 +1262,7 @@ xfs_dir2_sf_toino8(
/*
* Clean up the inode.
*/
kmem_free(buf);
kfree(buf);
dp->i_disk_size = newsize;
xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE | XFS_ILOG_DDATA);
}

21
fs/xfs/libxfs/xfs_format.h
View File

@ -477,15 +477,9 @@ xfs_is_quota_inode(struct xfs_sb *sbp, xfs_ino_t ino)
#define XFS_AGI_GOOD_VERSION(v) ((v) == XFS_AGI_VERSION)
/*
* Btree number 0 is bno, 1 is cnt, 2 is rmap. This value gives the size of the
* arrays below.
*/
#define XFS_BTNUM_AGF ((int)XFS_BTNUM_RMAPi + 1)
/*
* The second word of agf_levels in the first a.g. overlaps the EFS
* superblock's magic number. Since the magic numbers valid for EFS
* are > 64k, our value cannot be confused for an EFS superblock's.
* agf_cnt_level in the first AGF overlaps the EFS superblock's magic number.
* Since the magic numbers valid for EFS are > 64k, our value cannot be confused
* for an EFS superblock.
*/
typedef struct xfs_agf {
@ -499,8 +493,13 @@ typedef struct xfs_agf {
/*
* Freespace and rmap information
*/
__be32 agf_roots[XFS_BTNUM_AGF]; /* root blocks */
__be32 agf_levels[XFS_BTNUM_AGF]; /* btree levels */
__be32 agf_bno_root; /* bnobt root block */
__be32 agf_cnt_root; /* cntbt root block */
__be32 agf_rmap_root; /* rmapbt root block */
__be32 agf_bno_level; /* bnobt btree levels */
__be32 agf_cnt_level; /* cntbt btree levels */
__be32 agf_rmap_level; /* rmapbt btree levels */
__be32 agf_flfirst; /* first freelist block's index */
__be32 agf_fllast; /* last freelist block's index */

8
fs/xfs/libxfs/xfs_fs.h
View File

@ -195,6 +195,8 @@ struct xfs_fsop_geom {
#define XFS_FSOP_GEOM_SICK_PQUOTA (1 << 3) /* project quota */
#define XFS_FSOP_GEOM_SICK_RT_BITMAP (1 << 4) /* realtime bitmap */
#define XFS_FSOP_GEOM_SICK_RT_SUMMARY (1 << 5) /* realtime summary */
#define XFS_FSOP_GEOM_SICK_QUOTACHECK (1 << 6) /* quota counts */
#define XFS_FSOP_GEOM_SICK_NLINKS (1 << 7) /* inode link counts */
/* Output for XFS_FS_COUNTS */
typedef struct xfs_fsop_counts {
@ -292,6 +294,7 @@ struct xfs_ag_geometry {
#define XFS_AG_GEOM_SICK_FINOBT (1 << 7) /* free inode index */
#define XFS_AG_GEOM_SICK_RMAPBT (1 << 8) /* reverse mappings */
#define XFS_AG_GEOM_SICK_REFCNTBT (1 << 9) /* reference counts */
#define XFS_AG_GEOM_SICK_INODES (1 << 10) /* bad inodes were seen */
/*
* Structures for XFS_IOC_FSGROWFSDATA, XFS_IOC_FSGROWFSLOG & XFS_IOC_FSGROWFSRT
@ -709,9 +712,12 @@ struct xfs_scrub_metadata {
#define XFS_SCRUB_TYPE_GQUOTA 22 /* group quotas */
#define XFS_SCRUB_TYPE_PQUOTA 23 /* project quotas */
#define XFS_SCRUB_TYPE_FSCOUNTERS 24 /* fs summary counters */
#define XFS_SCRUB_TYPE_QUOTACHECK 25 /* quota counters */
#define XFS_SCRUB_TYPE_NLINKS 26 /* inode link counts */
#define XFS_SCRUB_TYPE_HEALTHY 27 /* everything checked out ok */
/* Number of scrub subcommands. */
#define XFS_SCRUB_TYPE_NR 25
#define XFS_SCRUB_TYPE_NR 28
/* i: Repair this metadata. */
#define XFS_SCRUB_IFLAG_REPAIR (1u << 0)

95
fs/xfs/libxfs/xfs_health.h
View File

@ -26,21 +26,40 @@
* and the "sick" field tells us if that piece was found to need repairs.
* Therefore we can conclude that for a given sick flag value:
*
* - checked && sick => metadata needs repair
* - checked && !sick => metadata is ok
* - !checked => has not been examined since mount
* - checked && sick => metadata needs repair
* - checked && !sick => metadata is ok
* - !checked && sick => errors have been observed during normal operation,
* but the metadata has not been checked thoroughly
* - !checked && !sick => has not been examined since mount
*
* Evidence of health problems can be sorted into three basic categories:
*
* a) Primary evidence, which signals that something is defective within the
* general grouping of metadata.
*
* b) Secondary evidence, which are side effects of primary problem but are
* not themselves problems. These can be forgotten when the primary
* health problems are addressed.
*
* c) Indirect evidence, which points to something being wrong in another
* group, but we had to release resources and this is all that's left of
* that state.
*/
struct xfs_mount;
struct xfs_perag;
struct xfs_inode;
struct xfs_fsop_geom;
struct xfs_btree_cur;
struct xfs_da_args;
/* Observable health issues for metadata spanning the entire filesystem. */
#define XFS_SICK_FS_COUNTERS (1 << 0) /* summary counters */
#define XFS_SICK_FS_UQUOTA (1 << 1) /* user quota */
#define XFS_SICK_FS_GQUOTA (1 << 2) /* group quota */
#define XFS_SICK_FS_PQUOTA (1 << 3) /* project quota */
#define XFS_SICK_FS_QUOTACHECK (1 << 4) /* quota counts */
#define XFS_SICK_FS_NLINKS (1 << 5) /* inode link counts */
/* Observable health issues for realtime volume metadata. */
#define XFS_SICK_RT_BITMAP (1 << 0) /* realtime bitmap */
@ -57,6 +76,7 @@ struct xfs_fsop_geom;
#define XFS_SICK_AG_FINOBT (1 << 7) /* free inode index */
#define XFS_SICK_AG_RMAPBT (1 << 8) /* reverse mappings */
#define XFS_SICK_AG_REFCNTBT (1 << 9) /* reference counts */
#define XFS_SICK_AG_INODES (1 << 10) /* inactivated bad inodes */
/* Observable health issues for inode metadata. */
#define XFS_SICK_INO_CORE (1 << 0) /* inode core */
@ -73,11 +93,16 @@ struct xfs_fsop_geom;
#define XFS_SICK_INO_DIR_ZAPPED (1 << 10) /* directory erased */
#define XFS_SICK_INO_SYMLINK_ZAPPED (1 << 11) /* symlink erased */
/* Don't propagate sick status to ag health summary during inactivation */
#define XFS_SICK_INO_FORGET (1 << 12)
/* Primary evidence of health problems in a given group. */
#define XFS_SICK_FS_PRIMARY (XFS_SICK_FS_COUNTERS | \
XFS_SICK_FS_UQUOTA | \
XFS_SICK_FS_GQUOTA | \
XFS_SICK_FS_PQUOTA)
XFS_SICK_FS_PQUOTA | \
XFS_SICK_FS_QUOTACHECK | \
XFS_SICK_FS_NLINKS)
#define XFS_SICK_RT_PRIMARY (XFS_SICK_RT_BITMAP | \
XFS_SICK_RT_SUMMARY)
@ -107,29 +132,86 @@ struct xfs_fsop_geom;
XFS_SICK_INO_DIR_ZAPPED | \
XFS_SICK_INO_SYMLINK_ZAPPED)
/* These functions must be provided by the xfs implementation. */
/* Secondary state related to (but not primary evidence of) health problems. */
#define XFS_SICK_FS_SECONDARY (0)
#define XFS_SICK_RT_SECONDARY (0)
#define XFS_SICK_AG_SECONDARY (0)
#define XFS_SICK_INO_SECONDARY (XFS_SICK_INO_FORGET)
/* Evidence of health problems elsewhere. */
#define XFS_SICK_FS_INDIRECT (0)
#define XFS_SICK_RT_INDIRECT (0)
#define XFS_SICK_AG_INDIRECT (XFS_SICK_AG_INODES)
#define XFS_SICK_INO_INDIRECT (0)
/* All health masks. */
#define XFS_SICK_FS_ALL (XFS_SICK_FS_PRIMARY | \
XFS_SICK_FS_SECONDARY | \
XFS_SICK_FS_INDIRECT)
#define XFS_SICK_RT_ALL (XFS_SICK_RT_PRIMARY | \
XFS_SICK_RT_SECONDARY | \
XFS_SICK_RT_INDIRECT)
#define XFS_SICK_AG_ALL (XFS_SICK_AG_PRIMARY | \
XFS_SICK_AG_SECONDARY | \
XFS_SICK_AG_INDIRECT)
#define XFS_SICK_INO_ALL (XFS_SICK_INO_PRIMARY | \
XFS_SICK_INO_SECONDARY | \
XFS_SICK_INO_INDIRECT | \
XFS_SICK_INO_ZAPPED)
/*
* These functions must be provided by the xfs implementation. Function
* behavior with respect to the first argument should be as follows:
*
* xfs_*_mark_sick: Set the sick flags and do not set checked flags.
* Runtime code should call this upon encountering
* a corruption.
*
* xfs_*_mark_corrupt: Set the sick and checked flags simultaneously.
* Fsck tools should call this when corruption is
* found.
*
* xfs_*_mark_healthy: Clear the sick flags and set the checked flags.
* Fsck tools should call this after correcting errors.
*
* xfs_*_measure_sickness: Return the sick and check status in the provided
* out parameters.
*/
void xfs_fs_mark_sick(struct xfs_mount *mp, unsigned int mask);
void xfs_fs_mark_corrupt(struct xfs_mount *mp, unsigned int mask);
void xfs_fs_mark_healthy(struct xfs_mount *mp, unsigned int mask);
void xfs_fs_measure_sickness(struct xfs_mount *mp, unsigned int *sick,
unsigned int *checked);
void xfs_rt_mark_sick(struct xfs_mount *mp, unsigned int mask);
void xfs_rt_mark_corrupt(struct xfs_mount *mp, unsigned int mask);
void xfs_rt_mark_healthy(struct xfs_mount *mp, unsigned int mask);
void xfs_rt_measure_sickness(struct xfs_mount *mp, unsigned int *sick,
unsigned int *checked);
void xfs_agno_mark_sick(struct xfs_mount *mp, xfs_agnumber_t agno,
unsigned int mask);
void xfs_ag_mark_sick(struct xfs_perag *pag, unsigned int mask);
void xfs_ag_mark_corrupt(struct xfs_perag *pag, unsigned int mask);
void xfs_ag_mark_healthy(struct xfs_perag *pag, unsigned int mask);
void xfs_ag_measure_sickness(struct xfs_perag *pag, unsigned int *sick,
unsigned int *checked);
void xfs_inode_mark_sick(struct xfs_inode *ip, unsigned int mask);
void xfs_inode_mark_corrupt(struct xfs_inode *ip, unsigned int mask);
void xfs_inode_mark_healthy(struct xfs_inode *ip, unsigned int mask);
void xfs_inode_measure_sickness(struct xfs_inode *ip, unsigned int *sick,
unsigned int *checked);
void xfs_health_unmount(struct xfs_mount *mp);
void xfs_bmap_mark_sick(struct xfs_inode *ip, int whichfork);
void xfs_btree_mark_sick(struct xfs_btree_cur *cur);
void xfs_dirattr_mark_sick(struct xfs_inode *ip, int whichfork);
void xfs_da_mark_sick(struct xfs_da_args *args);
/* Now some helpers. */
@ -197,4 +279,7 @@ void xfs_fsop_geom_health(struct xfs_mount *mp, struct xfs_fsop_geom *geo);
void xfs_ag_geom_health(struct xfs_perag *pag, struct xfs_ag_geometry *ageo);
void xfs_bulkstat_health(struct xfs_inode *ip, struct xfs_bulkstat *bs);
#define xfs_metadata_is_sick(error) \
(unlikely((error) == -EFSCORRUPTED || (error) == -EFSBADCRC))
#endif /* __XFS_HEALTH_H__ */

240
fs/xfs/libxfs/xfs_ialloc.c
View File

@ -27,6 +27,7 @@
#include "xfs_log.h"
#include "xfs_rmap.h"
#include "xfs_ag.h"
#include "xfs_health.h"
/*
* Lookup a record by ino in the btree given by cur.
@ -140,13 +141,13 @@ xfs_inobt_complain_bad_rec(
struct xfs_mount *mp = cur->bc_mp;
xfs_warn(mp,
"%s Inode BTree record corruption in AG %d detected at %pS!",
cur->bc_btnum == XFS_BTNUM_INO ? "Used" : "Free",
cur->bc_ag.pag->pag_agno, fa);
"%sbt record corruption in AG %d detected at %pS!",
cur->bc_ops->name, cur->bc_ag.pag->pag_agno, fa);
xfs_warn(mp,
"start inode 0x%x, count 0x%x, free 0x%x freemask 0x%llx, holemask 0x%x",
irec->ir_startino, irec->ir_count, irec->ir_freecount,
irec->ir_free, irec->ir_holemask);
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
@ -205,14 +206,17 @@ xfs_inobt_insert(
struct xfs_buf *agbp,
xfs_agino_t newino,
xfs_agino_t newlen,
xfs_btnum_t btnum)
bool is_finobt)
{
struct xfs_btree_cur *cur;
xfs_agino_t thisino;
int i;
int error;
cur = xfs_inobt_init_cursor(pag, tp, agbp, btnum);
if (is_finobt)
cur = xfs_finobt_init_cursor(pag, tp, agbp);
else
cur = xfs_inobt_init_cursor(pag, tp, agbp);
for (thisino = newino;
thisino < newino + newlen;
@ -528,16 +532,14 @@ __xfs_inobt_rec_merge(
}
/*
* Insert a new sparse inode chunk into the associated inode btree. The inode
* record for the sparse chunk is pre-aligned to a startino that should match
* any pre-existing sparse inode record in the tree. This allows sparse chunks
* to fill over time.
* Insert a new sparse inode chunk into the associated inode allocation btree.
* The inode record for the sparse chunk is pre-aligned to a startino that
* should match any pre-existing sparse inode record in the tree. This allows
* sparse chunks to fill over time.
*
* This function supports two modes of handling preexisting records depending on
* the merge flag. If merge is true, the provided record is merged with the
* If no preexisting record exists, the provided record is inserted.
* If there is a preexisting record, the provided record is merged with the
* existing record and updated in place. The merged record is returned in nrec.
* If merge is false, an existing record is replaced with the provided record.
* If no preexisting record exists, the provided record is always inserted.
*
* It is considered corruption if a merge is requested and not possible. Given
* the sparse inode alignment constraints, this should never happen.
@ -547,9 +549,7 @@ xfs_inobt_insert_sprec(
struct xfs_perag *pag,
struct xfs_trans *tp,
struct xfs_buf *agbp,
int btnum,
struct xfs_inobt_rec_incore *nrec, /* in/out: new/merged rec. */
bool merge) /* merge or replace */
struct xfs_inobt_rec_incore *nrec) /* in/out: new/merged rec. */
{
struct xfs_mount *mp = pag->pag_mount;
struct xfs_btree_cur *cur;
@ -557,7 +557,7 @@ xfs_inobt_insert_sprec(
int i;
struct xfs_inobt_rec_incore rec;
cur = xfs_inobt_init_cursor(pag, tp, agbp, btnum);
cur = xfs_inobt_init_cursor(pag, tp, agbp);
/* the new record is pre-aligned so we know where to look */
error = xfs_inobt_lookup(cur, nrec->ir_startino, XFS_LOOKUP_EQ, &i);
@ -571,6 +571,7 @@ xfs_inobt_insert_sprec(
if (error)
goto error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
@ -579,45 +580,45 @@ xfs_inobt_insert_sprec(
}
/*
* A record exists at this startino. Merge or replace the record
* depending on what we've been asked to do.
* A record exists at this startino. Merge the records.
*/
if (merge) {
error = xfs_inobt_get_rec(cur, &rec, &i);
if (error)
goto error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
error = -EFSCORRUPTED;
goto error;
}
if (XFS_IS_CORRUPT(mp, rec.ir_startino != nrec->ir_startino)) {
error = -EFSCORRUPTED;
goto error;
}
/*
* This should never fail. If we have coexisting records that
* cannot merge, something is seriously wrong.
*/
if (XFS_IS_CORRUPT(mp, !__xfs_inobt_can_merge(nrec, &rec))) {
error = -EFSCORRUPTED;
goto error;
}
trace_xfs_irec_merge_pre(mp, pag->pag_agno, rec.ir_startino,
rec.ir_holemask, nrec->ir_startino,
nrec->ir_holemask);
/* merge to nrec to output the updated record */
__xfs_inobt_rec_merge(nrec, &rec);
trace_xfs_irec_merge_post(mp, pag->pag_agno, nrec->ir_startino,
nrec->ir_holemask);
error = xfs_inobt_rec_check_count(mp, nrec);
if (error)
goto error;
error = xfs_inobt_get_rec(cur, &rec, &i);
if (error)
goto error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
if (XFS_IS_CORRUPT(mp, rec.ir_startino != nrec->ir_startino)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
/*
* This should never fail. If we have coexisting records that
* cannot merge, something is seriously wrong.
*/
if (XFS_IS_CORRUPT(mp, !__xfs_inobt_can_merge(nrec, &rec))) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
trace_xfs_irec_merge_pre(mp, pag->pag_agno, rec.ir_startino,
rec.ir_holemask, nrec->ir_startino,
nrec->ir_holemask);
/* merge to nrec to output the updated record */
__xfs_inobt_rec_merge(nrec, &rec);
trace_xfs_irec_merge_post(mp, pag->pag_agno, nrec->ir_startino,
nrec->ir_holemask);
error = xfs_inobt_rec_check_count(mp, nrec);
if (error)
goto error;
error = xfs_inobt_update(cur, nrec);
if (error)
@ -631,6 +632,59 @@ error:
return error;
}
/*
* Insert a new sparse inode chunk into the free inode btree. The inode
* record for the sparse chunk is pre-aligned to a startino that should match
* any pre-existing sparse inode record in the tree. This allows sparse chunks
* to fill over time.
*
* The new record is always inserted, overwriting a pre-existing record if
* there is one.
*/
STATIC int
xfs_finobt_insert_sprec(
struct xfs_perag *pag,
struct xfs_trans *tp,
struct xfs_buf *agbp,
struct xfs_inobt_rec_incore *nrec) /* in/out: new rec. */
{
struct xfs_mount *mp = pag->pag_mount;
struct xfs_btree_cur *cur;
int error;
int i;
cur = xfs_finobt_init_cursor(pag, tp, agbp);
/* the new record is pre-aligned so we know where to look */
error = xfs_inobt_lookup(cur, nrec->ir_startino, XFS_LOOKUP_EQ, &i);
if (error)
goto error;
/* if nothing there, insert a new record and return */
if (i == 0) {
error = xfs_inobt_insert_rec(cur, nrec->ir_holemask,
nrec->ir_count, nrec->ir_freecount,
nrec->ir_free, &i);
if (error)
goto error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
} else {
error = xfs_inobt_update(cur, nrec);
if (error)
goto error;
}
xfs_btree_del_cursor(cur, XFS_BTREE_NOERROR);
return 0;
error:
xfs_btree_del_cursor(cur, XFS_BTREE_ERROR);
return error;
}
/*
* Allocate new inodes in the allocation group specified by agbp. Returns 0 if
* inodes were allocated in this AG; -EAGAIN if there was no space in this AG so
@ -857,8 +911,7 @@ sparse_alloc:
* if necessary. If a merge does occur, rec is updated to the
* merged record.
*/
error = xfs_inobt_insert_sprec(pag, tp, agbp,
XFS_BTNUM_INO, &rec, true);
error = xfs_inobt_insert_sprec(pag, tp, agbp, &rec);
if (error == -EFSCORRUPTED) {
xfs_alert(args.mp,
"invalid sparse inode record: ino 0x%llx holemask 0x%x count %u",
@ -882,21 +935,19 @@ sparse_alloc:
* existing record with this one.
*/
if (xfs_has_finobt(args.mp)) {
error = xfs_inobt_insert_sprec(pag, tp, agbp,
XFS_BTNUM_FINO, &rec, false);
error = xfs_finobt_insert_sprec(pag, tp, agbp, &rec);
if (error)
return error;
}
} else {
/* full chunk - insert new records to both btrees */
error = xfs_inobt_insert(pag, tp, agbp, newino, newlen,
XFS_BTNUM_INO);
error = xfs_inobt_insert(pag, tp, agbp, newino, newlen, false);
if (error)
return error;
if (xfs_has_finobt(args.mp)) {
error = xfs_inobt_insert(pag, tp, agbp, newino,
newlen, XFS_BTNUM_FINO);
newlen, true);
if (error)
return error;
}
@ -949,8 +1000,10 @@ xfs_ialloc_next_rec(
error = xfs_inobt_get_rec(cur, rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
}
return 0;
@ -974,8 +1027,10 @@ xfs_ialloc_get_rec(
error = xfs_inobt_get_rec(cur, rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
}
return 0;
@ -1030,7 +1085,7 @@ xfs_dialloc_ag_inobt(
ASSERT(pag->pagi_freecount > 0);
restart_pagno:
cur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_INO);
cur = xfs_inobt_init_cursor(pag, tp, agbp);
/*
* If pagino is 0 (this is the root inode allocation) use newino.
* This must work because we've just allocated some.
@ -1053,6 +1108,7 @@ xfs_dialloc_ag_inobt(
if (error)
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1061,6 +1117,7 @@ xfs_dialloc_ag_inobt(
if (error)
goto error0;
if (XFS_IS_CORRUPT(mp, j != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1219,6 +1276,7 @@ xfs_dialloc_ag_inobt(
if (error)
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1228,6 +1286,7 @@ xfs_dialloc_ag_inobt(
if (error)
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1237,6 +1296,7 @@ xfs_dialloc_ag_inobt(
if (error)
goto error0;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1297,8 +1357,10 @@ xfs_dialloc_ag_finobt_near(
error = xfs_inobt_get_rec(lcur, rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(lcur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(lcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(lcur);
return -EFSCORRUPTED;
}
/*
* See if we've landed in the parent inode record. The finobt
@ -1322,12 +1384,14 @@ xfs_dialloc_ag_finobt_near(
if (error)
goto error_rcur;
if (XFS_IS_CORRUPT(lcur->bc_mp, j != 1)) {
xfs_btree_mark_sick(lcur);
error = -EFSCORRUPTED;
goto error_rcur;
}
}
if (XFS_IS_CORRUPT(lcur->bc_mp, i != 1 && j != 1)) {
xfs_btree_mark_sick(lcur);
error = -EFSCORRUPTED;
goto error_rcur;
}
@ -1383,8 +1447,10 @@ xfs_dialloc_ag_finobt_newino(
error = xfs_inobt_get_rec(cur, rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
return 0;
}
}
@ -1395,14 +1461,18 @@ xfs_dialloc_ag_finobt_newino(
error = xfs_inobt_lookup(cur, 0, XFS_LOOKUP_GE, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
error = xfs_inobt_get_rec(cur, rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
return 0;
}
@ -1424,14 +1494,18 @@ xfs_dialloc_ag_update_inobt(
error = xfs_inobt_lookup(cur, frec->ir_startino, XFS_LOOKUP_EQ, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
error = xfs_inobt_get_rec(cur, &rec, &i);
if (error)
return error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1))
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
ASSERT((XFS_AGINO_TO_OFFSET(cur->bc_mp, rec.ir_startino) %
XFS_INODES_PER_CHUNK) == 0);
@ -1440,8 +1514,10 @@ xfs_dialloc_ag_update_inobt(
if (XFS_IS_CORRUPT(cur->bc_mp,
rec.ir_free != frec->ir_free ||
rec.ir_freecount != frec->ir_freecount))
rec.ir_freecount != frec->ir_freecount)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
return xfs_inobt_update(cur, &rec);
}
@ -1483,7 +1559,7 @@ xfs_dialloc_ag(
if (!pagino)
pagino = be32_to_cpu(agi->agi_newino);
cur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_FINO);
cur = xfs_finobt_init_cursor(pag, tp, agbp);
error = xfs_check_agi_freecount(cur);
if (error)
@ -1526,7 +1602,7 @@ xfs_dialloc_ag(
* the original freecount. If all is well, make the equivalent update to
* the inobt using the finobt record and offset information.
*/
icur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_INO);
icur = xfs_inobt_init_cursor(pag, tp, agbp);
error = xfs_check_agi_freecount(icur);
if (error)
@ -1943,7 +2019,7 @@ xfs_difree_inobt(
/*
* Initialize the cursor.
*/
cur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_INO);
cur = xfs_inobt_init_cursor(pag, tp, agbp);
error = xfs_check_agi_freecount(cur);
if (error)
@ -1958,6 +2034,7 @@ xfs_difree_inobt(
goto error0;
}
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -1968,6 +2045,7 @@ xfs_difree_inobt(
goto error0;
}
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error0;
}
@ -2068,7 +2146,7 @@ xfs_difree_finobt(
int error;
int i;
cur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_FINO);
cur = xfs_finobt_init_cursor(pag, tp, agbp);
error = xfs_inobt_lookup(cur, ibtrec->ir_startino, XFS_LOOKUP_EQ, &i);
if (error)
@ -2080,6 +2158,7 @@ xfs_difree_finobt(
* something is out of sync.
*/
if (XFS_IS_CORRUPT(mp, ibtrec->ir_freecount != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
@ -2106,6 +2185,7 @@ xfs_difree_finobt(
if (error)
goto error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
@ -2116,6 +2196,7 @@ xfs_difree_finobt(
if (XFS_IS_CORRUPT(mp,
rec.ir_free != ibtrec->ir_free ||
rec.ir_freecount != ibtrec->ir_freecount)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto error;
}
@ -2265,7 +2346,7 @@ xfs_imap_lookup(
* we have a record, we need to ensure it contains the inode number
* we are looking up.
*/
cur = xfs_inobt_init_cursor(pag, tp, agbp, XFS_BTNUM_INO);
cur = xfs_inobt_init_cursor(pag, tp, agbp);
error = xfs_inobt_lookup(cur, agino, XFS_LOOKUP_LE, &i);
if (!error) {
if (i)
@ -2604,6 +2685,8 @@ xfs_read_agi(
error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp,
XFS_AG_DADDR(mp, pag->pag_agno, XFS_AGI_DADDR(mp)),
XFS_FSS_TO_BB(mp, 1), 0, agibpp, &xfs_agi_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_ag_mark_sick(pag, XFS_SICK_AG_AGI);
if (error)
return error;
if (tp)
@ -2765,7 +2848,7 @@ xfs_ialloc_count_inodes(
struct xfs_ialloc_count_inodes ci = {0};
int error;
ASSERT(cur->bc_btnum == XFS_BTNUM_INO);
ASSERT(xfs_btree_is_ino(cur->bc_ops));
error = xfs_btree_query_all(cur, xfs_ialloc_count_inodes_rec, &ci);
if (error)
return error;
@ -2982,7 +3065,7 @@ xfs_ialloc_check_shrink(
if (!xfs_has_sparseinodes(pag->pag_mount))
return 0;
cur = xfs_inobt_init_cursor(pag, tp, agibp, XFS_BTNUM_INO);
cur = xfs_inobt_init_cursor(pag, tp, agibp);
/* Look up the inobt record that would correspond to the new EOFS. */
agino = XFS_AGB_TO_AGINO(pag->pag_mount, new_length);
@ -2995,6 +3078,7 @@ xfs_ialloc_check_shrink(
goto out;
if (!has) {
xfs_ag_mark_sick(pag, XFS_SICK_AG_INOBT);
error = -EFSCORRUPTED;
goto out;
}

159
fs/xfs/libxfs/xfs_ialloc_btree.c
View File

@ -17,6 +17,7 @@
#include "xfs_ialloc_btree.h"
#include "xfs_alloc.h"
#include "xfs_error.h"
#include "xfs_health.h"
#include "xfs_trace.h"
#include "xfs_trans.h"
#include "xfs_rmap.h"
@ -37,7 +38,15 @@ xfs_inobt_dup_cursor(
struct xfs_btree_cur *cur)
{
return xfs_inobt_init_cursor(cur->bc_ag.pag, cur->bc_tp,
cur->bc_ag.agbp, cur->bc_btnum);
cur->bc_ag.agbp);
}
STATIC struct xfs_btree_cur *
xfs_finobt_dup_cursor(
struct xfs_btree_cur *cur)
{
return xfs_finobt_init_cursor(cur->bc_ag.pag, cur->bc_tp,
cur->bc_ag.agbp);
}
STATIC void
@ -81,9 +90,9 @@ xfs_inobt_mod_blockcount(
if (!xfs_has_inobtcounts(cur->bc_mp))
return;
if (cur->bc_btnum == XFS_BTNUM_FINO)
if (xfs_btree_is_fino(cur->bc_ops))
be32_add_cpu(&agi->agi_fblocks, howmuch);
else if (cur->bc_btnum == XFS_BTNUM_INO)
else
be32_add_cpu(&agi->agi_iblocks, howmuch);
xfs_ialloc_log_agi(cur->bc_tp, agbp, XFS_AGI_IBLOCKS);
}
@ -300,7 +309,7 @@ xfs_inobt_verify(
* xfs_perag_initialised_agi(pag)) if we ever do.
*/
if (xfs_has_crc(mp)) {
fa = xfs_btree_sblock_v5hdr_verify(bp);
fa = xfs_btree_agblock_v5hdr_verify(bp);
if (fa)
return fa;
}
@ -310,7 +319,7 @@ xfs_inobt_verify(
if (level >= M_IGEO(mp)->inobt_maxlevels)
return __this_address;
return xfs_btree_sblock_verify(bp,
return xfs_btree_agblock_verify(bp,
M_IGEO(mp)->inobt_mxr[level != 0]);
}
@ -320,7 +329,7 @@ xfs_inobt_read_verify(
{
xfs_failaddr_t fa;
if (!xfs_btree_sblock_verify_crc(bp))
if (!xfs_btree_agblock_verify_crc(bp))
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_inobt_verify(bp);
@ -344,7 +353,7 @@ xfs_inobt_write_verify(
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
return;
}
xfs_btree_sblock_calc_crc(bp);
xfs_btree_agblock_calc_crc(bp);
}
@ -398,9 +407,17 @@ xfs_inobt_keys_contiguous(
be32_to_cpu(key2->inobt.ir_startino));
}
static const struct xfs_btree_ops xfs_inobt_ops = {
const struct xfs_btree_ops xfs_inobt_ops = {
.name = "ino",
.type = XFS_BTREE_TYPE_AG,
.rec_len = sizeof(xfs_inobt_rec_t),
.key_len = sizeof(xfs_inobt_key_t),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.lru_refs = XFS_INO_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_ibt_2),
.sick_mask = XFS_SICK_AG_INOBT,
.dup_cursor = xfs_inobt_dup_cursor,
.set_root = xfs_inobt_set_root,
@ -420,11 +437,19 @@ static const struct xfs_btree_ops xfs_inobt_ops = {
.keys_contiguous = xfs_inobt_keys_contiguous,
};
static const struct xfs_btree_ops xfs_finobt_ops = {
const struct xfs_btree_ops xfs_finobt_ops = {
.name = "fino",
.type = XFS_BTREE_TYPE_AG,
.rec_len = sizeof(xfs_inobt_rec_t),
.key_len = sizeof(xfs_inobt_key_t),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.dup_cursor = xfs_inobt_dup_cursor,
.lru_refs = XFS_INO_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_fibt_2),
.sick_mask = XFS_SICK_AG_FINOBT,
.dup_cursor = xfs_finobt_dup_cursor,
.set_root = xfs_finobt_set_root,
.alloc_block = xfs_finobt_alloc_block,
.free_block = xfs_finobt_free_block,
@ -443,65 +468,54 @@ static const struct xfs_btree_ops xfs_finobt_ops = {
};
/*
* Initialize a new inode btree cursor.
* Create an inode btree cursor.
*
* For staging cursors tp and agbp are NULL.
*/
static struct xfs_btree_cur *
xfs_inobt_init_common(
struct xfs_perag *pag,
struct xfs_trans *tp, /* transaction pointer */
xfs_btnum_t btnum) /* ialloc or free ino btree */
{
struct xfs_mount *mp = pag->pag_mount;
struct xfs_btree_cur *cur;
cur = xfs_btree_alloc_cursor(mp, tp, btnum,
M_IGEO(mp)->inobt_maxlevels, xfs_inobt_cur_cache);
if (btnum == XFS_BTNUM_INO) {
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_ibt_2);
cur->bc_ops = &xfs_inobt_ops;
} else {
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_fibt_2);
cur->bc_ops = &xfs_finobt_ops;
}
if (xfs_has_crc(mp))
cur->bc_flags |= XFS_BTREE_CRC_BLOCKS;
cur->bc_ag.pag = xfs_perag_hold(pag);
return cur;
}
/* Create an inode btree cursor. */
struct xfs_btree_cur *
xfs_inobt_init_cursor(
struct xfs_perag *pag,
struct xfs_trans *tp,
struct xfs_buf *agbp,
xfs_btnum_t btnum)
struct xfs_buf *agbp)
{
struct xfs_mount *mp = pag->pag_mount;
struct xfs_btree_cur *cur;
struct xfs_agi *agi = agbp->b_addr;
cur = xfs_inobt_init_common(pag, tp, btnum);
if (btnum == XFS_BTNUM_INO)
cur->bc_nlevels = be32_to_cpu(agi->agi_level);
else
cur->bc_nlevels = be32_to_cpu(agi->agi_free_level);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_inobt_ops,
M_IGEO(mp)->inobt_maxlevels, xfs_inobt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.agbp = agbp;
if (agbp) {
struct xfs_agi *agi = agbp->b_addr;
cur->bc_nlevels = be32_to_cpu(agi->agi_level);
}
return cur;
}
/* Create an inode btree cursor with a fake root for staging. */
/*
* Create a free inode btree cursor.
*
* For staging cursors tp and agbp are NULL.
*/
struct xfs_btree_cur *
xfs_inobt_stage_cursor(
xfs_finobt_init_cursor(
struct xfs_perag *pag,
struct xbtree_afakeroot *afake,
xfs_btnum_t btnum)
struct xfs_trans *tp,
struct xfs_buf *agbp)
{
struct xfs_mount *mp = pag->pag_mount;
struct xfs_btree_cur *cur;
cur = xfs_inobt_init_common(pag, NULL, btnum);
xfs_btree_stage_afakeroot(cur, afake);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_finobt_ops,
M_IGEO(mp)->inobt_maxlevels, xfs_inobt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.agbp = agbp;
if (agbp) {
struct xfs_agi *agi = agbp->b_addr;
cur->bc_nlevels = be32_to_cpu(agi->agi_free_level);
}
return cur;
}
@ -521,7 +535,7 @@ xfs_inobt_commit_staged_btree(
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
if (cur->bc_btnum == XFS_BTNUM_INO) {
if (xfs_btree_is_ino(cur->bc_ops)) {
fields = XFS_AGI_ROOT | XFS_AGI_LEVEL;
agi->agi_root = cpu_to_be32(afake->af_root);
agi->agi_level = cpu_to_be32(afake->af_levels);
@ -530,7 +544,7 @@ xfs_inobt_commit_staged_btree(
fields |= XFS_AGI_IBLOCKS;
}
xfs_ialloc_log_agi(tp, agbp, fields);
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_inobt_ops);
xfs_btree_commit_afakeroot(cur, tp, agbp);
} else {
fields = XFS_AGI_FREE_ROOT | XFS_AGI_FREE_LEVEL;
agi->agi_free_root = cpu_to_be32(afake->af_root);
@ -540,7 +554,7 @@ xfs_inobt_commit_staged_btree(
fields |= XFS_AGI_IBLOCKS;
}
xfs_ialloc_log_agi(tp, agbp, fields);
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_finobt_ops);
xfs_btree_commit_afakeroot(cur, tp, agbp);
}
}
@ -721,45 +735,21 @@ xfs_inobt_max_size(
XFS_INODES_PER_CHUNK);
}
/* Read AGI and create inobt cursor. */
int
xfs_inobt_cur(
struct xfs_perag *pag,
struct xfs_trans *tp,
xfs_btnum_t which,
struct xfs_btree_cur **curpp,
struct xfs_buf **agi_bpp)
{
struct xfs_btree_cur *cur;
int error;
ASSERT(*agi_bpp == NULL);
ASSERT(*curpp == NULL);
error = xfs_ialloc_read_agi(pag, tp, agi_bpp);
if (error)
return error;
cur = xfs_inobt_init_cursor(pag, tp, *agi_bpp, which);
*curpp = cur;
return 0;
}
static int
xfs_inobt_count_blocks(
xfs_finobt_count_blocks(
struct xfs_perag *pag,
struct xfs_trans *tp,
xfs_btnum_t btnum,
xfs_extlen_t *tree_blocks)
{
struct xfs_buf *agbp = NULL;
struct xfs_btree_cur *cur = NULL;
struct xfs_btree_cur *cur;
int error;
error = xfs_inobt_cur(pag, tp, btnum, &cur, &agbp);
error = xfs_ialloc_read_agi(pag, tp, &agbp);
if (error)
return error;
cur = xfs_inobt_init_cursor(pag, tp, agbp);
error = xfs_btree_count_blocks(cur, tree_blocks);
xfs_btree_del_cursor(cur, error);
xfs_trans_brelse(tp, agbp);
@ -807,8 +797,7 @@ xfs_finobt_calc_reserves(
if (xfs_has_inobtcounts(pag->pag_mount))
error = xfs_finobt_read_blocks(pag, tp, &tree_len);
else
error = xfs_inobt_count_blocks(pag, tp, XFS_BTNUM_FINO,
&tree_len);
error = xfs_finobt_count_blocks(pag, tp, &tree_len);
if (error)
return error;

11
fs/xfs/libxfs/xfs_ialloc_btree.h
View File

@ -46,10 +46,10 @@ struct xfs_perag;
(maxrecs) * sizeof(xfs_inobt_key_t) + \
((index) - 1) * sizeof(xfs_inobt_ptr_t)))
extern struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_perag *pag,
struct xfs_trans *tp, struct xfs_buf *agbp, xfs_btnum_t btnum);
struct xfs_btree_cur *xfs_inobt_stage_cursor(struct xfs_perag *pag,
struct xbtree_afakeroot *afake, xfs_btnum_t btnum);
struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_perag *pag,
struct xfs_trans *tp, struct xfs_buf *agbp);
struct xfs_btree_cur *xfs_finobt_init_cursor(struct xfs_perag *pag,
struct xfs_trans *tp, struct xfs_buf *agbp);
extern int xfs_inobt_maxrecs(struct xfs_mount *, int, int);
/* ir_holemask to inode allocation bitmap conversion */
@ -66,9 +66,6 @@ int xfs_finobt_calc_reserves(struct xfs_perag *perag, struct xfs_trans *tp,
xfs_extlen_t *ask, xfs_extlen_t *used);
extern xfs_extlen_t xfs_iallocbt_calc_size(struct xfs_mount *mp,
unsigned long long len);
int xfs_inobt_cur(struct xfs_perag *pag, struct xfs_trans *tp,
xfs_btnum_t btnum, struct xfs_btree_cur **curpp,
struct xfs_buf **agi_bpp);
void xfs_inobt_commit_staged_btree(struct xfs_btree_cur *cur,
struct xfs_trans *tp, struct xfs_buf *agbp);

26
fs/xfs/libxfs/xfs_iext_tree.c
View File

@ -394,11 +394,18 @@ xfs_iext_leaf_key(
return leaf->recs[n].lo & XFS_IEXT_STARTOFF_MASK;
}
static inline void *
xfs_iext_alloc_node(
int size)
{
return kzalloc(size, GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
}
static void
xfs_iext_grow(
struct xfs_ifork *ifp)
{
struct xfs_iext_node *node = kmem_zalloc(NODE_SIZE, KM_NOFS);
struct xfs_iext_node *node = xfs_iext_alloc_node(NODE_SIZE);
int i;
if (ifp->if_height == 1) {
@ -454,7 +461,7 @@ xfs_iext_split_node(
int *nr_entries)
{
struct xfs_iext_node *node = *nodep;
struct xfs_iext_node *new = kmem_zalloc(NODE_SIZE, KM_NOFS);
struct xfs_iext_node *new = xfs_iext_alloc_node(NODE_SIZE);
const int nr_move = KEYS_PER_NODE / 2;
int nr_keep = nr_move + (KEYS_PER_NODE & 1);
int i = 0;
@ -542,7 +549,7 @@ xfs_iext_split_leaf(
int *nr_entries)
{
struct xfs_iext_leaf *leaf = cur->leaf;
struct xfs_iext_leaf *new = kmem_zalloc(NODE_SIZE, KM_NOFS);
struct xfs_iext_leaf *new = xfs_iext_alloc_node(NODE_SIZE);
const int nr_move = RECS_PER_LEAF / 2;
int nr_keep = nr_move + (RECS_PER_LEAF & 1);
int i;
@ -583,7 +590,7 @@ xfs_iext_alloc_root(
{
ASSERT(ifp->if_bytes == 0);
ifp->if_data = kmem_zalloc(sizeof(struct xfs_iext_rec), KM_NOFS);
ifp->if_data = xfs_iext_alloc_node(sizeof(struct xfs_iext_rec));
ifp->if_height = 1;
/* now that we have a node step into it */
@ -603,7 +610,8 @@ xfs_iext_realloc_root(
if (new_size / sizeof(struct xfs_iext_rec) == RECS_PER_LEAF)
new_size = NODE_SIZE;
new = krealloc(ifp->if_data, new_size, GFP_NOFS | __GFP_NOFAIL);
new = krealloc(ifp->if_data, new_size,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
memset(new + ifp->if_bytes, 0, new_size - ifp->if_bytes);
ifp->if_data = new;
cur->leaf = new;
@ -743,7 +751,7 @@ xfs_iext_remove_node(
again:
ASSERT(node->ptrs[pos]);
ASSERT(node->ptrs[pos] == victim);
kmem_free(victim);
kfree(victim);
nr_entries = xfs_iext_node_nr_entries(node, pos) - 1;
offset = node->keys[0];
@ -789,7 +797,7 @@ again:
ASSERT(node == ifp->if_data);
ifp->if_data = node->ptrs[0];
ifp->if_height--;
kmem_free(node);
kfree(node);
}
}
@ -863,7 +871,7 @@ xfs_iext_free_last_leaf(
struct xfs_ifork *ifp)
{
ifp->if_height--;
kmem_free(ifp->if_data);
kfree(ifp->if_data);
ifp->if_data = NULL;
}
@ -1044,7 +1052,7 @@ xfs_iext_destroy_node(
}
}
kmem_free(node);
kfree(node);
}
void

12
fs/xfs/libxfs/xfs_inode_buf.c
View File

@ -18,6 +18,7 @@
#include "xfs_trans.h"
#include "xfs_ialloc.h"
#include "xfs_dir2.h"
#include "xfs_health.h"
#include <linux/iversion.h>
@ -132,9 +133,14 @@ xfs_imap_to_bp(
struct xfs_imap *imap,
struct xfs_buf **bpp)
{
return xfs_trans_read_buf(mp, tp, mp->m_ddev_targp, imap->im_blkno,
imap->im_len, XBF_UNMAPPED, bpp,
&xfs_inode_buf_ops);
int error;
error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp, imap->im_blkno,
imap->im_len, XBF_UNMAPPED, bpp, &xfs_inode_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_agno_mark_sick(mp, xfs_daddr_to_agno(mp, imap->im_blkno),
XFS_SICK_AG_INODES);
return error;
}
static inline struct timespec64 xfs_inode_decode_bigtime(uint64_t ts)

49
fs/xfs/libxfs/xfs_inode_fork.c
View File

@ -25,6 +25,8 @@
#include "xfs_attr_leaf.h"
#include "xfs_types.h"
#include "xfs_errortag.h"
#include "xfs_health.h"
#include "xfs_symlink_remote.h"
struct kmem_cache *xfs_ifork_cache;
@ -50,7 +52,8 @@ xfs_init_local_fork(
mem_size++;
if (size) {
char *new_data = kmem_alloc(mem_size, KM_NOFS);
char *new_data = kmalloc(mem_size,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
memcpy(new_data, data, size);
if (zero_terminate)
@ -77,7 +80,7 @@ xfs_iformat_local(
/*
* If the size is unreasonable, then something
* is wrong and we just bail out rather than crash in
* kmem_alloc() or memcpy() below.
* kmalloc() or memcpy() below.
*/
if (unlikely(size > XFS_DFORK_SIZE(dip, ip->i_mount, whichfork))) {
xfs_warn(ip->i_mount,
@ -87,6 +90,7 @@ xfs_iformat_local(
xfs_inode_verifier_error(ip, -EFSCORRUPTED,
"xfs_iformat_local", dip, sizeof(*dip),
__this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return -EFSCORRUPTED;
}
@ -116,7 +120,7 @@ xfs_iformat_extents(
/*
* If the number of extents is unreasonable, then something is wrong and
* we just bail out rather than crash in kmem_alloc() or memcpy() below.
* we just bail out rather than crash in kmalloc() or memcpy() below.
*/
if (unlikely(size < 0 || size > XFS_DFORK_SIZE(dip, mp, whichfork))) {
xfs_warn(ip->i_mount, "corrupt inode %llu ((a)extents = %llu).",
@ -124,6 +128,7 @@ xfs_iformat_extents(
xfs_inode_verifier_error(ip, -EFSCORRUPTED,
"xfs_iformat_extents(1)", dip, sizeof(*dip),
__this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return -EFSCORRUPTED;
}
@ -143,6 +148,7 @@ xfs_iformat_extents(
xfs_inode_verifier_error(ip, -EFSCORRUPTED,
"xfs_iformat_extents(2)",
dp, sizeof(*dp), fa);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return xfs_bmap_complain_bad_rec(ip, whichfork,
fa, &new);
}
@ -201,11 +207,13 @@ xfs_iformat_btree(
xfs_inode_verifier_error(ip, -EFSCORRUPTED,
"xfs_iformat_btree", dfp, size,
__this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return -EFSCORRUPTED;
}
ifp->if_broot_bytes = size;
ifp->if_broot = kmem_alloc(size, KM_NOFS);
ifp->if_broot = kmalloc(size,
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
ASSERT(ifp->if_broot != NULL);
/*
* Copy and convert from the on-disk structure
@ -265,12 +273,14 @@ xfs_iformat_data_fork(
default:
xfs_inode_verifier_error(ip, -EFSCORRUPTED, __func__,
dip, sizeof(*dip), __this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return -EFSCORRUPTED;
}
break;
default:
xfs_inode_verifier_error(ip, -EFSCORRUPTED, __func__, dip,
sizeof(*dip), __this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
return -EFSCORRUPTED;
}
}
@ -342,6 +352,7 @@ xfs_iformat_attr_fork(
default:
xfs_inode_verifier_error(ip, error, __func__, dip,
sizeof(*dip), __this_address);
xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE);
error = -EFSCORRUPTED;
break;
}
@ -399,7 +410,8 @@ xfs_iroot_realloc(
*/
if (ifp->if_broot_bytes == 0) {
new_size = XFS_BMAP_BROOT_SPACE_CALC(mp, rec_diff);
ifp->if_broot = kmem_alloc(new_size, KM_NOFS);
ifp->if_broot = kmalloc(new_size,
GFP_KERNEL | __GFP_NOFAIL);
ifp->if_broot_bytes = (int)new_size;
return;
}
@ -414,7 +426,7 @@ xfs_iroot_realloc(
new_max = cur_max + rec_diff;
new_size = XFS_BMAP_BROOT_SPACE_CALC(mp, new_max);
ifp->if_broot = krealloc(ifp->if_broot, new_size,
GFP_NOFS | __GFP_NOFAIL);
GFP_KERNEL | __GFP_NOFAIL);
op = (char *)XFS_BMAP_BROOT_PTR_ADDR(mp, ifp->if_broot, 1,
ifp->if_broot_bytes);
np = (char *)XFS_BMAP_BROOT_PTR_ADDR(mp, ifp->if_broot, 1,
@ -440,7 +452,7 @@ xfs_iroot_realloc(
else
new_size = 0;
if (new_size > 0) {
new_broot = kmem_alloc(new_size, KM_NOFS);
new_broot = kmalloc(new_size, GFP_KERNEL | __GFP_NOFAIL);
/*
* First copy over the btree block header.
*/
@ -470,7 +482,7 @@ xfs_iroot_realloc(
(int)new_size);
memcpy(np, op, new_max * (uint)sizeof(xfs_fsblock_t));
}
kmem_free(ifp->if_broot);
kfree(ifp->if_broot);
ifp->if_broot = new_broot;
ifp->if_broot_bytes = (int)new_size;
if (ifp->if_broot)
@ -488,7 +500,7 @@ xfs_iroot_realloc(
*
* If the amount of space needed has decreased below the size of the
* inline buffer, then switch to using the inline buffer. Otherwise,
* use kmem_realloc() or kmem_alloc() to adjust the size of the buffer
* use krealloc() or kmalloc() to adjust the size of the buffer
* to what is needed.
*
* ip -- the inode whose if_data area is changing
@ -509,7 +521,7 @@ xfs_idata_realloc(
if (byte_diff) {
ifp->if_data = krealloc(ifp->if_data, new_size,
GFP_NOFS | __GFP_NOFAIL);
GFP_KERNEL | __GFP_NOFAIL);
if (new_size == 0)
ifp->if_data = NULL;
ifp->if_bytes = new_size;
@ -524,13 +536,13 @@ xfs_idestroy_fork(
struct xfs_ifork *ifp)
{
if (ifp->if_broot != NULL) {
kmem_free(ifp->if_broot);
kfree(ifp->if_broot);
ifp->if_broot = NULL;
}
switch (ifp->if_format) {
case XFS_DINODE_FMT_LOCAL:
kmem_free(ifp->if_data);
kfree(ifp->if_data);
ifp->if_data = NULL;
break;
case XFS_DINODE_FMT_EXTENTS:
@ -562,7 +574,7 @@ xfs_iextents_copy(
struct xfs_bmbt_irec rec;
int64_t copied = 0;
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED));
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED);
ASSERT(ifp->if_bytes > 0);
for_each_xfs_iext(ifp, &icur, &rec) {
@ -689,7 +701,7 @@ xfs_ifork_init_cow(
return;
ip->i_cowfp = kmem_cache_zalloc(xfs_ifork_cache,
GFP_NOFS | __GFP_NOFAIL);
GFP_KERNEL | __GFP_NOLOCKDEP | __GFP_NOFAIL);
ip->i_cowfp->if_format = XFS_DINODE_FMT_EXTENTS;
}
@ -802,3 +814,12 @@ xfs_iext_count_upgrade(
return 0;
}
/* Decide if a file mapping is on the realtime device or not. */
bool
xfs_ifork_is_realtime(
struct xfs_inode *ip,
int whichfork)
{
return XFS_IS_REALTIME_INODE(ip) && whichfork != XFS_ATTR_FORK;
}

1
fs/xfs/libxfs/xfs_inode_fork.h
View File

@ -260,6 +260,7 @@ int xfs_iext_count_may_overflow(struct xfs_inode *ip, int whichfork,
int nr_to_add);
int xfs_iext_count_upgrade(struct xfs_trans *tp, struct xfs_inode *ip,
uint nr_to_add);
bool xfs_ifork_is_realtime(struct xfs_inode *ip, int whichfork);
/* returns true if the fork has extents but they are not read in yet. */
static inline bool xfs_need_iread_extents(const struct xfs_ifork *ifp)

4
fs/xfs/libxfs/xfs_log_format.h
View File

@ -838,10 +838,12 @@ struct xfs_cud_log_format {
#define XFS_BMAP_EXTENT_ATTR_FORK (1U << 31)
#define XFS_BMAP_EXTENT_UNWRITTEN (1U << 30)
#define XFS_BMAP_EXTENT_REALTIME (1U << 29)
#define XFS_BMAP_EXTENT_FLAGS (XFS_BMAP_EXTENT_TYPE_MASK | \
XFS_BMAP_EXTENT_ATTR_FORK | \
XFS_BMAP_EXTENT_UNWRITTEN)
XFS_BMAP_EXTENT_UNWRITTEN | \
XFS_BMAP_EXTENT_REALTIME)
/*
* This is the structure used to lay out an bui log item in the

69
fs/xfs/libxfs/xfs_refcount.c
View File

@ -23,6 +23,7 @@
#include "xfs_refcount.h"
#include "xfs_rmap.h"
#include "xfs_ag.h"
#include "xfs_health.h"
struct kmem_cache *xfs_refcount_intent_cache;
@ -156,6 +157,7 @@ xfs_refcount_complain_bad_rec(
xfs_warn(mp,
"Start block 0x%x, block count 0x%x, references 0x%x",
irec->rc_startblock, irec->rc_blockcount, irec->rc_refcount);
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
@ -238,6 +240,7 @@ xfs_refcount_insert(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, *i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -268,12 +271,14 @@ xfs_refcount_delete(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
trace_xfs_refcount_delete(cur->bc_mp, cur->bc_ag.pag->pag_agno, &irec);
error = xfs_btree_delete(cur, i);
if (XFS_IS_CORRUPT(cur->bc_mp, *i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -398,6 +403,7 @@ xfs_refcount_split_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -425,6 +431,7 @@ xfs_refcount_split_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -470,6 +477,7 @@ xfs_refcount_merge_center_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -478,6 +486,7 @@ xfs_refcount_merge_center_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -487,6 +496,7 @@ xfs_refcount_merge_center_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -498,6 +508,7 @@ xfs_refcount_merge_center_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -542,6 +553,7 @@ xfs_refcount_merge_left_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -550,6 +562,7 @@ xfs_refcount_merge_left_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -561,6 +574,7 @@ xfs_refcount_merge_left_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -608,6 +622,7 @@ xfs_refcount_merge_right_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -616,6 +631,7 @@ xfs_refcount_merge_right_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -627,6 +643,7 @@ xfs_refcount_merge_right_extent(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -674,6 +691,7 @@ xfs_refcount_find_left_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -693,6 +711,7 @@ xfs_refcount_find_left_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -767,6 +786,7 @@ xfs_refcount_find_right_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -786,6 +806,7 @@ xfs_refcount_find_right_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1056,7 +1077,7 @@ xfs_refcount_still_have_space(
* to handle each of the shape changes to the refcount btree.
*/
overhead = xfs_allocfree_block_count(cur->bc_mp,
cur->bc_ag.refc.shape_changes);
cur->bc_refc.shape_changes);
overhead += cur->bc_mp->m_refc_maxlevels;
overhead *= cur->bc_mp->m_sb.sb_blocksize;
@ -1064,17 +1085,17 @@ xfs_refcount_still_have_space(
* Only allow 2 refcount extent updates per transaction if the
* refcount continue update "error" has been injected.
*/
if (cur->bc_ag.refc.nr_ops > 2 &&
if (cur->bc_refc.nr_ops > 2 &&
XFS_TEST_ERROR(false, cur->bc_mp,
XFS_ERRTAG_REFCOUNT_CONTINUE_UPDATE))
return false;
if (cur->bc_ag.refc.nr_ops == 0)
if (cur->bc_refc.nr_ops == 0)
return true;
else if (overhead > cur->bc_tp->t_log_res)
return false;
return cur->bc_tp->t_log_res - overhead >
cur->bc_ag.refc.nr_ops * XFS_REFCOUNT_ITEM_OVERHEAD;
return cur->bc_tp->t_log_res - overhead >
cur->bc_refc.nr_ops * XFS_REFCOUNT_ITEM_OVERHEAD;
}
/*
@ -1134,7 +1155,7 @@ xfs_refcount_adjust_extents(
* Either cover the hole (increment) or
* delete the range (decrement).
*/
cur->bc_ag.refc.nr_ops++;
cur->bc_refc.nr_ops++;
if (tmp.rc_refcount) {
error = xfs_refcount_insert(cur, &tmp,
&found_tmp);
@ -1142,6 +1163,7 @@ xfs_refcount_adjust_extents(
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp,
found_tmp != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1180,6 +1202,7 @@ xfs_refcount_adjust_extents(
*/
if (XFS_IS_CORRUPT(cur->bc_mp, ext.rc_blockcount == 0) ||
XFS_IS_CORRUPT(cur->bc_mp, ext.rc_blockcount > *aglen)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1193,7 +1216,7 @@ xfs_refcount_adjust_extents(
ext.rc_refcount += adj;
trace_xfs_refcount_modify_extent(cur->bc_mp,
cur->bc_ag.pag->pag_agno, &ext);
cur->bc_ag.refc.nr_ops++;
cur->bc_refc.nr_ops++;
if (ext.rc_refcount > 1) {
error = xfs_refcount_update(cur, &ext);
if (error)
@ -1203,6 +1226,7 @@ xfs_refcount_adjust_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1281,7 +1305,7 @@ xfs_refcount_adjust(
if (shape_changed)
shape_changes++;
if (shape_changes)
cur->bc_ag.refc.shape_changes++;
cur->bc_refc.shape_changes++;
/* Now that we've taken care of the ends, adjust the middle extents */
error = xfs_refcount_adjust_extents(cur, agbno, aglen, adj);
@ -1327,8 +1351,10 @@ xfs_refcount_continue_op(
struct xfs_perag *pag = cur->bc_ag.pag;
if (XFS_IS_CORRUPT(mp, !xfs_verify_agbext(pag, new_agbno,
ri->ri_blockcount)))
ri->ri_blockcount))) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
ri->ri_startblock = XFS_AGB_TO_FSB(mp, pag->pag_agno, new_agbno);
@ -1374,8 +1400,8 @@ xfs_refcount_finish_one(
*/
rcur = *pcur;
if (rcur != NULL && rcur->bc_ag.pag != ri->ri_pag) {
nr_ops = rcur->bc_ag.refc.nr_ops;
shape_changes = rcur->bc_ag.refc.shape_changes;
nr_ops = rcur->bc_refc.nr_ops;
shape_changes = rcur->bc_refc.shape_changes;
xfs_refcount_finish_one_cleanup(tp, rcur, 0);
rcur = NULL;
*pcur = NULL;
@ -1387,8 +1413,8 @@ xfs_refcount_finish_one(
return error;
rcur = xfs_refcountbt_init_cursor(mp, tp, agbp, ri->ri_pag);
rcur->bc_ag.refc.nr_ops = nr_ops;
rcur->bc_ag.refc.shape_changes = shape_changes;
rcur->bc_refc.nr_ops = nr_ops;
rcur->bc_refc.shape_changes = shape_changes;
}
*pcur = rcur;
@ -1449,7 +1475,7 @@ __xfs_refcount_add(
blockcount);
ri = kmem_cache_alloc(xfs_refcount_intent_cache,
GFP_NOFS | __GFP_NOFAIL);
GFP_KERNEL | __GFP_NOFAIL);
INIT_LIST_HEAD(&ri->ri_list);
ri->ri_type = type;
ri->ri_startblock = startblock;
@ -1535,6 +1561,7 @@ xfs_refcount_find_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1552,6 +1579,7 @@ xfs_refcount_find_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1585,6 +1613,7 @@ xfs_refcount_find_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1682,6 +1711,7 @@ xfs_refcount_adjust_cow_extents(
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec &&
ext.rc_domain != XFS_REFC_DOMAIN_COW)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1697,6 +1727,7 @@ xfs_refcount_adjust_cow_extents(
/* Adding a CoW reservation, there should be nothing here. */
if (XFS_IS_CORRUPT(cur->bc_mp,
agbno + aglen > ext.rc_startblock)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1714,6 +1745,7 @@ xfs_refcount_adjust_cow_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_tmp != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1721,14 +1753,17 @@ xfs_refcount_adjust_cow_extents(
case XFS_REFCOUNT_ADJUST_COW_FREE:
/* Removing a CoW reservation, there should be one extent. */
if (XFS_IS_CORRUPT(cur->bc_mp, ext.rc_startblock != agbno)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
if (XFS_IS_CORRUPT(cur->bc_mp, ext.rc_blockcount != aglen)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
if (XFS_IS_CORRUPT(cur->bc_mp, ext.rc_refcount != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1740,6 +1775,7 @@ xfs_refcount_adjust_cow_extents(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(cur->bc_mp, found_rec != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1889,8 +1925,10 @@ xfs_refcount_recover_extent(
struct xfs_refcount_recovery *rr;
if (XFS_IS_CORRUPT(cur->bc_mp,
be32_to_cpu(rec->refc.rc_refcount) != 1))
be32_to_cpu(rec->refc.rc_refcount) != 1)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
rr = kmalloc(sizeof(struct xfs_refcount_recovery),
GFP_KERNEL | __GFP_NOFAIL);
@ -1900,6 +1938,7 @@ xfs_refcount_recover_extent(
if (xfs_refcount_check_irec(cur->bc_ag.pag, &rr->rr_rrec) != NULL ||
XFS_IS_CORRUPT(cur->bc_mp,
rr->rr_rrec.rc_domain != XFS_REFC_DOMAIN_COW)) {
xfs_btree_mark_sick(cur);
kfree(rr);
return -EFSCORRUPTED;
}

80
fs/xfs/libxfs/xfs_refcount_btree.c
View File

@ -16,6 +16,7 @@
#include "xfs_refcount.h"
#include "xfs_alloc.h"
#include "xfs_error.h"
#include "xfs_health.h"
#include "xfs_trace.h"
#include "xfs_trans.h"
#include "xfs_bit.h"
@ -77,8 +78,6 @@ xfs_refcountbt_alloc_block(
xfs_refc_block(args.mp)));
if (error)
goto out_error;
trace_xfs_refcountbt_alloc_block(cur->bc_mp, cur->bc_ag.pag->pag_agno,
args.agbno, 1);
if (args.fsbno == NULLFSBLOCK) {
*stat = 0;
return 0;
@ -107,8 +106,6 @@ xfs_refcountbt_free_block(
struct xfs_agf *agf = agbp->b_addr;
xfs_fsblock_t fsbno = XFS_DADDR_TO_FSB(mp, xfs_buf_daddr(bp));
trace_xfs_refcountbt_free_block(cur->bc_mp, cur->bc_ag.pag->pag_agno,
XFS_FSB_TO_AGBNO(cur->bc_mp, fsbno), 1);
be32_add_cpu(&agf->agf_refcount_blocks, -1);
xfs_alloc_log_agf(cur->bc_tp, agbp, XFS_AGF_REFCOUNT_BLOCKS);
return xfs_free_extent_later(cur->bc_tp, fsbno, 1,
@ -220,7 +217,7 @@ xfs_refcountbt_verify(
if (!xfs_has_reflink(mp))
return __this_address;
fa = xfs_btree_sblock_v5hdr_verify(bp);
fa = xfs_btree_agblock_v5hdr_verify(bp);
if (fa)
return fa;
@ -242,7 +239,7 @@ xfs_refcountbt_verify(
} else if (level >= mp->m_refc_maxlevels)
return __this_address;
return xfs_btree_sblock_verify(bp, mp->m_refc_mxr[level != 0]);
return xfs_btree_agblock_verify(bp, mp->m_refc_mxr[level != 0]);
}
STATIC void
@ -251,7 +248,7 @@ xfs_refcountbt_read_verify(
{
xfs_failaddr_t fa;
if (!xfs_btree_sblock_verify_crc(bp))
if (!xfs_btree_agblock_verify_crc(bp))
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_refcountbt_verify(bp);
@ -275,7 +272,7 @@ xfs_refcountbt_write_verify(
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
return;
}
xfs_btree_sblock_calc_crc(bp);
xfs_btree_agblock_calc_crc(bp);
}
@ -321,9 +318,17 @@ xfs_refcountbt_keys_contiguous(
be32_to_cpu(key2->refc.rc_startblock));
}
static const struct xfs_btree_ops xfs_refcountbt_ops = {
const struct xfs_btree_ops xfs_refcountbt_ops = {
.name = "refcount",
.type = XFS_BTREE_TYPE_AG,
.rec_len = sizeof(struct xfs_refcount_rec),
.key_len = sizeof(struct xfs_refcount_key),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.lru_refs = XFS_REFC_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_refcbt_2),
.sick_mask = XFS_SICK_AG_REFCNTBT,
.dup_cursor = xfs_refcountbt_dup_cursor,
.set_root = xfs_refcountbt_set_root,
@ -344,32 +349,10 @@ static const struct xfs_btree_ops xfs_refcountbt_ops = {
};
/*
* Initialize a new refcount btree cursor.
* Create a new refcount btree cursor.
*
* For staging cursors tp and agbp are NULL.
*/
static struct xfs_btree_cur *
xfs_refcountbt_init_common(
struct xfs_mount *mp,
struct xfs_trans *tp,
struct xfs_perag *pag)
{
struct xfs_btree_cur *cur;
ASSERT(pag->pag_agno < mp->m_sb.sb_agcount);
cur = xfs_btree_alloc_cursor(mp, tp, XFS_BTNUM_REFC,
mp->m_refc_maxlevels, xfs_refcountbt_cur_cache);
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_refcbt_2);
cur->bc_flags |= XFS_BTREE_CRC_BLOCKS;
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.refc.nr_ops = 0;
cur->bc_ag.refc.shape_changes = 0;
cur->bc_ops = &xfs_refcountbt_ops;
return cur;
}
/* Create a btree cursor. */
struct xfs_btree_cur *
xfs_refcountbt_init_cursor(
struct xfs_mount *mp,
@ -377,26 +360,21 @@ xfs_refcountbt_init_cursor(
struct xfs_buf *agbp,
struct xfs_perag *pag)
{
struct xfs_agf *agf = agbp->b_addr;
struct xfs_btree_cur *cur;
cur = xfs_refcountbt_init_common(mp, tp, pag);
cur->bc_nlevels = be32_to_cpu(agf->agf_refcount_level);
ASSERT(pag->pag_agno < mp->m_sb.sb_agcount);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_refcountbt_ops,
mp->m_refc_maxlevels, xfs_refcountbt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_refc.nr_ops = 0;
cur->bc_refc.shape_changes = 0;
cur->bc_ag.agbp = agbp;
return cur;
}
if (agbp) {
struct xfs_agf *agf = agbp->b_addr;
/* Create a btree cursor with a fake root for staging. */
struct xfs_btree_cur *
xfs_refcountbt_stage_cursor(
struct xfs_mount *mp,
struct xbtree_afakeroot *afake,
struct xfs_perag *pag)
{
struct xfs_btree_cur *cur;
cur = xfs_refcountbt_init_common(mp, NULL, pag);
xfs_btree_stage_afakeroot(cur, afake);
cur->bc_nlevels = be32_to_cpu(agf->agf_refcount_level);
}
return cur;
}
@ -421,7 +399,7 @@ xfs_refcountbt_commit_staged_btree(
xfs_alloc_log_agf(tp, agbp, XFS_AGF_REFCOUNT_BLOCKS |
XFS_AGF_REFCOUNT_ROOT |
XFS_AGF_REFCOUNT_LEVEL);
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_refcountbt_ops);
xfs_btree_commit_afakeroot(cur, tp, agbp);
}
/* Calculate number of records in a refcount btree block. */

2
fs/xfs/libxfs/xfs_refcount_btree.h
View File

@ -48,8 +48,6 @@ struct xbtree_afakeroot;
extern struct xfs_btree_cur *xfs_refcountbt_init_cursor(struct xfs_mount *mp,
struct xfs_trans *tp, struct xfs_buf *agbp,
struct xfs_perag *pag);
struct xfs_btree_cur *xfs_refcountbt_stage_cursor(struct xfs_mount *mp,
struct xbtree_afakeroot *afake, struct xfs_perag *pag);
extern int xfs_refcountbt_maxrecs(int blocklen, bool leaf);
extern void xfs_refcountbt_compute_maxlevels(struct xfs_mount *mp);

284
fs/xfs/libxfs/xfs_rmap.c
View File

@ -23,6 +23,7 @@
#include "xfs_error.h"
#include "xfs_inode.h"
#include "xfs_ag.h"
#include "xfs_health.h"
struct kmem_cache *xfs_rmap_intent_cache;
@ -56,8 +57,10 @@ xfs_rmap_lookup_le(
error = xfs_rmap_get_rec(cur, irec, &get_stat);
if (error)
return error;
if (!get_stat)
if (!get_stat) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
return 0;
}
@ -132,6 +135,7 @@ xfs_rmap_insert(
if (error)
goto done;
if (XFS_IS_CORRUPT(rcur->bc_mp, i != 0)) {
xfs_btree_mark_sick(rcur);
error = -EFSCORRUPTED;
goto done;
}
@ -145,6 +149,7 @@ xfs_rmap_insert(
if (error)
goto done;
if (XFS_IS_CORRUPT(rcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(rcur);
error = -EFSCORRUPTED;
goto done;
}
@ -174,6 +179,7 @@ xfs_rmap_delete(
if (error)
goto done;
if (XFS_IS_CORRUPT(rcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(rcur);
error = -EFSCORRUPTED;
goto done;
}
@ -182,6 +188,7 @@ xfs_rmap_delete(
if (error)
goto done;
if (XFS_IS_CORRUPT(rcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(rcur);
error = -EFSCORRUPTED;
goto done;
}
@ -208,10 +215,10 @@ xfs_rmap_btrec_to_irec(
/* Simple checks for rmap records. */
xfs_failaddr_t
xfs_rmap_check_irec(
struct xfs_btree_cur *cur,
struct xfs_perag *pag,
const struct xfs_rmap_irec *irec)
{
struct xfs_mount *mp = cur->bc_mp;
struct xfs_mount *mp = pag->pag_mount;
bool is_inode;
bool is_unwritten;
bool is_bmbt;
@ -226,8 +233,8 @@ xfs_rmap_check_irec(
return __this_address;
} else {
/* check for valid extent range, including overflow */
if (!xfs_verify_agbext(cur->bc_ag.pag, irec->rm_startblock,
irec->rm_blockcount))
if (!xfs_verify_agbext(pag, irec->rm_startblock,
irec->rm_blockcount))
return __this_address;
}
@ -262,6 +269,16 @@ xfs_rmap_check_irec(
return NULL;
}
static inline xfs_failaddr_t
xfs_rmap_check_btrec(
struct xfs_btree_cur *cur,
const struct xfs_rmap_irec *irec)
{
if (xfs_btree_is_mem_rmap(cur->bc_ops))
return xfs_rmap_check_irec(cur->bc_mem.pag, irec);
return xfs_rmap_check_irec(cur->bc_ag.pag, irec);
}
static inline int
xfs_rmap_complain_bad_rec(
struct xfs_btree_cur *cur,
@ -270,13 +287,18 @@ xfs_rmap_complain_bad_rec(
{
struct xfs_mount *mp = cur->bc_mp;
xfs_warn(mp,
"Reverse Mapping BTree record corruption in AG %d detected at %pS!",
cur->bc_ag.pag->pag_agno, fa);
if (xfs_btree_is_mem_rmap(cur->bc_ops))
xfs_warn(mp,
"In-Memory Reverse Mapping BTree record corruption detected at %pS!", fa);
else
xfs_warn(mp,
"Reverse Mapping BTree record corruption in AG %d detected at %pS!",
cur->bc_ag.pag->pag_agno, fa);
xfs_warn(mp,
"Owner 0x%llx, flags 0x%x, start block 0x%x block count 0x%x",
irec->rm_owner, irec->rm_flags, irec->rm_startblock,
irec->rm_blockcount);
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
@ -299,7 +321,7 @@ xfs_rmap_get_rec(
fa = xfs_rmap_btrec_to_irec(rec, irec);
if (!fa)
fa = xfs_rmap_check_irec(cur, irec);
fa = xfs_rmap_check_btrec(cur, irec);
if (fa)
return xfs_rmap_complain_bad_rec(cur, fa, irec);
@ -512,7 +534,7 @@ xfs_rmap_lookup_le_range(
*/
static int
xfs_rmap_free_check_owner(
struct xfs_mount *mp,
struct xfs_btree_cur *cur,
uint64_t ltoff,
struct xfs_rmap_irec *rec,
xfs_filblks_t len,
@ -520,6 +542,7 @@ xfs_rmap_free_check_owner(
uint64_t offset,
unsigned int flags)
{
struct xfs_mount *mp = cur->bc_mp;
int error = 0;
if (owner == XFS_RMAP_OWN_UNKNOWN)
@ -529,12 +552,14 @@ xfs_rmap_free_check_owner(
if (XFS_IS_CORRUPT(mp,
(flags & XFS_RMAP_UNWRITTEN) !=
(rec->rm_flags & XFS_RMAP_UNWRITTEN))) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out;
}
/* Make sure the owner matches what we expect to find in the tree. */
if (XFS_IS_CORRUPT(mp, owner != rec->rm_owner)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out;
}
@ -546,16 +571,19 @@ xfs_rmap_free_check_owner(
if (flags & XFS_RMAP_BMBT_BLOCK) {
if (XFS_IS_CORRUPT(mp,
!(rec->rm_flags & XFS_RMAP_BMBT_BLOCK))) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out;
}
} else {
if (XFS_IS_CORRUPT(mp, rec->rm_offset > offset)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out;
}
if (XFS_IS_CORRUPT(mp,
offset + len > ltoff + rec->rm_blockcount)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out;
}
@ -618,6 +646,7 @@ xfs_rmap_unmap(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -639,6 +668,7 @@ xfs_rmap_unmap(
if (XFS_IS_CORRUPT(mp,
bno <
ltrec.rm_startblock + ltrec.rm_blockcount)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -665,6 +695,7 @@ xfs_rmap_unmap(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -677,12 +708,13 @@ xfs_rmap_unmap(
ltrec.rm_startblock > bno ||
ltrec.rm_startblock + ltrec.rm_blockcount <
bno + len)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
/* Check owner information. */
error = xfs_rmap_free_check_owner(mp, ltoff, &ltrec, len, owner,
error = xfs_rmap_free_check_owner(cur, ltoff, &ltrec, len, owner,
offset, flags);
if (error)
goto out_error;
@ -697,6 +729,7 @@ xfs_rmap_unmap(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -788,6 +821,86 @@ out_error:
return error;
}
#ifdef CONFIG_XFS_LIVE_HOOKS
/*
* Use a static key here to reduce the overhead of rmapbt live updates. If
* the compiler supports jump labels, the static branch will be replaced by a
* nop sled when there are no hook users. Online fsck is currently the only
* caller, so this is a reasonable tradeoff.
*
* Note: Patching the kernel code requires taking the cpu hotplug lock. Other
* parts of the kernel allocate memory with that lock held, which means that
* XFS callers cannot hold any locks that might be used by memory reclaim or
* writeback when calling the static_branch_{inc,dec} functions.
*/
DEFINE_STATIC_XFS_HOOK_SWITCH(xfs_rmap_hooks_switch);
void
xfs_rmap_hook_disable(void)
{
xfs_hooks_switch_off(&xfs_rmap_hooks_switch);
}
void
xfs_rmap_hook_enable(void)
{
xfs_hooks_switch_on(&xfs_rmap_hooks_switch);
}
/* Call downstream hooks for a reverse mapping update. */
static inline void
xfs_rmap_update_hook(
struct xfs_trans *tp,
struct xfs_perag *pag,
enum xfs_rmap_intent_type op,
xfs_agblock_t startblock,
xfs_extlen_t blockcount,
bool unwritten,
const struct xfs_owner_info *oinfo)
{
if (xfs_hooks_switched_on(&xfs_rmap_hooks_switch)) {
struct xfs_rmap_update_params p = {
.startblock = startblock,
.blockcount = blockcount,
.unwritten = unwritten,
.oinfo = *oinfo, /* struct copy */
};
if (pag)
xfs_hooks_call(&pag->pag_rmap_update_hooks, op, &p);
}
}
/* Call the specified function during a reverse mapping update. */
int
xfs_rmap_hook_add(
struct xfs_perag *pag,
struct xfs_rmap_hook *hook)
{
return xfs_hooks_add(&pag->pag_rmap_update_hooks, &hook->rmap_hook);
}
/* Stop calling the specified function during a reverse mapping update. */
void
xfs_rmap_hook_del(
struct xfs_perag *pag,
struct xfs_rmap_hook *hook)
{
xfs_hooks_del(&pag->pag_rmap_update_hooks, &hook->rmap_hook);
}
/* Configure rmap update hook functions. */
void
xfs_rmap_hook_setup(
struct xfs_rmap_hook *hook,
notifier_fn_t mod_fn)
{
xfs_hook_setup(&hook->rmap_hook, mod_fn);
}
#else
# define xfs_rmap_update_hook(t, p, o, s, b, u, oi) do { } while (0)
#endif /* CONFIG_XFS_LIVE_HOOKS */
/*
* Remove a reference to an extent in the rmap btree.
*/
@ -808,7 +921,7 @@ xfs_rmap_free(
return 0;
cur = xfs_rmapbt_init_cursor(mp, tp, agbp, pag);
xfs_rmap_update_hook(tp, pag, XFS_RMAP_UNMAP, bno, len, false, oinfo);
error = xfs_rmap_unmap(cur, bno, len, false, oinfo);
xfs_btree_del_cursor(cur, error);
@ -900,6 +1013,7 @@ xfs_rmap_map(
if (XFS_IS_CORRUPT(mp,
have_lt != 0 &&
ltrec.rm_startblock + ltrec.rm_blockcount > bno)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -917,10 +1031,12 @@ xfs_rmap_map(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, have_gt != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
if (XFS_IS_CORRUPT(mp, bno + len > gtrec.rm_startblock)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -974,6 +1090,7 @@ xfs_rmap_map(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1021,6 +1138,7 @@ xfs_rmap_map(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -1055,6 +1173,7 @@ xfs_rmap_alloc(
return 0;
cur = xfs_rmapbt_init_cursor(mp, tp, agbp, pag);
xfs_rmap_update_hook(tp, pag, XFS_RMAP_MAP, bno, len, false, oinfo);
error = xfs_rmap_map(cur, bno, len, false, oinfo);
xfs_btree_del_cursor(cur, error);
@ -1116,6 +1235,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1153,12 +1273,14 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
if (XFS_IS_CORRUPT(mp,
LEFT.rm_startblock + LEFT.rm_blockcount >
bno)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1181,6 +1303,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1193,10 +1316,12 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
if (XFS_IS_CORRUPT(mp, bno + len > RIGHT.rm_startblock)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1227,6 +1352,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1246,6 +1372,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1257,6 +1384,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1264,6 +1392,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1275,6 +1404,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1282,6 +1412,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1305,6 +1436,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1312,6 +1444,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1331,6 +1464,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1342,6 +1476,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1349,6 +1484,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1419,6 +1555,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1461,6 +1598,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1476,6 +1614,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1509,6 +1648,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1522,6 +1662,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 0)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1534,6 +1675,7 @@ xfs_rmap_convert(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1606,6 +1748,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1634,6 +1777,7 @@ xfs_rmap_convert_shared(
if (XFS_IS_CORRUPT(mp,
LEFT.rm_startblock + LEFT.rm_blockcount >
bno)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1652,10 +1796,12 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
if (XFS_IS_CORRUPT(mp, bno + len > RIGHT.rm_startblock)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1706,6 +1852,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1732,6 +1879,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1758,6 +1906,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1781,6 +1930,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1816,6 +1966,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1861,6 +2012,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1896,6 +2048,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -1934,6 +2087,7 @@ xfs_rmap_convert_shared(
if (error)
goto done;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto done;
}
@ -2023,6 +2177,7 @@ xfs_rmap_unmap_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2033,12 +2188,14 @@ xfs_rmap_unmap_shared(
ltrec.rm_startblock > bno ||
ltrec.rm_startblock + ltrec.rm_blockcount <
bno + len)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
/* Make sure the owner matches what we expect to find in the tree. */
if (XFS_IS_CORRUPT(mp, owner != ltrec.rm_owner)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2047,16 +2204,19 @@ xfs_rmap_unmap_shared(
if (XFS_IS_CORRUPT(mp,
(flags & XFS_RMAP_UNWRITTEN) !=
(ltrec.rm_flags & XFS_RMAP_UNWRITTEN))) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
/* Check the offset. */
if (XFS_IS_CORRUPT(mp, ltrec.rm_offset > offset)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
if (XFS_IS_CORRUPT(mp, offset > ltoff + ltrec.rm_blockcount)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2113,6 +2273,7 @@ xfs_rmap_unmap_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2142,6 +2303,7 @@ xfs_rmap_unmap_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2221,6 +2383,7 @@ xfs_rmap_map_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, have_gt != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2273,6 +2436,7 @@ xfs_rmap_map_shared(
if (error)
goto out_error;
if (XFS_IS_CORRUPT(mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
goto out_error;
}
@ -2335,15 +2499,12 @@ xfs_rmap_map_raw(
{
struct xfs_owner_info oinfo;
oinfo.oi_owner = rmap->rm_owner;
oinfo.oi_offset = rmap->rm_offset;
oinfo.oi_flags = 0;
if (rmap->rm_flags & XFS_RMAP_ATTR_FORK)
oinfo.oi_flags |= XFS_OWNER_INFO_ATTR_FORK;
if (rmap->rm_flags & XFS_RMAP_BMBT_BLOCK)
oinfo.oi_flags |= XFS_OWNER_INFO_BMBT_BLOCK;
xfs_owner_info_pack(&oinfo, rmap->rm_owner, rmap->rm_offset,
rmap->rm_flags);
if (rmap->rm_flags || XFS_RMAP_NON_INODE_OWNER(rmap->rm_owner))
if ((rmap->rm_flags & (XFS_RMAP_ATTR_FORK | XFS_RMAP_BMBT_BLOCK |
XFS_RMAP_UNWRITTEN)) ||
XFS_RMAP_NON_INODE_OWNER(rmap->rm_owner))
return xfs_rmap_map(cur, rmap->rm_startblock,
rmap->rm_blockcount,
rmap->rm_flags & XFS_RMAP_UNWRITTEN,
@ -2373,7 +2534,7 @@ xfs_rmap_query_range_helper(
fa = xfs_rmap_btrec_to_irec(rec, &irec);
if (!fa)
fa = xfs_rmap_check_irec(cur, &irec);
fa = xfs_rmap_check_btrec(cur, &irec);
if (fa)
return xfs_rmap_complain_bad_rec(cur, fa, &irec);
@ -2428,6 +2589,38 @@ xfs_rmap_finish_one_cleanup(
xfs_trans_brelse(tp, agbp);
}
/* Commit an rmap operation into the ondisk tree. */
int
__xfs_rmap_finish_intent(
struct xfs_btree_cur *rcur,
enum xfs_rmap_intent_type op,
xfs_agblock_t bno,
xfs_extlen_t len,
const struct xfs_owner_info *oinfo,
bool unwritten)
{
switch (op) {
case XFS_RMAP_ALLOC:
case XFS_RMAP_MAP:
return xfs_rmap_map(rcur, bno, len, unwritten, oinfo);
case XFS_RMAP_MAP_SHARED:
return xfs_rmap_map_shared(rcur, bno, len, unwritten, oinfo);
case XFS_RMAP_FREE:
case XFS_RMAP_UNMAP:
return xfs_rmap_unmap(rcur, bno, len, unwritten, oinfo);
case XFS_RMAP_UNMAP_SHARED:
return xfs_rmap_unmap_shared(rcur, bno, len, unwritten, oinfo);
case XFS_RMAP_CONVERT:
return xfs_rmap_convert(rcur, bno, len, !unwritten, oinfo);
case XFS_RMAP_CONVERT_SHARED:
return xfs_rmap_convert_shared(rcur, bno, len, !unwritten,
oinfo);
default:
ASSERT(0);
return -EFSCORRUPTED;
}
}
/*
* Process one of the deferred rmap operations. We pass back the
* btree cursor to maintain our lock on the rmapbt between calls.
@ -2476,10 +2669,14 @@ xfs_rmap_finish_one(
* allocate blocks.
*/
error = xfs_free_extent_fix_freelist(tp, ri->ri_pag, &agbp);
if (error)
if (error) {
xfs_ag_mark_sick(ri->ri_pag, XFS_SICK_AG_AGFL);
return error;
if (XFS_IS_CORRUPT(tp->t_mountp, !agbp))
}
if (XFS_IS_CORRUPT(tp->t_mountp, !agbp)) {
xfs_ag_mark_sick(ri->ri_pag, XFS_SICK_AG_AGFL);
return -EFSCORRUPTED;
}
rcur = xfs_rmapbt_init_cursor(mp, tp, agbp, ri->ri_pag);
}
@ -2490,39 +2687,14 @@ xfs_rmap_finish_one(
unwritten = ri->ri_bmap.br_state == XFS_EXT_UNWRITTEN;
bno = XFS_FSB_TO_AGBNO(rcur->bc_mp, ri->ri_bmap.br_startblock);
switch (ri->ri_type) {
case XFS_RMAP_ALLOC:
case XFS_RMAP_MAP:
error = xfs_rmap_map(rcur, bno, ri->ri_bmap.br_blockcount,
unwritten, &oinfo);
break;
case XFS_RMAP_MAP_SHARED:
error = xfs_rmap_map_shared(rcur, bno,
ri->ri_bmap.br_blockcount, unwritten, &oinfo);
break;
case XFS_RMAP_FREE:
case XFS_RMAP_UNMAP:
error = xfs_rmap_unmap(rcur, bno, ri->ri_bmap.br_blockcount,
unwritten, &oinfo);
break;
case XFS_RMAP_UNMAP_SHARED:
error = xfs_rmap_unmap_shared(rcur, bno,
ri->ri_bmap.br_blockcount, unwritten, &oinfo);
break;
case XFS_RMAP_CONVERT:
error = xfs_rmap_convert(rcur, bno, ri->ri_bmap.br_blockcount,
!unwritten, &oinfo);
break;
case XFS_RMAP_CONVERT_SHARED:
error = xfs_rmap_convert_shared(rcur, bno,
ri->ri_bmap.br_blockcount, !unwritten, &oinfo);
break;
default:
ASSERT(0);
error = -EFSCORRUPTED;
}
error = __xfs_rmap_finish_intent(rcur, ri->ri_type, bno,
ri->ri_bmap.br_blockcount, &oinfo, unwritten);
if (error)
return error;
return error;
xfs_rmap_update_hook(tp, ri->ri_pag, ri->ri_type, bno,
ri->ri_bmap.br_blockcount, unwritten, &oinfo);
return 0;
}
/*
@ -2559,7 +2731,7 @@ __xfs_rmap_add(
bmap->br_blockcount,
bmap->br_state);
ri = kmem_cache_alloc(xfs_rmap_intent_cache, GFP_NOFS | __GFP_NOFAIL);
ri = kmem_cache_alloc(xfs_rmap_intent_cache, GFP_KERNEL | __GFP_NOFAIL);
INIT_LIST_HEAD(&ri->ri_list);
ri->ri_type = type;
ri->ri_owner = owner;

31
fs/xfs/libxfs/xfs_rmap.h
View File

@ -186,6 +186,10 @@ void xfs_rmap_finish_one_cleanup(struct xfs_trans *tp,
struct xfs_btree_cur *rcur, int error);
int xfs_rmap_finish_one(struct xfs_trans *tp, struct xfs_rmap_intent *ri,
struct xfs_btree_cur **pcur);
int __xfs_rmap_finish_intent(struct xfs_btree_cur *rcur,
enum xfs_rmap_intent_type op, xfs_agblock_t bno,
xfs_extlen_t len, const struct xfs_owner_info *oinfo,
bool unwritten);
int xfs_rmap_lookup_le_range(struct xfs_btree_cur *cur, xfs_agblock_t bno,
uint64_t owner, uint64_t offset, unsigned int flags,
@ -195,7 +199,7 @@ int xfs_rmap_compare(const struct xfs_rmap_irec *a,
union xfs_btree_rec;
xfs_failaddr_t xfs_rmap_btrec_to_irec(const union xfs_btree_rec *rec,
struct xfs_rmap_irec *irec);
xfs_failaddr_t xfs_rmap_check_irec(struct xfs_btree_cur *cur,
xfs_failaddr_t xfs_rmap_check_irec(struct xfs_perag *pag,
const struct xfs_rmap_irec *irec);
int xfs_rmap_has_records(struct xfs_btree_cur *cur, xfs_agblock_t bno,
@ -235,4 +239,29 @@ extern struct kmem_cache *xfs_rmap_intent_cache;
int __init xfs_rmap_intent_init_cache(void);
void xfs_rmap_intent_destroy_cache(void);
/*
* Parameters for tracking reverse mapping changes. The hook function arg
* parameter is enum xfs_rmap_intent_type, and the rest is below.
*/
struct xfs_rmap_update_params {
xfs_agblock_t startblock;
xfs_extlen_t blockcount;
struct xfs_owner_info oinfo;
bool unwritten;
};
#ifdef CONFIG_XFS_LIVE_HOOKS
struct xfs_rmap_hook {
struct xfs_hook rmap_hook;
};
void xfs_rmap_hook_disable(void);
void xfs_rmap_hook_enable(void);
int xfs_rmap_hook_add(struct xfs_perag *pag, struct xfs_rmap_hook *hook);
void xfs_rmap_hook_del(struct xfs_perag *pag, struct xfs_rmap_hook *hook);
void xfs_rmap_hook_setup(struct xfs_rmap_hook *hook, notifier_fn_t mod_fn);
#endif
#endif /* __XFS_RMAP_H__ */

239
fs/xfs/libxfs/xfs_rmap_btree.c
View File

@ -16,11 +16,14 @@
#include "xfs_btree_staging.h"
#include "xfs_rmap.h"
#include "xfs_rmap_btree.h"
#include "xfs_health.h"
#include "xfs_trace.h"
#include "xfs_error.h"
#include "xfs_extent_busy.h"
#include "xfs_ag.h"
#include "xfs_ag_resv.h"
#include "xfs_buf_mem.h"
#include "xfs_btree_mem.h"
static struct kmem_cache *xfs_rmapbt_cur_cache;
@ -65,13 +68,12 @@ xfs_rmapbt_set_root(
{
struct xfs_buf *agbp = cur->bc_ag.agbp;
struct xfs_agf *agf = agbp->b_addr;
int btnum = cur->bc_btnum;
ASSERT(ptr->s != 0);
agf->agf_roots[btnum] = ptr->s;
be32_add_cpu(&agf->agf_levels[btnum], inc);
cur->bc_ag.pag->pagf_levels[btnum] += inc;
agf->agf_rmap_root = ptr->s;
be32_add_cpu(&agf->agf_rmap_level, inc);
cur->bc_ag.pag->pagf_rmap_level += inc;
xfs_alloc_log_agf(cur->bc_tp, agbp, XFS_AGF_ROOTS | XFS_AGF_LEVELS);
}
@ -94,8 +96,6 @@ xfs_rmapbt_alloc_block(
&bno, 1);
if (error)
return error;
trace_xfs_rmapbt_alloc_block(cur->bc_mp, pag->pag_agno, bno, 1);
if (bno == NULLAGBLOCK) {
*stat = 0;
return 0;
@ -125,8 +125,6 @@ xfs_rmapbt_free_block(
int error;
bno = xfs_daddr_to_agbno(cur->bc_mp, xfs_buf_daddr(bp));
trace_xfs_rmapbt_free_block(cur->bc_mp, pag->pag_agno,
bno, 1);
be32_add_cpu(&agf->agf_rmap_blocks, -1);
xfs_alloc_log_agf(cur->bc_tp, agbp, XFS_AGF_RMAP_BLOCKS);
error = xfs_alloc_put_freelist(pag, cur->bc_tp, agbp, NULL, bno, 1);
@ -226,7 +224,7 @@ xfs_rmapbt_init_ptr_from_cur(
ASSERT(cur->bc_ag.pag->pag_agno == be32_to_cpu(agf->agf_seqno));
ptr->s = agf->agf_roots[cur->bc_btnum];
ptr->s = agf->agf_rmap_root;
}
/*
@ -340,18 +338,29 @@ xfs_rmapbt_verify(
if (!xfs_has_rmapbt(mp))
return __this_address;
fa = xfs_btree_sblock_v5hdr_verify(bp);
fa = xfs_btree_agblock_v5hdr_verify(bp);
if (fa)
return fa;
level = be16_to_cpu(block->bb_level);
if (pag && xfs_perag_initialised_agf(pag)) {
if (level >= pag->pagf_levels[XFS_BTNUM_RMAPi])
unsigned int maxlevel = pag->pagf_rmap_level;
#ifdef CONFIG_XFS_ONLINE_REPAIR
/*
* Online repair could be rewriting the free space btrees, so
* we'll validate against the larger of either tree while this
* is going on.
*/
maxlevel = max_t(unsigned int, maxlevel,
pag->pagf_repair_rmap_level);
#endif
if (level >= maxlevel)
return __this_address;
} else if (level >= mp->m_rmap_maxlevels)
return __this_address;
return xfs_btree_sblock_verify(bp, mp->m_rmap_mxr[level != 0]);
return xfs_btree_agblock_verify(bp, mp->m_rmap_mxr[level != 0]);
}
static void
@ -360,7 +369,7 @@ xfs_rmapbt_read_verify(
{
xfs_failaddr_t fa;
if (!xfs_btree_sblock_verify_crc(bp))
if (!xfs_btree_agblock_verify_crc(bp))
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_rmapbt_verify(bp);
@ -384,7 +393,7 @@ xfs_rmapbt_write_verify(
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
return;
}
xfs_btree_sblock_calc_crc(bp);
xfs_btree_agblock_calc_crc(bp);
}
@ -476,9 +485,19 @@ xfs_rmapbt_keys_contiguous(
be32_to_cpu(key2->rmap.rm_startblock));
}
static const struct xfs_btree_ops xfs_rmapbt_ops = {
const struct xfs_btree_ops xfs_rmapbt_ops = {
.name = "rmap",
.type = XFS_BTREE_TYPE_AG,
.geom_flags = XFS_BTGEO_OVERLAPPING,
.rec_len = sizeof(struct xfs_rmap_rec),
/* Overlapping btree; 2 keys per pointer. */
.key_len = 2 * sizeof(struct xfs_rmap_key),
.ptr_len = XFS_BTREE_SHORT_PTR_LEN,
.lru_refs = XFS_RMAP_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_rmap_2),
.sick_mask = XFS_SICK_AG_RMAPBT,
.dup_cursor = xfs_rmapbt_dup_cursor,
.set_root = xfs_rmapbt_set_root,
@ -498,26 +517,11 @@ static const struct xfs_btree_ops xfs_rmapbt_ops = {
.keys_contiguous = xfs_rmapbt_keys_contiguous,
};
static struct xfs_btree_cur *
xfs_rmapbt_init_common(
struct xfs_mount *mp,
struct xfs_trans *tp,
struct xfs_perag *pag)
{
struct xfs_btree_cur *cur;
/* Overlapping btree; 2 keys per pointer. */
cur = xfs_btree_alloc_cursor(mp, tp, XFS_BTNUM_RMAP,
mp->m_rmap_maxlevels, xfs_rmapbt_cur_cache);
cur->bc_flags = XFS_BTREE_CRC_BLOCKS | XFS_BTREE_OVERLAPPING;
cur->bc_statoff = XFS_STATS_CALC_INDEX(xs_rmap_2);
cur->bc_ops = &xfs_rmapbt_ops;
cur->bc_ag.pag = xfs_perag_hold(pag);
return cur;
}
/* Create a new reverse mapping btree cursor. */
/*
* Create a new reverse mapping btree cursor.
*
* For staging cursors tp and agbp are NULL.
*/
struct xfs_btree_cur *
xfs_rmapbt_init_cursor(
struct xfs_mount *mp,
@ -525,29 +529,165 @@ xfs_rmapbt_init_cursor(
struct xfs_buf *agbp,
struct xfs_perag *pag)
{
struct xfs_agf *agf = agbp->b_addr;
struct xfs_btree_cur *cur;
cur = xfs_rmapbt_init_common(mp, tp, pag);
cur->bc_nlevels = be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_rmapbt_ops,
mp->m_rmap_maxlevels, xfs_rmapbt_cur_cache);
cur->bc_ag.pag = xfs_perag_hold(pag);
cur->bc_ag.agbp = agbp;
if (agbp) {
struct xfs_agf *agf = agbp->b_addr;
cur->bc_nlevels = be32_to_cpu(agf->agf_rmap_level);
}
return cur;
}
/* Create a new reverse mapping btree cursor with a fake root for staging. */
#ifdef CONFIG_XFS_BTREE_IN_MEM
static inline unsigned int
xfs_rmapbt_mem_block_maxrecs(
unsigned int blocklen,
bool leaf)
{
if (leaf)
return blocklen / sizeof(struct xfs_rmap_rec);
return blocklen /
(2 * sizeof(struct xfs_rmap_key) + sizeof(__be64));
}
/*
* Validate an in-memory rmap btree block. Callers are allowed to generate an
* in-memory btree even if the ondisk feature is not enabled.
*/
static xfs_failaddr_t
xfs_rmapbt_mem_verify(
struct xfs_buf *bp)
{
struct xfs_btree_block *block = XFS_BUF_TO_BLOCK(bp);
xfs_failaddr_t fa;
unsigned int level;
unsigned int maxrecs;
if (!xfs_verify_magic(bp, block->bb_magic))
return __this_address;
fa = xfs_btree_fsblock_v5hdr_verify(bp, XFS_RMAP_OWN_UNKNOWN);
if (fa)
return fa;
level = be16_to_cpu(block->bb_level);
if (level >= xfs_rmapbt_maxlevels_ondisk())
return __this_address;
maxrecs = xfs_rmapbt_mem_block_maxrecs(
XFBNO_BLOCKSIZE - XFS_BTREE_LBLOCK_CRC_LEN, level == 0);
return xfs_btree_memblock_verify(bp, maxrecs);
}
static void
xfs_rmapbt_mem_rw_verify(
struct xfs_buf *bp)
{
xfs_failaddr_t fa = xfs_rmapbt_mem_verify(bp);
if (fa)
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
}
/* skip crc checks on in-memory btrees to save time */
static const struct xfs_buf_ops xfs_rmapbt_mem_buf_ops = {
.name = "xfs_rmapbt_mem",
.magic = { 0, cpu_to_be32(XFS_RMAP_CRC_MAGIC) },
.verify_read = xfs_rmapbt_mem_rw_verify,
.verify_write = xfs_rmapbt_mem_rw_verify,
.verify_struct = xfs_rmapbt_mem_verify,
};
const struct xfs_btree_ops xfs_rmapbt_mem_ops = {
.name = "mem_rmap",
.type = XFS_BTREE_TYPE_MEM,
.geom_flags = XFS_BTGEO_OVERLAPPING,
.rec_len = sizeof(struct xfs_rmap_rec),
/* Overlapping btree; 2 keys per pointer. */
.key_len = 2 * sizeof(struct xfs_rmap_key),
.ptr_len = XFS_BTREE_LONG_PTR_LEN,
.lru_refs = XFS_RMAP_BTREE_REF,
.statoff = XFS_STATS_CALC_INDEX(xs_rmap_mem_2),
.dup_cursor = xfbtree_dup_cursor,
.set_root = xfbtree_set_root,
.alloc_block = xfbtree_alloc_block,
.free_block = xfbtree_free_block,
.get_minrecs = xfbtree_get_minrecs,
.get_maxrecs = xfbtree_get_maxrecs,
.init_key_from_rec = xfs_rmapbt_init_key_from_rec,
.init_high_key_from_rec = xfs_rmapbt_init_high_key_from_rec,
.init_rec_from_cur = xfs_rmapbt_init_rec_from_cur,
.init_ptr_from_cur = xfbtree_init_ptr_from_cur,
.key_diff = xfs_rmapbt_key_diff,
.buf_ops = &xfs_rmapbt_mem_buf_ops,
.diff_two_keys = xfs_rmapbt_diff_two_keys,
.keys_inorder = xfs_rmapbt_keys_inorder,
.recs_inorder = xfs_rmapbt_recs_inorder,
.keys_contiguous = xfs_rmapbt_keys_contiguous,
};
/* Create a cursor for an in-memory btree. */
struct xfs_btree_cur *
xfs_rmapbt_stage_cursor(
struct xfs_mount *mp,
struct xbtree_afakeroot *afake,
struct xfs_perag *pag)
xfs_rmapbt_mem_cursor(
struct xfs_perag *pag,
struct xfs_trans *tp,
struct xfbtree *xfbt)
{
struct xfs_btree_cur *cur;
struct xfs_mount *mp = pag->pag_mount;
cur = xfs_rmapbt_init_common(mp, NULL, pag);
xfs_btree_stage_afakeroot(cur, afake);
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_rmapbt_mem_ops,
xfs_rmapbt_maxlevels_ondisk(), xfs_rmapbt_cur_cache);
cur->bc_mem.xfbtree = xfbt;
cur->bc_nlevels = xfbt->nlevels;
cur->bc_mem.pag = xfs_perag_hold(pag);
return cur;
}
/* Create an in-memory rmap btree. */
int
xfs_rmapbt_mem_init(
struct xfs_mount *mp,
struct xfbtree *xfbt,
struct xfs_buftarg *btp,
xfs_agnumber_t agno)
{
xfbt->owner = agno;
return xfbtree_init(mp, xfbt, btp, &xfs_rmapbt_mem_ops);
}
/* Compute the max possible height for reverse mapping btrees in memory. */
static unsigned int
xfs_rmapbt_mem_maxlevels(void)
{
unsigned int minrecs[2];
unsigned int blocklen;
blocklen = XFBNO_BLOCKSIZE - XFS_BTREE_LBLOCK_CRC_LEN;
minrecs[0] = xfs_rmapbt_mem_block_maxrecs(blocklen, true) / 2;
minrecs[1] = xfs_rmapbt_mem_block_maxrecs(blocklen, false) / 2;
/*
* How tall can an in-memory rmap btree become if we filled the entire
* AG with rmap records?
*/
return xfs_btree_compute_maxlevels(minrecs,
XFS_MAX_AG_BYTES / sizeof(struct xfs_rmap_rec));
}
#else
# define xfs_rmapbt_mem_maxlevels() (0)
#endif /* CONFIG_XFS_BTREE_IN_MEM */
/*
* Install a new reverse mapping btree root. Caller is responsible for
* invalidating and freeing the old btree blocks.
@ -563,12 +703,12 @@ xfs_rmapbt_commit_staged_btree(
ASSERT(cur->bc_flags & XFS_BTREE_STAGING);
agf->agf_roots[cur->bc_btnum] = cpu_to_be32(afake->af_root);
agf->agf_levels[cur->bc_btnum] = cpu_to_be32(afake->af_levels);
agf->agf_rmap_root = cpu_to_be32(afake->af_root);
agf->agf_rmap_level = cpu_to_be32(afake->af_levels);
agf->agf_rmap_blocks = cpu_to_be32(afake->af_blocks);
xfs_alloc_log_agf(tp, agbp, XFS_AGF_ROOTS | XFS_AGF_LEVELS |
XFS_AGF_RMAP_BLOCKS);
xfs_btree_commit_afakeroot(cur, tp, agbp, &xfs_rmapbt_ops);
xfs_btree_commit_afakeroot(cur, tp, agbp);
}
/* Calculate number of records in a reverse mapping btree block. */
@ -618,7 +758,8 @@ xfs_rmapbt_maxlevels_ondisk(void)
* like if it consumes almost all the blocks in the AG due to maximal
* sharing factor.
*/
return xfs_btree_space_to_height(minrecs, XFS_MAX_CRC_AG_BLOCKS);
return max(xfs_btree_space_to_height(minrecs, XFS_MAX_CRC_AG_BLOCKS),
xfs_rmapbt_mem_maxlevels());
}
/* Compute the maximum height of an rmap btree. */

8
fs/xfs/libxfs/xfs_rmap_btree.h
View File

@ -10,6 +10,7 @@ struct xfs_buf;
struct xfs_btree_cur;
struct xfs_mount;
struct xbtree_afakeroot;
struct xfbtree;
/* rmaps only exist on crc enabled filesystems */
#define XFS_RMAP_BLOCK_LEN XFS_BTREE_SBLOCK_CRC_LEN
@ -44,8 +45,6 @@ struct xbtree_afakeroot;
struct xfs_btree_cur *xfs_rmapbt_init_cursor(struct xfs_mount *mp,
struct xfs_trans *tp, struct xfs_buf *bp,
struct xfs_perag *pag);
struct xfs_btree_cur *xfs_rmapbt_stage_cursor(struct xfs_mount *mp,
struct xbtree_afakeroot *afake, struct xfs_perag *pag);
void xfs_rmapbt_commit_staged_btree(struct xfs_btree_cur *cur,
struct xfs_trans *tp, struct xfs_buf *agbp);
int xfs_rmapbt_maxrecs(int blocklen, int leaf);
@ -64,4 +63,9 @@ unsigned int xfs_rmapbt_maxlevels_ondisk(void);
int __init xfs_rmapbt_init_cur_cache(void);
void xfs_rmapbt_destroy_cur_cache(void);
struct xfs_btree_cur *xfs_rmapbt_mem_cursor(struct xfs_perag *pag,
struct xfs_trans *tp, struct xfbtree *xfbtree);
int xfs_rmapbt_mem_init(struct xfs_mount *mp, struct xfbtree *xfbtree,
struct xfs_buftarg *btp, xfs_agnumber_t agno);
#endif /* __XFS_RMAP_BTREE_H__ */

11
fs/xfs/libxfs/xfs_rtbitmap.c
View File

@ -17,6 +17,7 @@
#include "xfs_rtalloc.h"
#include "xfs_error.h"
#include "xfs_rtbitmap.h"
#include "xfs_health.h"
/*
* Realtime allocator bitmap functions shared with userspace.
@ -115,13 +116,19 @@ xfs_rtbuf_get(
if (error)
return error;
if (XFS_IS_CORRUPT(mp, nmap == 0 || !xfs_bmap_is_written_extent(&map)))
if (XFS_IS_CORRUPT(mp, nmap == 0 || !xfs_bmap_is_written_extent(&map))) {
xfs_rt_mark_sick(mp, issum ? XFS_SICK_RT_SUMMARY :
XFS_SICK_RT_BITMAP);
return -EFSCORRUPTED;
}
ASSERT(map.br_startblock != NULLFSBLOCK);
error = xfs_trans_read_buf(mp, args->tp, mp->m_ddev_targp,
XFS_FSB_TO_DADDR(mp, map.br_startblock),
mp->m_bsize, 0, &bp, &xfs_rtbuf_ops);
if (xfs_metadata_is_sick(error))
xfs_rt_mark_sick(mp, issum ? XFS_SICK_RT_SUMMARY :
XFS_SICK_RT_BITMAP);
if (error)
return error;
@ -934,7 +941,7 @@ xfs_rtfree_extent(
struct timespec64 atime;
ASSERT(mp->m_rbmip->i_itemp != NULL);
ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL));
xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL);
error = xfs_rtcheck_alloc_range(&args, start, len);
if (error)

2
fs/xfs/libxfs/xfs_sb.c
View File

@ -1290,6 +1290,8 @@ xfs_sb_read_secondary(
error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp,
XFS_AG_DADDR(mp, agno, XFS_SB_BLOCK(mp)),
XFS_FSS_TO_BB(mp, 1), 0, &bp, &xfs_sb_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_agno_mark_sick(mp, agno, XFS_SICK_AG_SB);
if (error)
return error;
xfs_buf_set_ref(bp, XFS_SSB_REF);

67
fs/xfs/libxfs/xfs_shared.h
View File

@ -43,6 +43,60 @@ extern const struct xfs_buf_ops xfs_sb_buf_ops;
extern const struct xfs_buf_ops xfs_sb_quiet_buf_ops;
extern const struct xfs_buf_ops xfs_symlink_buf_ops;
/* btree ops */
extern const struct xfs_btree_ops xfs_bnobt_ops;
extern const struct xfs_btree_ops xfs_cntbt_ops;
extern const struct xfs_btree_ops xfs_inobt_ops;
extern const struct xfs_btree_ops xfs_finobt_ops;
extern const struct xfs_btree_ops xfs_bmbt_ops;
extern const struct xfs_btree_ops xfs_refcountbt_ops;
extern const struct xfs_btree_ops xfs_rmapbt_ops;
extern const struct xfs_btree_ops xfs_rmapbt_mem_ops;
static inline bool xfs_btree_is_bno(const struct xfs_btree_ops *ops)
{
return ops == &xfs_bnobt_ops;
}
static inline bool xfs_btree_is_cnt(const struct xfs_btree_ops *ops)
{
return ops == &xfs_cntbt_ops;
}
static inline bool xfs_btree_is_bmap(const struct xfs_btree_ops *ops)
{
return ops == &xfs_bmbt_ops;
}
static inline bool xfs_btree_is_ino(const struct xfs_btree_ops *ops)
{
return ops == &xfs_inobt_ops;
}
static inline bool xfs_btree_is_fino(const struct xfs_btree_ops *ops)
{
return ops == &xfs_finobt_ops;
}
static inline bool xfs_btree_is_refcount(const struct xfs_btree_ops *ops)
{
return ops == &xfs_refcountbt_ops;
}
static inline bool xfs_btree_is_rmap(const struct xfs_btree_ops *ops)
{
return ops == &xfs_rmapbt_ops;
}
#ifdef CONFIG_XFS_BTREE_IN_MEM
static inline bool xfs_btree_is_mem_rmap(const struct xfs_btree_ops *ops)
{
return ops == &xfs_rmapbt_mem_ops;
}
#else
# define xfs_btree_is_mem_rmap(...) (false)
#endif
/* log size calculation functions */
int xfs_log_calc_unit_res(struct xfs_mount *mp, int unit_bytes);
int xfs_log_calc_minimum_size(struct xfs_mount *);
@ -128,19 +182,6 @@ void xfs_log_get_max_trans_res(struct xfs_mount *mp,
#define XFS_ICHGTIME_CHG 0x2 /* inode field change timestamp */
#define XFS_ICHGTIME_CREATE 0x4 /* inode create timestamp */
/*
* Symlink decoding/encoding functions
*/
int xfs_symlink_blocks(struct xfs_mount *mp, int pathlen);
int xfs_symlink_hdr_set(struct xfs_mount *mp, xfs_ino_t ino, uint32_t offset,
uint32_t size, struct xfs_buf *bp);
bool xfs_symlink_hdr_ok(xfs_ino_t ino, uint32_t offset,
uint32_t size, struct xfs_buf *bp);
void xfs_symlink_local_to_remote(struct xfs_trans *tp, struct xfs_buf *bp,
struct xfs_inode *ip, struct xfs_ifork *ifp);
xfs_failaddr_t xfs_symlink_shortform_verify(void *sfp, int64_t size);
/* Computed inode geometry for the filesystem. */
struct xfs_ino_geometry {
/* Maximum inode count in this filesystem. */

155
fs/xfs/libxfs/xfs_symlink_remote.c
View File

@ -16,7 +16,10 @@
#include "xfs_trans.h"
#include "xfs_buf_item.h"
#include "xfs_log.h"
#include "xfs_symlink_remote.h"
#include "xfs_bit.h"
#include "xfs_bmap.h"
#include "xfs_health.h"
/*
* Each contiguous block has a header, so it is not just a simple pathlen
@ -227,3 +230,153 @@ xfs_symlink_shortform_verify(
return __this_address;
return NULL;
}
/* Read a remote symlink target into the buffer. */
int
xfs_symlink_remote_read(
struct xfs_inode *ip,
char *link)
{
struct xfs_mount *mp = ip->i_mount;
struct xfs_bmbt_irec mval[XFS_SYMLINK_MAPS];
struct xfs_buf *bp;
xfs_daddr_t d;
char *cur_chunk;
int pathlen = ip->i_disk_size;
int nmaps = XFS_SYMLINK_MAPS;
int byte_cnt;
int n;
int error = 0;
int fsblocks = 0;
int offset;
xfs_assert_ilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL);
fsblocks = xfs_symlink_blocks(mp, pathlen);
error = xfs_bmapi_read(ip, 0, fsblocks, mval, &nmaps, 0);
if (error)
goto out;
offset = 0;
for (n = 0; n < nmaps; n++) {
d = XFS_FSB_TO_DADDR(mp, mval[n].br_startblock);
byte_cnt = XFS_FSB_TO_B(mp, mval[n].br_blockcount);
error = xfs_buf_read(mp->m_ddev_targp, d, BTOBB(byte_cnt), 0,
&bp, &xfs_symlink_buf_ops);
if (xfs_metadata_is_sick(error))
xfs_inode_mark_sick(ip, XFS_SICK_INO_SYMLINK);
if (error)
return error;
byte_cnt = XFS_SYMLINK_BUF_SPACE(mp, byte_cnt);
if (pathlen < byte_cnt)
byte_cnt = pathlen;
cur_chunk = bp->b_addr;
if (xfs_has_crc(mp)) {
if (!xfs_symlink_hdr_ok(ip->i_ino, offset,
byte_cnt, bp)) {
xfs_inode_mark_sick(ip, XFS_SICK_INO_SYMLINK);
error = -EFSCORRUPTED;
xfs_alert(mp,
"symlink header does not match required off/len/owner (0x%x/0x%x,0x%llx)",
offset, byte_cnt, ip->i_ino);
xfs_buf_relse(bp);
goto out;
}
cur_chunk += sizeof(struct xfs_dsymlink_hdr);
}
memcpy(link + offset, cur_chunk, byte_cnt);
pathlen -= byte_cnt;
offset += byte_cnt;
xfs_buf_relse(bp);
}
ASSERT(pathlen == 0);
link[ip->i_disk_size] = '\0';
error = 0;
out:
return error;
}
/* Write the symlink target into the inode. */
int
xfs_symlink_write_target(
struct xfs_trans *tp,
struct xfs_inode *ip,
const char *target_path,
int pathlen,
xfs_fsblock_t fs_blocks,
uint resblks)
{
struct xfs_bmbt_irec mval[XFS_SYMLINK_MAPS];
struct xfs_mount *mp = tp->t_mountp;
const char *cur_chunk;
struct xfs_buf *bp;
xfs_daddr_t d;
int byte_cnt;
int nmaps;
int offset = 0;
int n;
int error;
/*
* If the symlink will fit into the inode, write it inline.
*/
if (pathlen <= xfs_inode_data_fork_size(ip)) {
xfs_init_local_fork(ip, XFS_DATA_FORK, target_path, pathlen);
ip->i_disk_size = pathlen;
ip->i_df.if_format = XFS_DINODE_FMT_LOCAL;
xfs_trans_log_inode(tp, ip, XFS_ILOG_DDATA | XFS_ILOG_CORE);
return 0;
}
nmaps = XFS_SYMLINK_MAPS;
error = xfs_bmapi_write(tp, ip, 0, fs_blocks, XFS_BMAPI_METADATA,
resblks, mval, &nmaps);
if (error)
return error;
ip->i_disk_size = pathlen;
xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
cur_chunk = target_path;
offset = 0;
for (n = 0; n < nmaps; n++) {
char *buf;
d = XFS_FSB_TO_DADDR(mp, mval[n].br_startblock);
byte_cnt = XFS_FSB_TO_B(mp, mval[n].br_blockcount);
error = xfs_trans_get_buf(tp, mp->m_ddev_targp, d,
BTOBB(byte_cnt), 0, &bp);
if (error)
return error;
bp->b_ops = &xfs_symlink_buf_ops;
byte_cnt = XFS_SYMLINK_BUF_SPACE(mp, byte_cnt);
byte_cnt = min(byte_cnt, pathlen);
buf = bp->b_addr;
buf += xfs_symlink_hdr_set(mp, ip->i_ino, offset, byte_cnt,
bp);
memcpy(buf, cur_chunk, byte_cnt);
cur_chunk += byte_cnt;
pathlen -= byte_cnt;
offset += byte_cnt;
xfs_trans_buf_set_type(tp, bp, XFS_BLFT_SYMLINK_BUF);
xfs_trans_log_buf(tp, bp, 0, (buf + byte_cnt - 1) -
(char *)bp->b_addr);
}
ASSERT(pathlen == 0);
return 0;
}

26
fs/xfs/libxfs/xfs_symlink_remote.h Normal file
View File

@ -0,0 +1,26 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (c) 2000-2005 Silicon Graphics, Inc.
* Copyright (c) 2013 Red Hat, Inc.
* All Rights Reserved.
*/
#ifndef __XFS_SYMLINK_REMOTE_H
#define __XFS_SYMLINK_REMOTE_H
/*
* Symlink decoding/encoding functions
*/
int xfs_symlink_blocks(struct xfs_mount *mp, int pathlen);
int xfs_symlink_hdr_set(struct xfs_mount *mp, xfs_ino_t ino, uint32_t offset,
uint32_t size, struct xfs_buf *bp);
bool xfs_symlink_hdr_ok(xfs_ino_t ino, uint32_t offset,
uint32_t size, struct xfs_buf *bp);
void xfs_symlink_local_to_remote(struct xfs_trans *tp, struct xfs_buf *bp,
struct xfs_inode *ip, struct xfs_ifork *ifp);
xfs_failaddr_t xfs_symlink_shortform_verify(void *sfp, int64_t size);
int xfs_symlink_remote_read(struct xfs_inode *ip, char *link);
int xfs_symlink_write_target(struct xfs_trans *tp, struct xfs_inode *ip,
const char *target_path, int pathlen, xfs_fsblock_t fs_blocks,
uint resblks);
#endif /* __XFS_SYMLINK_REMOTE_H */

6
fs/xfs/libxfs/xfs_trans_inode.c
View File

@ -31,7 +31,7 @@ xfs_trans_ijoin(
{
struct xfs_inode_log_item *iip;
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL);
if (ip->i_itemp == NULL)
xfs_inode_item_init(ip, ip->i_mount);
iip = ip->i_itemp;
@ -60,7 +60,7 @@ xfs_trans_ichgtime(
struct timespec64 tv;
ASSERT(tp);
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL);
tv = current_time(inode);
@ -90,7 +90,7 @@ xfs_trans_log_inode(
struct inode *inode = VFS_I(ip);
ASSERT(iip);
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL);
ASSERT(!xfs_iflags_test(ip, XFS_ISTALE));
tp->t_flags |= XFS_TRANS_DIRTY;

26
fs/xfs/libxfs/xfs_types.h
View File

@ -80,11 +80,13 @@ typedef void * xfs_failaddr_t;
/*
* Inode fork identifiers.
*/
#define XFS_DATA_FORK 0
#define XFS_ATTR_FORK 1
#define XFS_COW_FORK 2
#define XFS_STAGING_FORK (-1) /* fake fork for staging a btree */
#define XFS_DATA_FORK (0)
#define XFS_ATTR_FORK (1)
#define XFS_COW_FORK (2)
#define XFS_WHICHFORK_STRINGS \
{ XFS_STAGING_FORK, "staging" }, \
{ XFS_DATA_FORK, "data" }, \
{ XFS_ATTR_FORK, "attr" }, \
{ XFS_COW_FORK, "cow" }
@ -114,24 +116,6 @@ typedef enum {
{ XFS_LOOKUP_LEi, "le" }, \
{ XFS_LOOKUP_GEi, "ge" }
/*
* This enum is used in string mapping in xfs_trace.h and scrub/trace.h;
* please keep the TRACE_DEFINE_ENUMs for it up to date.
*/
typedef enum {
XFS_BTNUM_BNOi, XFS_BTNUM_CNTi, XFS_BTNUM_RMAPi, XFS_BTNUM_BMAPi,
XFS_BTNUM_INOi, XFS_BTNUM_FINOi, XFS_BTNUM_REFCi, XFS_BTNUM_MAX
} xfs_btnum_t;
#define XFS_BTNUM_STRINGS \
{ XFS_BTNUM_BNOi, "bnobt" }, \
{ XFS_BTNUM_CNTi, "cntbt" }, \
{ XFS_BTNUM_RMAPi, "rmapbt" }, \
{ XFS_BTNUM_BMAPi, "bmbt" }, \
{ XFS_BTNUM_INOi, "inobt" }, \
{ XFS_BTNUM_FINOi, "finobt" }, \
{ XFS_BTNUM_REFCi, "refcbt" }
struct xfs_name {
const unsigned char *name;
int len;

78
fs/xfs/mrlock.h
View File

@ -1,78 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (c) 2000-2006 Silicon Graphics, Inc.
* All Rights Reserved.
*/
#ifndef __XFS_SUPPORT_MRLOCK_H__
#define __XFS_SUPPORT_MRLOCK_H__
#include <linux/rwsem.h>
typedef struct {
struct rw_semaphore mr_lock;
#if defined(DEBUG) || defined(XFS_WARN)
int mr_writer;
#endif
} mrlock_t;
#if defined(DEBUG) || defined(XFS_WARN)
#define mrinit(mrp, name) \
do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0)
#else
#define mrinit(mrp, name) \
do { init_rwsem(&(mrp)->mr_lock); } while (0)
#endif
#define mrlock_init(mrp, t,n,s) mrinit(mrp, n)
#define mrfree(mrp) do { } while (0)
static inline void mraccess_nested(mrlock_t *mrp, int subclass)
{
down_read_nested(&mrp->mr_lock, subclass);
}
static inline void mrupdate_nested(mrlock_t *mrp, int subclass)
{
down_write_nested(&mrp->mr_lock, subclass);
#if defined(DEBUG) || defined(XFS_WARN)
mrp->mr_writer = 1;
#endif
}
static inline int mrtryaccess(mrlock_t *mrp)
{
return down_read_trylock(&mrp->mr_lock);
}
static inline int mrtryupdate(mrlock_t *mrp)
{
if (!down_write_trylock(&mrp->mr_lock))
return 0;
#if defined(DEBUG) || defined(XFS_WARN)
mrp->mr_writer = 1;
#endif
return 1;
}
static inline void mrunlock_excl(mrlock_t *mrp)
{
#if defined(DEBUG) || defined(XFS_WARN)
mrp->mr_writer = 0;
#endif
up_write(&mrp->mr_lock);
}
static inline void mrunlock_shared(mrlock_t *mrp)
{
up_read(&mrp->mr_lock);
}
static inline void mrdemote(mrlock_t *mrp)
{
#if defined(DEBUG) || defined(XFS_WARN)
mrp->mr_writer = 0;
#endif
downgrade_write(&mrp->mr_lock);
}
#endif /* __XFS_SUPPORT_MRLOCK_H__ */

5
fs/xfs/scrub/agb_bitmap.h
View File

@ -65,4 +65,9 @@ int xagb_bitmap_set_btblocks(struct xagb_bitmap *bitmap,
int xagb_bitmap_set_btcur_path(struct xagb_bitmap *bitmap,
struct xfs_btree_cur *cur);
static inline uint32_t xagb_bitmap_count_set_regions(struct xagb_bitmap *b)
{
return xbitmap32_count_set_regions(&b->agbitmap);
}
#endif /* __XFS_SCRUB_AGB_BITMAP_H__ */

12
fs/xfs/scrub/agheader.c
View File

@ -556,28 +556,28 @@ xchk_agf(
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
/* Check the AGF btree roots and levels */
agbno = be32_to_cpu(agf->agf_roots[XFS_BTNUM_BNO]);
agbno = be32_to_cpu(agf->agf_bno_root);
if (!xfs_verify_agbno(pag, agbno))
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
agbno = be32_to_cpu(agf->agf_roots[XFS_BTNUM_CNT]);
agbno = be32_to_cpu(agf->agf_cnt_root);
if (!xfs_verify_agbno(pag, agbno))
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
level = be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]);
level = be32_to_cpu(agf->agf_bno_level);
if (level <= 0 || level > mp->m_alloc_maxlevels)
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
level = be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]);
level = be32_to_cpu(agf->agf_cnt_level);
if (level <= 0 || level > mp->m_alloc_maxlevels)
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
if (xfs_has_rmapbt(mp)) {
agbno = be32_to_cpu(agf->agf_roots[XFS_BTNUM_RMAP]);
agbno = be32_to_cpu(agf->agf_rmap_root);
if (!xfs_verify_agbno(pag, agbno))
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
level = be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]);
level = be32_to_cpu(agf->agf_rmap_level);
if (level <= 0 || level > mp->m_rmap_maxlevels)
xchk_block_set_corrupt(sc, sc->sa.agf_bp);
}

47
fs/xfs/scrub/agheader_repair.c
View File

@ -174,8 +174,7 @@ xrep_agf_find_btrees(
* We relied on the rmapbt to reconstruct the AGF. If we get a
* different root then something's seriously wrong.
*/
if (fab[XREP_AGF_RMAPBT].root !=
be32_to_cpu(old_agf->agf_roots[XFS_BTNUM_RMAPi]))
if (fab[XREP_AGF_RMAPBT].root != be32_to_cpu(old_agf->agf_rmap_root))
return -EFSCORRUPTED;
/* We must find the refcountbt root if that feature is enabled. */
@ -224,20 +223,14 @@ xrep_agf_set_roots(
struct xfs_agf *agf,
struct xrep_find_ag_btree *fab)
{
agf->agf_roots[XFS_BTNUM_BNOi] =
cpu_to_be32(fab[XREP_AGF_BNOBT].root);
agf->agf_levels[XFS_BTNUM_BNOi] =
cpu_to_be32(fab[XREP_AGF_BNOBT].height);
agf->agf_bno_root = cpu_to_be32(fab[XREP_AGF_BNOBT].root);
agf->agf_bno_level = cpu_to_be32(fab[XREP_AGF_BNOBT].height);
agf->agf_roots[XFS_BTNUM_CNTi] =
cpu_to_be32(fab[XREP_AGF_CNTBT].root);
agf->agf_levels[XFS_BTNUM_CNTi] =
cpu_to_be32(fab[XREP_AGF_CNTBT].height);
agf->agf_cnt_root = cpu_to_be32(fab[XREP_AGF_CNTBT].root);
agf->agf_cnt_level = cpu_to_be32(fab[XREP_AGF_CNTBT].height);
agf->agf_roots[XFS_BTNUM_RMAPi] =
cpu_to_be32(fab[XREP_AGF_RMAPBT].root);
agf->agf_levels[XFS_BTNUM_RMAPi] =
cpu_to_be32(fab[XREP_AGF_RMAPBT].height);
agf->agf_rmap_root = cpu_to_be32(fab[XREP_AGF_RMAPBT].root);
agf->agf_rmap_level = cpu_to_be32(fab[XREP_AGF_RMAPBT].height);
if (xfs_has_reflink(sc->mp)) {
agf->agf_refcount_root =
@ -262,8 +255,7 @@ xrep_agf_calc_from_btrees(
int error;
/* Update the AGF counters from the bnobt. */
cur = xfs_allocbt_init_cursor(mp, sc->tp, agf_bp,
sc->sa.pag, XFS_BTNUM_BNO);
cur = xfs_bnobt_init_cursor(mp, sc->tp, agf_bp, sc->sa.pag);
error = xfs_alloc_query_all(cur, xrep_agf_walk_allocbt, &raa);
if (error)
goto err;
@ -276,8 +268,7 @@ xrep_agf_calc_from_btrees(
agf->agf_longest = cpu_to_be32(raa.longest);
/* Update the AGF counters from the cntbt. */
cur = xfs_allocbt_init_cursor(mp, sc->tp, agf_bp,
sc->sa.pag, XFS_BTNUM_CNT);
cur = xfs_cntbt_init_cursor(mp, sc->tp, agf_bp, sc->sa.pag);
error = xfs_btree_count_blocks(cur, &blocks);
if (error)
goto err;
@ -333,12 +324,9 @@ xrep_agf_commit_new(
pag->pagf_btreeblks = be32_to_cpu(agf->agf_btreeblks);
pag->pagf_freeblks = be32_to_cpu(agf->agf_freeblks);
pag->pagf_longest = be32_to_cpu(agf->agf_longest);
pag->pagf_levels[XFS_BTNUM_BNOi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNOi]);
pag->pagf_levels[XFS_BTNUM_CNTi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNTi]);
pag->pagf_levels[XFS_BTNUM_RMAPi] =
be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAPi]);
pag->pagf_bno_level = be32_to_cpu(agf->agf_bno_level);
pag->pagf_cnt_level = be32_to_cpu(agf->agf_cnt_level);
pag->pagf_rmap_level = be32_to_cpu(agf->agf_rmap_level);
pag->pagf_refcount_level = be32_to_cpu(agf->agf_refcount_level);
set_bit(XFS_AGSTATE_AGF_INIT, &pag->pag_opstate);
@ -559,16 +547,14 @@ xrep_agfl_collect_blocks(
goto out_bmp;
/* Find all blocks currently being used by the bnobt. */
cur = xfs_allocbt_init_cursor(mp, sc->tp, agf_bp,
sc->sa.pag, XFS_BTNUM_BNO);
cur = xfs_bnobt_init_cursor(mp, sc->tp, agf_bp, sc->sa.pag);
error = xagb_bitmap_set_btblocks(&ra.agmetablocks, cur);
xfs_btree_del_cursor(cur, error);
if (error)
goto out_bmp;
/* Find all blocks currently being used by the cntbt. */
cur = xfs_allocbt_init_cursor(mp, sc->tp, agf_bp,
sc->sa.pag, XFS_BTNUM_CNT);
cur = xfs_cntbt_init_cursor(mp, sc->tp, agf_bp, sc->sa.pag);
error = xagb_bitmap_set_btblocks(&ra.agmetablocks, cur);
xfs_btree_del_cursor(cur, error);
if (error)
@ -908,7 +894,7 @@ xrep_agi_calc_from_btrees(
xfs_agino_t freecount;
int error;
cur = xfs_inobt_init_cursor(sc->sa.pag, sc->tp, agi_bp, XFS_BTNUM_INO);
cur = xfs_inobt_init_cursor(sc->sa.pag, sc->tp, agi_bp);
error = xfs_ialloc_count_inodes(cur, &count, &freecount);
if (error)
goto err;
@ -928,8 +914,7 @@ xrep_agi_calc_from_btrees(
if (xfs_has_finobt(mp) && xfs_has_inobtcounts(mp)) {
xfs_agblock_t blocks;
cur = xfs_inobt_init_cursor(sc->sa.pag, sc->tp, agi_bp,
XFS_BTNUM_FINO);
cur = xfs_finobt_init_cursor(sc->sa.pag, sc->tp, agi_bp);
error = xfs_btree_count_blocks(cur, &blocks);
if (error)
goto err;

27
fs/xfs/scrub/alloc_repair.c
View File

@ -687,8 +687,8 @@ xrep_abt_reset_counters(
* height values before re-initializing the perag info from the updated
* AGF to capture all the new values.
*/
pag->pagf_repair_levels[XFS_BTNUM_BNOi] = pag->pagf_levels[XFS_BTNUM_BNOi];
pag->pagf_repair_levels[XFS_BTNUM_CNTi] = pag->pagf_levels[XFS_BTNUM_CNTi];
pag->pagf_repair_bno_level = pag->pagf_bno_level;
pag->pagf_repair_cnt_level = pag->pagf_cnt_level;
/* Reinitialize with the values we just logged. */
return xrep_reinit_pagf(sc);
@ -735,10 +735,11 @@ xrep_abt_build_new_trees(
ra->new_cntbt.bload.claim_block = xrep_abt_claim_block;
/* Allocate cursors for the staged btrees. */
bno_cur = xfs_allocbt_stage_cursor(sc->mp, &ra->new_bnobt.afake,
pag, XFS_BTNUM_BNO);
cnt_cur = xfs_allocbt_stage_cursor(sc->mp, &ra->new_cntbt.afake,
pag, XFS_BTNUM_CNT);
bno_cur = xfs_bnobt_init_cursor(sc->mp, NULL, NULL, pag);
xfs_btree_stage_afakeroot(bno_cur, &ra->new_bnobt.afake);
cnt_cur = xfs_cntbt_init_cursor(sc->mp, NULL, NULL, pag);
xfs_btree_stage_afakeroot(cnt_cur, &ra->new_cntbt.afake);
/* Last chance to abort before we start committing fixes. */
if (xchk_should_terminate(sc, &error))
@ -765,10 +766,8 @@ xrep_abt_build_new_trees(
* height so that we don't trip the verifiers when writing the new
* btree blocks to disk.
*/
pag->pagf_repair_levels[XFS_BTNUM_BNOi] =
ra->new_bnobt.bload.btree_height;
pag->pagf_repair_levels[XFS_BTNUM_CNTi] =
ra->new_cntbt.bload.btree_height;
pag->pagf_repair_bno_level = ra->new_bnobt.bload.btree_height;
pag->pagf_repair_cnt_level = ra->new_cntbt.bload.btree_height;
/* Load the free space by length tree. */
ra->array_cur = XFARRAY_CURSOR_INIT;
@ -807,8 +806,8 @@ xrep_abt_build_new_trees(
return xrep_roll_ag_trans(sc);
err_levels:
pag->pagf_repair_levels[XFS_BTNUM_BNOi] = 0;
pag->pagf_repair_levels[XFS_BTNUM_CNTi] = 0;
pag->pagf_repair_bno_level = 0;
pag->pagf_repair_cnt_level = 0;
err_cur:
xfs_btree_del_cursor(cnt_cur, error);
xfs_btree_del_cursor(bno_cur, error);
@ -838,8 +837,8 @@ xrep_abt_remove_old_trees(
* Now that we've zapped all the old allocbt blocks we can turn off
* the alternate height mechanism.
*/
pag->pagf_repair_levels[XFS_BTNUM_BNOi] = 0;
pag->pagf_repair_levels[XFS_BTNUM_CNTi] = 0;
pag->pagf_repair_bno_level = 0;
pag->pagf_repair_cnt_level = 0;
return 0;
}

14
fs/xfs/scrub/bitmap.c
View File

@ -566,3 +566,17 @@ xbitmap32_test(
*len = bn->bn_start - start;
return false;
}
/* Count the number of set regions in this bitmap. */
uint32_t
xbitmap32_count_set_regions(
struct xbitmap32 *bitmap)
{
struct xbitmap32_node *bn;
uint32_t nr = 0;
for_each_xbitmap32_extent(bn, bitmap)
nr++;
return nr;
}

2
fs/xfs/scrub/bitmap.h
View File

@ -62,4 +62,6 @@ int xbitmap32_walk(struct xbitmap32 *bitmap, xbitmap32_walk_fn fn,
bool xbitmap32_empty(struct xbitmap32 *bitmap);
bool xbitmap32_test(struct xbitmap32 *bitmap, uint32_t start, uint32_t *len);
uint32_t xbitmap32_count_set_regions(struct xbitmap32 *bitmap);
#endif /* __XFS_SCRUB_BITMAP_H__ */

2
fs/xfs/scrub/bmap.c
View File

@ -924,7 +924,7 @@ xchk_bmap(
if (!ifp)
return -ENOENT;
info.is_rt = whichfork == XFS_DATA_FORK && XFS_IS_REALTIME_INODE(ip);
info.is_rt = xfs_ifork_is_realtime(ip, whichfork);
info.whichfork = whichfork;
info.is_shared = whichfork == XFS_DATA_FORK && xfs_is_reflink_inode(ip);
info.sc = sc;

8
fs/xfs/scrub/bmap_repair.c
View File

@ -639,7 +639,13 @@ xrep_bmap_build_new_fork(
rb->new_bmapbt.bload.get_records = xrep_bmap_get_records;
rb->new_bmapbt.bload.claim_block = xrep_bmap_claim_block;
rb->new_bmapbt.bload.iroot_size = xrep_bmap_iroot_size;
bmap_cur = xfs_bmbt_stage_cursor(sc->mp, sc->ip, ifake);
/*
* Allocate a new bmap btree cursor for reloading an inode block mapping
* data structure.
*/
bmap_cur = xfs_bmbt_init_cursor(sc->mp, NULL, sc->ip, XFS_STAGING_FORK);
xfs_btree_stage_ifakeroot(bmap_cur, ifake);
/*
* Figure out the size and format of the new fork, then fill it with

58
fs/xfs/scrub/btree.c
View File

@ -47,7 +47,7 @@ __xchk_btree_process_error(
*error = 0;
fallthrough;
default:
if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE)
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE)
trace_xchk_ifork_btree_op_error(sc, cur, level,
*error, ret_ip);
else
@ -91,7 +91,7 @@ __xchk_btree_set_corrupt(
{
sc->sm->sm_flags |= errflag;
if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE)
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE)
trace_xchk_ifork_btree_error(sc, cur, level,
ret_ip);
else
@ -168,7 +168,7 @@ xchk_btree_rec(
if (xfs_btree_keycmp_lt(cur, &key, keyp))
xchk_btree_set_corrupt(bs->sc, cur, 1);
if (!(cur->bc_flags & XFS_BTREE_OVERLAPPING))
if (!(cur->bc_ops->geom_flags & XFS_BTGEO_OVERLAPPING))
return;
/* Is high_key(rec) no larger than the parent high key? */
@ -215,7 +215,7 @@ xchk_btree_key(
if (xfs_btree_keycmp_lt(cur, key, keyp))
xchk_btree_set_corrupt(bs->sc, cur, level);
if (!(cur->bc_flags & XFS_BTREE_OVERLAPPING))
if (!(cur->bc_ops->geom_flags & XFS_BTGEO_OVERLAPPING))
return;
/* Is this block's high key no larger than the parent high key? */
@ -236,22 +236,18 @@ xchk_btree_ptr_ok(
int level,
union xfs_btree_ptr *ptr)
{
bool res;
/* A btree rooted in an inode has no block pointer to the root. */
if ((bs->cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) &&
if (bs->cur->bc_ops->type == XFS_BTREE_TYPE_INODE &&
level == bs->cur->bc_nlevels)
return true;
/* Otherwise, check the pointers. */
if (bs->cur->bc_flags & XFS_BTREE_LONG_PTRS)
res = xfs_btree_check_lptr(bs->cur, be64_to_cpu(ptr->l), level);
else
res = xfs_btree_check_sptr(bs->cur, be32_to_cpu(ptr->s), level);
if (!res)
if (__xfs_btree_check_ptr(bs->cur, ptr, 0, level)) {
xchk_btree_set_corrupt(bs->sc, bs->cur, level);
return false;
}
return res;
return true;
}
/* Check that a btree block's sibling matches what we expect it. */
@ -374,18 +370,21 @@ xchk_btree_check_block_owner(
{
xfs_agnumber_t agno;
xfs_agblock_t agbno;
xfs_btnum_t btnum;
bool init_sa;
int error = 0;
if (!bs->cur)
return 0;
btnum = bs->cur->bc_btnum;
agno = xfs_daddr_to_agno(bs->cur->bc_mp, daddr);
agbno = xfs_daddr_to_agbno(bs->cur->bc_mp, daddr);
init_sa = bs->cur->bc_flags & XFS_BTREE_LONG_PTRS;
/*
* If the btree being examined is not itself a per-AG btree, initialize
* sc->sa so that we can check for the presence of an ownership record
* in the rmap btree for the AG containing the block.
*/
init_sa = bs->cur->bc_ops->type != XFS_BTREE_TYPE_AG;
if (init_sa) {
error = xchk_ag_init_existing(bs->sc, agno, &bs->sc->sa);
if (!xchk_btree_xref_process_error(bs->sc, bs->cur,
@ -399,11 +398,11 @@ xchk_btree_check_block_owner(
* have to nullify it (to shut down further block owner checks) if
* self-xref encounters problems.
*/
if (!bs->sc->sa.bno_cur && btnum == XFS_BTNUM_BNO)
if (!bs->sc->sa.bno_cur && xfs_btree_is_bno(bs->cur->bc_ops))
bs->cur = NULL;
xchk_xref_is_only_owned_by(bs->sc, agbno, 1, bs->oinfo);
if (!bs->sc->sa.rmap_cur && btnum == XFS_BTNUM_RMAP)
if (!bs->sc->sa.rmap_cur && xfs_btree_is_rmap(bs->cur->bc_ops))
bs->cur = NULL;
out_free:
@ -429,7 +428,7 @@ xchk_btree_check_owner(
* up.
*/
if (bp == NULL) {
if (!(cur->bc_flags & XFS_BTREE_ROOT_IN_INODE))
if (cur->bc_ops->type != XFS_BTREE_TYPE_INODE)
xchk_btree_set_corrupt(bs->sc, bs->cur, level);
return 0;
}
@ -442,7 +441,7 @@ xchk_btree_check_owner(
* duplicate cursors. Therefore, save the buffer daddr for
* later scanning.
*/
if (cur->bc_btnum == XFS_BTNUM_BNO || cur->bc_btnum == XFS_BTNUM_RMAP) {
if (xfs_btree_is_bno(cur->bc_ops) || xfs_btree_is_rmap(cur->bc_ops)) {
struct check_owner *co;
co = kmalloc(sizeof(struct check_owner), XCHK_GFP_FLAGS);
@ -475,7 +474,7 @@ xchk_btree_check_iroot_minrecs(
* existing filesystems, so instead we disable the check for data fork
* bmap btrees when there's an attr fork.
*/
if (bs->cur->bc_btnum == XFS_BTNUM_BMAP &&
if (xfs_btree_is_bmap(bs->cur->bc_ops) &&
bs->cur->bc_ino.whichfork == XFS_DATA_FORK &&
xfs_inode_has_attr_fork(bs->sc->ip))
return false;
@ -508,7 +507,7 @@ xchk_btree_check_minrecs(
* child block might be less than the standard minrecs, but that's ok
* provided that there's only one direct child of the root.
*/
if ((cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) &&
if (cur->bc_ops->type == XFS_BTREE_TYPE_INODE &&
level == cur->bc_nlevels - 2) {
struct xfs_btree_block *root_block;
struct xfs_buf *root_bp;
@ -562,7 +561,7 @@ xchk_btree_block_check_keys(
return;
}
if (!(cur->bc_flags & XFS_BTREE_OVERLAPPING))
if (!(cur->bc_ops->geom_flags & XFS_BTGEO_OVERLAPPING))
return;
/* Make sure the high key of this block matches the parent. */
@ -585,7 +584,6 @@ xchk_btree_get_block(
struct xfs_btree_block **pblock,
struct xfs_buf **pbp)
{
xfs_failaddr_t failed_at;
int error;
*pblock = NULL;
@ -597,13 +595,7 @@ xchk_btree_get_block(
return error;
xfs_btree_get_block(bs->cur, level, pbp);
if (bs->cur->bc_flags & XFS_BTREE_LONG_PTRS)
failed_at = __xfs_btree_check_lblock(bs->cur, *pblock,
level, *pbp);
else
failed_at = __xfs_btree_check_sblock(bs->cur, *pblock,
level, *pbp);
if (failed_at) {
if (__xfs_btree_check_block(bs->cur, *pblock, level, *pbp)) {
xchk_btree_set_corrupt(bs->sc, bs->cur, level);
return 0;
}
@ -664,7 +656,7 @@ xchk_btree_block_keys(
if (xfs_btree_keycmp_ne(cur, &block_keys, parent_keys))
xchk_btree_set_corrupt(bs->sc, cur, 1);
if (!(cur->bc_flags & XFS_BTREE_OVERLAPPING))
if (!(cur->bc_ops->geom_flags & XFS_BTGEO_OVERLAPPING))
return;
/* Get high keys */
@ -728,7 +720,7 @@ xchk_btree(
* error codes for us.
*/
level = cur->bc_nlevels - 1;
cur->bc_ops->init_ptr_from_cur(cur, &ptr);
xfs_btree_init_ptr_from_cur(cur, &ptr);
if (!xchk_btree_ptr_ok(bs, cur->bc_nlevels, &ptr))
goto out;
error = xchk_btree_get_block(bs, level, &ptr, &block, &bp);

133
fs/xfs/scrub/common.c
View File

@ -29,6 +29,8 @@
#include "xfs_attr.h"
#include "xfs_reflink.h"
#include "xfs_ag.h"
#include "xfs_error.h"
#include "xfs_quota.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/trace.h"
@ -82,6 +84,15 @@ __xchk_process_error(
sc->ip ? sc->ip : XFS_I(file_inode(sc->file)),
sc->sm, *error);
break;
case -ECANCELED:
/*
* ECANCELED here means that the caller set one of the scrub
* outcome flags (corrupt, xfail, xcorrupt) and wants to exit
* quickly. Set error to zero and do not continue.
*/
trace_xchk_op_error(sc, agno, bno, *error, ret_ip);
*error = 0;
break;
case -EFSBADCRC:
case -EFSCORRUPTED:
/* Note the badness but don't abort. */
@ -89,8 +100,7 @@ __xchk_process_error(
*error = 0;
fallthrough;
default:
trace_xchk_op_error(sc, agno, bno, *error,
ret_ip);
trace_xchk_op_error(sc, agno, bno, *error, ret_ip);
break;
}
return false;
@ -136,6 +146,16 @@ __xchk_fblock_process_error(
/* Used to restart an op with deadlock avoidance. */
trace_xchk_deadlock_retry(sc->ip, sc->sm, *error);
break;
case -ECANCELED:
/*
* ECANCELED here means that the caller set one of the scrub
* outcome flags (corrupt, xfail, xcorrupt) and wants to exit
* quickly. Set error to zero and do not continue.
*/
trace_xchk_file_op_error(sc, whichfork, offset, *error,
ret_ip);
*error = 0;
break;
case -EFSBADCRC:
case -EFSCORRUPTED:
/* Note the badness but don't abort. */
@ -227,6 +247,19 @@ xchk_block_set_corrupt(
trace_xchk_block_error(sc, xfs_buf_daddr(bp), __return_address);
}
#ifdef CONFIG_XFS_QUOTA
/* Record a corrupt quota counter. */
void
xchk_qcheck_set_corrupt(
struct xfs_scrub *sc,
unsigned int dqtype,
xfs_dqid_t id)
{
sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT;
trace_xchk_qcheck_error(sc, dqtype, id, __return_address);
}
#endif
/* Record a corruption while cross-referencing. */
void
xchk_block_xref_set_corrupt(
@ -427,7 +460,7 @@ xchk_perag_read_headers(
* Grab the AG headers for the attached perag structure and wait for pending
* intents to drain.
*/
static int
int
xchk_perag_drain_and_lock(
struct xfs_scrub *sc)
{
@ -555,46 +588,50 @@ xchk_ag_btcur_init(
{
struct xfs_mount *mp = sc->mp;
if (sa->agf_bp &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_BNO)) {
if (sa->agf_bp) {
/* Set up a bnobt cursor for cross-referencing. */
sa->bno_cur = xfs_allocbt_init_cursor(mp, sc->tp, sa->agf_bp,
sa->pag, XFS_BTNUM_BNO);
}
if (sa->agf_bp &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_CNT)) {
/* Set up a cntbt cursor for cross-referencing. */
sa->cnt_cur = xfs_allocbt_init_cursor(mp, sc->tp, sa->agf_bp,
sa->pag, XFS_BTNUM_CNT);
}
/* Set up a inobt cursor for cross-referencing. */
if (sa->agi_bp &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_INO)) {
sa->ino_cur = xfs_inobt_init_cursor(sa->pag, sc->tp, sa->agi_bp,
XFS_BTNUM_INO);
}
/* Set up a finobt cursor for cross-referencing. */
if (sa->agi_bp && xfs_has_finobt(mp) &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_FINO)) {
sa->fino_cur = xfs_inobt_init_cursor(sa->pag, sc->tp, sa->agi_bp,
XFS_BTNUM_FINO);
}
/* Set up a rmapbt cursor for cross-referencing. */
if (sa->agf_bp && xfs_has_rmapbt(mp) &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_RMAP)) {
sa->rmap_cur = xfs_rmapbt_init_cursor(mp, sc->tp, sa->agf_bp,
sa->bno_cur = xfs_bnobt_init_cursor(mp, sc->tp, sa->agf_bp,
sa->pag);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->bno_cur,
XFS_SCRUB_TYPE_BNOBT);
/* Set up a cntbt cursor for cross-referencing. */
sa->cnt_cur = xfs_cntbt_init_cursor(mp, sc->tp, sa->agf_bp,
sa->pag);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->cnt_cur,
XFS_SCRUB_TYPE_CNTBT);
/* Set up a rmapbt cursor for cross-referencing. */
if (xfs_has_rmapbt(mp)) {
sa->rmap_cur = xfs_rmapbt_init_cursor(mp, sc->tp,
sa->agf_bp, sa->pag);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->rmap_cur,
XFS_SCRUB_TYPE_RMAPBT);
}
/* Set up a refcountbt cursor for cross-referencing. */
if (xfs_has_reflink(mp)) {
sa->refc_cur = xfs_refcountbt_init_cursor(mp, sc->tp,
sa->agf_bp, sa->pag);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->refc_cur,
XFS_SCRUB_TYPE_REFCNTBT);
}
}
/* Set up a refcountbt cursor for cross-referencing. */
if (sa->agf_bp && xfs_has_reflink(mp) &&
xchk_ag_btree_healthy_enough(sc, sa->pag, XFS_BTNUM_REFC)) {
sa->refc_cur = xfs_refcountbt_init_cursor(mp, sc->tp,
sa->agf_bp, sa->pag);
if (sa->agi_bp) {
/* Set up a inobt cursor for cross-referencing. */
sa->ino_cur = xfs_inobt_init_cursor(sa->pag, sc->tp,
sa->agi_bp);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->ino_cur,
XFS_SCRUB_TYPE_INOBT);
/* Set up a finobt cursor for cross-referencing. */
if (xfs_has_finobt(mp)) {
sa->fino_cur = xfs_finobt_init_cursor(sa->pag, sc->tp,
sa->agi_bp);
xchk_ag_btree_del_cursor_if_sick(sc, &sa->fino_cur,
XFS_SCRUB_TYPE_FINOBT);
}
}
}
@ -653,6 +690,13 @@ xchk_trans_cancel(
sc->tp = NULL;
}
int
xchk_trans_alloc_empty(
struct xfs_scrub *sc)
{
return xfs_trans_alloc_empty(sc->mp, &sc->tp);
}
/*
* Grab an empty transaction so that we can re-grab locked buffers if
* one of our btrees turns out to be cyclic.
@ -672,7 +716,7 @@ xchk_trans_alloc(
return xfs_trans_alloc(sc->mp, &M_RES(sc->mp)->tr_itruncate,
resblks, 0, 0, &sc->tp);
return xfs_trans_alloc_empty(sc->mp, &sc->tp);
return xchk_trans_alloc_empty(sc);
}
/* Set us up with a transaction and an empty context. */
@ -1259,6 +1303,15 @@ xchk_fsgates_enable(
if (scrub_fsgates & XCHK_FSGATES_DRAIN)
xfs_drain_wait_enable();
if (scrub_fsgates & XCHK_FSGATES_QUOTA)
xfs_dqtrx_hook_enable();
if (scrub_fsgates & XCHK_FSGATES_DIRENTS)
xfs_dir_hook_enable();
if (scrub_fsgates & XCHK_FSGATES_RMAP)
xfs_rmap_hook_enable();
sc->flags |= scrub_fsgates;
}

13
fs/xfs/scrub/common.h
View File

@ -32,6 +32,7 @@ xchk_should_terminate(
}
int xchk_trans_alloc(struct xfs_scrub *sc, uint resblks);
int xchk_trans_alloc_empty(struct xfs_scrub *sc);
void xchk_trans_cancel(struct xfs_scrub *sc);
bool xchk_process_error(struct xfs_scrub *sc, xfs_agnumber_t agno,
@ -54,6 +55,10 @@ void xchk_block_set_corrupt(struct xfs_scrub *sc,
void xchk_ino_set_corrupt(struct xfs_scrub *sc, xfs_ino_t ino);
void xchk_fblock_set_corrupt(struct xfs_scrub *sc, int whichfork,
xfs_fileoff_t offset);
#ifdef CONFIG_XFS_QUOTA
void xchk_qcheck_set_corrupt(struct xfs_scrub *sc, unsigned int dqtype,
xfs_dqid_t id);
#endif
void xchk_block_xref_set_corrupt(struct xfs_scrub *sc,
struct xfs_buf *bp);
@ -105,6 +110,7 @@ xchk_setup_rtsummary(struct xfs_scrub *sc)
#ifdef CONFIG_XFS_QUOTA
int xchk_ino_dqattach(struct xfs_scrub *sc);
int xchk_setup_quota(struct xfs_scrub *sc);
int xchk_setup_quotacheck(struct xfs_scrub *sc);
#else
static inline int
xchk_ino_dqattach(struct xfs_scrub *sc)
@ -116,12 +122,19 @@ xchk_setup_quota(struct xfs_scrub *sc)
{
return -ENOENT;
}
static inline int
xchk_setup_quotacheck(struct xfs_scrub *sc)
{
return -ENOENT;
}
#endif
int xchk_setup_fscounters(struct xfs_scrub *sc);
int xchk_setup_nlinks(struct xfs_scrub *sc);
void xchk_ag_free(struct xfs_scrub *sc, struct xchk_ag *sa);
int xchk_ag_init(struct xfs_scrub *sc, xfs_agnumber_t agno,
struct xchk_ag *sa);
int xchk_perag_drain_and_lock(struct xfs_scrub *sc);
/*
* Grab all AG resources, treating the inability to grab the perag structure as

2
fs/xfs/scrub/cow_repair.c
View File

@ -609,6 +609,6 @@ xrep_bmap_cow(
out_bitmap:
xfsb_bitmap_destroy(&xc->old_cowfork_fsblocks);
xoff_bitmap_destroy(&xc->bad_fileoffs);
kmem_free(xc);
kfree(xc);
return error;
}

4
fs/xfs/scrub/dir.c
View File

@ -93,11 +93,11 @@ xchk_dir_actor(
return -ECANCELED;
}
if (!strncmp(".", name->name, name->len)) {
if (xfs_dir2_samename(name, &xfs_name_dot)) {
/* If this is "." then check that the inum matches the dir. */
if (ino != dp->i_ino)
xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
} else if (!strncmp("..", name->name, name->len)) {
} else if (xfs_dir2_samename(name, &xfs_name_dotdot)) {
/*
* If this is ".." in the root inode, check that the inum
* matches this dir.

29
fs/xfs/scrub/fscounters.c
View File

@ -22,6 +22,7 @@
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/trace.h"
#include "scrub/fscounters.h"
/*
* FS Summary Counters
@ -48,17 +49,6 @@
* our tolerance for mismatch between expected and actual counter values.
*/
struct xchk_fscounters {
struct xfs_scrub *sc;
uint64_t icount;
uint64_t ifree;
uint64_t fdblocks;
uint64_t frextents;
unsigned long long icount_min;
unsigned long long icount_max;
bool frozen;
};
/*
* Since the expected value computation is lockless but only browses incore
* values, the percpu counters should be fairly close to each other. However,
@ -235,14 +225,19 @@ xchk_setup_fscounters(
* Pause all writer activity in the filesystem while we're scrubbing to
* reduce the likelihood of background perturbations to the counters
* throwing off our calculations.
*
* If we're repairing, we need to prevent any other thread from
* changing the global fs summary counters while we're repairing them.
* This requires the fs to be frozen, which will disable background
* reclaim and purge all inactive inodes.
*/
if (sc->flags & XCHK_TRY_HARDER) {
if ((sc->flags & XCHK_TRY_HARDER) || xchk_could_repair(sc)) {
error = xchk_fscounters_freeze(sc);
if (error)
return error;
}
return xfs_trans_alloc_empty(sc->mp, &sc->tp);
return xchk_trans_alloc_empty(sc);
}
/*
@ -254,7 +249,9 @@ xchk_setup_fscounters(
* set the INCOMPLETE flag even when a negative errno is returned. This care
* must be taken with certain errno values (i.e. EFSBADCRC, EFSCORRUPTED,
* ECANCELED) that are absorbed into a scrub state flag update by
* xchk_*_process_error.
* xchk_*_process_error. Scrub and repair share the same incore data
* structures, so the INCOMPLETE flag is critical to prevent a repair based on
* insufficient information.
*/
/* Count free space btree blocks manually for pre-lazysbcount filesystems. */
@ -482,6 +479,10 @@ xchk_fscount_within_range(
if (curr_value == expected)
return true;
/* We require exact matches when repair is running. */
if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR)
return false;
min_value = min(old_value, curr_value);
max_value = max(old_value, curr_value);

20
fs/xfs/scrub/fscounters.h Normal file
View File

@ -0,0 +1,20 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_FSCOUNTERS_H__
#define __XFS_SCRUB_FSCOUNTERS_H__
struct xchk_fscounters {
struct xfs_scrub *sc;
uint64_t icount;
uint64_t ifree;
uint64_t fdblocks;
uint64_t frextents;
unsigned long long icount_min;
unsigned long long icount_max;
bool frozen;
};
#endif /* __XFS_SCRUB_FSCOUNTERS_H__ */

72
fs/xfs/scrub/fscounters_repair.c Normal file
View File

@ -0,0 +1,72 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2018-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_defer.h"
#include "xfs_btree.h"
#include "xfs_bit.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_sb.h"
#include "xfs_inode.h"
#include "xfs_alloc.h"
#include "xfs_ialloc.h"
#include "xfs_rmap.h"
#include "xfs_health.h"
#include "scrub/xfs_scrub.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/trace.h"
#include "scrub/repair.h"
#include "scrub/fscounters.h"
/*
* FS Summary Counters
* ===================
*
* We correct errors in the filesystem summary counters by setting them to the
* values computed during the obligatory scrub phase. However, we must be
* careful not to allow any other thread to change the counters while we're
* computing and setting new values. To achieve this, we freeze the
* filesystem for the whole operation if the REPAIR flag is set. The checking
* function is stricter when we've frozen the fs.
*/
/*
* Reset the superblock counters. Caller is responsible for freezing the
* filesystem during the calculation and reset phases.
*/
int
xrep_fscounters(
struct xfs_scrub *sc)
{
struct xfs_mount *mp = sc->mp;
struct xchk_fscounters *fsc = sc->buf;
/*
* Reinitialize the in-core counters from what we computed. We froze
* the filesystem, so there shouldn't be anyone else trying to modify
* these counters.
*/
if (!fsc->frozen) {
ASSERT(fsc->frozen);
return -EFSCORRUPTED;
}
trace_xrep_reset_counters(mp, fsc);
percpu_counter_set(&mp->m_icount, fsc->icount);
percpu_counter_set(&mp->m_ifree, fsc->ifree);
percpu_counter_set(&mp->m_fdblocks, fsc->fdblocks);
percpu_counter_set(&mp->m_frextents, fsc->frextents);
mp->m_sb.sb_frextents = fsc->frextents;
return 0;
}

140
fs/xfs/scrub/health.c
View File

@ -14,6 +14,7 @@
#include "xfs_health.h"
#include "scrub/scrub.h"
#include "scrub/health.h"
#include "scrub/common.h"
/*
* Scrub and In-Core Filesystem Health Assessments
@ -105,6 +106,8 @@ static const struct xchk_health_map type_to_health_flag[XFS_SCRUB_TYPE_NR] = {
[XFS_SCRUB_TYPE_GQUOTA] = { XHG_FS, XFS_SICK_FS_GQUOTA },
[XFS_SCRUB_TYPE_PQUOTA] = { XHG_FS, XFS_SICK_FS_PQUOTA },
[XFS_SCRUB_TYPE_FSCOUNTERS] = { XHG_FS, XFS_SICK_FS_COUNTERS },
[XFS_SCRUB_TYPE_QUOTACHECK] = { XHG_FS, XFS_SICK_FS_QUOTACHECK },
[XFS_SCRUB_TYPE_NLINKS] = { XHG_FS, XFS_SICK_FS_NLINKS },
};
/* Return the health status mask for this scrub type. */
@ -147,6 +150,24 @@ xchk_file_looks_zapped(
return xfs_inode_has_sickness(sc->ip, mask);
}
/*
* Scrub gave the filesystem a clean bill of health, so clear all the indirect
* markers of past problems (at least for the fs and ags) so that we can be
* healthy again.
*/
STATIC void
xchk_mark_all_healthy(
struct xfs_mount *mp)
{
struct xfs_perag *pag;
xfs_agnumber_t agno;
xfs_fs_mark_healthy(mp, XFS_SICK_FS_INDIRECT);
xfs_rt_mark_healthy(mp, XFS_SICK_RT_INDIRECT);
for_each_perag(mp, agno, pag)
xfs_ag_mark_healthy(pag, XFS_SICK_AG_INDIRECT);
}
/*
* Update filesystem health assessments based on what we found and did.
*
@ -164,6 +185,18 @@ xchk_update_health(
struct xfs_perag *pag;
bool bad;
/*
* The HEALTHY scrub type is a request from userspace to clear all the
* indirect flags after a clean scan of the entire filesystem. As such
* there's no sick flag defined for it, so we branch here ahead of the
* mask check.
*/
if (sc->sm->sm_type == XFS_SCRUB_TYPE_HEALTHY &&
!(sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)) {
xchk_mark_all_healthy(sc->mp);
return;
}
if (!sc->sick_mask)
return;
@ -173,7 +206,7 @@ xchk_update_health(
case XHG_AG:
pag = xfs_perag_get(sc->mp, sc->sm->sm_agno);
if (bad)
xfs_ag_mark_sick(pag, sc->sick_mask);
xfs_ag_mark_corrupt(pag, sc->sick_mask);
else
xfs_ag_mark_healthy(pag, sc->sick_mask);
xfs_perag_put(pag);
@ -181,20 +214,30 @@ xchk_update_health(
case XHG_INO:
if (!sc->ip)
return;
if (bad)
xfs_inode_mark_sick(sc->ip, sc->sick_mask);
else
if (bad) {
unsigned int mask = sc->sick_mask;
/*
* If we're coming in for repairs then we don't want
* sickness flags to propagate to the incore health
* status if the inode gets inactivated before we can
* fix it.
*/
if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR)
mask |= XFS_SICK_INO_FORGET;
xfs_inode_mark_corrupt(sc->ip, mask);
} else
xfs_inode_mark_healthy(sc->ip, sc->sick_mask);
break;
case XHG_FS:
if (bad)
xfs_fs_mark_sick(sc->mp, sc->sick_mask);
xfs_fs_mark_corrupt(sc->mp, sc->sick_mask);
else
xfs_fs_mark_healthy(sc->mp, sc->sick_mask);
break;
case XHG_RT:
if (bad)
xfs_rt_mark_sick(sc->mp, sc->sick_mask);
xfs_rt_mark_corrupt(sc->mp, sc->sick_mask);
else
xfs_rt_mark_healthy(sc->mp, sc->sick_mask);
break;
@ -205,13 +248,13 @@ xchk_update_health(
}
/* Is the given per-AG btree healthy enough for scanning? */
bool
xchk_ag_btree_healthy_enough(
void
xchk_ag_btree_del_cursor_if_sick(
struct xfs_scrub *sc,
struct xfs_perag *pag,
xfs_btnum_t btnum)
struct xfs_btree_cur **curp,
unsigned int sm_type)
{
unsigned int mask = 0;
unsigned int mask = (*curp)->bc_ops->sick_mask;
/*
* We always want the cursor if it's the same type as whatever we're
@ -220,41 +263,8 @@ xchk_ag_btree_healthy_enough(
* Otherwise, we're only interested in the btree for cross-referencing.
* If we know the btree is bad then don't bother, just set XFAIL.
*/
switch (btnum) {
case XFS_BTNUM_BNO:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_BNOBT)
return true;
mask = XFS_SICK_AG_BNOBT;
break;
case XFS_BTNUM_CNT:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_CNTBT)
return true;
mask = XFS_SICK_AG_CNTBT;
break;
case XFS_BTNUM_INO:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_INOBT)
return true;
mask = XFS_SICK_AG_INOBT;
break;
case XFS_BTNUM_FINO:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_FINOBT)
return true;
mask = XFS_SICK_AG_FINOBT;
break;
case XFS_BTNUM_RMAP:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_RMAPBT)
return true;
mask = XFS_SICK_AG_RMAPBT;
break;
case XFS_BTNUM_REFC:
if (sc->sm->sm_type == XFS_SCRUB_TYPE_REFCNTBT)
return true;
mask = XFS_SICK_AG_REFCNTBT;
break;
default:
ASSERT(0);
return true;
}
if (sc->sm->sm_type == sm_type)
return;
/*
* If we just repaired some AG metadata, sc->sick_mask will reflect all
@ -266,10 +276,42 @@ xchk_ag_btree_healthy_enough(
type_to_health_flag[sc->sm->sm_type].group == XHG_AG)
mask &= ~sc->sick_mask;
if (xfs_ag_has_sickness(pag, mask)) {
if (xfs_ag_has_sickness((*curp)->bc_ag.pag, mask)) {
sc->sm->sm_flags |= XFS_SCRUB_OFLAG_XFAIL;
return false;
xfs_btree_del_cursor(*curp, XFS_BTREE_NOERROR);
*curp = NULL;
}
}
/*
* Quick scan to double-check that there isn't any evidence of lingering
* primary health problems. If we're still clear, then the health update will
* take care of clearing the indirect evidence.
*/
int
xchk_health_record(
struct xfs_scrub *sc)
{
struct xfs_mount *mp = sc->mp;
struct xfs_perag *pag;
xfs_agnumber_t agno;
unsigned int sick;
unsigned int checked;
xfs_fs_measure_sickness(mp, &sick, &checked);
if (sick & XFS_SICK_FS_PRIMARY)
xchk_set_corrupt(sc);
xfs_rt_measure_sickness(mp, &sick, &checked);
if (sick & XFS_SICK_RT_PRIMARY)
xchk_set_corrupt(sc);
for_each_perag(mp, agno, pag) {
xfs_ag_measure_sickness(pag, &sick, &checked);
if (sick & XFS_SICK_AG_PRIMARY)
xchk_set_corrupt(sc);
}
return true;
return 0;
}

5
fs/xfs/scrub/health.h
View File

@ -8,9 +8,10 @@
unsigned int xchk_health_mask_for_scrub_type(__u32 scrub_type);
void xchk_update_health(struct xfs_scrub *sc);
bool xchk_ag_btree_healthy_enough(struct xfs_scrub *sc, struct xfs_perag *pag,
xfs_btnum_t btnum);
void xchk_ag_btree_del_cursor_if_sick(struct xfs_scrub *sc,
struct xfs_btree_cur **curp, unsigned int sm_type);
void xchk_mark_healthy_if_clean(struct xfs_scrub *sc, unsigned int mask);
bool xchk_file_looks_zapped(struct xfs_scrub *sc, unsigned int mask);
int xchk_health_record(struct xfs_scrub *sc);
#endif /* __XFS_SCRUB_HEALTH_H__ */

20
fs/xfs/scrub/ialloc.c
View File

@ -76,7 +76,7 @@ xchk_inobt_xref_finobt(
int has_record;
int error;
ASSERT(cur->bc_btnum == XFS_BTNUM_FINO);
ASSERT(xfs_btree_is_fino(cur->bc_ops));
error = xfs_inobt_lookup(cur, agino, XFS_LOOKUP_LE, &has_record);
if (error)
@ -179,7 +179,7 @@ xchk_finobt_xref_inobt(
int has_record;
int error;
ASSERT(cur->bc_btnum == XFS_BTNUM_INO);
ASSERT(xfs_btree_is_ino(cur->bc_ops));
error = xfs_inobt_lookup(cur, agino, XFS_LOOKUP_LE, &has_record);
if (error)
@ -514,7 +514,7 @@ xchk_iallocbt_rec_alignment(
* Otherwise, we expect that the finobt record is aligned to the
* cluster alignment as told by the superblock.
*/
if (bs->cur->bc_btnum == XFS_BTNUM_FINO) {
if (xfs_btree_is_fino(bs->cur->bc_ops)) {
unsigned int imask;
imask = min_t(unsigned int, XFS_INODES_PER_CHUNK,
@ -649,8 +649,7 @@ out:
*/
STATIC void
xchk_iallocbt_xref_rmap_btreeblks(
struct xfs_scrub *sc,
int which)
struct xfs_scrub *sc)
{
xfs_filblks_t blocks;
xfs_extlen_t inobt_blocks = 0;
@ -688,7 +687,6 @@ xchk_iallocbt_xref_rmap_btreeblks(
STATIC void
xchk_iallocbt_xref_rmap_inodes(
struct xfs_scrub *sc,
int which,
unsigned long long inodes)
{
xfs_filblks_t blocks;
@ -719,17 +717,14 @@ xchk_iallocbt(
.next_startino = NULLAGINO,
.next_cluster_ino = NULLAGINO,
};
xfs_btnum_t which;
int error;
switch (sc->sm->sm_type) {
case XFS_SCRUB_TYPE_INOBT:
cur = sc->sa.ino_cur;
which = XFS_BTNUM_INO;
break;
case XFS_SCRUB_TYPE_FINOBT:
cur = sc->sa.fino_cur;
which = XFS_BTNUM_FINO;
break;
default:
ASSERT(0);
@ -741,7 +736,7 @@ xchk_iallocbt(
if (error)
return error;
xchk_iallocbt_xref_rmap_btreeblks(sc, which);
xchk_iallocbt_xref_rmap_btreeblks(sc);
/*
* If we're scrubbing the inode btree, inode_blocks is the number of
@ -750,9 +745,8 @@ xchk_iallocbt(
* knows about. We can't do this for the finobt since it only points
* to inode chunks with free inodes.
*/
if (which == XFS_BTNUM_INO)
xchk_iallocbt_xref_rmap_inodes(sc, which, iabt.inodes);
if (sc->sm->sm_type == XFS_SCRUB_TYPE_INOBT)
xchk_iallocbt_xref_rmap_inodes(sc, iabt.inodes);
return error;
}

10
fs/xfs/scrub/ialloc_repair.c
View File

@ -369,7 +369,7 @@ xrep_ibt_check_inode_ext(
* On a sparse inode fs, this cluster could be part of a sparse chunk.
* Sparse clusters must be aligned to sparse chunk alignment.
*/
if (xfs_has_sparseinodes(mp) &&
if (xfs_has_sparseinodes(mp) && mp->m_sb.sb_spino_align &&
(!IS_ALIGNED(agbno, mp->m_sb.sb_spino_align) ||
!IS_ALIGNED(agbno + len, mp->m_sb.sb_spino_align)))
return -EFSCORRUPTED;
@ -663,8 +663,8 @@ xrep_ibt_build_new_trees(
ri->new_inobt.bload.claim_block = xrep_ibt_claim_block;
ri->new_inobt.bload.get_records = xrep_ibt_get_records;
ino_cur = xfs_inobt_stage_cursor(sc->sa.pag, &ri->new_inobt.afake,
XFS_BTNUM_INO);
ino_cur = xfs_inobt_init_cursor(sc->sa.pag, NULL, NULL);
xfs_btree_stage_afakeroot(ino_cur, &ri->new_inobt.afake);
error = xfs_btree_bload_compute_geometry(ino_cur, &ri->new_inobt.bload,
xfarray_length(ri->inode_records));
if (error)
@ -684,8 +684,8 @@ xrep_ibt_build_new_trees(
ri->new_finobt.bload.claim_block = xrep_fibt_claim_block;
ri->new_finobt.bload.get_records = xrep_fibt_get_records;
fino_cur = xfs_inobt_stage_cursor(sc->sa.pag,
&ri->new_finobt.afake, XFS_BTNUM_FINO);
fino_cur = xfs_finobt_init_cursor(sc->sa.pag, NULL, NULL);
xfs_btree_stage_afakeroot(fino_cur, &ri->new_finobt.afake);
error = xfs_btree_bload_compute_geometry(fino_cur,
&ri->new_finobt.bload, ri->finobt_recs);
if (error)

237
fs/xfs/scrub/inode_repair.c
View File

@ -37,12 +37,15 @@
#include "xfs_attr_leaf.h"
#include "xfs_log_priv.h"
#include "xfs_health.h"
#include "xfs_symlink_remote.h"
#include "scrub/xfs_scrub.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/btree.h"
#include "scrub/trace.h"
#include "scrub/repair.h"
#include "scrub/iscan.h"
#include "scrub/readdir.h"
/*
* Inode Record Repair
@ -126,6 +129,10 @@ struct xrep_inode {
/* Must we remove all access from this file? */
bool zap_acls;
/* Inode scanner to see if we can find the ftype from dirents */
struct xchk_iscan ftype_iscan;
uint8_t alleged_ftype;
};
/*
@ -227,26 +234,233 @@ xrep_dinode_header(
dip->di_gen = cpu_to_be32(sc->sm->sm_gen);
}
/* Turn di_mode into /something/ recognizable. */
STATIC void
/*
* If this directory entry points to the scrub target inode, then the directory
* we're scanning is the parent of the scrub target inode.
*/
STATIC int
xrep_dinode_findmode_dirent(
struct xfs_scrub *sc,
struct xfs_inode *dp,
xfs_dir2_dataptr_t dapos,
const struct xfs_name *name,
xfs_ino_t ino,
void *priv)
{
struct xrep_inode *ri = priv;
int error = 0;
if (xchk_should_terminate(ri->sc, &error))
return error;
if (ino != sc->sm->sm_ino)
return 0;
/* Ignore garbage directory entry names. */
if (name->len == 0 || !xfs_dir2_namecheck(name->name, name->len))
return -EFSCORRUPTED;
/* Don't pick up dot or dotdot entries; we only want child dirents. */
if (xfs_dir2_samename(name, &xfs_name_dotdot) ||
xfs_dir2_samename(name, &xfs_name_dot))
return 0;
/*
* Uhoh, more than one parent for this inode and they don't agree on
* the file type?
*/
if (ri->alleged_ftype != XFS_DIR3_FT_UNKNOWN &&
ri->alleged_ftype != name->type) {
trace_xrep_dinode_findmode_dirent_inval(ri->sc, dp, name->type,
ri->alleged_ftype);
return -EFSCORRUPTED;
}
/* We found a potential parent; remember the ftype. */
trace_xrep_dinode_findmode_dirent(ri->sc, dp, name->type);
ri->alleged_ftype = name->type;
return 0;
}
/*
* If this is a directory, walk the dirents looking for any that point to the
* scrub target inode.
*/
STATIC int
xrep_dinode_findmode_walk_directory(
struct xrep_inode *ri,
struct xfs_inode *dp)
{
struct xfs_scrub *sc = ri->sc;
unsigned int lock_mode;
int error = 0;
/*
* Scan the directory to see if there it contains an entry pointing to
* the directory that we are repairing.
*/
lock_mode = xfs_ilock_data_map_shared(dp);
/*
* If this directory is known to be sick, we cannot scan it reliably
* and must abort.
*/
if (xfs_inode_has_sickness(dp, XFS_SICK_INO_CORE |
XFS_SICK_INO_BMBTD |
XFS_SICK_INO_DIR)) {
error = -EFSCORRUPTED;
goto out_unlock;
}
/*
* We cannot complete our parent pointer scan if a directory looks as
* though it has been zapped by the inode record repair code.
*/
if (xchk_dir_looks_zapped(dp)) {
error = -EBUSY;
goto out_unlock;
}
error = xchk_dir_walk(sc, dp, xrep_dinode_findmode_dirent, ri);
if (error)
goto out_unlock;
out_unlock:
xfs_iunlock(dp, lock_mode);
return error;
}
/*
* Try to find the mode of the inode being repaired by looking for directories
* that point down to this file.
*/
STATIC int
xrep_dinode_find_mode(
struct xrep_inode *ri,
uint16_t *mode)
{
struct xfs_scrub *sc = ri->sc;
struct xfs_inode *dp;
int error;
/* No ftype means we have no other metadata to consult. */
if (!xfs_has_ftype(sc->mp)) {
*mode = S_IFREG;
return 0;
}
/*
* Scan all directories for parents that might point down to this
* inode. Skip the inode being repaired during the scan since it
* cannot be its own parent. Note that we still hold the AGI locked
* so there's a real possibility that _iscan_iter can return EBUSY.
*/
xchk_iscan_start(sc, 5000, 100, &ri->ftype_iscan);
ri->ftype_iscan.skip_ino = sc->sm->sm_ino;
ri->alleged_ftype = XFS_DIR3_FT_UNKNOWN;
while ((error = xchk_iscan_iter(&ri->ftype_iscan, &dp)) == 1) {
if (S_ISDIR(VFS_I(dp)->i_mode))
error = xrep_dinode_findmode_walk_directory(ri, dp);
xchk_iscan_mark_visited(&ri->ftype_iscan, dp);
xchk_irele(sc, dp);
if (error < 0)
break;
if (xchk_should_terminate(sc, &error))
break;
}
xchk_iscan_iter_finish(&ri->ftype_iscan);
xchk_iscan_teardown(&ri->ftype_iscan);
if (error == -EBUSY) {
if (ri->alleged_ftype != XFS_DIR3_FT_UNKNOWN) {
/*
* If we got an EBUSY after finding at least one
* dirent, that means the scan found an inode on the
* inactivation list and could not open it. Accept the
* alleged ftype and install a new mode below.
*/
error = 0;
} else if (!(sc->flags & XCHK_TRY_HARDER)) {
/*
* Otherwise, retry the operation one time to see if
* the reason for the delay is an inode from the same
* cluster buffer waiting on the inactivation list.
*/
error = -EDEADLOCK;
}
}
if (error)
return error;
/*
* Convert the discovered ftype into the file mode. If all else fails,
* return S_IFREG.
*/
switch (ri->alleged_ftype) {
case XFS_DIR3_FT_DIR:
*mode = S_IFDIR;
break;
case XFS_DIR3_FT_WHT:
case XFS_DIR3_FT_CHRDEV:
*mode = S_IFCHR;
break;
case XFS_DIR3_FT_BLKDEV:
*mode = S_IFBLK;
break;
case XFS_DIR3_FT_FIFO:
*mode = S_IFIFO;
break;
case XFS_DIR3_FT_SOCK:
*mode = S_IFSOCK;
break;
case XFS_DIR3_FT_SYMLINK:
*mode = S_IFLNK;
break;
default:
*mode = S_IFREG;
break;
}
return 0;
}
/* Turn di_mode into /something/ recognizable. Returns true if we succeed. */
STATIC int
xrep_dinode_mode(
struct xrep_inode *ri,
struct xfs_dinode *dip)
{
struct xfs_scrub *sc = ri->sc;
uint16_t mode = be16_to_cpu(dip->di_mode);
int error;
trace_xrep_dinode_mode(sc, dip);
if (mode == 0 || xfs_mode_to_ftype(mode) != XFS_DIR3_FT_UNKNOWN)
return;
return 0;
/* Try to fix the mode. If we cannot, then leave everything alone. */
error = xrep_dinode_find_mode(ri, &mode);
switch (error) {
case -EINTR:
case -EBUSY:
case -EDEADLOCK:
/* temporary failure or fatal signal */
return error;
case 0:
/* found mode */
break;
default:
/* some other error, assume S_IFREG */
mode = S_IFREG;
break;
}
/* bad mode, so we set it to a file that only root can read */
mode = S_IFREG;
dip->di_mode = cpu_to_be16(mode);
dip->di_uid = 0;
dip->di_gid = 0;
ri->zap_acls = true;
return 0;
}
/* Fix any conflicting flags that the verifiers complain about. */
@ -1107,12 +1321,15 @@ xrep_dinode_core(
/* Fix everything the verifier will complain about. */
dip = xfs_buf_offset(bp, ri->imap.im_boffset);
xrep_dinode_header(sc, dip);
xrep_dinode_mode(ri, dip);
iget_error = xrep_dinode_mode(ri, dip);
if (iget_error)
goto write;
xrep_dinode_flags(sc, dip, ri->rt_extents > 0);
xrep_dinode_size(ri, dip);
xrep_dinode_extsize_hints(sc, dip);
xrep_dinode_zap_forks(ri, dip);
write:
/* Write out the inode. */
trace_xrep_dinode_fixed(sc, dip);
xfs_dinode_calc_crc(sc->mp, dip);
@ -1128,7 +1345,8 @@ xrep_dinode_core(
* accessing the inode. If iget fails, we still need to commit the
* changes.
*/
iget_error = xchk_iget(sc, ino, &sc->ip);
if (!iget_error)
iget_error = xchk_iget(sc, ino, &sc->ip);
if (!iget_error)
xchk_ilock(sc, XFS_IOLOCK_EXCL);
@ -1496,6 +1714,13 @@ xrep_inode(
ASSERT(ri != NULL);
error = xrep_dinode_problems(ri);
if (error == -EBUSY) {
/*
* Directory scan to recover inode mode encountered a
* busy inode, so we did not continue repairing things.
*/
return 0;
}
if (error)
return error;

767
fs/xfs/scrub/iscan.c Normal file
View File

@ -0,0 +1,767 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_inode.h"
#include "xfs_btree.h"
#include "xfs_ialloc.h"
#include "xfs_ialloc_btree.h"
#include "xfs_ag.h"
#include "xfs_error.h"
#include "xfs_bit.h"
#include "xfs_icache.h"
#include "scrub/scrub.h"
#include "scrub/iscan.h"
#include "scrub/common.h"
#include "scrub/trace.h"
/*
* Live File Scan
* ==============
*
* Live file scans walk every inode in a live filesystem. This is more or
* less like a regular iwalk, except that when we're advancing the scan cursor,
* we must ensure that inodes cannot be added or deleted anywhere between the
* old cursor value and the new cursor value. If we're advancing the cursor
* by one inode, the caller must hold that inode; if we're finding the next
* inode to scan, we must grab the AGI and hold it until we've updated the
* scan cursor.
*
* Callers are expected to use this code to scan all files in the filesystem to
* construct a new metadata index of some kind. The scan races against other
* live updates, which means there must be a provision to update the new index
* when updates are made to inodes that already been scanned. The iscan lock
* can be used in live update hook code to stop the scan and protect this data
* structure.
*
* To keep the new index up to date with other metadata updates being made to
* the live filesystem, it is assumed that the caller will add hooks as needed
* to be notified when a metadata update occurs. The inode scanner must tell
* the hook code when an inode has been visited with xchk_iscan_mark_visit.
* Hook functions can use xchk_iscan_want_live_update to decide if the
* scanner's observations must be updated.
*/
/*
* If the inobt record @rec covers @iscan->skip_ino, mark the inode free so
* that the scan ignores that inode.
*/
STATIC void
xchk_iscan_mask_skipino(
struct xchk_iscan *iscan,
struct xfs_perag *pag,
struct xfs_inobt_rec_incore *rec,
xfs_agino_t lastrecino)
{
struct xfs_scrub *sc = iscan->sc;
struct xfs_mount *mp = sc->mp;
xfs_agnumber_t skip_agno = XFS_INO_TO_AGNO(mp, iscan->skip_ino);
xfs_agnumber_t skip_agino = XFS_INO_TO_AGINO(mp, iscan->skip_ino);
if (pag->pag_agno != skip_agno)
return;
if (skip_agino < rec->ir_startino)
return;
if (skip_agino > lastrecino)
return;
rec->ir_free |= xfs_inobt_maskn(skip_agino - rec->ir_startino, 1);
}
/*
* Set *cursor to the next allocated inode after whatever it's set to now.
* If there are no more inodes in this AG, cursor is set to NULLAGINO.
*/
STATIC int
xchk_iscan_find_next(
struct xchk_iscan *iscan,
struct xfs_buf *agi_bp,
struct xfs_perag *pag,
xfs_inofree_t *allocmaskp,
xfs_agino_t *cursor,
uint8_t *nr_inodesp)
{
struct xfs_scrub *sc = iscan->sc;
struct xfs_inobt_rec_incore rec;
struct xfs_btree_cur *cur;
struct xfs_mount *mp = sc->mp;
struct xfs_trans *tp = sc->tp;
xfs_agnumber_t agno = pag->pag_agno;
xfs_agino_t lastino = NULLAGINO;
xfs_agino_t first, last;
xfs_agino_t agino = *cursor;
int has_rec;
int error;
/* If the cursor is beyond the end of this AG, move to the next one. */
xfs_agino_range(mp, agno, &first, &last);
if (agino > last) {
*cursor = NULLAGINO;
return 0;
}
/*
* Look up the inode chunk for the current cursor position. If there
* is no chunk here, we want the next one.
*/
cur = xfs_inobt_init_cursor(pag, tp, agi_bp);
error = xfs_inobt_lookup(cur, agino, XFS_LOOKUP_LE, &has_rec);
if (!error && !has_rec)
error = xfs_btree_increment(cur, 0, &has_rec);
for (; !error; error = xfs_btree_increment(cur, 0, &has_rec)) {
xfs_inofree_t allocmask;
/*
* If we've run out of inobt records in this AG, move the
* cursor on to the next AG and exit. The caller can try
* again with the next AG.
*/
if (!has_rec) {
*cursor = NULLAGINO;
break;
}
error = xfs_inobt_get_rec(cur, &rec, &has_rec);
if (error)
break;
if (!has_rec) {
error = -EFSCORRUPTED;
break;
}
/* Make sure that we always move forward. */
if (lastino != NULLAGINO &&
XFS_IS_CORRUPT(mp, lastino >= rec.ir_startino)) {
error = -EFSCORRUPTED;
break;
}
lastino = rec.ir_startino + XFS_INODES_PER_CHUNK - 1;
/*
* If this record only covers inodes that come before the
* cursor, advance to the next record.
*/
if (rec.ir_startino + XFS_INODES_PER_CHUNK <= agino)
continue;
if (iscan->skip_ino)
xchk_iscan_mask_skipino(iscan, pag, &rec, lastino);
/*
* If the incoming lookup put us in the middle of an inobt
* record, mark it and the previous inodes "free" so that the
* search for allocated inodes will start at the cursor.
* We don't care about ir_freecount here.
*/
if (agino >= rec.ir_startino)
rec.ir_free |= xfs_inobt_maskn(0,
agino + 1 - rec.ir_startino);
/*
* If there are allocated inodes in this chunk, find them
* and update the scan cursor.
*/
allocmask = ~rec.ir_free;
if (hweight64(allocmask) > 0) {
int next = xfs_lowbit64(allocmask);
ASSERT(next >= 0);
*cursor = rec.ir_startino + next;
*allocmaskp = allocmask >> next;
*nr_inodesp = XFS_INODES_PER_CHUNK - next;
break;
}
}
xfs_btree_del_cursor(cur, error);
return error;
}
/*
* Advance both the scan and the visited cursors.
*
* The inumber address space for a given filesystem is sparse, which means that
* the scan cursor can jump a long ways in a single iter() call. There are no
* inodes in these sparse areas, so we must move the visited cursor forward at
* the same time so that the scan user can receive live updates for inodes that
* may get created once we release the AGI buffer.
*/
static inline void
xchk_iscan_move_cursor(
struct xchk_iscan *iscan,
xfs_agnumber_t agno,
xfs_agino_t agino)
{
struct xfs_scrub *sc = iscan->sc;
struct xfs_mount *mp = sc->mp;
xfs_ino_t cursor, visited;
BUILD_BUG_ON(XFS_MAXINUMBER == NULLFSINO);
/*
* Special-case ino == 0 here so that we never set visited_ino to
* NULLFSINO when wrapping around EOFS, for that will let through all
* live updates.
*/
cursor = XFS_AGINO_TO_INO(mp, agno, agino);
if (cursor == 0)
visited = XFS_MAXINUMBER;
else
visited = cursor - 1;
mutex_lock(&iscan->lock);
iscan->cursor_ino = cursor;
iscan->__visited_ino = visited;
trace_xchk_iscan_move_cursor(iscan);
mutex_unlock(&iscan->lock);
}
/*
* Prepare to return agno/agino to the iscan caller by moving the lastino
* cursor to the previous inode. Do this while we still hold the AGI so that
* no other threads can create or delete inodes in this AG.
*/
static inline void
xchk_iscan_finish(
struct xchk_iscan *iscan)
{
mutex_lock(&iscan->lock);
iscan->cursor_ino = NULLFSINO;
/* All live updates will be applied from now on */
iscan->__visited_ino = NULLFSINO;
mutex_unlock(&iscan->lock);
}
/*
* Advance ino to the next inode that the inobt thinks is allocated, being
* careful to jump to the next AG if we've reached the right end of this AG's
* inode btree. Advancing ino effectively means that we've pushed the inode
* scan forward, so set the iscan cursor to (ino - 1) so that our live update
* predicates will track inode allocations in that part of the inode number
* key space once we release the AGI buffer.
*
* Returns 1 if there's a new inode to examine, 0 if we've run out of inodes,
* -ECANCELED if the live scan aborted, or the usual negative errno.
*/
STATIC int
xchk_iscan_advance(
struct xchk_iscan *iscan,
struct xfs_perag **pagp,
struct xfs_buf **agi_bpp,
xfs_inofree_t *allocmaskp,
uint8_t *nr_inodesp)
{
struct xfs_scrub *sc = iscan->sc;
struct xfs_mount *mp = sc->mp;
struct xfs_buf *agi_bp;
struct xfs_perag *pag;
xfs_agnumber_t agno;
xfs_agino_t agino;
int ret;
ASSERT(iscan->cursor_ino >= iscan->__visited_ino);
do {
if (xchk_iscan_aborted(iscan))
return -ECANCELED;
agno = XFS_INO_TO_AGNO(mp, iscan->cursor_ino);
pag = xfs_perag_get(mp, agno);
if (!pag)
return -ECANCELED;
ret = xfs_ialloc_read_agi(pag, sc->tp, &agi_bp);
if (ret)
goto out_pag;
agino = XFS_INO_TO_AGINO(mp, iscan->cursor_ino);
ret = xchk_iscan_find_next(iscan, agi_bp, pag, allocmaskp,
&agino, nr_inodesp);
if (ret)
goto out_buf;
if (agino != NULLAGINO) {
/*
* Found the next inode in this AG, so return it along
* with the AGI buffer and the perag structure to
* ensure it cannot go away.
*/
xchk_iscan_move_cursor(iscan, agno, agino);
*agi_bpp = agi_bp;
*pagp = pag;
return 1;
}
/*
* Did not find any more inodes in this AG, move on to the next
* AG.
*/
agno = (agno + 1) % mp->m_sb.sb_agcount;
xchk_iscan_move_cursor(iscan, agno, 0);
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
trace_xchk_iscan_advance_ag(iscan);
} while (iscan->cursor_ino != iscan->scan_start_ino);
xchk_iscan_finish(iscan);
return 0;
out_buf:
xfs_trans_brelse(sc->tp, agi_bp);
out_pag:
xfs_perag_put(pag);
return ret;
}
/*
* Grabbing the inode failed, so we need to back up the scan and ask the caller
* to try to _advance the scan again. Returns -EBUSY if we've run out of retry
* opportunities, -ECANCELED if the process has a fatal signal pending, or
* -EAGAIN if we should try again.
*/
STATIC int
xchk_iscan_iget_retry(
struct xchk_iscan *iscan,
bool wait)
{
ASSERT(iscan->cursor_ino == iscan->__visited_ino + 1);
if (!iscan->iget_timeout ||
time_is_before_jiffies(iscan->__iget_deadline))
return -EBUSY;
if (wait) {
unsigned long relax;
/*
* Sleep for a period of time to let the rest of the system
* catch up. If we return early, someone sent a kill signal to
* the calling process.
*/
relax = msecs_to_jiffies(iscan->iget_retry_delay);
trace_xchk_iscan_iget_retry_wait(iscan);
if (schedule_timeout_killable(relax) ||
xchk_iscan_aborted(iscan))
return -ECANCELED;
}
iscan->cursor_ino--;
return -EAGAIN;
}
/*
* Grab an inode as part of an inode scan. While scanning this inode, the
* caller must ensure that no other threads can modify the inode until a call
* to xchk_iscan_visit succeeds.
*
* Returns the number of incore inodes grabbed; -EAGAIN if the caller should
* call again xchk_iscan_advance; -EBUSY if we couldn't grab an inode;
* -ECANCELED if there's a fatal signal pending; or some other negative errno.
*/
STATIC int
xchk_iscan_iget(
struct xchk_iscan *iscan,
struct xfs_perag *pag,
struct xfs_buf *agi_bp,
xfs_inofree_t allocmask,
uint8_t nr_inodes)
{
struct xfs_scrub *sc = iscan->sc;
struct xfs_mount *mp = sc->mp;
xfs_ino_t ino = iscan->cursor_ino;
unsigned int idx = 0;
unsigned int i;
int error;
ASSERT(iscan->__inodes[0] == NULL);
/* Fill the first slot in the inode array. */
error = xfs_iget(sc->mp, sc->tp, ino, XFS_IGET_NORETRY, 0,
&iscan->__inodes[idx]);
trace_xchk_iscan_iget(iscan, error);
if (error == -ENOENT || error == -EAGAIN) {
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
/*
* It's possible that this inode has lost all of its links but
* hasn't yet been inactivated. If we don't have a transaction
* or it's not writable, flush the inodegc workers and wait.
*/
xfs_inodegc_flush(mp);
return xchk_iscan_iget_retry(iscan, true);
}
if (error == -EINVAL) {
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
/*
* We thought the inode was allocated, but the inode btree
* lookup failed, which means that it was freed since the last
* time we advanced the cursor. Back up and try again. This
* should never happen since still hold the AGI buffer from the
* inobt check, but we need to be careful about infinite loops.
*/
return xchk_iscan_iget_retry(iscan, false);
}
if (error) {
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
return error;
}
idx++;
ino++;
allocmask >>= 1;
/*
* Now that we've filled the first slot in __inodes, try to fill the
* rest of the batch with consecutively ordered inodes. to reduce the
* number of _iter calls. Make a bitmap of unallocated inodes from the
* zeroes in the inuse bitmap; these inodes will not be scanned, but
* the _want_live_update predicate will pass through all live updates.
*
* If we can't iget an allocated inode, stop and return what we have.
*/
mutex_lock(&iscan->lock);
iscan->__batch_ino = ino - 1;
iscan->__skipped_inomask = 0;
mutex_unlock(&iscan->lock);
for (i = 1; i < nr_inodes; i++, ino++, allocmask >>= 1) {
if (!(allocmask & 1)) {
ASSERT(!(iscan->__skipped_inomask & (1ULL << i)));
mutex_lock(&iscan->lock);
iscan->cursor_ino = ino;
iscan->__skipped_inomask |= (1ULL << i);
mutex_unlock(&iscan->lock);
continue;
}
ASSERT(iscan->__inodes[idx] == NULL);
error = xfs_iget(sc->mp, sc->tp, ino, XFS_IGET_NORETRY, 0,
&iscan->__inodes[idx]);
if (error)
break;
mutex_lock(&iscan->lock);
iscan->cursor_ino = ino;
mutex_unlock(&iscan->lock);
idx++;
}
trace_xchk_iscan_iget_batch(sc->mp, iscan, nr_inodes, idx);
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
return idx;
}
/*
* Advance the visit cursor to reflect skipped inodes beyond whatever we
* scanned.
*/
STATIC void
xchk_iscan_finish_batch(
struct xchk_iscan *iscan)
{
xfs_ino_t highest_skipped;
mutex_lock(&iscan->lock);
if (iscan->__batch_ino != NULLFSINO) {
highest_skipped = iscan->__batch_ino +
xfs_highbit64(iscan->__skipped_inomask);
iscan->__visited_ino = max(iscan->__visited_ino,
highest_skipped);
trace_xchk_iscan_skip(iscan);
}
iscan->__batch_ino = NULLFSINO;
iscan->__skipped_inomask = 0;
mutex_unlock(&iscan->lock);
}
/*
* Advance the inode scan cursor to the next allocated inode and return up to
* 64 consecutive allocated inodes starting with the cursor position.
*/
STATIC int
xchk_iscan_iter_batch(
struct xchk_iscan *iscan)
{
struct xfs_scrub *sc = iscan->sc;
int ret;
xchk_iscan_finish_batch(iscan);
if (iscan->iget_timeout)
iscan->__iget_deadline = jiffies +
msecs_to_jiffies(iscan->iget_timeout);
do {
struct xfs_buf *agi_bp = NULL;
struct xfs_perag *pag = NULL;
xfs_inofree_t allocmask = 0;
uint8_t nr_inodes = 0;
ret = xchk_iscan_advance(iscan, &pag, &agi_bp, &allocmask,
&nr_inodes);
if (ret != 1)
return ret;
if (xchk_iscan_aborted(iscan)) {
xfs_trans_brelse(sc->tp, agi_bp);
xfs_perag_put(pag);
ret = -ECANCELED;
break;
}
ret = xchk_iscan_iget(iscan, pag, agi_bp, allocmask, nr_inodes);
} while (ret == -EAGAIN);
return ret;
}
/*
* Advance the inode scan cursor to the next allocated inode and return the
* incore inode structure associated with it.
*
* Returns 1 if there's a new inode to examine, 0 if we've run out of inodes,
* -ECANCELED if the live scan aborted, -EBUSY if the incore inode could not be
* grabbed, or the usual negative errno.
*
* If the function returns -EBUSY and the caller can handle skipping an inode,
* it may call this function again to continue the scan with the next allocated
* inode.
*/
int
xchk_iscan_iter(
struct xchk_iscan *iscan,
struct xfs_inode **ipp)
{
unsigned int i;
int error;
/* Find a cached inode, or go get another batch. */
for (i = 0; i < XFS_INODES_PER_CHUNK; i++) {
if (iscan->__inodes[i])
goto foundit;
}
error = xchk_iscan_iter_batch(iscan);
if (error <= 0)
return error;
ASSERT(iscan->__inodes[0] != NULL);
i = 0;
foundit:
/* Give the caller our reference. */
*ipp = iscan->__inodes[i];
iscan->__inodes[i] = NULL;
return 1;
}
/* Clean up an xfs_iscan_iter call by dropping any inodes that we still hold. */
void
xchk_iscan_iter_finish(
struct xchk_iscan *iscan)
{
struct xfs_scrub *sc = iscan->sc;
unsigned int i;
for (i = 0; i < XFS_INODES_PER_CHUNK; i++) {
if (iscan->__inodes[i]) {
xchk_irele(sc, iscan->__inodes[i]);
iscan->__inodes[i] = NULL;
}
}
}
/* Mark this inode scan finished and release resources. */
void
xchk_iscan_teardown(
struct xchk_iscan *iscan)
{
xchk_iscan_iter_finish(iscan);
xchk_iscan_finish(iscan);
mutex_destroy(&iscan->lock);
}
/* Pick an AG from which to start a scan. */
static inline xfs_ino_t
xchk_iscan_rotor(
struct xfs_mount *mp)
{
static atomic_t agi_rotor;
unsigned int r = atomic_inc_return(&agi_rotor) - 1;
/*
* Rotoring *backwards* through the AGs, so we add one here before
* subtracting from the agcount to arrive at an AG number.
*/
r = (r % mp->m_sb.sb_agcount) + 1;
return XFS_AGINO_TO_INO(mp, mp->m_sb.sb_agcount - r, 0);
}
/*
* Set ourselves up to start an inode scan. If the @iget_timeout and
* @iget_retry_delay parameters are set, the scan will try to iget each inode
* for @iget_timeout milliseconds. If an iget call indicates that the inode is
* waiting to be inactivated, the CPU will relax for @iget_retry_delay
* milliseconds after pushing the inactivation workers.
*/
void
xchk_iscan_start(
struct xfs_scrub *sc,
unsigned int iget_timeout,
unsigned int iget_retry_delay,
struct xchk_iscan *iscan)
{
xfs_ino_t start_ino;
start_ino = xchk_iscan_rotor(sc->mp);
iscan->__batch_ino = NULLFSINO;
iscan->__skipped_inomask = 0;
iscan->sc = sc;
clear_bit(XCHK_ISCAN_OPSTATE_ABORTED, &iscan->__opstate);
iscan->iget_timeout = iget_timeout;
iscan->iget_retry_delay = iget_retry_delay;
iscan->__visited_ino = start_ino;
iscan->cursor_ino = start_ino;
iscan->scan_start_ino = start_ino;
mutex_init(&iscan->lock);
memset(iscan->__inodes, 0, sizeof(iscan->__inodes));
trace_xchk_iscan_start(iscan, start_ino);
}
/*
* Mark this inode as having been visited. Callers must hold a sufficiently
* exclusive lock on the inode to prevent concurrent modifications.
*/
void
xchk_iscan_mark_visited(
struct xchk_iscan *iscan,
struct xfs_inode *ip)
{
mutex_lock(&iscan->lock);
iscan->__visited_ino = ip->i_ino;
trace_xchk_iscan_visit(iscan);
mutex_unlock(&iscan->lock);
}
/*
* Did we skip this inode because it wasn't allocated when we loaded the batch?
* If so, it is newly allocated and will not be scanned. All live updates to
* this inode must be passed to the caller to maintain scan correctness.
*/
static inline bool
xchk_iscan_skipped(
const struct xchk_iscan *iscan,
xfs_ino_t ino)
{
if (iscan->__batch_ino == NULLFSINO)
return false;
if (ino < iscan->__batch_ino)
return false;
if (ino >= iscan->__batch_ino + XFS_INODES_PER_CHUNK)
return false;
return iscan->__skipped_inomask & (1ULL << (ino - iscan->__batch_ino));
}
/*
* Do we need a live update for this inode? This is true if the scanner thread
* has visited this inode and the scan hasn't been aborted due to errors.
* Callers must hold a sufficiently exclusive lock on the inode to prevent
* scanners from reading any inode metadata.
*/
bool
xchk_iscan_want_live_update(
struct xchk_iscan *iscan,
xfs_ino_t ino)
{
bool ret = false;
if (xchk_iscan_aborted(iscan))
return false;
mutex_lock(&iscan->lock);
trace_xchk_iscan_want_live_update(iscan, ino);
/* Scan is finished, caller should receive all updates. */
if (iscan->__visited_ino == NULLFSINO) {
ret = true;
goto unlock;
}
/*
* No inodes have been visited yet, so the visited cursor points at the
* start of the scan range. The caller should not receive any updates.
*/
if (iscan->scan_start_ino == iscan->__visited_ino) {
ret = false;
goto unlock;
}
/*
* This inode was not allocated at the time of the iscan batch.
* The caller should receive all updates.
*/
if (xchk_iscan_skipped(iscan, ino)) {
ret = true;
goto unlock;
}
/*
* The visited cursor hasn't yet wrapped around the end of the FS. If
* @ino is inside the starred range, the caller should receive updates:
*
* 0 ------------ S ************ V ------------ EOFS
*/
if (iscan->scan_start_ino <= iscan->__visited_ino) {
if (ino >= iscan->scan_start_ino &&
ino <= iscan->__visited_ino)
ret = true;
goto unlock;
}
/*
* The visited cursor wrapped around the end of the FS. If @ino is
* inside the starred range, the caller should receive updates:
*
* 0 ************ V ------------ S ************ EOFS
*/
if (ino >= iscan->scan_start_ino || ino <= iscan->__visited_ino)
ret = true;
unlock:
mutex_unlock(&iscan->lock);
return ret;
}

84
fs/xfs/scrub/iscan.h Normal file
View File

@ -0,0 +1,84 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_ISCAN_H__
#define __XFS_SCRUB_ISCAN_H__
struct xchk_iscan {
struct xfs_scrub *sc;
/* Lock to protect the scan cursor. */
struct mutex lock;
/*
* This is the first inode in the inumber address space that we
* examined. When the scan wraps around back to here, the scan is
* finished.
*/
xfs_ino_t scan_start_ino;
/* This is the inode that will be examined next. */
xfs_ino_t cursor_ino;
/* If nonzero and non-NULL, skip this inode when scanning. */
xfs_ino_t skip_ino;
/*
* This is the last inode that we've successfully scanned, either
* because the caller scanned it, or we moved the cursor past an empty
* part of the inode address space. Scan callers should only use the
* xchk_iscan_visit function to modify this.
*/
xfs_ino_t __visited_ino;
/* Operational state of the livescan. */
unsigned long __opstate;
/* Give up on iterating @cursor_ino if we can't iget it by this time. */
unsigned long __iget_deadline;
/* Amount of time (in ms) that we will try to iget an inode. */
unsigned int iget_timeout;
/* Wait this many ms to retry an iget. */
unsigned int iget_retry_delay;
/*
* The scan grabs batches of inodes and stashes them here before
* handing them out with _iter. Unallocated inodes are set in the
* mask so that all updates to that inode are selected for live
* update propagation.
*/
xfs_ino_t __batch_ino;
xfs_inofree_t __skipped_inomask;
struct xfs_inode *__inodes[XFS_INODES_PER_CHUNK];
};
/* Set if the scan has been aborted due to some event in the fs. */
#define XCHK_ISCAN_OPSTATE_ABORTED (1)
static inline bool
xchk_iscan_aborted(const struct xchk_iscan *iscan)
{
return test_bit(XCHK_ISCAN_OPSTATE_ABORTED, &iscan->__opstate);
}
static inline void
xchk_iscan_abort(struct xchk_iscan *iscan)
{
set_bit(XCHK_ISCAN_OPSTATE_ABORTED, &iscan->__opstate);
}
void xchk_iscan_start(struct xfs_scrub *sc, unsigned int iget_timeout,
unsigned int iget_retry_delay, struct xchk_iscan *iscan);
void xchk_iscan_teardown(struct xchk_iscan *iscan);
int xchk_iscan_iter(struct xchk_iscan *iscan, struct xfs_inode **ipp);
void xchk_iscan_iter_finish(struct xchk_iscan *iscan);
void xchk_iscan_mark_visited(struct xchk_iscan *iscan, struct xfs_inode *ip);
bool xchk_iscan_want_live_update(struct xchk_iscan *iscan, xfs_ino_t ino);
#endif /* __XFS_SCRUB_ISCAN_H__ */

14
fs/xfs/scrub/newbt.c
View File

@ -239,7 +239,11 @@ xrep_newbt_alloc_ag_blocks(
xrep_newbt_validate_ag_alloc_hint(xnr);
error = xfs_alloc_vextent_near_bno(&args, xnr->alloc_hint);
if (xnr->alloc_vextent)
error = xnr->alloc_vextent(sc, &args, xnr->alloc_hint);
else
error = xfs_alloc_vextent_near_bno(&args,
xnr->alloc_hint);
if (error)
return error;
if (args.fsbno == NULLFSBLOCK)
@ -309,7 +313,11 @@ xrep_newbt_alloc_file_blocks(
xrep_newbt_validate_file_alloc_hint(xnr);
error = xfs_alloc_vextent_start_ag(&args, xnr->alloc_hint);
if (xnr->alloc_vextent)
error = xnr->alloc_vextent(sc, &args, xnr->alloc_hint);
else
error = xfs_alloc_vextent_start_ag(&args,
xnr->alloc_hint);
if (error)
return error;
if (args.fsbno == NULLFSBLOCK)
@ -535,7 +543,7 @@ xrep_newbt_claim_block(
trace_xrep_newbt_claim_block(mp, resv->pag->pag_agno, agbno, 1,
xnr->oinfo.oi_owner);
if (cur->bc_flags & XFS_BTREE_LONG_PTRS)
if (cur->bc_ops->ptr_len == XFS_BTREE_LONG_PTR_LEN)
ptr->l = cpu_to_be64(XFS_AGB_TO_FSB(mp, resv->pag->pag_agno,
agbno));
else

7
fs/xfs/scrub/newbt.h
View File

@ -6,6 +6,8 @@
#ifndef __XFS_SCRUB_NEWBT_H__
#define __XFS_SCRUB_NEWBT_H__
struct xfs_alloc_arg;
struct xrep_newbt_resv {
/* Link to list of extents that we've reserved. */
struct list_head list;
@ -28,6 +30,11 @@ struct xrep_newbt_resv {
struct xrep_newbt {
struct xfs_scrub *sc;
/* Custom allocation function, or NULL for xfs_alloc_vextent */
int (*alloc_vextent)(struct xfs_scrub *sc,
struct xfs_alloc_arg *args,
xfs_fsblock_t alloc_hint);
/* List of extents that we've reserved. */
struct list_head resv_list;

930
fs/xfs/scrub/nlinks.c Normal file
View File

@ -0,0 +1,930 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_inode.h"
#include "xfs_icache.h"
#include "xfs_iwalk.h"
#include "xfs_ialloc.h"
#include "xfs_dir2.h"
#include "xfs_dir2_priv.h"
#include "xfs_ag.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/repair.h"
#include "scrub/xfile.h"
#include "scrub/xfarray.h"
#include "scrub/iscan.h"
#include "scrub/nlinks.h"
#include "scrub/trace.h"
#include "scrub/readdir.h"
/*
* Live Inode Link Count Checking
* ==============================
*
* Inode link counts are "summary" metadata, in the sense that they are
* computed as the number of directory entries referencing each file on the
* filesystem. Therefore, we compute the correct link counts by creating a
* shadow link count structure and walking every inode.
*/
/* Set us up to scrub inode link counts. */
int
xchk_setup_nlinks(
struct xfs_scrub *sc)
{
xchk_fsgates_enable(sc, XCHK_FSGATES_DIRENTS);
sc->buf = kzalloc(sizeof(struct xchk_nlink_ctrs), XCHK_GFP_FLAGS);
if (!sc->buf)
return -ENOMEM;
return xchk_setup_fs(sc);
}
/*
* Part 1: Collecting file link counts. For each file, we create a shadow link
* counting structure, then walk the entire directory tree, incrementing parent
* and child link counts for each directory entry seen.
*
* To avoid false corruption reports in part 2, any failure in this part must
* set the INCOMPLETE flag even when a negative errno is returned. This care
* must be taken with certain errno values (i.e. EFSBADCRC, EFSCORRUPTED,
* ECANCELED) that are absorbed into a scrub state flag update by
* xchk_*_process_error. Scrub and repair share the same incore data
* structures, so the INCOMPLETE flag is critical to prevent a repair based on
* insufficient information.
*
* Because we are scanning a live filesystem, it's possible that another thread
* will try to update the link counts for an inode that we've already scanned.
* This will cause our counts to be incorrect. Therefore, we hook all
* directory entry updates because that is when link count updates occur. By
* shadowing transaction updates in this manner, live nlink check can ensure by
* locking the inode and the shadow structure that its own copies are not out
* of date. Because the hook code runs in a different process context from the
* scrub code and the scrub state flags are not accessed atomically, failures
* in the hook code must abort the iscan and the scrubber must notice the
* aborted scan and set the incomplete flag.
*
* Note that we use jump labels and srcu notifier hooks to minimize the
* overhead when live nlinks is /not/ running. Locking order for nlink
* observations is inode ILOCK -> iscan_lock/xchk_nlink_ctrs lock.
*/
/*
* Add a delta to an nlink counter, clamping the value to U32_MAX. Because
* XFS_MAXLINK < U32_MAX, the checking code will produce the correct results
* even if we lose some precision.
*/
static inline void
careful_add(
xfs_nlink_t *nlinkp,
int delta)
{
uint64_t new_value = (uint64_t)(*nlinkp) + delta;
BUILD_BUG_ON(XFS_MAXLINK > U32_MAX);
*nlinkp = min_t(uint64_t, new_value, U32_MAX);
}
/* Update incore link count information. Caller must hold the nlinks lock. */
STATIC int
xchk_nlinks_update_incore(
struct xchk_nlink_ctrs *xnc,
xfs_ino_t ino,
int parents_delta,
int backrefs_delta,
int children_delta)
{
struct xchk_nlink nl;
int error;
if (!xnc->nlinks)
return 0;
error = xfarray_load_sparse(xnc->nlinks, ino, &nl);
if (error)
return error;
trace_xchk_nlinks_update_incore(xnc->sc->mp, ino, &nl, parents_delta,
backrefs_delta, children_delta);
careful_add(&nl.parents, parents_delta);
careful_add(&nl.backrefs, backrefs_delta);
careful_add(&nl.children, children_delta);
nl.flags |= XCHK_NLINK_WRITTEN;
error = xfarray_store(xnc->nlinks, ino, &nl);
if (error == -EFBIG) {
/*
* EFBIG means we tried to store data at too high a byte offset
* in the sparse array. IOWs, we cannot complete the check and
* must notify userspace that the check was incomplete.
*/
error = -ECANCELED;
}
return error;
}
/*
* Apply a link count change from the regular filesystem into our shadow link
* count structure based on a directory update in progress.
*/
STATIC int
xchk_nlinks_live_update(
struct notifier_block *nb,
unsigned long action,
void *data)
{
struct xfs_dir_update_params *p = data;
struct xchk_nlink_ctrs *xnc;
int error;
xnc = container_of(nb, struct xchk_nlink_ctrs, dhook.dirent_hook.nb);
trace_xchk_nlinks_live_update(xnc->sc->mp, p->dp, action, p->ip->i_ino,
p->delta, p->name->name, p->name->len);
/*
* If we've already scanned @dp, update the number of parents that link
* to @ip. If @ip is a subdirectory, update the number of child links
* going out of @dp.
*/
if (xchk_iscan_want_live_update(&xnc->collect_iscan, p->dp->i_ino)) {
mutex_lock(&xnc->lock);
error = xchk_nlinks_update_incore(xnc, p->ip->i_ino, p->delta,
0, 0);
if (!error && S_ISDIR(VFS_IC(p->ip)->i_mode))
error = xchk_nlinks_update_incore(xnc, p->dp->i_ino, 0,
0, p->delta);
mutex_unlock(&xnc->lock);
if (error)
goto out_abort;
}
/*
* If @ip is a subdirectory and we've already scanned it, update the
* number of backrefs pointing to @dp.
*/
if (S_ISDIR(VFS_IC(p->ip)->i_mode) &&
xchk_iscan_want_live_update(&xnc->collect_iscan, p->ip->i_ino)) {
mutex_lock(&xnc->lock);
error = xchk_nlinks_update_incore(xnc, p->dp->i_ino, 0,
p->delta, 0);
mutex_unlock(&xnc->lock);
if (error)
goto out_abort;
}
return NOTIFY_DONE;
out_abort:
xchk_iscan_abort(&xnc->collect_iscan);
return NOTIFY_DONE;
}
/* Bump the observed link count for the inode referenced by this entry. */
STATIC int
xchk_nlinks_collect_dirent(
struct xfs_scrub *sc,
struct xfs_inode *dp,
xfs_dir2_dataptr_t dapos,
const struct xfs_name *name,
xfs_ino_t ino,
void *priv)
{
struct xchk_nlink_ctrs *xnc = priv;
bool dot = false, dotdot = false;
int error;
/* Does this name make sense? */
if (name->len == 0 || !xfs_dir2_namecheck(name->name, name->len)) {
error = -ECANCELED;
goto out_abort;
}
if (name->len == 1 && name->name[0] == '.')
dot = true;
else if (name->len == 2 && name->name[0] == '.' &&
name->name[1] == '.')
dotdot = true;
/* Don't accept a '.' entry that points somewhere else. */
if (dot && ino != dp->i_ino) {
error = -ECANCELED;
goto out_abort;
}
/* Don't accept an invalid inode number. */
if (!xfs_verify_dir_ino(sc->mp, ino)) {
error = -ECANCELED;
goto out_abort;
}
/* Update the shadow link counts if we haven't already failed. */
if (xchk_iscan_aborted(&xnc->collect_iscan)) {
error = -ECANCELED;
goto out_incomplete;
}
trace_xchk_nlinks_collect_dirent(sc->mp, dp, ino, name);
mutex_lock(&xnc->lock);
/*
* If this is a dotdot entry, it is a back link from dp to ino. How
* we handle this depends on whether or not dp is the root directory.
*
* The root directory is its own parent, so we pretend the dotdot entry
* establishes the "parent" of the root directory. Increment the
* number of parents of the root directory.
*
* Otherwise, increment the number of backrefs pointing back to ino.
*/
if (dotdot) {
if (dp == sc->mp->m_rootip)
error = xchk_nlinks_update_incore(xnc, ino, 1, 0, 0);
else
error = xchk_nlinks_update_incore(xnc, ino, 0, 1, 0);
if (error)
goto out_unlock;
}
/*
* If this dirent is a forward link from dp to ino, increment the
* number of parents linking into ino.
*/
if (!dot && !dotdot) {
error = xchk_nlinks_update_incore(xnc, ino, 1, 0, 0);
if (error)
goto out_unlock;
}
/*
* If this dirent is a forward link to a subdirectory, increment the
* number of child links of dp.
*/
if (!dot && !dotdot && name->type == XFS_DIR3_FT_DIR) {
error = xchk_nlinks_update_incore(xnc, dp->i_ino, 0, 0, 1);
if (error)
goto out_unlock;
}
mutex_unlock(&xnc->lock);
return 0;
out_unlock:
mutex_unlock(&xnc->lock);
out_abort:
xchk_iscan_abort(&xnc->collect_iscan);
out_incomplete:
xchk_set_incomplete(sc);
return error;
}
/* Walk a directory to bump the observed link counts of the children. */
STATIC int
xchk_nlinks_collect_dir(
struct xchk_nlink_ctrs *xnc,
struct xfs_inode *dp)
{
struct xfs_scrub *sc = xnc->sc;
unsigned int lock_mode;
int error = 0;
/* Prevent anyone from changing this directory while we walk it. */
xfs_ilock(dp, XFS_IOLOCK_SHARED);
lock_mode = xfs_ilock_data_map_shared(dp);
/*
* The dotdot entry of an unlinked directory still points to the last
* parent, but the parent no longer links to this directory. Skip the
* directory to avoid overcounting.
*/
if (VFS_I(dp)->i_nlink == 0)
goto out_unlock;
/*
* We cannot count file links if the directory looks as though it has
* been zapped by the inode record repair code.
*/
if (xchk_dir_looks_zapped(dp)) {
error = -EBUSY;
goto out_abort;
}
error = xchk_dir_walk(sc, dp, xchk_nlinks_collect_dirent, xnc);
if (error == -ECANCELED) {
error = 0;
goto out_unlock;
}
if (error)
goto out_abort;
xchk_iscan_mark_visited(&xnc->collect_iscan, dp);
goto out_unlock;
out_abort:
xchk_set_incomplete(sc);
xchk_iscan_abort(&xnc->collect_iscan);
out_unlock:
xfs_iunlock(dp, lock_mode);
xfs_iunlock(dp, XFS_IOLOCK_SHARED);
return error;
}
/* If this looks like a valid pointer, count it. */
static inline int
xchk_nlinks_collect_metafile(
struct xchk_nlink_ctrs *xnc,
xfs_ino_t ino)
{
if (!xfs_verify_ino(xnc->sc->mp, ino))
return 0;
trace_xchk_nlinks_collect_metafile(xnc->sc->mp, ino);
return xchk_nlinks_update_incore(xnc, ino, 1, 0, 0);
}
/* Bump the link counts of metadata files rooted in the superblock. */
STATIC int
xchk_nlinks_collect_metafiles(
struct xchk_nlink_ctrs *xnc)
{
struct xfs_mount *mp = xnc->sc->mp;
int error = -ECANCELED;
if (xchk_iscan_aborted(&xnc->collect_iscan))
goto out_incomplete;
mutex_lock(&xnc->lock);
error = xchk_nlinks_collect_metafile(xnc, mp->m_sb.sb_rbmino);
if (error)
goto out_abort;
error = xchk_nlinks_collect_metafile(xnc, mp->m_sb.sb_rsumino);
if (error)
goto out_abort;
error = xchk_nlinks_collect_metafile(xnc, mp->m_sb.sb_uquotino);
if (error)
goto out_abort;
error = xchk_nlinks_collect_metafile(xnc, mp->m_sb.sb_gquotino);
if (error)
goto out_abort;
error = xchk_nlinks_collect_metafile(xnc, mp->m_sb.sb_pquotino);
if (error)
goto out_abort;
mutex_unlock(&xnc->lock);
return 0;
out_abort:
mutex_unlock(&xnc->lock);
xchk_iscan_abort(&xnc->collect_iscan);
out_incomplete:
xchk_set_incomplete(xnc->sc);
return error;
}
/* Advance the collection scan cursor for this non-directory file. */
static inline int
xchk_nlinks_collect_file(
struct xchk_nlink_ctrs *xnc,
struct xfs_inode *ip)
{
xfs_ilock(ip, XFS_IOLOCK_SHARED);
xchk_iscan_mark_visited(&xnc->collect_iscan, ip);
xfs_iunlock(ip, XFS_IOLOCK_SHARED);
return 0;
}
/* Walk all directories and count inode links. */
STATIC int
xchk_nlinks_collect(
struct xchk_nlink_ctrs *xnc)
{
struct xfs_scrub *sc = xnc->sc;
struct xfs_inode *ip;
int error;
/* Count the rt and quota files that are rooted in the superblock. */
error = xchk_nlinks_collect_metafiles(xnc);
if (error)
return error;
/*
* Set up for a potentially lengthy filesystem scan by reducing our
* transaction resource usage for the duration. Specifically:
*
* Cancel the transaction to release the log grant space while we scan
* the filesystem.
*
* Create a new empty transaction to eliminate the possibility of the
* inode scan deadlocking on cyclical metadata.
*
* We pass the empty transaction to the file scanning function to avoid
* repeatedly cycling empty transactions. This can be done even though
* we take the IOLOCK to quiesce the file because empty transactions
* do not take sb_internal.
*/
xchk_trans_cancel(sc);
error = xchk_trans_alloc_empty(sc);
if (error)
return error;
while ((error = xchk_iscan_iter(&xnc->collect_iscan, &ip)) == 1) {
if (S_ISDIR(VFS_I(ip)->i_mode))
error = xchk_nlinks_collect_dir(xnc, ip);
else
error = xchk_nlinks_collect_file(xnc, ip);
xchk_irele(sc, ip);
if (error)
break;
if (xchk_should_terminate(sc, &error))
break;
}
xchk_iscan_iter_finish(&xnc->collect_iscan);
if (error) {
xchk_set_incomplete(sc);
/*
* If we couldn't grab an inode that was busy with a state
* change, change the error code so that we exit to userspace
* as quickly as possible.
*/
if (error == -EBUSY)
return -ECANCELED;
return error;
}
/*
* Switch out for a real transaction in preparation for building a new
* tree.
*/
xchk_trans_cancel(sc);
return xchk_setup_fs(sc);
}
/*
* Part 2: Comparing file link counters. Walk each inode and compare the link
* counts against our shadow information; and then walk each shadow link count
* structure (that wasn't covered in the first part), comparing it against the
* file.
*/
/* Read the observed link count for comparison with the actual inode. */
STATIC int
xchk_nlinks_comparison_read(
struct xchk_nlink_ctrs *xnc,
xfs_ino_t ino,
struct xchk_nlink *obs)
{
struct xchk_nlink nl;
int error;
error = xfarray_load_sparse(xnc->nlinks, ino, &nl);
if (error)
return error;
nl.flags |= (XCHK_NLINK_COMPARE_SCANNED | XCHK_NLINK_WRITTEN);
error = xfarray_store(xnc->nlinks, ino, &nl);
if (error == -EFBIG) {
/*
* EFBIG means we tried to store data at too high a byte offset
* in the sparse array. IOWs, we cannot complete the check and
* must notify userspace that the check was incomplete. This
* shouldn't really happen outside of the collection phase.
*/
xchk_set_incomplete(xnc->sc);
return -ECANCELED;
}
if (error)
return error;
/* Copy the counters, but do not expose the internal state. */
obs->parents = nl.parents;
obs->backrefs = nl.backrefs;
obs->children = nl.children;
obs->flags = 0;
return 0;
}
/* Check our link count against an inode. */
STATIC int
xchk_nlinks_compare_inode(
struct xchk_nlink_ctrs *xnc,
struct xfs_inode *ip)
{
struct xchk_nlink obs;
struct xfs_scrub *sc = xnc->sc;
uint64_t total_links;
unsigned int actual_nlink;
int error;
xfs_ilock(ip, XFS_ILOCK_SHARED);
mutex_lock(&xnc->lock);
if (xchk_iscan_aborted(&xnc->collect_iscan)) {
xchk_set_incomplete(xnc->sc);
error = -ECANCELED;
goto out_scanlock;
}
error = xchk_nlinks_comparison_read(xnc, ip->i_ino, &obs);
if (error)
goto out_scanlock;
/*
* If we don't have ftype to get an accurate count of the subdirectory
* entries in this directory, take advantage of the fact that on a
* consistent ftype=0 filesystem, the number of subdirectory
* backreferences (dotdot entries) pointing towards this directory
* should be equal to the number of subdirectory entries in the
* directory.
*/
if (!xfs_has_ftype(sc->mp) && S_ISDIR(VFS_I(ip)->i_mode))
obs.children = obs.backrefs;
total_links = xchk_nlink_total(ip, &obs);
actual_nlink = VFS_I(ip)->i_nlink;
trace_xchk_nlinks_compare_inode(sc->mp, ip, &obs);
/*
* If we found so many parents that we'd overflow i_nlink, we must flag
* this as a corruption. The VFS won't let users increase the link
* count, but it will let them decrease it.
*/
if (total_links > XFS_MAXLINK) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
/* Link counts should match. */
if (total_links != actual_nlink) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
if (S_ISDIR(VFS_I(ip)->i_mode) && actual_nlink > 0) {
/*
* The collection phase ignores directories with zero link
* count, so we ignore them here too.
*
* The number of subdirectory backreferences (dotdot entries)
* pointing towards this directory should be equal to the
* number of subdirectory entries in the directory.
*/
if (obs.children != obs.backrefs)
xchk_ino_xref_set_corrupt(sc, ip->i_ino);
} else {
/*
* Non-directories and unlinked directories should not have
* back references.
*/
if (obs.backrefs != 0) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
/*
* Non-directories and unlinked directories should not have
* children.
*/
if (obs.children != 0) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
}
if (ip == sc->mp->m_rootip) {
/*
* For the root of a directory tree, both the '.' and '..'
* entries should point to the root directory. The dotdot
* entry is counted as a parent of the root /and/ a backref of
* the root directory.
*/
if (obs.parents != 1) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
} else if (actual_nlink > 0) {
/*
* Linked files that are not the root directory should have at
* least one parent.
*/
if (obs.parents == 0) {
xchk_ino_set_corrupt(sc, ip->i_ino);
goto out_corrupt;
}
}
out_corrupt:
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
error = -ECANCELED;
out_scanlock:
mutex_unlock(&xnc->lock);
xfs_iunlock(ip, XFS_ILOCK_SHARED);
return error;
}
/*
* Check our link count against an inode that wasn't checked previously. This
* is intended to catch directories with dangling links, though we could be
* racing with inode allocation in other threads.
*/
STATIC int
xchk_nlinks_compare_inum(
struct xchk_nlink_ctrs *xnc,
xfs_ino_t ino)
{
struct xchk_nlink obs;
struct xfs_mount *mp = xnc->sc->mp;
struct xfs_trans *tp = xnc->sc->tp;
struct xfs_buf *agi_bp;
struct xfs_inode *ip;
int error;
/*
* The first iget failed, so try again with the variant that returns
* either an incore inode or the AGI buffer. If the function returns
* EINVAL/ENOENT, it should have passed us the AGI buffer so that we
* can guarantee that the inode won't be allocated while we check for
* a zero link count in the observed link count data.
*/
error = xchk_iget_agi(xnc->sc, ino, &agi_bp, &ip);
if (!error) {
/* Actually got an inode, so use the inode compare. */
error = xchk_nlinks_compare_inode(xnc, ip);
xchk_irele(xnc->sc, ip);
return error;
}
if (error == -ENOENT || error == -EINVAL) {
/* No inode was found. Check for zero link count below. */
error = 0;
}
if (error)
goto out_agi;
/* Ensure that we have protected against inode allocation/freeing. */
if (agi_bp == NULL) {
ASSERT(agi_bp != NULL);
xchk_set_incomplete(xnc->sc);
return -ECANCELED;
}
if (xchk_iscan_aborted(&xnc->collect_iscan)) {
xchk_set_incomplete(xnc->sc);
error = -ECANCELED;
goto out_agi;
}
mutex_lock(&xnc->lock);
error = xchk_nlinks_comparison_read(xnc, ino, &obs);
if (error)
goto out_scanlock;
trace_xchk_nlinks_check_zero(mp, ino, &obs);
/*
* If we can't grab the inode, the link count had better be zero. We
* still hold the AGI to prevent inode allocation/freeing.
*/
if (xchk_nlink_total(NULL, &obs) != 0) {
xchk_ino_set_corrupt(xnc->sc, ino);
error = -ECANCELED;
}
out_scanlock:
mutex_unlock(&xnc->lock);
out_agi:
if (agi_bp)
xfs_trans_brelse(tp, agi_bp);
return error;
}
/*
* Try to visit every inode in the filesystem to compare the link count. Move
* on if we can't grab an inode, since we'll revisit unchecked nlink records in
* the second part.
*/
static int
xchk_nlinks_compare_iter(
struct xchk_nlink_ctrs *xnc,
struct xfs_inode **ipp)
{
int error;
do {
error = xchk_iscan_iter(&xnc->compare_iscan, ipp);
} while (error == -EBUSY);
return error;
}
/* Compare the link counts we observed against the live information. */
STATIC int
xchk_nlinks_compare(
struct xchk_nlink_ctrs *xnc)
{
struct xchk_nlink nl;
struct xfs_scrub *sc = xnc->sc;
struct xfs_inode *ip;
xfarray_idx_t cur = XFARRAY_CURSOR_INIT;
int error;
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
return 0;
/*
* Create a new empty transaction so that we can advance the iscan
* cursor without deadlocking if the inobt has a cycle and push on the
* inactivation workqueue.
*/
xchk_trans_cancel(sc);
error = xchk_trans_alloc_empty(sc);
if (error)
return error;
/*
* Use the inobt to walk all allocated inodes to compare the link
* counts. Inodes skipped by _compare_iter will be tried again in the
* next phase of the scan.
*/
xchk_iscan_start(sc, 0, 0, &xnc->compare_iscan);
while ((error = xchk_nlinks_compare_iter(xnc, &ip)) == 1) {
error = xchk_nlinks_compare_inode(xnc, ip);
xchk_iscan_mark_visited(&xnc->compare_iscan, ip);
xchk_irele(sc, ip);
if (error)
break;
if (xchk_should_terminate(sc, &error))
break;
}
xchk_iscan_iter_finish(&xnc->compare_iscan);
xchk_iscan_teardown(&xnc->compare_iscan);
if (error)
return error;
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
return 0;
/*
* Walk all the non-null nlink observations that weren't checked in the
* previous step.
*/
mutex_lock(&xnc->lock);
while ((error = xfarray_iter(xnc->nlinks, &cur, &nl)) == 1) {
xfs_ino_t ino = cur - 1;
if (nl.flags & XCHK_NLINK_COMPARE_SCANNED)
continue;
mutex_unlock(&xnc->lock);
error = xchk_nlinks_compare_inum(xnc, ino);
if (error)
return error;
if (xchk_should_terminate(xnc->sc, &error))
return error;
mutex_lock(&xnc->lock);
}
mutex_unlock(&xnc->lock);
return error;
}
/* Tear down everything associated with a nlinks check. */
static void
xchk_nlinks_teardown_scan(
void *priv)
{
struct xchk_nlink_ctrs *xnc = priv;
/* Discourage any hook functions that might be running. */
xchk_iscan_abort(&xnc->collect_iscan);
xfs_dir_hook_del(xnc->sc->mp, &xnc->dhook);
xfarray_destroy(xnc->nlinks);
xnc->nlinks = NULL;
xchk_iscan_teardown(&xnc->collect_iscan);
mutex_destroy(&xnc->lock);
xnc->sc = NULL;
}
/*
* Scan all inodes in the entire filesystem to generate link count data. If
* the scan is successful, the counts will be left alive for a repair. If any
* error occurs, we'll tear everything down.
*/
STATIC int
xchk_nlinks_setup_scan(
struct xfs_scrub *sc,
struct xchk_nlink_ctrs *xnc)
{
struct xfs_mount *mp = sc->mp;
char *descr;
unsigned long long max_inos;
xfs_agnumber_t last_agno = mp->m_sb.sb_agcount - 1;
xfs_agino_t first_agino, last_agino;
int error;
ASSERT(xnc->sc == NULL);
xnc->sc = sc;
mutex_init(&xnc->lock);
/* Retry iget every tenth of a second for up to 30 seconds. */
xchk_iscan_start(sc, 30000, 100, &xnc->collect_iscan);
/*
* Set up enough space to store an nlink record for the highest
* possible inode number in this system.
*/
xfs_agino_range(mp, last_agno, &first_agino, &last_agino);
max_inos = XFS_AGINO_TO_INO(mp, last_agno, last_agino) + 1;
descr = xchk_xfile_descr(sc, "file link counts");
error = xfarray_create(descr, min(XFS_MAXINUMBER + 1, max_inos),
sizeof(struct xchk_nlink), &xnc->nlinks);
kfree(descr);
if (error)
goto out_teardown;
/*
* Hook into the directory entry code so that we can capture updates to
* file link counts. The hook only triggers for inodes that were
* already scanned, and the scanner thread takes each inode's ILOCK,
* which means that any in-progress inode updates will finish before we
* can scan the inode.
*/
ASSERT(sc->flags & XCHK_FSGATES_DIRENTS);
xfs_dir_hook_setup(&xnc->dhook, xchk_nlinks_live_update);
error = xfs_dir_hook_add(mp, &xnc->dhook);
if (error)
goto out_teardown;
/* Use deferred cleanup to pass the inode link count data to repair. */
sc->buf_cleanup = xchk_nlinks_teardown_scan;
return 0;
out_teardown:
xchk_nlinks_teardown_scan(xnc);
return error;
}
/* Scrub the link count of all inodes on the filesystem. */
int
xchk_nlinks(
struct xfs_scrub *sc)
{
struct xchk_nlink_ctrs *xnc = sc->buf;
int error = 0;
/* Set ourselves up to check link counts on the live filesystem. */
error = xchk_nlinks_setup_scan(sc, xnc);
if (error)
return error;
/* Walk all inodes, picking up link count information. */
error = xchk_nlinks_collect(xnc);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
/* Fail fast if we're not playing with a full dataset. */
if (xchk_iscan_aborted(&xnc->collect_iscan))
xchk_set_incomplete(sc);
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE)
return 0;
/* Compare link counts. */
error = xchk_nlinks_compare(xnc);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
/* Check one last time for an incomplete dataset. */
if (xchk_iscan_aborted(&xnc->collect_iscan))
xchk_set_incomplete(sc);
return 0;
}

102
fs/xfs/scrub/nlinks.h Normal file
View File

@ -0,0 +1,102 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_NLINKS_H__
#define __XFS_SCRUB_NLINKS_H__
/* Live link count control structure. */
struct xchk_nlink_ctrs {
struct xfs_scrub *sc;
/* Shadow link count data and its mutex. */
struct xfarray *nlinks;
struct mutex lock;
/*
* The collection step uses a separate iscan context from the compare
* step because the collection iscan coordinates live updates to the
* observation data while this scanner is running. The compare iscan
* is secondary and can be reinitialized as needed.
*/
struct xchk_iscan collect_iscan;
struct xchk_iscan compare_iscan;
/*
* Hook into directory updates so that we can receive live updates
* from other writer threads.
*/
struct xfs_dir_hook dhook;
};
/*
* In-core link counts for a given inode in the filesystem.
*
* For an empty rootdir, the directory entries and the field to which they are
* accounted are as follows:
*
* Root directory:
*
* . points to self (root.child)
* .. points to self (root.parent)
* f1 points to a child file (f1.parent)
* d1 points to a child dir (d1.parent, root.child)
*
* Subdirectory d1:
*
* . points to self (d1.child)
* .. points to root dir (root.backref)
* f2 points to child file (f2.parent)
* f3 points to root.f1 (f1.parent)
*
* root.nlink == 3 (root.dot, root.dotdot, root.d1)
* d1.nlink == 2 (root.d1, d1.dot)
* f1.nlink == 2 (root.f1, d1.f3)
* f2.nlink == 1 (d1.f2)
*/
struct xchk_nlink {
/* Count of forward links from parent directories to this file. */
xfs_nlink_t parents;
/*
* Count of back links to this parent directory from child
* subdirectories.
*/
xfs_nlink_t backrefs;
/*
* Count of forward links from this directory to all child files and
* the number of dot entries. Should be zero for non-directories.
*/
xfs_nlink_t children;
/* Record state flags */
unsigned int flags;
};
/*
* This incore link count has been written at least once. We never want to
* store an xchk_nlink that looks uninitialized.
*/
#define XCHK_NLINK_WRITTEN (1U << 0)
/* Already checked this link count record. */
#define XCHK_NLINK_COMPARE_SCANNED (1U << 1)
/* Already made a repair with this link count record. */
#define XREP_NLINK_DIRTY (1U << 2)
/* Compute total link count, using large enough variables to detect overflow. */
static inline uint64_t
xchk_nlink_total(struct xfs_inode *ip, const struct xchk_nlink *live)
{
uint64_t ret = live->parents;
/* Add one link count for the dot entry of any linked directory. */
if (ip && S_ISDIR(VFS_I(ip)->i_mode) && VFS_I(ip)->i_nlink)
ret++;
return ret + live->children;
}
#endif /* __XFS_SCRUB_NLINKS_H__ */

223
fs/xfs/scrub/nlinks_repair.c Normal file
View File

@ -0,0 +1,223 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2021-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_inode.h"
#include "xfs_icache.h"
#include "xfs_bmap_util.h"
#include "xfs_iwalk.h"
#include "xfs_ialloc.h"
#include "xfs_sb.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/repair.h"
#include "scrub/xfile.h"
#include "scrub/xfarray.h"
#include "scrub/iscan.h"
#include "scrub/nlinks.h"
#include "scrub/trace.h"
/*
* Live Inode Link Count Repair
* ============================
*
* Use the live inode link count information that we collected to replace the
* nlink values of the incore inodes. A scrub->repair cycle should have left
* the live data and hooks active, so this is safe so long as we make sure the
* inode is locked.
*/
/*
* Correct the link count of the given inode. Because we have to grab locks
* and resources in a certain order, it's possible that this will be a no-op.
*/
STATIC int
xrep_nlinks_repair_inode(
struct xchk_nlink_ctrs *xnc)
{
struct xchk_nlink obs;
struct xfs_scrub *sc = xnc->sc;
struct xfs_mount *mp = sc->mp;
struct xfs_inode *ip = sc->ip;
uint64_t total_links;
uint64_t actual_nlink;
bool dirty = false;
int error;
xchk_ilock(sc, XFS_IOLOCK_EXCL);
error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, 0, 0, 0, &sc->tp);
if (error)
return error;
xchk_ilock(sc, XFS_ILOCK_EXCL);
xfs_trans_ijoin(sc->tp, ip, 0);
mutex_lock(&xnc->lock);
if (xchk_iscan_aborted(&xnc->collect_iscan)) {
error = -ECANCELED;
goto out_scanlock;
}
error = xfarray_load_sparse(xnc->nlinks, ip->i_ino, &obs);
if (error)
goto out_scanlock;
/*
* We're done accessing the shared scan data, so we can drop the lock.
* We still hold @ip's ILOCK, so its link count cannot change.
*/
mutex_unlock(&xnc->lock);
total_links = xchk_nlink_total(ip, &obs);
actual_nlink = VFS_I(ip)->i_nlink;
/*
* Non-directories cannot have directories pointing up to them.
*
* We previously set error to zero, but set it again because one static
* checker author fears that programmers will fail to maintain this
* invariant and built their tool to flag this as a security risk. A
* different tool author made their bot complain about the redundant
* store. This is a never-ending and stupid battle; both tools missed
* *actual bugs* elsewhere; and I no longer care.
*/
if (!S_ISDIR(VFS_I(ip)->i_mode) && obs.children != 0) {
trace_xrep_nlinks_unfixable_inode(mp, ip, &obs);
error = 0;
goto out_trans;
}
/*
* We did not find any links to this inode. If the inode agrees, we
* have nothing further to do. If not, the inode has a nonzero link
* count and we don't have anywhere to graft the child onto. Dropping
* a live inode's link count to zero can cause unexpected shutdowns in
* inactivation, so leave it alone.
*/
if (total_links == 0) {
if (actual_nlink != 0)
trace_xrep_nlinks_unfixable_inode(mp, ip, &obs);
goto out_trans;
}
/* Commit the new link count if it changed. */
if (total_links != actual_nlink) {
if (total_links > XFS_MAXLINK) {
trace_xrep_nlinks_unfixable_inode(mp, ip, &obs);
goto out_trans;
}
trace_xrep_nlinks_update_inode(mp, ip, &obs);
set_nlink(VFS_I(ip), total_links);
dirty = true;
}
if (!dirty) {
error = 0;
goto out_trans;
}
xfs_trans_log_inode(sc->tp, ip, XFS_ILOG_CORE);
error = xrep_trans_commit(sc);
xchk_iunlock(sc, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
return error;
out_scanlock:
mutex_unlock(&xnc->lock);
out_trans:
xchk_trans_cancel(sc);
xchk_iunlock(sc, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
return error;
}
/*
* Try to visit every inode in the filesystem for repairs. Move on if we can't
* grab an inode, since we're still making forward progress.
*/
static int
xrep_nlinks_iter(
struct xchk_nlink_ctrs *xnc,
struct xfs_inode **ipp)
{
int error;
do {
error = xchk_iscan_iter(&xnc->compare_iscan, ipp);
} while (error == -EBUSY);
return error;
}
/* Commit the new inode link counters. */
int
xrep_nlinks(
struct xfs_scrub *sc)
{
struct xchk_nlink_ctrs *xnc = sc->buf;
int error;
/*
* We need ftype for an accurate count of the number of child
* subdirectory links. Child subdirectories with a back link (dotdot
* entry) but no forward link are unfixable, so we cannot repair the
* link count of the parent directory based on the back link count
* alone. Filesystems without ftype support are rare (old V4) so we
* just skip out here.
*/
if (!xfs_has_ftype(sc->mp))
return -EOPNOTSUPP;
/*
* Use the inobt to walk all allocated inodes to compare and fix the
* link counts. Retry iget every tenth of a second for up to 30
* seconds -- even if repair misses a few inodes, we still try to fix
* as many of them as we can.
*/
xchk_iscan_start(sc, 30000, 100, &xnc->compare_iscan);
ASSERT(sc->ip == NULL);
while ((error = xrep_nlinks_iter(xnc, &sc->ip)) == 1) {
/*
* Commit the scrub transaction so that we can create repair
* transactions with the correct reservations.
*/
xchk_trans_cancel(sc);
error = xrep_nlinks_repair_inode(xnc);
xchk_iscan_mark_visited(&xnc->compare_iscan, sc->ip);
xchk_irele(sc, sc->ip);
sc->ip = NULL;
if (error)
break;
if (xchk_should_terminate(sc, &error))
break;
/*
* Create a new empty transaction so that we can advance the
* iscan cursor without deadlocking if the inobt has a cycle.
* We can only push the inactivation workqueues with an empty
* transaction.
*/
error = xchk_trans_alloc_empty(sc);
if (error)
break;
}
xchk_iscan_iter_finish(&xnc->compare_iscan);
xchk_iscan_teardown(&xnc->compare_iscan);
return error;
}

867
fs/xfs/scrub/quotacheck.c Normal file
View File

@ -0,0 +1,867 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2020-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_inode.h"
#include "xfs_quota.h"
#include "xfs_qm.h"
#include "xfs_icache.h"
#include "xfs_bmap_util.h"
#include "xfs_ialloc.h"
#include "xfs_ag.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/repair.h"
#include "scrub/xfile.h"
#include "scrub/xfarray.h"
#include "scrub/iscan.h"
#include "scrub/quota.h"
#include "scrub/quotacheck.h"
#include "scrub/trace.h"
/*
* Live Quotacheck
* ===============
*
* Quota counters are "summary" metadata, in the sense that they are computed
* as the summation of the block usage counts for every file on the filesystem.
* Therefore, we compute the correct icount, bcount, and rtbcount values by
* creating a shadow quota counter structure and walking every inode.
*/
/* Track the quota deltas for a dquot in a transaction. */
struct xqcheck_dqtrx {
xfs_dqtype_t q_type;
xfs_dqid_t q_id;
int64_t icount_delta;
int64_t bcount_delta;
int64_t delbcnt_delta;
int64_t rtbcount_delta;
int64_t delrtb_delta;
};
#define XQCHECK_MAX_NR_DQTRXS (XFS_QM_TRANS_DQTYPES * XFS_QM_TRANS_MAXDQS)
/*
* Track the quota deltas for all dquots attached to a transaction if the
* quota deltas are being applied to an inode that we already scanned.
*/
struct xqcheck_dqacct {
struct rhash_head hash;
uintptr_t tx_id;
struct xqcheck_dqtrx dqtrx[XQCHECK_MAX_NR_DQTRXS];
unsigned int refcount;
};
/* Free a shadow dquot accounting structure. */
static void
xqcheck_dqacct_free(
void *ptr,
void *arg)
{
struct xqcheck_dqacct *dqa = ptr;
kfree(dqa);
}
/* Set us up to scrub quota counters. */
int
xchk_setup_quotacheck(
struct xfs_scrub *sc)
{
if (!XFS_IS_QUOTA_ON(sc->mp))
return -ENOENT;
xchk_fsgates_enable(sc, XCHK_FSGATES_QUOTA);
sc->buf = kzalloc(sizeof(struct xqcheck), XCHK_GFP_FLAGS);
if (!sc->buf)
return -ENOMEM;
return xchk_setup_fs(sc);
}
/*
* Part 1: Collecting dquot resource usage counts. For each xfs_dquot attached
* to each inode, we create a shadow dquot, and compute the inode count and add
* the data/rt block usage from what we see.
*
* To avoid false corruption reports in part 2, any failure in this part must
* set the INCOMPLETE flag even when a negative errno is returned. This care
* must be taken with certain errno values (i.e. EFSBADCRC, EFSCORRUPTED,
* ECANCELED) that are absorbed into a scrub state flag update by
* xchk_*_process_error. Scrub and repair share the same incore data
* structures, so the INCOMPLETE flag is critical to prevent a repair based on
* insufficient information.
*
* Because we are scanning a live filesystem, it's possible that another thread
* will try to update the quota counters for an inode that we've already
* scanned. This will cause our counts to be incorrect. Therefore, we hook
* the live transaction code in two places: (1) when the callers update the
* per-transaction dqtrx structure to log quota counter updates; and (2) when
* transaction commit actually logs those updates to the incore dquot. By
* shadowing transaction updates in this manner, live quotacheck can ensure
* by locking the dquot and the shadow structure that its own copies are not
* out of date. Because the hook code runs in a different process context from
* the scrub code and the scrub state flags are not accessed atomically,
* failures in the hook code must abort the iscan and the scrubber must notice
* the aborted scan and set the incomplete flag.
*
* Note that we use srcu notifier hooks to minimize the overhead when live
* quotacheck is /not/ running.
*/
/* Update an incore dquot counter information from a live update. */
static int
xqcheck_update_incore_counts(
struct xqcheck *xqc,
struct xfarray *counts,
xfs_dqid_t id,
int64_t inodes,
int64_t nblks,
int64_t rtblks)
{
struct xqcheck_dquot xcdq;
int error;
error = xfarray_load_sparse(counts, id, &xcdq);
if (error)
return error;
xcdq.flags |= XQCHECK_DQUOT_WRITTEN;
xcdq.icount += inodes;
xcdq.bcount += nblks;
xcdq.rtbcount += rtblks;
error = xfarray_store(counts, id, &xcdq);
if (error == -EFBIG) {
/*
* EFBIG means we tried to store data at too high a byte offset
* in the sparse array. IOWs, we cannot complete the check and
* must notify userspace that the check was incomplete.
*/
error = -ECANCELED;
}
return error;
}
/* Decide if this is the shadow dquot accounting structure for a transaction. */
static int
xqcheck_dqacct_obj_cmpfn(
struct rhashtable_compare_arg *arg,
const void *obj)
{
const uintptr_t *tx_idp = arg->key;
const struct xqcheck_dqacct *dqa = obj;
if (dqa->tx_id != *tx_idp)
return 1;
return 0;
}
static const struct rhashtable_params xqcheck_dqacct_hash_params = {
.min_size = 32,
.key_len = sizeof(uintptr_t),
.key_offset = offsetof(struct xqcheck_dqacct, tx_id),
.head_offset = offsetof(struct xqcheck_dqacct, hash),
.automatic_shrinking = true,
.obj_cmpfn = xqcheck_dqacct_obj_cmpfn,
};
/* Find a shadow dqtrx slot for the given dquot. */
STATIC struct xqcheck_dqtrx *
xqcheck_get_dqtrx(
struct xqcheck_dqacct *dqa,
xfs_dqtype_t q_type,
xfs_dqid_t q_id)
{
int i;
for (i = 0; i < XQCHECK_MAX_NR_DQTRXS; i++) {
if (dqa->dqtrx[i].q_type == 0 ||
(dqa->dqtrx[i].q_type == q_type &&
dqa->dqtrx[i].q_id == q_id))
return &dqa->dqtrx[i];
}
return NULL;
}
/*
* Create and fill out a quota delta tracking structure to shadow the updates
* going on in the regular quota code.
*/
static int
xqcheck_mod_live_ino_dqtrx(
struct notifier_block *nb,
unsigned long action,
void *data)
{
struct xfs_mod_ino_dqtrx_params *p = data;
struct xqcheck *xqc;
struct xqcheck_dqacct *dqa;
struct xqcheck_dqtrx *dqtrx;
int error;
xqc = container_of(nb, struct xqcheck, qhook.mod_hook.nb);
/* Skip quota reservation fields. */
switch (action) {
case XFS_TRANS_DQ_BCOUNT:
case XFS_TRANS_DQ_DELBCOUNT:
case XFS_TRANS_DQ_ICOUNT:
case XFS_TRANS_DQ_RTBCOUNT:
case XFS_TRANS_DQ_DELRTBCOUNT:
break;
default:
return NOTIFY_DONE;
}
/* Ignore dqtrx updates for quota types we don't care about. */
switch (p->q_type) {
case XFS_DQTYPE_USER:
if (!xqc->ucounts)
return NOTIFY_DONE;
break;
case XFS_DQTYPE_GROUP:
if (!xqc->gcounts)
return NOTIFY_DONE;
break;
case XFS_DQTYPE_PROJ:
if (!xqc->pcounts)
return NOTIFY_DONE;
break;
default:
return NOTIFY_DONE;
}
/* Skip inodes that haven't been scanned yet. */
if (!xchk_iscan_want_live_update(&xqc->iscan, p->ino))
return NOTIFY_DONE;
/* Make a shadow quota accounting tracker for this transaction. */
mutex_lock(&xqc->lock);
dqa = rhashtable_lookup_fast(&xqc->shadow_dquot_acct, &p->tx_id,
xqcheck_dqacct_hash_params);
if (!dqa) {
dqa = kzalloc(sizeof(struct xqcheck_dqacct), XCHK_GFP_FLAGS);
if (!dqa)
goto out_abort;
dqa->tx_id = p->tx_id;
error = rhashtable_insert_fast(&xqc->shadow_dquot_acct,
&dqa->hash, xqcheck_dqacct_hash_params);
if (error)
goto out_abort;
}
/* Find the shadow dqtrx (or an empty slot) here. */
dqtrx = xqcheck_get_dqtrx(dqa, p->q_type, p->q_id);
if (!dqtrx)
goto out_abort;
if (dqtrx->q_type == 0) {
dqtrx->q_type = p->q_type;
dqtrx->q_id = p->q_id;
dqa->refcount++;
}
/* Update counter */
switch (action) {
case XFS_TRANS_DQ_BCOUNT:
dqtrx->bcount_delta += p->delta;
break;
case XFS_TRANS_DQ_DELBCOUNT:
dqtrx->delbcnt_delta += p->delta;
break;
case XFS_TRANS_DQ_ICOUNT:
dqtrx->icount_delta += p->delta;
break;
case XFS_TRANS_DQ_RTBCOUNT:
dqtrx->rtbcount_delta += p->delta;
break;
case XFS_TRANS_DQ_DELRTBCOUNT:
dqtrx->delrtb_delta += p->delta;
break;
}
mutex_unlock(&xqc->lock);
return NOTIFY_DONE;
out_abort:
xchk_iscan_abort(&xqc->iscan);
mutex_unlock(&xqc->lock);
return NOTIFY_DONE;
}
/*
* Apply the transaction quota deltas to our shadow quota accounting info when
* the regular quota code are doing the same.
*/
static int
xqcheck_apply_live_dqtrx(
struct notifier_block *nb,
unsigned long action,
void *data)
{
struct xfs_apply_dqtrx_params *p = data;
struct xqcheck *xqc;
struct xqcheck_dqacct *dqa;
struct xqcheck_dqtrx *dqtrx;
struct xfarray *counts;
int error;
xqc = container_of(nb, struct xqcheck, qhook.apply_hook.nb);
/* Map the dquot type to an incore counter object. */
switch (p->q_type) {
case XFS_DQTYPE_USER:
counts = xqc->ucounts;
break;
case XFS_DQTYPE_GROUP:
counts = xqc->gcounts;
break;
case XFS_DQTYPE_PROJ:
counts = xqc->pcounts;
break;
default:
return NOTIFY_DONE;
}
if (xchk_iscan_aborted(&xqc->iscan) || counts == NULL)
return NOTIFY_DONE;
/*
* Find the shadow dqtrx for this transaction and dquot, if any deltas
* need to be applied here. If not, we're finished early.
*/
mutex_lock(&xqc->lock);
dqa = rhashtable_lookup_fast(&xqc->shadow_dquot_acct, &p->tx_id,
xqcheck_dqacct_hash_params);
if (!dqa)
goto out_unlock;
dqtrx = xqcheck_get_dqtrx(dqa, p->q_type, p->q_id);
if (!dqtrx || dqtrx->q_type == 0)
goto out_unlock;
/* Update our shadow dquot if we're committing. */
if (action == XFS_APPLY_DQTRX_COMMIT) {
error = xqcheck_update_incore_counts(xqc, counts, p->q_id,
dqtrx->icount_delta,
dqtrx->bcount_delta + dqtrx->delbcnt_delta,
dqtrx->rtbcount_delta + dqtrx->delrtb_delta);
if (error)
goto out_abort;
}
/* Free the shadow accounting structure if that was the last user. */
dqa->refcount--;
if (dqa->refcount == 0) {
error = rhashtable_remove_fast(&xqc->shadow_dquot_acct,
&dqa->hash, xqcheck_dqacct_hash_params);
if (error)
goto out_abort;
xqcheck_dqacct_free(dqa, NULL);
}
mutex_unlock(&xqc->lock);
return NOTIFY_DONE;
out_abort:
xchk_iscan_abort(&xqc->iscan);
out_unlock:
mutex_unlock(&xqc->lock);
return NOTIFY_DONE;
}
/* Record this inode's quota usage in our shadow quota counter data. */
STATIC int
xqcheck_collect_inode(
struct xqcheck *xqc,
struct xfs_inode *ip)
{
struct xfs_trans *tp = xqc->sc->tp;
xfs_filblks_t nblks, rtblks;
uint ilock_flags = 0;
xfs_dqid_t id;
bool isreg = S_ISREG(VFS_I(ip)->i_mode);
int error = 0;
if (xfs_is_quota_inode(&tp->t_mountp->m_sb, ip->i_ino)) {
/*
* Quota files are never counted towards quota, so we do not
* need to take the lock.
*/
xchk_iscan_mark_visited(&xqc->iscan, ip);
return 0;
}
/* Figure out the data / rt device block counts. */
xfs_ilock(ip, XFS_IOLOCK_SHARED);
if (isreg)
xfs_ilock(ip, XFS_MMAPLOCK_SHARED);
if (XFS_IS_REALTIME_INODE(ip)) {
/*
* Read in the data fork for rt files so that _count_blocks
* can count the number of blocks allocated from the rt volume.
* Inodes do not track that separately.
*/
ilock_flags = xfs_ilock_data_map_shared(ip);
error = xfs_iread_extents(tp, ip, XFS_DATA_FORK);
if (error)
goto out_abort;
} else {
ilock_flags = XFS_ILOCK_SHARED;
xfs_ilock(ip, XFS_ILOCK_SHARED);
}
xfs_inode_count_blocks(tp, ip, &nblks, &rtblks);
if (xchk_iscan_aborted(&xqc->iscan)) {
error = -ECANCELED;
goto out_incomplete;
}
/* Update the shadow dquot counters. */
mutex_lock(&xqc->lock);
if (xqc->ucounts) {
id = xfs_qm_id_for_quotatype(ip, XFS_DQTYPE_USER);
error = xqcheck_update_incore_counts(xqc, xqc->ucounts, id, 1,
nblks, rtblks);
if (error)
goto out_mutex;
}
if (xqc->gcounts) {
id = xfs_qm_id_for_quotatype(ip, XFS_DQTYPE_GROUP);
error = xqcheck_update_incore_counts(xqc, xqc->gcounts, id, 1,
nblks, rtblks);
if (error)
goto out_mutex;
}
if (xqc->pcounts) {
id = xfs_qm_id_for_quotatype(ip, XFS_DQTYPE_PROJ);
error = xqcheck_update_incore_counts(xqc, xqc->pcounts, id, 1,
nblks, rtblks);
if (error)
goto out_mutex;
}
mutex_unlock(&xqc->lock);
xchk_iscan_mark_visited(&xqc->iscan, ip);
goto out_ilock;
out_mutex:
mutex_unlock(&xqc->lock);
out_abort:
xchk_iscan_abort(&xqc->iscan);
out_incomplete:
xchk_set_incomplete(xqc->sc);
out_ilock:
xfs_iunlock(ip, ilock_flags);
if (isreg)
xfs_iunlock(ip, XFS_MMAPLOCK_SHARED);
xfs_iunlock(ip, XFS_IOLOCK_SHARED);
return error;
}
/* Walk all the allocated inodes and run a quota scan on them. */
STATIC int
xqcheck_collect_counts(
struct xqcheck *xqc)
{
struct xfs_scrub *sc = xqc->sc;
struct xfs_inode *ip;
int error;
/*
* Set up for a potentially lengthy filesystem scan by reducing our
* transaction resource usage for the duration. Specifically:
*
* Cancel the transaction to release the log grant space while we scan
* the filesystem.
*
* Create a new empty transaction to eliminate the possibility of the
* inode scan deadlocking on cyclical metadata.
*
* We pass the empty transaction to the file scanning function to avoid
* repeatedly cycling empty transactions. This can be done without
* risk of deadlock between sb_internal and the IOLOCK (we take the
* IOLOCK to quiesce the file before scanning) because empty
* transactions do not take sb_internal.
*/
xchk_trans_cancel(sc);
error = xchk_trans_alloc_empty(sc);
if (error)
return error;
while ((error = xchk_iscan_iter(&xqc->iscan, &ip)) == 1) {
error = xqcheck_collect_inode(xqc, ip);
xchk_irele(sc, ip);
if (error)
break;
if (xchk_should_terminate(sc, &error))
break;
}
xchk_iscan_iter_finish(&xqc->iscan);
if (error) {
xchk_set_incomplete(sc);
/*
* If we couldn't grab an inode that was busy with a state
* change, change the error code so that we exit to userspace
* as quickly as possible.
*/
if (error == -EBUSY)
return -ECANCELED;
return error;
}
/*
* Switch out for a real transaction in preparation for building a new
* tree.
*/
xchk_trans_cancel(sc);
return xchk_setup_fs(sc);
}
/*
* Part 2: Comparing dquot resource counters. Walk each xfs_dquot, comparing
* the resource usage counters against our shadow dquots; and then walk each
* shadow dquot (that wasn't covered in the first part), comparing it against
* the xfs_dquot.
*/
/*
* Check the dquot data against what we observed. Caller must hold the dquot
* lock.
*/
STATIC int
xqcheck_compare_dquot(
struct xqcheck *xqc,
xfs_dqtype_t dqtype,
struct xfs_dquot *dq)
{
struct xqcheck_dquot xcdq;
struct xfarray *counts = xqcheck_counters_for(xqc, dqtype);
int error;
if (xchk_iscan_aborted(&xqc->iscan)) {
xchk_set_incomplete(xqc->sc);
return -ECANCELED;
}
mutex_lock(&xqc->lock);
error = xfarray_load_sparse(counts, dq->q_id, &xcdq);
if (error)
goto out_unlock;
if (xcdq.icount != dq->q_ino.count)
xchk_qcheck_set_corrupt(xqc->sc, dqtype, dq->q_id);
if (xcdq.bcount != dq->q_blk.count)
xchk_qcheck_set_corrupt(xqc->sc, dqtype, dq->q_id);
if (xcdq.rtbcount != dq->q_rtb.count)
xchk_qcheck_set_corrupt(xqc->sc, dqtype, dq->q_id);
xcdq.flags |= (XQCHECK_DQUOT_COMPARE_SCANNED | XQCHECK_DQUOT_WRITTEN);
error = xfarray_store(counts, dq->q_id, &xcdq);
if (error == -EFBIG) {
/*
* EFBIG means we tried to store data at too high a byte offset
* in the sparse array. IOWs, we cannot complete the check and
* must notify userspace that the check was incomplete. This
* should never happen outside of the collection phase.
*/
xchk_set_incomplete(xqc->sc);
error = -ECANCELED;
}
mutex_unlock(&xqc->lock);
if (error)
return error;
if (xqc->sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
return -ECANCELED;
return 0;
out_unlock:
mutex_unlock(&xqc->lock);
return error;
}
/*
* Walk all the observed dquots, and make sure there's a matching incore
* dquot and that its counts match ours.
*/
STATIC int
xqcheck_walk_observations(
struct xqcheck *xqc,
xfs_dqtype_t dqtype)
{
struct xqcheck_dquot xcdq;
struct xfs_dquot *dq;
struct xfarray *counts = xqcheck_counters_for(xqc, dqtype);
xfarray_idx_t cur = XFARRAY_CURSOR_INIT;
int error;
mutex_lock(&xqc->lock);
while ((error = xfarray_iter(counts, &cur, &xcdq)) == 1) {
xfs_dqid_t id = cur - 1;
if (xcdq.flags & XQCHECK_DQUOT_COMPARE_SCANNED)
continue;
mutex_unlock(&xqc->lock);
error = xfs_qm_dqget(xqc->sc->mp, id, dqtype, false, &dq);
if (error == -ENOENT) {
xchk_qcheck_set_corrupt(xqc->sc, dqtype, id);
return 0;
}
if (error)
return error;
error = xqcheck_compare_dquot(xqc, dqtype, dq);
xfs_qm_dqput(dq);
if (error)
return error;
if (xchk_should_terminate(xqc->sc, &error))
return error;
mutex_lock(&xqc->lock);
}
mutex_unlock(&xqc->lock);
return error;
}
/* Compare the quota counters we observed against the live dquots. */
STATIC int
xqcheck_compare_dqtype(
struct xqcheck *xqc,
xfs_dqtype_t dqtype)
{
struct xchk_dqiter cursor = { };
struct xfs_scrub *sc = xqc->sc;
struct xfs_dquot *dq;
int error;
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
return 0;
/* If the quota CHKD flag is cleared, we need to repair this quota. */
if (!(xfs_quota_chkd_flag(dqtype) & sc->mp->m_qflags)) {
xchk_qcheck_set_corrupt(xqc->sc, dqtype, 0);
return 0;
}
/* Compare what we observed against the actual dquots. */
xchk_dqiter_init(&cursor, sc, dqtype);
while ((error = xchk_dquot_iter(&cursor, &dq)) == 1) {
error = xqcheck_compare_dquot(xqc, dqtype, dq);
xfs_qm_dqput(dq);
if (error)
break;
}
if (error)
return error;
/* Walk all the observed dquots and compare to the incore ones. */
return xqcheck_walk_observations(xqc, dqtype);
}
/* Tear down everything associated with a quotacheck. */
static void
xqcheck_teardown_scan(
void *priv)
{
struct xqcheck *xqc = priv;
struct xfs_quotainfo *qi = xqc->sc->mp->m_quotainfo;
/* Discourage any hook functions that might be running. */
xchk_iscan_abort(&xqc->iscan);
/*
* As noted above, the apply hook is responsible for cleaning up the
* shadow dquot accounting data when a transaction completes. The mod
* hook must be removed before the apply hook so that we don't
* mistakenly leave an active shadow account for the mod hook to get
* its hands on. No hooks should be running after these functions
* return.
*/
xfs_dqtrx_hook_del(qi, &xqc->qhook);
if (xqc->shadow_dquot_acct.key_len) {
rhashtable_free_and_destroy(&xqc->shadow_dquot_acct,
xqcheck_dqacct_free, NULL);
xqc->shadow_dquot_acct.key_len = 0;
}
if (xqc->pcounts) {
xfarray_destroy(xqc->pcounts);
xqc->pcounts = NULL;
}
if (xqc->gcounts) {
xfarray_destroy(xqc->gcounts);
xqc->gcounts = NULL;
}
if (xqc->ucounts) {
xfarray_destroy(xqc->ucounts);
xqc->ucounts = NULL;
}
xchk_iscan_teardown(&xqc->iscan);
mutex_destroy(&xqc->lock);
xqc->sc = NULL;
}
/*
* Scan all inodes in the entire filesystem to generate quota counter data.
* If the scan is successful, the quota data will be left alive for a repair.
* If any error occurs, we'll tear everything down.
*/
STATIC int
xqcheck_setup_scan(
struct xfs_scrub *sc,
struct xqcheck *xqc)
{
char *descr;
struct xfs_quotainfo *qi = sc->mp->m_quotainfo;
unsigned long long max_dquots = XFS_DQ_ID_MAX + 1ULL;
int error;
ASSERT(xqc->sc == NULL);
xqc->sc = sc;
mutex_init(&xqc->lock);
/* Retry iget every tenth of a second for up to 30 seconds. */
xchk_iscan_start(sc, 30000, 100, &xqc->iscan);
error = -ENOMEM;
if (xfs_this_quota_on(sc->mp, XFS_DQTYPE_USER)) {
descr = xchk_xfile_descr(sc, "user dquot records");
error = xfarray_create(descr, max_dquots,
sizeof(struct xqcheck_dquot), &xqc->ucounts);
kfree(descr);
if (error)
goto out_teardown;
}
if (xfs_this_quota_on(sc->mp, XFS_DQTYPE_GROUP)) {
descr = xchk_xfile_descr(sc, "group dquot records");
error = xfarray_create(descr, max_dquots,
sizeof(struct xqcheck_dquot), &xqc->gcounts);
kfree(descr);
if (error)
goto out_teardown;
}
if (xfs_this_quota_on(sc->mp, XFS_DQTYPE_PROJ)) {
descr = xchk_xfile_descr(sc, "project dquot records");
error = xfarray_create(descr, max_dquots,
sizeof(struct xqcheck_dquot), &xqc->pcounts);
kfree(descr);
if (error)
goto out_teardown;
}
/*
* Set up hash table to map transactions to our internal shadow dqtrx
* structures.
*/
error = rhashtable_init(&xqc->shadow_dquot_acct,
&xqcheck_dqacct_hash_params);
if (error)
goto out_teardown;
/*
* Hook into the quota code. The hook only triggers for inodes that
* were already scanned, and the scanner thread takes each inode's
* ILOCK, which means that any in-progress inode updates will finish
* before we can scan the inode.
*
* The apply hook (which removes the shadow dquot accounting struct)
* must be installed before the mod hook so that we never fail to catch
* the end of a quota update sequence and leave stale shadow data.
*/
ASSERT(sc->flags & XCHK_FSGATES_QUOTA);
xfs_dqtrx_hook_setup(&xqc->qhook, xqcheck_mod_live_ino_dqtrx,
xqcheck_apply_live_dqtrx);
error = xfs_dqtrx_hook_add(qi, &xqc->qhook);
if (error)
goto out_teardown;
/* Use deferred cleanup to pass the quota count data to repair. */
sc->buf_cleanup = xqcheck_teardown_scan;
return 0;
out_teardown:
xqcheck_teardown_scan(xqc);
return error;
}
/* Scrub all counters for a given quota type. */
int
xchk_quotacheck(
struct xfs_scrub *sc)
{
struct xqcheck *xqc = sc->buf;
int error = 0;
/* Check quota counters on the live filesystem. */
error = xqcheck_setup_scan(sc, xqc);
if (error)
return error;
/* Walk all inodes, picking up quota information. */
error = xqcheck_collect_counts(xqc);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
/* Fail fast if we're not playing with a full dataset. */
if (xchk_iscan_aborted(&xqc->iscan))
xchk_set_incomplete(sc);
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE)
return 0;
/* Compare quota counters. */
if (xqc->ucounts) {
error = xqcheck_compare_dqtype(xqc, XFS_DQTYPE_USER);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
}
if (xqc->gcounts) {
error = xqcheck_compare_dqtype(xqc, XFS_DQTYPE_GROUP);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
}
if (xqc->pcounts) {
error = xqcheck_compare_dqtype(xqc, XFS_DQTYPE_PROJ);
if (!xchk_xref_process_error(sc, 0, 0, &error))
return error;
}
/* Check one last time for an incomplete dataset. */
if (xchk_iscan_aborted(&xqc->iscan))
xchk_set_incomplete(sc);
return 0;
}

76
fs/xfs/scrub/quotacheck.h Normal file
View File

@ -0,0 +1,76 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2020-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_QUOTACHECK_H__
#define __XFS_SCRUB_QUOTACHECK_H__
/* Quota counters for live quotacheck. */
struct xqcheck_dquot {
/* block usage count */
int64_t bcount;
/* inode usage count */
int64_t icount;
/* realtime block usage count */
int64_t rtbcount;
/* Record state */
unsigned int flags;
};
/*
* This incore dquot record has been written at least once. We never want to
* store an xqcheck_dquot that looks uninitialized.
*/
#define XQCHECK_DQUOT_WRITTEN (1U << 0)
/* Already checked this dquot. */
#define XQCHECK_DQUOT_COMPARE_SCANNED (1U << 1)
/* Already repaired this dquot. */
#define XQCHECK_DQUOT_REPAIR_SCANNED (1U << 2)
/* Live quotacheck control structure. */
struct xqcheck {
struct xfs_scrub *sc;
/* Shadow dquot counter data. */
struct xfarray *ucounts;
struct xfarray *gcounts;
struct xfarray *pcounts;
/* Lock protecting quotacheck count observations */
struct mutex lock;
struct xchk_iscan iscan;
/* Hooks into the quota code. */
struct xfs_dqtrx_hook qhook;
/* Shadow quota delta tracking structure. */
struct rhashtable shadow_dquot_acct;
};
/* Return the incore counter array for a given quota type. */
static inline struct xfarray *
xqcheck_counters_for(
struct xqcheck *xqc,
xfs_dqtype_t dqtype)
{
switch (dqtype) {
case XFS_DQTYPE_USER:
return xqc->ucounts;
case XFS_DQTYPE_GROUP:
return xqc->gcounts;
case XFS_DQTYPE_PROJ:
return xqc->pcounts;
}
ASSERT(0);
return NULL;
}
#endif /* __XFS_SCRUB_QUOTACHECK_H__ */

261
fs/xfs/scrub/quotacheck_repair.c Normal file
View File

@ -0,0 +1,261 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2020-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_inode.h"
#include "xfs_quota.h"
#include "xfs_qm.h"
#include "xfs_icache.h"
#include "xfs_bmap_util.h"
#include "xfs_iwalk.h"
#include "xfs_ialloc.h"
#include "xfs_sb.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/repair.h"
#include "scrub/xfile.h"
#include "scrub/xfarray.h"
#include "scrub/iscan.h"
#include "scrub/quota.h"
#include "scrub/quotacheck.h"
#include "scrub/trace.h"
/*
* Live Quotacheck Repair
* ======================
*
* Use the live quota counter information that we collected to replace the
* counter values in the incore dquots. A scrub->repair cycle should have left
* the live data and hooks active, so this is safe so long as we make sure the
* dquot is locked.
*/
/* Commit new counters to a dquot. */
static int
xqcheck_commit_dquot(
struct xqcheck *xqc,
xfs_dqtype_t dqtype,
struct xfs_dquot *dq)
{
struct xqcheck_dquot xcdq;
struct xfarray *counts = xqcheck_counters_for(xqc, dqtype);
int64_t delta;
bool dirty = false;
int error = 0;
/* Unlock the dquot just long enough to allocate a transaction. */
xfs_dqunlock(dq);
error = xchk_trans_alloc(xqc->sc, 0);
xfs_dqlock(dq);
if (error)
return error;
xfs_trans_dqjoin(xqc->sc->tp, dq);
if (xchk_iscan_aborted(&xqc->iscan)) {
error = -ECANCELED;
goto out_cancel;
}
mutex_lock(&xqc->lock);
error = xfarray_load_sparse(counts, dq->q_id, &xcdq);
if (error)
goto out_unlock;
/* Adjust counters as needed. */
delta = (int64_t)xcdq.icount - dq->q_ino.count;
if (delta) {
dq->q_ino.reserved += delta;
dq->q_ino.count += delta;
dirty = true;
}
delta = (int64_t)xcdq.bcount - dq->q_blk.count;
if (delta) {
dq->q_blk.reserved += delta;
dq->q_blk.count += delta;
dirty = true;
}
delta = (int64_t)xcdq.rtbcount - dq->q_rtb.count;
if (delta) {
dq->q_rtb.reserved += delta;
dq->q_rtb.count += delta;
dirty = true;
}
xcdq.flags |= (XQCHECK_DQUOT_REPAIR_SCANNED | XQCHECK_DQUOT_WRITTEN);
error = xfarray_store(counts, dq->q_id, &xcdq);
if (error == -EFBIG) {
/*
* EFBIG means we tried to store data at too high a byte offset
* in the sparse array. IOWs, we cannot complete the repair
* and must cancel the whole operation. This should never
* happen, but we need to catch it anyway.
*/
error = -ECANCELED;
}
mutex_unlock(&xqc->lock);
if (error || !dirty)
goto out_cancel;
trace_xrep_quotacheck_dquot(xqc->sc->mp, dq->q_type, dq->q_id);
/* Commit the dirty dquot to disk. */
dq->q_flags |= XFS_DQFLAG_DIRTY;
if (dq->q_id)
xfs_qm_adjust_dqtimers(dq);
xfs_trans_log_dquot(xqc->sc->tp, dq);
/*
* Transaction commit unlocks the dquot, so we must re-lock it so that
* the caller can put the reference (which apparently requires a locked
* dquot).
*/
error = xrep_trans_commit(xqc->sc);
xfs_dqlock(dq);
return error;
out_unlock:
mutex_unlock(&xqc->lock);
out_cancel:
xchk_trans_cancel(xqc->sc);
/* Re-lock the dquot so the caller can put the reference. */
xfs_dqlock(dq);
return error;
}
/* Commit new quota counters for a particular quota type. */
STATIC int
xqcheck_commit_dqtype(
struct xqcheck *xqc,
unsigned int dqtype)
{
struct xchk_dqiter cursor = { };
struct xqcheck_dquot xcdq;
struct xfs_scrub *sc = xqc->sc;
struct xfs_mount *mp = sc->mp;
struct xfarray *counts = xqcheck_counters_for(xqc, dqtype);
struct xfs_dquot *dq;
xfarray_idx_t cur = XFARRAY_CURSOR_INIT;
int error;
/*
* Update the counters of every dquot that the quota file knows about.
*/
xchk_dqiter_init(&cursor, sc, dqtype);
while ((error = xchk_dquot_iter(&cursor, &dq)) == 1) {
error = xqcheck_commit_dquot(xqc, dqtype, dq);
xfs_qm_dqput(dq);
if (error)
break;
}
if (error)
return error;
/*
* Make a second pass to deal with the dquots that we know about but
* the quota file previously did not know about.
*/
mutex_lock(&xqc->lock);
while ((error = xfarray_iter(counts, &cur, &xcdq)) == 1) {
xfs_dqid_t id = cur - 1;
if (xcdq.flags & XQCHECK_DQUOT_REPAIR_SCANNED)
continue;
mutex_unlock(&xqc->lock);
/*
* Grab the dquot, allowing for dquot block allocation in a
* separate transaction. We committed the scrub transaction
* in a previous step, so we will not be creating nested
* transactions here.
*/
error = xfs_qm_dqget(mp, id, dqtype, true, &dq);
if (error)
return error;
error = xqcheck_commit_dquot(xqc, dqtype, dq);
xfs_qm_dqput(dq);
if (error)
return error;
mutex_lock(&xqc->lock);
}
mutex_unlock(&xqc->lock);
return error;
}
/* Figure out quota CHKD flags for the running quota types. */
static inline unsigned int
xqcheck_chkd_flags(
struct xfs_mount *mp)
{
unsigned int ret = 0;
if (XFS_IS_UQUOTA_ON(mp))
ret |= XFS_UQUOTA_CHKD;
if (XFS_IS_GQUOTA_ON(mp))
ret |= XFS_GQUOTA_CHKD;
if (XFS_IS_PQUOTA_ON(mp))
ret |= XFS_PQUOTA_CHKD;
return ret;
}
/* Commit the new dquot counters. */
int
xrep_quotacheck(
struct xfs_scrub *sc)
{
struct xqcheck *xqc = sc->buf;
unsigned int qflags = xqcheck_chkd_flags(sc->mp);
int error;
/*
* Clear the CHKD flag for the running quota types and commit the scrub
* transaction so that we can allocate new quota block mappings if we
* have to. If we crash after this point, the sb still has the CHKD
* flags cleared, so mount quotacheck will fix all of this up.
*/
xrep_update_qflags(sc, qflags, 0);
error = xrep_trans_commit(sc);
if (error)
return error;
/* Commit the new counters to the dquots. */
if (xqc->ucounts) {
error = xqcheck_commit_dqtype(xqc, XFS_DQTYPE_USER);
if (error)
return error;
}
if (xqc->gcounts) {
error = xqcheck_commit_dqtype(xqc, XFS_DQTYPE_GROUP);
if (error)
return error;
}
if (xqc->pcounts) {
error = xqcheck_commit_dqtype(xqc, XFS_DQTYPE_PROJ);
if (error)
return error;
}
/* Set the CHKD flags now that we've fixed quota counts. */
error = xchk_trans_alloc(sc, 0);
if (error)
return error;
xrep_update_qflags(sc, 0, qflags);
return xrep_trans_commit(sc);
}

307
fs/xfs/scrub/rcbag.c Normal file
View File

@ -0,0 +1,307 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2022-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_log_format.h"
#include "xfs_trans.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_defer.h"
#include "xfs_btree.h"
#include "xfs_buf_mem.h"
#include "xfs_btree_mem.h"
#include "xfs_error.h"
#include "scrub/scrub.h"
#include "scrub/rcbag_btree.h"
#include "scrub/rcbag.h"
#include "scrub/trace.h"
struct rcbag {
struct xfs_mount *mp;
struct xfbtree xfbtree;
uint64_t nr_items;
};
int
rcbag_init(
struct xfs_mount *mp,
struct xfs_buftarg *btp,
struct rcbag **bagp)
{
struct rcbag *bag;
int error;
bag = kzalloc(sizeof(struct rcbag), XCHK_GFP_FLAGS);
if (!bag)
return -ENOMEM;
bag->nr_items = 0;
bag->mp = mp;
error = rcbagbt_mem_init(mp, &bag->xfbtree, btp);
if (error)
goto out_bag;
*bagp = bag;
return 0;
out_bag:
kfree(bag);
return error;
}
void
rcbag_free(
struct rcbag **bagp)
{
struct rcbag *bag = *bagp;
xfbtree_destroy(&bag->xfbtree);
kfree(bag);
*bagp = NULL;
}
/* Track an rmap in the refcount bag. */
int
rcbag_add(
struct rcbag *bag,
struct xfs_trans *tp,
const struct xfs_rmap_irec *rmap)
{
struct rcbag_rec bagrec;
struct xfs_mount *mp = bag->mp;
struct xfs_btree_cur *cur;
int has;
int error;
cur = rcbagbt_mem_cursor(mp, tp, &bag->xfbtree);
error = rcbagbt_lookup_eq(cur, rmap, &has);
if (error)
goto out_cur;
if (has) {
error = rcbagbt_get_rec(cur, &bagrec, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
bagrec.rbg_refcount++;
error = rcbagbt_update(cur, &bagrec);
if (error)
goto out_cur;
} else {
bagrec.rbg_startblock = rmap->rm_startblock;
bagrec.rbg_blockcount = rmap->rm_blockcount;
bagrec.rbg_refcount = 1;
error = rcbagbt_insert(cur, &bagrec, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
}
xfs_btree_del_cursor(cur, 0);
error = xfbtree_trans_commit(&bag->xfbtree, tp);
if (error)
return error;
bag->nr_items++;
return 0;
out_cur:
xfs_btree_del_cursor(cur, error);
xfbtree_trans_cancel(&bag->xfbtree, tp);
return error;
}
/* Return the number of records in the bag. */
uint64_t
rcbag_count(
const struct rcbag *rcbag)
{
return rcbag->nr_items;
}
static inline uint32_t rcbag_rec_next_bno(const struct rcbag_rec *r)
{
return r->rbg_startblock + r->rbg_blockcount;
}
/*
* Find the next block where the refcount changes, given the next rmap we
* looked at and the ones we're already tracking.
*/
int
rcbag_next_edge(
struct rcbag *bag,
struct xfs_trans *tp,
const struct xfs_rmap_irec *next_rmap,
bool next_valid,
uint32_t *next_bnop)
{
struct rcbag_rec bagrec;
struct xfs_mount *mp = bag->mp;
struct xfs_btree_cur *cur;
uint32_t next_bno = NULLAGBLOCK;
int has;
int error;
if (next_valid)
next_bno = next_rmap->rm_startblock;
cur = rcbagbt_mem_cursor(mp, tp, &bag->xfbtree);
error = xfs_btree_goto_left_edge(cur);
if (error)
goto out_cur;
while (true) {
error = xfs_btree_increment(cur, 0, &has);
if (error)
goto out_cur;
if (!has)
break;
error = rcbagbt_get_rec(cur, &bagrec, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
next_bno = min(next_bno, rcbag_rec_next_bno(&bagrec));
}
/*
* We should have found /something/ because either next_rrm is the next
* interesting rmap to look at after emitting this refcount extent, or
* there are other rmaps in rmap_bag contributing to the current
* sharing count. But if something is seriously wrong, bail out.
*/
if (next_bno == NULLAGBLOCK) {
error = -EFSCORRUPTED;
goto out_cur;
}
xfs_btree_del_cursor(cur, 0);
*next_bnop = next_bno;
return 0;
out_cur:
xfs_btree_del_cursor(cur, error);
return error;
}
/* Pop all refcount bag records that end at next_bno */
int
rcbag_remove_ending_at(
struct rcbag *bag,
struct xfs_trans *tp,
uint32_t next_bno)
{
struct rcbag_rec bagrec;
struct xfs_mount *mp = bag->mp;
struct xfs_btree_cur *cur;
int has;
int error;
/* go to the right edge of the tree */
cur = rcbagbt_mem_cursor(mp, tp, &bag->xfbtree);
memset(&cur->bc_rec, 0xFF, sizeof(cur->bc_rec));
error = xfs_btree_lookup(cur, XFS_LOOKUP_GE, &has);
if (error)
goto out_cur;
while (true) {
error = xfs_btree_decrement(cur, 0, &has);
if (error)
goto out_cur;
if (!has)
break;
error = rcbagbt_get_rec(cur, &bagrec, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
if (rcbag_rec_next_bno(&bagrec) != next_bno)
continue;
error = xfs_btree_delete(cur, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
bag->nr_items -= bagrec.rbg_refcount;
}
xfs_btree_del_cursor(cur, 0);
return xfbtree_trans_commit(&bag->xfbtree, tp);
out_cur:
xfs_btree_del_cursor(cur, error);
xfbtree_trans_cancel(&bag->xfbtree, tp);
return error;
}
/* Dump the rcbag. */
void
rcbag_dump(
struct rcbag *bag,
struct xfs_trans *tp)
{
struct rcbag_rec bagrec;
struct xfs_mount *mp = bag->mp;
struct xfs_btree_cur *cur;
unsigned long long nr = 0;
int has;
int error;
cur = rcbagbt_mem_cursor(mp, tp, &bag->xfbtree);
error = xfs_btree_goto_left_edge(cur);
if (error)
goto out_cur;
while (true) {
error = xfs_btree_increment(cur, 0, &has);
if (error)
goto out_cur;
if (!has)
break;
error = rcbagbt_get_rec(cur, &bagrec, &has);
if (error)
goto out_cur;
if (!has) {
error = -EFSCORRUPTED;
goto out_cur;
}
xfs_err(bag->mp, "[%llu]: bno 0x%x fsbcount 0x%x refcount 0x%llx\n",
nr++,
(unsigned int)bagrec.rbg_startblock,
(unsigned int)bagrec.rbg_blockcount,
(unsigned long long)bagrec.rbg_refcount);
}
out_cur:
xfs_btree_del_cursor(cur, error);
}

28
fs/xfs/scrub/rcbag.h Normal file
View File

@ -0,0 +1,28 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2022-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_RCBAG_H__
#define __XFS_SCRUB_RCBAG_H__
struct xfs_mount;
struct rcbag;
struct xfs_buftarg;
int rcbag_init(struct xfs_mount *mp, struct xfs_buftarg *btp,
struct rcbag **bagp);
void rcbag_free(struct rcbag **bagp);
int rcbag_add(struct rcbag *bag, struct xfs_trans *tp,
const struct xfs_rmap_irec *rmap);
uint64_t rcbag_count(const struct rcbag *bag);
int rcbag_next_edge(struct rcbag *bag, struct xfs_trans *tp,
const struct xfs_rmap_irec *next_rmap, bool next_valid,
uint32_t *next_bnop);
int rcbag_remove_ending_at(struct rcbag *bag, struct xfs_trans *tp,
uint32_t next_bno);
void rcbag_dump(struct rcbag *bag, struct xfs_trans *tp);
#endif /* __XFS_SCRUB_RCBAG_H__ */

370
fs/xfs/scrub/rcbag_btree.c Normal file
View File

@ -0,0 +1,370 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2022-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#include "xfs.h"
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_defer.h"
#include "xfs_btree.h"
#include "xfs_buf_mem.h"
#include "xfs_btree_mem.h"
#include "xfs_error.h"
#include "scrub/rcbag_btree.h"
#include "scrub/trace.h"
static struct kmem_cache *rcbagbt_cur_cache;
STATIC void
rcbagbt_init_key_from_rec(
union xfs_btree_key *key,
const union xfs_btree_rec *rec)
{
struct rcbag_key *bag_key = (struct rcbag_key *)key;
const struct rcbag_rec *bag_rec = (const struct rcbag_rec *)rec;
BUILD_BUG_ON(sizeof(struct rcbag_key) > sizeof(union xfs_btree_key));
BUILD_BUG_ON(sizeof(struct rcbag_rec) > sizeof(union xfs_btree_rec));
bag_key->rbg_startblock = bag_rec->rbg_startblock;
bag_key->rbg_blockcount = bag_rec->rbg_blockcount;
}
STATIC void
rcbagbt_init_rec_from_cur(
struct xfs_btree_cur *cur,
union xfs_btree_rec *rec)
{
struct rcbag_rec *bag_rec = (struct rcbag_rec *)rec;
struct rcbag_rec *bag_irec = (struct rcbag_rec *)&cur->bc_rec;
bag_rec->rbg_startblock = bag_irec->rbg_startblock;
bag_rec->rbg_blockcount = bag_irec->rbg_blockcount;
bag_rec->rbg_refcount = bag_irec->rbg_refcount;
}
STATIC int64_t
rcbagbt_key_diff(
struct xfs_btree_cur *cur,
const union xfs_btree_key *key)
{
struct rcbag_rec *rec = (struct rcbag_rec *)&cur->bc_rec;
const struct rcbag_key *kp = (const struct rcbag_key *)key;
if (kp->rbg_startblock > rec->rbg_startblock)
return 1;
if (kp->rbg_startblock < rec->rbg_startblock)
return -1;
if (kp->rbg_blockcount > rec->rbg_blockcount)
return 1;
if (kp->rbg_blockcount < rec->rbg_blockcount)
return -1;
return 0;
}
STATIC int64_t
rcbagbt_diff_two_keys(
struct xfs_btree_cur *cur,
const union xfs_btree_key *k1,
const union xfs_btree_key *k2,
const union xfs_btree_key *mask)
{
const struct rcbag_key *kp1 = (const struct rcbag_key *)k1;
const struct rcbag_key *kp2 = (const struct rcbag_key *)k2;
ASSERT(mask == NULL);
if (kp1->rbg_startblock > kp2->rbg_startblock)
return 1;
if (kp1->rbg_startblock < kp2->rbg_startblock)
return -1;
if (kp1->rbg_blockcount > kp2->rbg_blockcount)
return 1;
if (kp1->rbg_blockcount < kp2->rbg_blockcount)
return -1;
return 0;
}
STATIC int
rcbagbt_keys_inorder(
struct xfs_btree_cur *cur,
const union xfs_btree_key *k1,
const union xfs_btree_key *k2)
{
const struct rcbag_key *kp1 = (const struct rcbag_key *)k1;
const struct rcbag_key *kp2 = (const struct rcbag_key *)k2;
if (kp1->rbg_startblock > kp2->rbg_startblock)
return 0;
if (kp1->rbg_startblock < kp2->rbg_startblock)
return 1;
if (kp1->rbg_blockcount > kp2->rbg_blockcount)
return 0;
if (kp1->rbg_blockcount < kp2->rbg_blockcount)
return 1;
return 0;
}
STATIC int
rcbagbt_recs_inorder(
struct xfs_btree_cur *cur,
const union xfs_btree_rec *r1,
const union xfs_btree_rec *r2)
{
const struct rcbag_rec *rp1 = (const struct rcbag_rec *)r1;
const struct rcbag_rec *rp2 = (const struct rcbag_rec *)r2;
if (rp1->rbg_startblock > rp2->rbg_startblock)
return 0;
if (rp1->rbg_startblock < rp2->rbg_startblock)
return 1;
if (rp1->rbg_blockcount > rp2->rbg_blockcount)
return 0;
if (rp1->rbg_blockcount < rp2->rbg_blockcount)
return 1;
return 0;
}
static xfs_failaddr_t
rcbagbt_verify(
struct xfs_buf *bp)
{
struct xfs_mount *mp = bp->b_mount;
struct xfs_btree_block *block = XFS_BUF_TO_BLOCK(bp);
xfs_failaddr_t fa;
unsigned int level;
unsigned int maxrecs;
if (!xfs_verify_magic(bp, block->bb_magic))
return __this_address;
fa = xfs_btree_fsblock_v5hdr_verify(bp, XFS_RMAP_OWN_UNKNOWN);
if (fa)
return fa;
level = be16_to_cpu(block->bb_level);
if (level >= rcbagbt_maxlevels_possible())
return __this_address;
maxrecs = rcbagbt_maxrecs(mp, XFBNO_BLOCKSIZE, level == 0);
return xfs_btree_memblock_verify(bp, maxrecs);
}
static void
rcbagbt_rw_verify(
struct xfs_buf *bp)
{
xfs_failaddr_t fa = rcbagbt_verify(bp);
if (fa)
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
}
/* skip crc checks on in-memory btrees to save time */
static const struct xfs_buf_ops rcbagbt_mem_buf_ops = {
.name = "rcbagbt_mem",
.magic = { 0, cpu_to_be32(RCBAG_MAGIC) },
.verify_read = rcbagbt_rw_verify,
.verify_write = rcbagbt_rw_verify,
.verify_struct = rcbagbt_verify,
};
static const struct xfs_btree_ops rcbagbt_mem_ops = {
.name = "rcbag",
.type = XFS_BTREE_TYPE_MEM,
.rec_len = sizeof(struct rcbag_rec),
.key_len = sizeof(struct rcbag_key),
.ptr_len = XFS_BTREE_LONG_PTR_LEN,
.lru_refs = 1,
.statoff = XFS_STATS_CALC_INDEX(xs_rcbag_2),
.dup_cursor = xfbtree_dup_cursor,
.set_root = xfbtree_set_root,
.alloc_block = xfbtree_alloc_block,
.free_block = xfbtree_free_block,
.get_minrecs = xfbtree_get_minrecs,
.get_maxrecs = xfbtree_get_maxrecs,
.init_key_from_rec = rcbagbt_init_key_from_rec,
.init_rec_from_cur = rcbagbt_init_rec_from_cur,
.init_ptr_from_cur = xfbtree_init_ptr_from_cur,
.key_diff = rcbagbt_key_diff,
.buf_ops = &rcbagbt_mem_buf_ops,
.diff_two_keys = rcbagbt_diff_two_keys,
.keys_inorder = rcbagbt_keys_inorder,
.recs_inorder = rcbagbt_recs_inorder,
};
/* Create a cursor for an in-memory btree. */
struct xfs_btree_cur *
rcbagbt_mem_cursor(
struct xfs_mount *mp,
struct xfs_trans *tp,
struct xfbtree *xfbtree)
{
struct xfs_btree_cur *cur;
cur = xfs_btree_alloc_cursor(mp, tp, &rcbagbt_mem_ops,
rcbagbt_maxlevels_possible(), rcbagbt_cur_cache);
cur->bc_mem.xfbtree = xfbtree;
cur->bc_nlevels = xfbtree->nlevels;
return cur;
}
/* Create an in-memory refcount bag btree. */
int
rcbagbt_mem_init(
struct xfs_mount *mp,
struct xfbtree *xfbt,
struct xfs_buftarg *btp)
{
xfbt->owner = 0;
return xfbtree_init(mp, xfbt, btp, &rcbagbt_mem_ops);
}
/* Calculate number of records in a refcount bag btree block. */
static inline unsigned int
rcbagbt_block_maxrecs(
unsigned int blocklen,
bool leaf)
{
if (leaf)
return blocklen / sizeof(struct rcbag_rec);
return blocklen /
(sizeof(struct rcbag_key) + sizeof(rcbag_ptr_t));
}
/*
* Calculate number of records in an refcount bag btree block.
*/
unsigned int
rcbagbt_maxrecs(
struct xfs_mount *mp,
unsigned int blocklen,
bool leaf)
{
blocklen -= RCBAG_BLOCK_LEN;
return rcbagbt_block_maxrecs(blocklen, leaf);
}
/* Compute the max possible height for refcount bag btrees. */
unsigned int
rcbagbt_maxlevels_possible(void)
{
unsigned int minrecs[2];
unsigned int blocklen;
blocklen = XFBNO_BLOCKSIZE - XFS_BTREE_LBLOCK_CRC_LEN;
minrecs[0] = rcbagbt_block_maxrecs(blocklen, true) / 2;
minrecs[1] = rcbagbt_block_maxrecs(blocklen, false) / 2;
return xfs_btree_space_to_height(minrecs, ULLONG_MAX);
}
/* Calculate the refcount bag btree size for some records. */
unsigned long long
rcbagbt_calc_size(
unsigned long long nr_records)
{
unsigned int minrecs[2];
unsigned int blocklen;
blocklen = XFBNO_BLOCKSIZE - XFS_BTREE_LBLOCK_CRC_LEN;
minrecs[0] = rcbagbt_block_maxrecs(blocklen, true) / 2;
minrecs[1] = rcbagbt_block_maxrecs(blocklen, false) / 2;
return xfs_btree_calc_size(minrecs, nr_records);
}
int __init
rcbagbt_init_cur_cache(void)
{
rcbagbt_cur_cache = kmem_cache_create("xfs_rcbagbt_cur",
xfs_btree_cur_sizeof(rcbagbt_maxlevels_possible()),
0, 0, NULL);
if (!rcbagbt_cur_cache)
return -ENOMEM;
return 0;
}
void
rcbagbt_destroy_cur_cache(void)
{
kmem_cache_destroy(rcbagbt_cur_cache);
rcbagbt_cur_cache = NULL;
}
/* Look up the refcount bag record corresponding to this reverse mapping. */
int
rcbagbt_lookup_eq(
struct xfs_btree_cur *cur,
const struct xfs_rmap_irec *rmap,
int *success)
{
struct rcbag_rec *rec = (struct rcbag_rec *)&cur->bc_rec;
rec->rbg_startblock = rmap->rm_startblock;
rec->rbg_blockcount = rmap->rm_blockcount;
return xfs_btree_lookup(cur, XFS_LOOKUP_EQ, success);
}
/* Get the data from the pointed-to record. */
int
rcbagbt_get_rec(
struct xfs_btree_cur *cur,
struct rcbag_rec *rec,
int *has)
{
union xfs_btree_rec *btrec;
int error;
error = xfs_btree_get_rec(cur, &btrec, has);
if (error || !(*has))
return error;
memcpy(rec, btrec, sizeof(struct rcbag_rec));
return 0;
}
/* Update the record referred to by cur to the value given. */
int
rcbagbt_update(
struct xfs_btree_cur *cur,
const struct rcbag_rec *rec)
{
union xfs_btree_rec btrec;
memcpy(&btrec, rec, sizeof(struct rcbag_rec));
return xfs_btree_update(cur, &btrec);
}
/* Update the record referred to by cur to the value given. */
int
rcbagbt_insert(
struct xfs_btree_cur *cur,
const struct rcbag_rec *rec,
int *success)
{
struct rcbag_rec *btrec = (struct rcbag_rec *)&cur->bc_rec;
memcpy(btrec, rec, sizeof(struct rcbag_rec));
return xfs_btree_insert(cur, success);
}

81
fs/xfs/scrub/rcbag_btree.h Normal file
View File

@ -0,0 +1,81 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2022-2024 Oracle. All Rights Reserved.
* Author: Darrick J. Wong <djwong@kernel.org>
*/
#ifndef __XFS_SCRUB_RCBAG_BTREE_H__
#define __XFS_SCRUB_RCBAG_BTREE_H__
#ifdef CONFIG_XFS_BTREE_IN_MEM
struct xfs_buf;
struct xfs_btree_cur;
struct xfs_mount;
#define RCBAG_MAGIC 0x74826671 /* 'JRBG' */
struct rcbag_key {
uint32_t rbg_startblock;
uint32_t rbg_blockcount;
};
struct rcbag_rec {
uint32_t rbg_startblock;
uint32_t rbg_blockcount;
uint64_t rbg_refcount;
};
typedef __be64 rcbag_ptr_t;
/* reflinks only exist on crc enabled filesystems */
#define RCBAG_BLOCK_LEN XFS_BTREE_LBLOCK_CRC_LEN
/*
* Record, key, and pointer address macros for btree blocks.
*
* (note that some of these may appear unused, but they are used in userspace)
*/
#define RCBAG_REC_ADDR(block, index) \
((struct rcbag_rec *) \
((char *)(block) + RCBAG_BLOCK_LEN + \
(((index) - 1) * sizeof(struct rcbag_rec))))
#define RCBAG_KEY_ADDR(block, index) \
((struct rcbag_key *) \
((char *)(block) + RCBAG_BLOCK_LEN + \
((index) - 1) * sizeof(struct rcbag_key)))
#define RCBAG_PTR_ADDR(block, index, maxrecs) \
((rcbag_ptr_t *) \
((char *)(block) + RCBAG_BLOCK_LEN + \
(maxrecs) * sizeof(struct rcbag_key) + \
((index) - 1) * sizeof(rcbag_ptr_t)))
unsigned int rcbagbt_maxrecs(struct xfs_mount *mp, unsigned int blocklen,
bool leaf);
unsigned long long rcbagbt_calc_size(unsigned long long nr_records);
unsigned int rcbagbt_maxlevels_possible(void);
int __init rcbagbt_init_cur_cache(void);
void rcbagbt_destroy_cur_cache(void);
struct xfs_btree_cur *rcbagbt_mem_cursor(struct xfs_mount *mp,
struct xfs_trans *tp, struct xfbtree *xfbtree);
int rcbagbt_mem_init(struct xfs_mount *mp, struct xfbtree *xfbtree,
struct xfs_buftarg *btp);
int rcbagbt_lookup_eq(struct xfs_btree_cur *cur,
const struct xfs_rmap_irec *rmap, int *success);
int rcbagbt_get_rec(struct xfs_btree_cur *cur, struct rcbag_rec *rec, int *has);
int rcbagbt_update(struct xfs_btree_cur *cur, const struct rcbag_rec *rec);
int rcbagbt_insert(struct xfs_btree_cur *cur, const struct rcbag_rec *rec,
int *success);
#else
# define rcbagbt_init_cur_cache() 0
# define rcbagbt_destroy_cur_cache() ((void)0)
#endif /* CONFIG_XFS_BTREE_IN_MEM */
#endif /* __XFS_SCRUB_RCBAG_BTREE_H__ */

4
fs/xfs/scrub/readdir.c
View File

@ -281,7 +281,7 @@ xchk_dir_walk(
return -EIO;
ASSERT(S_ISDIR(VFS_I(dp)->i_mode));
ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL));
xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL);
if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL)
return xchk_dir_walk_sf(sc, dp, dirent_fn, priv);
@ -332,7 +332,7 @@ xchk_dir_lookup(
return -EIO;
ASSERT(S_ISDIR(VFS_I(dp)->i_mode));
ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL));
xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL);
if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) {
error = xfs_dir2_sf_lookup(&args);

2
fs/xfs/scrub/reap.c
View File

@ -114,7 +114,7 @@ xreap_put_freelist(
int error;
/* Make sure there's space on the freelist. */
error = xrep_fix_freelist(sc, true);
error = xrep_fix_freelist(sc, 0);
if (error)
return error;

12
fs/xfs/scrub/refcount.c
View File

@ -7,8 +7,10 @@
#include "xfs_fs.h"
#include "xfs_shared.h"
#include "xfs_format.h"
#include "xfs_log_format.h"
#include "xfs_trans_resv.h"
#include "xfs_mount.h"
#include "xfs_trans.h"
#include "xfs_ag.h"
#include "xfs_btree.h"
#include "xfs_rmap.h"
@ -17,6 +19,7 @@
#include "scrub/common.h"
#include "scrub/btree.h"
#include "scrub/trace.h"
#include "scrub/repair.h"
/*
* Set us up to scrub reference count btrees.
@ -27,6 +30,15 @@ xchk_setup_ag_refcountbt(
{
if (xchk_need_intent_drain(sc))
xchk_fsgates_enable(sc, XCHK_FSGATES_DRAIN);
if (xchk_could_repair(sc)) {
int error;
error = xrep_setup_ag_refcountbt(sc);
if (error)
return error;
}
return xchk_setup_ag_btree(sc, false);
}

177
fs/xfs/scrub/refcount_repair.c
View File

@ -25,6 +25,7 @@
#include "xfs_refcount_btree.h"
#include "xfs_error.h"
#include "xfs_ag.h"
#include "xfs_health.h"
#include "scrub/xfs_scrub.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
@ -37,6 +38,7 @@
#include "scrub/xfarray.h"
#include "scrub/newbt.h"
#include "scrub/reap.h"
#include "scrub/rcbag.h"
/*
* Rebuilding the Reference Count Btree
@ -97,12 +99,6 @@
* insert all the records.
*/
/* The only parts of the rmap that we care about for computing refcounts. */
struct xrep_refc_rmap {
xfs_agblock_t startblock;
xfs_extlen_t blockcount;
} __packed;
struct xrep_refc {
/* refcount extents */
struct xfarray *refcount_records;
@ -122,6 +118,20 @@ struct xrep_refc {
xfs_extlen_t btblocks;
};
/* Set us up to repair refcount btrees. */
int
xrep_setup_ag_refcountbt(
struct xfs_scrub *sc)
{
char *descr;
int error;
descr = xchk_xfile_ag_descr(sc, "rmap record bag");
error = xrep_setup_xfbtree(sc, descr);
kfree(descr);
return error;
}
/* Check for any obvious conflicts with this shared/CoW staging extent. */
STATIC int
xrep_refc_check_ext(
@ -223,10 +233,9 @@ xrep_refc_rmap_shareable(
STATIC int
xrep_refc_walk_rmaps(
struct xrep_refc *rr,
struct xrep_refc_rmap *rrm,
struct xfs_rmap_irec *rmap,
bool *have_rec)
{
struct xfs_rmap_irec rmap;
struct xfs_btree_cur *cur = rr->sc->sa.rmap_cur;
struct xfs_mount *mp = cur->bc_mp;
int have_gt;
@ -250,29 +259,30 @@ xrep_refc_walk_rmaps(
if (!have_gt)
return 0;
error = xfs_rmap_get_rec(cur, &rmap, &have_gt);
error = xfs_rmap_get_rec(cur, rmap, &have_gt);
if (error)
return error;
if (XFS_IS_CORRUPT(mp, !have_gt))
if (XFS_IS_CORRUPT(mp, !have_gt)) {
xfs_btree_mark_sick(cur);
return -EFSCORRUPTED;
}
if (rmap.rm_owner == XFS_RMAP_OWN_COW) {
error = xrep_refc_stash_cow(rr, rmap.rm_startblock,
rmap.rm_blockcount);
if (rmap->rm_owner == XFS_RMAP_OWN_COW) {
error = xrep_refc_stash_cow(rr, rmap->rm_startblock,
rmap->rm_blockcount);
if (error)
return error;
} else if (rmap.rm_owner == XFS_RMAP_OWN_REFC) {
} else if (rmap->rm_owner == XFS_RMAP_OWN_REFC) {
/* refcountbt block, dump it when we're done. */
rr->btblocks += rmap.rm_blockcount;
rr->btblocks += rmap->rm_blockcount;
error = xagb_bitmap_set(&rr->old_refcountbt_blocks,
rmap.rm_startblock, rmap.rm_blockcount);
rmap->rm_startblock,
rmap->rm_blockcount);
if (error)
return error;
}
} while (!xrep_refc_rmap_shareable(mp, &rmap));
} while (!xrep_refc_rmap_shareable(mp, rmap));
rrm->startblock = rmap.rm_startblock;
rrm->blockcount = rmap.rm_blockcount;
*have_rec = true;
return 0;
}
@ -354,45 +364,6 @@ xrep_refc_sort_records(
return error;
}
#define RRM_NEXT(r) ((r).startblock + (r).blockcount)
/*
* Find the next block where the refcount changes, given the next rmap we
* looked at and the ones we're already tracking.
*/
static inline int
xrep_refc_next_edge(
struct xfarray *rmap_bag,
struct xrep_refc_rmap *next_rrm,
bool next_valid,
xfs_agblock_t *nbnop)
{
struct xrep_refc_rmap rrm;
xfarray_idx_t array_cur = XFARRAY_CURSOR_INIT;
xfs_agblock_t nbno = NULLAGBLOCK;
int error;
if (next_valid)
nbno = next_rrm->startblock;
while ((error = xfarray_iter(rmap_bag, &array_cur, &rrm)) == 1)
nbno = min_t(xfs_agblock_t, nbno, RRM_NEXT(rrm));
if (error)
return error;
/*
* We should have found /something/ because either next_rrm is the next
* interesting rmap to look at after emitting this refcount extent, or
* there are other rmaps in rmap_bag contributing to the current
* sharing count. But if something is seriously wrong, bail out.
*/
if (nbno == NULLAGBLOCK)
return -EFSCORRUPTED;
*nbnop = nbno;
return 0;
}
/*
* Walk forward through the rmap btree to collect all rmaps starting at
* @bno in @rmap_bag. These represent the file(s) that share ownership of
@ -402,22 +373,21 @@ xrep_refc_next_edge(
static int
xrep_refc_push_rmaps_at(
struct xrep_refc *rr,
struct xfarray *rmap_bag,
struct rcbag *rcstack,
xfs_agblock_t bno,
struct xrep_refc_rmap *rrm,
bool *have,
uint64_t *stack_sz)
struct xfs_rmap_irec *rmap,
bool *have)
{
struct xfs_scrub *sc = rr->sc;
int have_gt;
int error;
while (*have && rrm->startblock == bno) {
error = xfarray_store_anywhere(rmap_bag, rrm);
while (*have && rmap->rm_startblock == bno) {
error = rcbag_add(rcstack, rr->sc->tp, rmap);
if (error)
return error;
(*stack_sz)++;
error = xrep_refc_walk_rmaps(rr, rrm, have);
error = xrep_refc_walk_rmaps(rr, rmap, have);
if (error)
return error;
}
@ -425,8 +395,10 @@ xrep_refc_push_rmaps_at(
error = xfs_btree_decrement(sc->sa.rmap_cur, 0, &have_gt);
if (error)
return error;
if (XFS_IS_CORRUPT(sc->mp, !have_gt))
if (XFS_IS_CORRUPT(sc->mp, !have_gt)) {
xfs_btree_mark_sick(sc->sa.rmap_cur);
return -EFSCORRUPTED;
}
return 0;
}
@ -436,12 +408,9 @@ STATIC int
xrep_refc_find_refcounts(
struct xrep_refc *rr)
{
struct xrep_refc_rmap rrm;
struct xfs_scrub *sc = rr->sc;
struct xfarray *rmap_bag;
char *descr;
uint64_t old_stack_sz;
uint64_t stack_sz = 0;
struct rcbag *rcstack;
uint64_t old_stack_height;
xfs_agblock_t sbno;
xfs_agblock_t cbno;
xfs_agblock_t nbno;
@ -451,14 +420,11 @@ xrep_refc_find_refcounts(
xrep_ag_btcur_init(sc, &sc->sa);
/*
* Set up a sparse array to store all the rmap records that we're
* tracking to generate a reference count record. If this exceeds
* Set up a bag to store all the rmap records that we're tracking to
* generate a reference count record. If the size of the bag exceeds
* MAXREFCOUNT, we clamp rc_refcount.
*/
descr = xchk_xfile_ag_descr(sc, "rmap record bag");
error = xfarray_create(descr, 0, sizeof(struct xrep_refc_rmap),
&rmap_bag);
kfree(descr);
error = rcbag_init(sc->mp, sc->xmbtp, &rcstack);
if (error)
goto out_cur;
@ -469,62 +435,54 @@ xrep_refc_find_refcounts(
/* Process reverse mappings into refcount data. */
while (xfs_btree_has_more_records(sc->sa.rmap_cur)) {
struct xfs_rmap_irec rmap;
/* Push all rmaps with pblk == sbno onto the stack */
error = xrep_refc_walk_rmaps(rr, &rrm, &have);
error = xrep_refc_walk_rmaps(rr, &rmap, &have);
if (error)
goto out_bag;
if (!have)
break;
sbno = cbno = rrm.startblock;
error = xrep_refc_push_rmaps_at(rr, rmap_bag, sbno,
&rrm, &have, &stack_sz);
sbno = cbno = rmap.rm_startblock;
error = xrep_refc_push_rmaps_at(rr, rcstack, sbno, &rmap,
&have);
if (error)
goto out_bag;
/* Set nbno to the bno of the next refcount change */
error = xrep_refc_next_edge(rmap_bag, &rrm, have, &nbno);
error = rcbag_next_edge(rcstack, sc->tp, &rmap, have, &nbno);
if (error)
goto out_bag;
ASSERT(nbno > sbno);
old_stack_sz = stack_sz;
old_stack_height = rcbag_count(rcstack);
/* While stack isn't empty... */
while (stack_sz) {
xfarray_idx_t array_cur = XFARRAY_CURSOR_INIT;
while (rcbag_count(rcstack) > 0) {
/* Pop all rmaps that end at nbno */
while ((error = xfarray_iter(rmap_bag, &array_cur,
&rrm)) == 1) {
if (RRM_NEXT(rrm) != nbno)
continue;
error = xfarray_unset(rmap_bag, array_cur - 1);
if (error)
goto out_bag;
stack_sz--;
}
error = rcbag_remove_ending_at(rcstack, sc->tp, nbno);
if (error)
goto out_bag;
/* Push array items that start at nbno */
error = xrep_refc_walk_rmaps(rr, &rrm, &have);
error = xrep_refc_walk_rmaps(rr, &rmap, &have);
if (error)
goto out_bag;
if (have) {
error = xrep_refc_push_rmaps_at(rr, rmap_bag,
nbno, &rrm, &have, &stack_sz);
error = xrep_refc_push_rmaps_at(rr, rcstack,
nbno, &rmap, &have);
if (error)
goto out_bag;
}
/* Emit refcount if necessary */
ASSERT(nbno > cbno);
if (stack_sz != old_stack_sz) {
if (old_stack_sz > 1) {
if (rcbag_count(rcstack) != old_stack_height) {
if (old_stack_height > 1) {
error = xrep_refc_stash(rr,
XFS_REFC_DOMAIN_SHARED,
cbno, nbno - cbno,
old_stack_sz);
old_stack_height);
if (error)
goto out_bag;
}
@ -532,13 +490,13 @@ xrep_refc_find_refcounts(
}
/* Stack empty, go find the next rmap */
if (stack_sz == 0)
if (rcbag_count(rcstack) == 0)
break;
old_stack_sz = stack_sz;
old_stack_height = rcbag_count(rcstack);
sbno = nbno;
/* Set nbno to the bno of the next refcount change */
error = xrep_refc_next_edge(rmap_bag, &rrm, have,
error = rcbag_next_edge(rcstack, sc->tp, &rmap, have,
&nbno);
if (error)
goto out_bag;
@ -547,14 +505,13 @@ xrep_refc_find_refcounts(
}
}
ASSERT(stack_sz == 0);
ASSERT(rcbag_count(rcstack) == 0);
out_bag:
xfarray_destroy(rmap_bag);
rcbag_free(&rcstack);
out_cur:
xchk_ag_btcur_free(&sc->sa);
return error;
}
#undef RRM_NEXT
/* Retrieve refcountbt data for bulk load. */
STATIC int
@ -653,8 +610,8 @@ xrep_refc_build_new_tree(
rr->new_btree.bload.claim_block = xrep_refc_claim_block;
/* Compute how many blocks we'll need. */
refc_cur = xfs_refcountbt_stage_cursor(sc->mp, &rr->new_btree.afake,
pag);
refc_cur = xfs_refcountbt_init_cursor(sc->mp, NULL, NULL, pag);
xfs_btree_stage_afakeroot(refc_cur, &rr->new_btree.afake);
error = xfs_btree_bload_compute_geometry(refc_cur,
&rr->new_btree.bload,
xfarray_length(rr->refcount_records));

120
fs/xfs/scrub/repair.c
View File

@ -30,12 +30,15 @@
#include "xfs_errortag.h"
#include "xfs_error.h"
#include "xfs_reflink.h"
#include "xfs_health.h"
#include "xfs_buf_mem.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/trace.h"
#include "scrub/repair.h"
#include "scrub/bitmap.h"
#include "scrub/stats.h"
#include "scrub/xfile.h"
/*
* Attempt to repair some metadata, if the metadata is corrupt and userspace
@ -400,7 +403,7 @@ xrep_calc_ag_resblks(
int
xrep_fix_freelist(
struct xfs_scrub *sc,
bool can_shrink)
int alloc_flags)
{
struct xfs_alloc_arg args = {0};
@ -410,8 +413,7 @@ xrep_fix_freelist(
args.alignment = 1;
args.pag = sc->sa.pag;
return xfs_alloc_fix_freelist(&args,
can_shrink ? 0 : XFS_ALLOC_FLAG_NOSHRINK);
return xfs_alloc_fix_freelist(&args, alloc_flags);
}
/*
@ -687,6 +689,44 @@ xrep_find_ag_btree_roots(
}
#ifdef CONFIG_XFS_QUOTA
/* Update some quota flags in the superblock. */
void
xrep_update_qflags(
struct xfs_scrub *sc,
unsigned int clear_flags,
unsigned int set_flags)
{
struct xfs_mount *mp = sc->mp;
struct xfs_buf *bp;
mutex_lock(&mp->m_quotainfo->qi_quotaofflock);
if ((mp->m_qflags & clear_flags) == 0 &&
(mp->m_qflags & set_flags) == set_flags)
goto no_update;
mp->m_qflags &= ~clear_flags;
mp->m_qflags |= set_flags;
spin_lock(&mp->m_sb_lock);
mp->m_sb.sb_qflags &= ~clear_flags;
mp->m_sb.sb_qflags |= set_flags;
spin_unlock(&mp->m_sb_lock);
/*
* Update the quota flags in the ondisk superblock without touching
* the summary counters. We have not quiesced inode chunk allocation,
* so we cannot coordinate with updates to the icount and ifree percpu
* counters.
*/
bp = xfs_trans_getsb(sc->tp);
xfs_sb_to_disk(bp->b_addr, &mp->m_sb);
xfs_trans_buf_set_type(sc->tp, bp, XFS_BLFT_SB_BUF);
xfs_trans_log_buf(sc->tp, bp, 0, sizeof(struct xfs_dsb) - 1);
no_update:
mutex_unlock(&sc->mp->m_quotainfo->qi_quotaofflock);
}
/* Force a quotacheck the next time we mount. */
void
xrep_force_quotacheck(
@ -699,13 +739,7 @@ xrep_force_quotacheck(
if (!(flag & sc->mp->m_qflags))
return;
mutex_lock(&sc->mp->m_quotainfo->qi_quotaofflock);
sc->mp->m_qflags &= ~flag;
spin_lock(&sc->mp->m_sb_lock);
sc->mp->m_sb.sb_qflags &= ~flag;
spin_unlock(&sc->mp->m_sb_lock);
xfs_log_sb(sc->tp);
mutex_unlock(&sc->mp->m_quotainfo->qi_quotaofflock);
xrep_update_qflags(sc, flag, 0);
}
/*
@ -799,20 +833,20 @@ xrep_ag_btcur_init(
/* Set up a bnobt cursor for cross-referencing. */
if (sc->sm->sm_type != XFS_SCRUB_TYPE_BNOBT &&
sc->sm->sm_type != XFS_SCRUB_TYPE_CNTBT) {
sa->bno_cur = xfs_allocbt_init_cursor(mp, sc->tp, sa->agf_bp,
sc->sa.pag, XFS_BTNUM_BNO);
sa->cnt_cur = xfs_allocbt_init_cursor(mp, sc->tp, sa->agf_bp,
sc->sa.pag, XFS_BTNUM_CNT);
sa->bno_cur = xfs_bnobt_init_cursor(mp, sc->tp, sa->agf_bp,
sc->sa.pag);
sa->cnt_cur = xfs_cntbt_init_cursor(mp, sc->tp, sa->agf_bp,
sc->sa.pag);
}
/* Set up a inobt cursor for cross-referencing. */
if (sc->sm->sm_type != XFS_SCRUB_TYPE_INOBT &&
sc->sm->sm_type != XFS_SCRUB_TYPE_FINOBT) {
sa->ino_cur = xfs_inobt_init_cursor(sc->sa.pag, sc->tp,
sa->agi_bp, XFS_BTNUM_INO);
sa->agi_bp);
if (xfs_has_finobt(mp))
sa->fino_cur = xfs_inobt_init_cursor(sc->sa.pag,
sc->tp, sa->agi_bp, XFS_BTNUM_FINO);
sa->fino_cur = xfs_finobt_init_cursor(sc->sa.pag,
sc->tp, sa->agi_bp);
}
/* Set up a rmapbt cursor for cross-referencing. */
@ -1115,3 +1149,55 @@ xrep_metadata_inode_forks(
return 0;
}
/*
* Set up an in-memory buffer cache so that we can use the xfbtree. Allocating
* a shmem file might take loks, so we cannot be in transaction context. Park
* our resources in the scrub context and let the teardown function take care
* of them at the right time.
*/
int
xrep_setup_xfbtree(
struct xfs_scrub *sc,
const char *descr)
{
ASSERT(sc->tp == NULL);
return xmbuf_alloc(sc->mp, descr, &sc->xmbtp);
}
/*
* Create a dummy transaction for use in a live update hook function. This
* function MUST NOT be called from regular repair code because the current
* process' transaction is saved via the cookie.
*/
int
xrep_trans_alloc_hook_dummy(
struct xfs_mount *mp,
void **cookiep,
struct xfs_trans **tpp)
{
int error;
*cookiep = current->journal_info;
current->journal_info = NULL;
error = xfs_trans_alloc_empty(mp, tpp);
if (!error)
return 0;
current->journal_info = *cookiep;
*cookiep = NULL;
return error;
}
/* Cancel a dummy transaction used by a live update hook function. */
void
xrep_trans_cancel_hook_dummy(
void **cookiep,
struct xfs_trans *tp)
{
xfs_trans_cancel(tp);
current->journal_info = *cookiep;
*cookiep = NULL;
}

23
fs/xfs/scrub/repair.h
View File

@ -51,7 +51,7 @@ struct xbitmap;
struct xagb_bitmap;
struct xfsb_bitmap;
int xrep_fix_freelist(struct xfs_scrub *sc, bool can_shrink);
int xrep_fix_freelist(struct xfs_scrub *sc, int alloc_flags);
struct xrep_find_ag_btree {
/* in: rmap owner of the btree we're looking for */
@ -72,6 +72,8 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,
struct xrep_find_ag_btree *btree_info, struct xfs_buf *agfl_bp);
#ifdef CONFIG_XFS_QUOTA
void xrep_update_qflags(struct xfs_scrub *sc, unsigned int clear_flags,
unsigned int set_flags);
void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
int xrep_ino_dqattach(struct xfs_scrub *sc);
#else
@ -79,11 +81,15 @@ int xrep_ino_dqattach(struct xfs_scrub *sc);
# define xrep_ino_dqattach(sc) (0)
#endif /* CONFIG_XFS_QUOTA */
int xrep_setup_xfbtree(struct xfs_scrub *sc, const char *descr);
int xrep_ino_ensure_extent_count(struct xfs_scrub *sc, int whichfork,
xfs_extnum_t nextents);
int xrep_reset_perag_resv(struct xfs_scrub *sc);
int xrep_bmap(struct xfs_scrub *sc, int whichfork, bool allow_unwritten);
int xrep_metadata_inode_forks(struct xfs_scrub *sc);
int xrep_setup_ag_rmapbt(struct xfs_scrub *sc);
int xrep_setup_ag_refcountbt(struct xfs_scrub *sc);
/* Repair setup functions */
int xrep_setup_ag_allocbt(struct xfs_scrub *sc);
@ -109,11 +115,14 @@ int xrep_agfl(struct xfs_scrub *sc);
int xrep_agi(struct xfs_scrub *sc);
int xrep_allocbt(struct xfs_scrub *sc);
int xrep_iallocbt(struct xfs_scrub *sc);
int xrep_rmapbt(struct xfs_scrub *sc);
int xrep_refcountbt(struct xfs_scrub *sc);
int xrep_inode(struct xfs_scrub *sc);
int xrep_bmap_data(struct xfs_scrub *sc);
int xrep_bmap_attr(struct xfs_scrub *sc);
int xrep_bmap_cow(struct xfs_scrub *sc);
int xrep_nlinks(struct xfs_scrub *sc);
int xrep_fscounters(struct xfs_scrub *sc);
#ifdef CONFIG_XFS_RT
int xrep_rtbitmap(struct xfs_scrub *sc);
@ -123,13 +132,19 @@ int xrep_rtbitmap(struct xfs_scrub *sc);
#ifdef CONFIG_XFS_QUOTA
int xrep_quota(struct xfs_scrub *sc);
int xrep_quotacheck(struct xfs_scrub *sc);
#else
# define xrep_quota xrep_notsupported
# define xrep_quotacheck xrep_notsupported
#endif /* CONFIG_XFS_QUOTA */
int xrep_reinit_pagf(struct xfs_scrub *sc);
int xrep_reinit_pagi(struct xfs_scrub *sc);
int xrep_trans_alloc_hook_dummy(struct xfs_mount *mp, void **cookiep,
struct xfs_trans **tpp);
void xrep_trans_cancel_hook_dummy(void **cookiep, struct xfs_trans *tp);
#else
#define xrep_ino_dqattach(sc) (0)
@ -171,6 +186,8 @@ xrep_setup_nothing(
return 0;
}
#define xrep_setup_ag_allocbt xrep_setup_nothing
#define xrep_setup_ag_rmapbt xrep_setup_nothing
#define xrep_setup_ag_refcountbt xrep_setup_nothing
#define xrep_setup_inode(sc, imap) ((void)0)
@ -184,6 +201,7 @@ xrep_setup_nothing(
#define xrep_agi xrep_notsupported
#define xrep_allocbt xrep_notsupported
#define xrep_iallocbt xrep_notsupported
#define xrep_rmapbt xrep_notsupported
#define xrep_refcountbt xrep_notsupported
#define xrep_inode xrep_notsupported
#define xrep_bmap_data xrep_notsupported
@ -191,6 +209,9 @@ xrep_setup_nothing(
#define xrep_bmap_cow xrep_notsupported
#define xrep_rtbitmap xrep_notsupported
#define xrep_quota xrep_notsupported
#define xrep_quotacheck xrep_notsupported
#define xrep_nlinks xrep_notsupported
#define xrep_fscounters xrep_notsupported
#endif /* CONFIG_XFS_ONLINE_REPAIR */

Some files were not shown because too many files have changed in this diff Show More