2007-06-12 13:07:21 +00:00
|
|
|
/*
|
|
|
|
* Copyright (C) 2007 Oracle. All rights reserved.
|
|
|
|
*
|
|
|
|
* This program is free software; you can redistribute it and/or
|
|
|
|
* modify it under the terms of the GNU General Public
|
|
|
|
* License v2 as published by the Free Software Foundation.
|
|
|
|
*
|
|
|
|
* This program is distributed in the hope that it will be useful,
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
* General Public License for more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU General Public
|
|
|
|
* License along with this program; if not, write to the
|
|
|
|
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
|
|
|
|
* Boston, MA 021110-1307, USA.
|
|
|
|
*/
|
|
|
|
|
2007-03-22 16:13:20 +00:00
|
|
|
#include <linux/fs.h>
|
2007-03-28 17:57:48 +00:00
|
|
|
#include <linux/blkdev.h>
|
2007-03-28 23:44:27 +00:00
|
|
|
#include <linux/scatterlist.h>
|
2007-03-30 12:47:31 +00:00
|
|
|
#include <linux/swap.h>
|
2007-04-09 14:42:37 +00:00
|
|
|
#include <linux/radix-tree.h>
|
2007-05-02 19:53:43 +00:00
|
|
|
#include <linux/writeback.h>
|
2009-01-06 02:25:51 +00:00
|
|
|
#include <linux/buffer_head.h>
|
2008-04-09 20:28:12 +00:00
|
|
|
#include <linux/workqueue.h>
|
2008-06-25 20:01:31 +00:00
|
|
|
#include <linux/kthread.h>
|
2008-11-20 15:22:27 +00:00
|
|
|
#include <linux/freezer.h>
|
2009-04-19 12:02:41 +00:00
|
|
|
#include <linux/crc32c.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 08:04:11 +00:00
|
|
|
#include <linux/slab.h>
|
2010-11-22 03:20:49 +00:00
|
|
|
#include <linux/migrate.h>
|
2011-05-06 13:33:15 +00:00
|
|
|
#include <linux/ratelimit.h>
|
2011-03-18 22:56:43 +00:00
|
|
|
#include <asm/unaligned.h>
|
2008-11-20 15:22:27 +00:00
|
|
|
#include "compat.h"
|
2007-02-02 14:18:22 +00:00
|
|
|
#include "ctree.h"
|
|
|
|
#include "disk-io.h"
|
2007-03-16 20:20:31 +00:00
|
|
|
#include "transaction.h"
|
2007-04-09 14:42:37 +00:00
|
|
|
#include "btrfs_inode.h"
|
2008-03-24 19:01:56 +00:00
|
|
|
#include "volumes.h"
|
2007-10-15 20:15:53 +00:00
|
|
|
#include "print-tree.h"
|
2008-06-11 20:50:36 +00:00
|
|
|
#include "async-thread.h"
|
2008-06-25 20:01:30 +00:00
|
|
|
#include "locking.h"
|
2008-09-05 20:13:11 +00:00
|
|
|
#include "tree-log.h"
|
2009-04-03 13:47:43 +00:00
|
|
|
#include "free-space-cache.h"
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
#include "inode-map.h"
|
2011-11-09 12:44:05 +00:00
|
|
|
#include "check-integrity.h"
|
2012-06-04 18:03:51 +00:00
|
|
|
#include "rcu-string.h"
|
2012-11-06 12:15:27 +00:00
|
|
|
#include "dev-replace.h"
|
2013-01-29 23:40:14 +00:00
|
|
|
#include "raid56.h"
|
2007-02-02 14:18:22 +00:00
|
|
|
|
2012-09-25 18:25:58 +00:00
|
|
|
#ifdef CONFIG_X86
|
|
|
|
#include <asm/cpufeature.h>
|
|
|
|
#endif
|
|
|
|
|
2008-01-24 21:13:08 +00:00
|
|
|
static struct extent_io_ops btree_extent_io_ops;
|
2008-06-11 20:50:36 +00:00
|
|
|
static void end_workqueue_fn(struct btrfs_work *work);
|
2009-09-21 19:56:00 +00:00
|
|
|
static void free_fs_root(struct btrfs_root *root);
|
2012-03-05 23:06:18 +00:00
|
|
|
static int btrfs_check_super_valid(struct btrfs_fs_info *fs_info,
|
2011-01-06 11:30:25 +00:00
|
|
|
int read_only);
|
2013-02-13 16:09:14 +00:00
|
|
|
static void btrfs_destroy_ordered_operations(struct btrfs_transaction *t,
|
|
|
|
struct btrfs_root *root);
|
2012-03-01 13:56:26 +00:00
|
|
|
static void btrfs_destroy_ordered_extents(struct btrfs_root *root);
|
2011-01-06 11:30:25 +00:00
|
|
|
static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,
|
|
|
|
struct btrfs_root *root);
|
2012-03-01 13:56:26 +00:00
|
|
|
static void btrfs_destroy_pending_snapshots(struct btrfs_transaction *t);
|
|
|
|
static void btrfs_destroy_delalloc_inodes(struct btrfs_root *root);
|
2011-01-06 11:30:25 +00:00
|
|
|
static int btrfs_destroy_marked_extents(struct btrfs_root *root,
|
|
|
|
struct extent_io_tree *dirty_pages,
|
|
|
|
int mark);
|
|
|
|
static int btrfs_destroy_pinned_extent(struct btrfs_root *root,
|
|
|
|
struct extent_io_tree *pinned_extents);
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* end_io_wq structs are used to do processing in task context when an IO is
|
|
|
|
* complete. This is used during reads to verify checksums, and it is used
|
|
|
|
* by writes to insert metadata for new file extents after IO is complete.
|
|
|
|
*/
|
2008-04-09 20:28:12 +00:00
|
|
|
struct end_io_wq {
|
|
|
|
struct bio *bio;
|
|
|
|
bio_end_io_t *end_io;
|
|
|
|
void *private;
|
|
|
|
struct btrfs_fs_info *info;
|
|
|
|
int error;
|
2008-04-09 20:28:12 +00:00
|
|
|
int metadata;
|
2008-04-09 20:28:12 +00:00
|
|
|
struct list_head list;
|
2008-06-11 20:50:36 +00:00
|
|
|
struct btrfs_work work;
|
2008-04-09 20:28:12 +00:00
|
|
|
};
|
2007-11-08 02:08:01 +00:00
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* async submit bios are used to offload expensive checksumming
|
|
|
|
* onto the worker threads. They checksum file and metadata bios
|
|
|
|
* just before they are sent down the IO stack.
|
|
|
|
*/
|
2008-04-16 15:14:51 +00:00
|
|
|
struct async_submit_bio {
|
|
|
|
struct inode *inode;
|
|
|
|
struct bio *bio;
|
|
|
|
struct list_head list;
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
extent_submit_bio_hook_t *submit_bio_start;
|
|
|
|
extent_submit_bio_hook_t *submit_bio_done;
|
2008-04-16 15:14:51 +00:00
|
|
|
int rw;
|
|
|
|
int mirror_num;
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
unsigned long bio_flags;
|
2010-05-25 13:48:28 +00:00
|
|
|
/*
|
|
|
|
* bio_offset is optional, can be used if the pages in the bio
|
|
|
|
* can't tell us where in the file the bio should go
|
|
|
|
*/
|
|
|
|
u64 bio_offset;
|
2008-06-11 20:50:36 +00:00
|
|
|
struct btrfs_work work;
|
2012-03-12 15:03:00 +00:00
|
|
|
int error;
|
2008-04-16 15:14:51 +00:00
|
|
|
};
|
|
|
|
|
2011-07-26 20:11:19 +00:00
|
|
|
/*
|
|
|
|
* Lockdep class keys for extent_buffer->lock's in this root. For a given
|
|
|
|
* eb, the lockdep key is determined by the btrfs_root it belongs to and
|
|
|
|
* the level the eb occupies in the tree.
|
|
|
|
*
|
|
|
|
* Different roots are used for different purposes and may nest inside each
|
|
|
|
* other and they require separate keysets. As lockdep keys should be
|
|
|
|
* static, assign keysets according to the purpose of the root as indicated
|
|
|
|
* by btrfs_root->objectid. This ensures that all special purpose roots
|
|
|
|
* have separate keysets.
|
2009-02-12 19:09:45 +00:00
|
|
|
*
|
2011-07-26 20:11:19 +00:00
|
|
|
* Lock-nesting across peer nodes is always done with the immediate parent
|
|
|
|
* node locked thus preventing deadlock. As lockdep doesn't know this, use
|
|
|
|
* subclass to avoid triggering lockdep warning in such cases.
|
2009-02-12 19:09:45 +00:00
|
|
|
*
|
2011-07-26 20:11:19 +00:00
|
|
|
* The key is set by the readpage_end_io_hook after the buffer has passed
|
|
|
|
* csum validation but before the pages are unlocked. It is also set by
|
|
|
|
* btrfs_init_new_buffer on freshly allocated blocks.
|
2009-02-12 19:09:45 +00:00
|
|
|
*
|
2011-07-26 20:11:19 +00:00
|
|
|
* We also add a check to make sure the highest level of the tree is the
|
|
|
|
* same as our lockdep setup here. If BTRFS_MAX_LEVEL changes, this code
|
|
|
|
* needs update as well.
|
2009-02-12 19:09:45 +00:00
|
|
|
*/
|
|
|
|
#ifdef CONFIG_DEBUG_LOCK_ALLOC
|
|
|
|
# if BTRFS_MAX_LEVEL != 8
|
|
|
|
# error
|
|
|
|
# endif
|
2011-07-26 20:11:19 +00:00
|
|
|
|
|
|
|
static struct btrfs_lockdep_keyset {
|
|
|
|
u64 id; /* root objectid */
|
|
|
|
const char *name_stem; /* lock name stem */
|
|
|
|
char names[BTRFS_MAX_LEVEL + 1][20];
|
|
|
|
struct lock_class_key keys[BTRFS_MAX_LEVEL + 1];
|
|
|
|
} btrfs_lockdep_keysets[] = {
|
|
|
|
{ .id = BTRFS_ROOT_TREE_OBJECTID, .name_stem = "root" },
|
|
|
|
{ .id = BTRFS_EXTENT_TREE_OBJECTID, .name_stem = "extent" },
|
|
|
|
{ .id = BTRFS_CHUNK_TREE_OBJECTID, .name_stem = "chunk" },
|
|
|
|
{ .id = BTRFS_DEV_TREE_OBJECTID, .name_stem = "dev" },
|
|
|
|
{ .id = BTRFS_FS_TREE_OBJECTID, .name_stem = "fs" },
|
|
|
|
{ .id = BTRFS_CSUM_TREE_OBJECTID, .name_stem = "csum" },
|
|
|
|
{ .id = BTRFS_ORPHAN_OBJECTID, .name_stem = "orphan" },
|
|
|
|
{ .id = BTRFS_TREE_LOG_OBJECTID, .name_stem = "log" },
|
|
|
|
{ .id = BTRFS_TREE_RELOC_OBJECTID, .name_stem = "treloc" },
|
|
|
|
{ .id = BTRFS_DATA_RELOC_TREE_OBJECTID, .name_stem = "dreloc" },
|
|
|
|
{ .id = 0, .name_stem = "tree" },
|
2009-02-12 19:09:45 +00:00
|
|
|
};
|
2011-07-26 20:11:19 +00:00
|
|
|
|
|
|
|
void __init btrfs_init_lockdep(void)
|
|
|
|
{
|
|
|
|
int i, j;
|
|
|
|
|
|
|
|
/* initialize lockdep class names */
|
|
|
|
for (i = 0; i < ARRAY_SIZE(btrfs_lockdep_keysets); i++) {
|
|
|
|
struct btrfs_lockdep_keyset *ks = &btrfs_lockdep_keysets[i];
|
|
|
|
|
|
|
|
for (j = 0; j < ARRAY_SIZE(ks->names); j++)
|
|
|
|
snprintf(ks->names[j], sizeof(ks->names[j]),
|
|
|
|
"btrfs-%s-%02d", ks->name_stem, j);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
void btrfs_set_buffer_lockdep_class(u64 objectid, struct extent_buffer *eb,
|
|
|
|
int level)
|
|
|
|
{
|
|
|
|
struct btrfs_lockdep_keyset *ks;
|
|
|
|
|
|
|
|
BUG_ON(level >= ARRAY_SIZE(ks->keys));
|
|
|
|
|
|
|
|
/* find the matching keyset, id 0 is the default entry */
|
|
|
|
for (ks = btrfs_lockdep_keysets; ks->id; ks++)
|
|
|
|
if (ks->id == objectid)
|
|
|
|
break;
|
|
|
|
|
|
|
|
lockdep_set_class_and_name(&eb->lock,
|
|
|
|
&ks->keys[level], ks->names[level]);
|
|
|
|
}
|
|
|
|
|
2009-02-12 19:09:45 +00:00
|
|
|
#endif
|
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* extents on the btree inode are pretty simple, there's one extent
|
|
|
|
* that covers the entire device
|
|
|
|
*/
|
2008-12-02 14:54:17 +00:00
|
|
|
static struct extent_map *btree_get_extent(struct inode *inode,
|
2011-04-19 12:29:38 +00:00
|
|
|
struct page *page, size_t pg_offset, u64 start, u64 len,
|
2008-12-02 14:54:17 +00:00
|
|
|
int create)
|
2007-04-11 19:53:25 +00:00
|
|
|
{
|
2007-10-15 20:14:19 +00:00
|
|
|
struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
|
|
|
|
struct extent_map *em;
|
|
|
|
int ret;
|
|
|
|
|
2009-09-02 20:24:52 +00:00
|
|
|
read_lock(&em_tree->lock);
|
2008-01-24 21:13:08 +00:00
|
|
|
em = lookup_extent_mapping(em_tree, start, len);
|
2008-05-07 15:43:44 +00:00
|
|
|
if (em) {
|
|
|
|
em->bdev =
|
|
|
|
BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
|
2009-09-02 20:24:52 +00:00
|
|
|
read_unlock(&em_tree->lock);
|
2007-10-15 20:14:19 +00:00
|
|
|
goto out;
|
2008-05-07 15:43:44 +00:00
|
|
|
}
|
2009-09-02 20:24:52 +00:00
|
|
|
read_unlock(&em_tree->lock);
|
2008-04-18 14:29:50 +00:00
|
|
|
|
2011-04-20 22:48:27 +00:00
|
|
|
em = alloc_extent_map();
|
2007-10-15 20:14:19 +00:00
|
|
|
if (!em) {
|
|
|
|
em = ERR_PTR(-ENOMEM);
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
em->start = 0;
|
2008-04-18 18:17:20 +00:00
|
|
|
em->len = (u64)-1;
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
em->block_len = (u64)-1;
|
2007-10-15 20:14:19 +00:00
|
|
|
em->block_start = 0;
|
2008-05-07 15:43:44 +00:00
|
|
|
em->bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
|
2008-01-24 21:13:08 +00:00
|
|
|
|
2009-09-02 20:24:52 +00:00
|
|
|
write_lock(&em_tree->lock);
|
2007-10-15 20:14:19 +00:00
|
|
|
ret = add_extent_mapping(em_tree, em);
|
|
|
|
if (ret == -EEXIST) {
|
|
|
|
free_extent_map(em);
|
2008-04-18 14:29:50 +00:00
|
|
|
em = lookup_extent_mapping(em_tree, start, len);
|
2012-09-13 09:32:54 +00:00
|
|
|
if (!em)
|
2012-09-13 09:32:32 +00:00
|
|
|
em = ERR_PTR(-EIO);
|
2007-10-15 20:14:19 +00:00
|
|
|
} else if (ret) {
|
2008-04-18 14:29:50 +00:00
|
|
|
free_extent_map(em);
|
2012-09-13 09:32:32 +00:00
|
|
|
em = ERR_PTR(ret);
|
2007-10-15 20:14:19 +00:00
|
|
|
}
|
2009-09-02 20:24:52 +00:00
|
|
|
write_unlock(&em_tree->lock);
|
2008-04-18 14:29:50 +00:00
|
|
|
|
2007-10-15 20:14:19 +00:00
|
|
|
out:
|
|
|
|
return em;
|
2007-04-11 19:53:25 +00:00
|
|
|
}
|
|
|
|
|
2007-10-15 20:19:22 +00:00
|
|
|
u32 btrfs_csum_data(struct btrfs_root *root, char *data, u32 seed, size_t len)
|
|
|
|
{
|
2009-04-19 12:02:41 +00:00
|
|
|
return crc32c(seed, data, len);
|
2007-10-15 20:19:22 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void btrfs_csum_final(u32 crc, char *result)
|
|
|
|
{
|
2011-03-18 22:56:43 +00:00
|
|
|
put_unaligned_le32(~crc, result);
|
2007-10-15 20:19:22 +00:00
|
|
|
}
|
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* compute the csum for a btree block, and either verify it or write it
|
|
|
|
* into the csum field of the block.
|
|
|
|
*/
|
2007-10-15 20:19:22 +00:00
|
|
|
static int csum_tree_block(struct btrfs_root *root, struct extent_buffer *buf,
|
|
|
|
int verify)
|
|
|
|
{
|
2011-04-13 13:41:04 +00:00
|
|
|
u16 csum_size = btrfs_super_csum_size(root->fs_info->super_copy);
|
2008-12-02 12:17:45 +00:00
|
|
|
char *result = NULL;
|
2007-10-15 20:19:22 +00:00
|
|
|
unsigned long len;
|
|
|
|
unsigned long cur_len;
|
|
|
|
unsigned long offset = BTRFS_CSUM_SIZE;
|
|
|
|
char *kaddr;
|
|
|
|
unsigned long map_start;
|
|
|
|
unsigned long map_len;
|
|
|
|
int err;
|
|
|
|
u32 crc = ~(u32)0;
|
2008-12-02 12:17:45 +00:00
|
|
|
unsigned long inline_result;
|
2007-10-15 20:19:22 +00:00
|
|
|
|
|
|
|
len = buf->len - offset;
|
2009-01-06 02:25:51 +00:00
|
|
|
while (len > 0) {
|
2007-10-15 20:19:22 +00:00
|
|
|
err = map_private_extent_buffer(buf, offset, 32,
|
2011-07-19 16:04:14 +00:00
|
|
|
&kaddr, &map_start, &map_len);
|
2009-01-06 02:25:51 +00:00
|
|
|
if (err)
|
2007-10-15 20:19:22 +00:00
|
|
|
return 1;
|
|
|
|
cur_len = min(len, map_len - (offset - map_start));
|
|
|
|
crc = btrfs_csum_data(root, kaddr + offset - map_start,
|
|
|
|
crc, cur_len);
|
|
|
|
len -= cur_len;
|
|
|
|
offset += cur_len;
|
|
|
|
}
|
2008-12-02 12:17:45 +00:00
|
|
|
if (csum_size > sizeof(inline_result)) {
|
|
|
|
result = kzalloc(csum_size * sizeof(char), GFP_NOFS);
|
|
|
|
if (!result)
|
|
|
|
return 1;
|
|
|
|
} else {
|
|
|
|
result = (char *)&inline_result;
|
|
|
|
}
|
|
|
|
|
2007-10-15 20:19:22 +00:00
|
|
|
btrfs_csum_final(crc, result);
|
|
|
|
|
|
|
|
if (verify) {
|
2008-12-02 12:17:45 +00:00
|
|
|
if (memcmp_extent_buffer(buf, result, 0, csum_size)) {
|
2008-01-08 20:46:27 +00:00
|
|
|
u32 val;
|
|
|
|
u32 found = 0;
|
2008-12-02 12:17:45 +00:00
|
|
|
memcpy(&found, result, csum_size);
|
2008-01-08 20:46:27 +00:00
|
|
|
|
2008-12-02 12:17:45 +00:00
|
|
|
read_extent_buffer(buf, &val, 0, csum_size);
|
2011-05-06 13:33:15 +00:00
|
|
|
printk_ratelimited(KERN_INFO "btrfs: %s checksum verify "
|
2009-04-27 11:29:05 +00:00
|
|
|
"failed on %llu wanted %X found %X "
|
|
|
|
"level %d\n",
|
|
|
|
root->fs_info->sb->s_id,
|
|
|
|
(unsigned long long)buf->start, val, found,
|
|
|
|
btrfs_header_level(buf));
|
2008-12-02 12:17:45 +00:00
|
|
|
if (result != (char *)&inline_result)
|
|
|
|
kfree(result);
|
2007-10-15 20:19:22 +00:00
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
} else {
|
2008-12-02 12:17:45 +00:00
|
|
|
write_extent_buffer(buf, result, 0, csum_size);
|
2007-10-15 20:19:22 +00:00
|
|
|
}
|
2008-12-02 12:17:45 +00:00
|
|
|
if (result != (char *)&inline_result)
|
|
|
|
kfree(result);
|
2007-10-15 20:19:22 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* we can't consider a given block up to date unless the transid of the
|
|
|
|
* block matches the transid in the parent node's pointer. This is how we
|
|
|
|
* detect blocks that either didn't get written at all or got written
|
|
|
|
* in the wrong place.
|
|
|
|
*/
|
2008-05-12 17:39:03 +00:00
|
|
|
static int verify_parent_transid(struct extent_io_tree *io_tree,
|
2012-05-06 11:23:47 +00:00
|
|
|
struct extent_buffer *eb, u64 parent_transid,
|
|
|
|
int atomic)
|
2008-05-12 17:39:03 +00:00
|
|
|
{
|
2010-02-03 19:33:23 +00:00
|
|
|
struct extent_state *cached_state = NULL;
|
2008-05-12 17:39:03 +00:00
|
|
|
int ret;
|
|
|
|
|
|
|
|
if (!parent_transid || btrfs_header_generation(eb) == parent_transid)
|
|
|
|
return 0;
|
|
|
|
|
2012-05-06 11:23:47 +00:00
|
|
|
if (atomic)
|
|
|
|
return -EAGAIN;
|
|
|
|
|
2010-02-03 19:33:23 +00:00
|
|
|
lock_extent_bits(io_tree, eb->start, eb->start + eb->len - 1,
|
2012-03-01 13:57:19 +00:00
|
|
|
0, &cached_state);
|
2012-03-13 13:38:00 +00:00
|
|
|
if (extent_buffer_uptodate(eb) &&
|
2008-05-12 17:39:03 +00:00
|
|
|
btrfs_header_generation(eb) == parent_transid) {
|
|
|
|
ret = 0;
|
|
|
|
goto out;
|
|
|
|
}
|
2011-05-06 13:33:15 +00:00
|
|
|
printk_ratelimited("parent transid verify failed on %llu wanted %llu "
|
2009-04-27 11:29:05 +00:00
|
|
|
"found %llu\n",
|
|
|
|
(unsigned long long)eb->start,
|
|
|
|
(unsigned long long)parent_transid,
|
|
|
|
(unsigned long long)btrfs_header_generation(eb));
|
2008-05-12 17:39:03 +00:00
|
|
|
ret = 1;
|
2012-03-13 13:38:00 +00:00
|
|
|
clear_extent_buffer_uptodate(eb);
|
2008-07-30 14:29:12 +00:00
|
|
|
out:
|
2010-02-03 19:33:23 +00:00
|
|
|
unlock_extent_cached(io_tree, eb->start, eb->start + eb->len - 1,
|
|
|
|
&cached_state, GFP_NOFS);
|
2008-05-12 17:39:03 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
|
|
|
* helper to read a given tree block, doing retries as required when
|
|
|
|
* the checksums don't match and we have alternate mirrors to try.
|
|
|
|
*/
|
2008-04-09 20:28:12 +00:00
|
|
|
static int btree_read_extent_buffer_pages(struct btrfs_root *root,
|
|
|
|
struct extent_buffer *eb,
|
2008-05-12 16:59:19 +00:00
|
|
|
u64 start, u64 parent_transid)
|
2008-04-09 20:28:12 +00:00
|
|
|
{
|
|
|
|
struct extent_io_tree *io_tree;
|
2012-03-27 01:57:36 +00:00
|
|
|
int failed = 0;
|
2008-04-09 20:28:12 +00:00
|
|
|
int ret;
|
|
|
|
int num_copies = 0;
|
|
|
|
int mirror_num = 0;
|
2012-03-27 01:57:36 +00:00
|
|
|
int failed_mirror = 0;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2011-03-16 17:42:43 +00:00
|
|
|
clear_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags);
|
2008-04-09 20:28:12 +00:00
|
|
|
io_tree = &BTRFS_I(root->fs_info->btree_inode)->io_tree;
|
|
|
|
while (1) {
|
2011-06-10 12:06:53 +00:00
|
|
|
ret = read_extent_buffer_pages(io_tree, eb, start,
|
|
|
|
WAIT_COMPLETE,
|
2008-04-09 20:28:12 +00:00
|
|
|
btree_get_extent, mirror_num);
|
2012-08-10 14:58:21 +00:00
|
|
|
if (!ret) {
|
|
|
|
if (!verify_parent_transid(io_tree, eb,
|
2012-05-06 11:23:47 +00:00
|
|
|
parent_transid, 0))
|
2012-08-10 14:58:21 +00:00
|
|
|
break;
|
|
|
|
else
|
|
|
|
ret = -EIO;
|
|
|
|
}
|
2009-01-06 02:25:51 +00:00
|
|
|
|
2011-03-16 17:42:43 +00:00
|
|
|
/*
|
|
|
|
* This buffer's crc is fine, but its contents are corrupted, so
|
|
|
|
* there is no reason to read the other copies, they won't be
|
|
|
|
* any less wrong.
|
|
|
|
*/
|
|
|
|
if (test_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags))
|
2012-03-27 01:57:36 +00:00
|
|
|
break;
|
|
|
|
|
2012-11-05 13:59:07 +00:00
|
|
|
num_copies = btrfs_num_copies(root->fs_info,
|
2008-04-09 20:28:12 +00:00
|
|
|
eb->start, eb->len);
|
2008-04-28 20:40:52 +00:00
|
|
|
if (num_copies == 1)
|
2012-03-27 01:57:36 +00:00
|
|
|
break;
|
2008-04-28 20:40:52 +00:00
|
|
|
|
2012-04-16 13:42:26 +00:00
|
|
|
if (!failed_mirror) {
|
|
|
|
failed = 1;
|
|
|
|
failed_mirror = eb->read_mirror;
|
|
|
|
}
|
|
|
|
|
2008-04-09 20:28:12 +00:00
|
|
|
mirror_num++;
|
2012-03-27 01:57:36 +00:00
|
|
|
if (mirror_num == failed_mirror)
|
|
|
|
mirror_num++;
|
|
|
|
|
2008-04-28 20:40:52 +00:00
|
|
|
if (mirror_num > num_copies)
|
2012-03-27 01:57:36 +00:00
|
|
|
break;
|
2008-04-09 20:28:12 +00:00
|
|
|
}
|
2012-03-27 01:57:36 +00:00
|
|
|
|
2012-07-10 13:30:17 +00:00
|
|
|
if (failed && !ret && failed_mirror)
|
2012-03-27 01:57:36 +00:00
|
|
|
repair_eb_io_failure(root, eb, failed_mirror);
|
|
|
|
|
|
|
|
return ret;
|
2008-04-09 20:28:12 +00:00
|
|
|
}
|
2007-10-15 20:19:22 +00:00
|
|
|
|
2008-09-29 19:18:18 +00:00
|
|
|
/*
|
2009-01-06 02:25:51 +00:00
|
|
|
* checksum a dirty tree block before IO. This has extra checks to make sure
|
|
|
|
* we only fill in the checksum field in the first page of a multi-page block
|
2008-09-29 19:18:18 +00:00
|
|
|
*/
|
2009-01-06 02:25:51 +00:00
|
|
|
|
2008-12-02 14:54:17 +00:00
|
|
|
static int csum_dirty_buffer(struct btrfs_root *root, struct page *page)
|
2007-10-15 20:19:22 +00:00
|
|
|
{
|
2008-01-24 21:13:08 +00:00
|
|
|
struct extent_io_tree *tree;
|
2012-12-21 09:17:45 +00:00
|
|
|
u64 start = page_offset(page);
|
2007-10-15 20:19:22 +00:00
|
|
|
u64 found_start;
|
|
|
|
struct extent_buffer *eb;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2008-01-24 21:13:08 +00:00
|
|
|
tree = &BTRFS_I(page->mapping->host)->io_tree;
|
2007-10-15 20:19:22 +00:00
|
|
|
|
2012-03-07 21:20:05 +00:00
|
|
|
eb = (struct extent_buffer *)page->private;
|
|
|
|
if (page != eb->pages[0])
|
|
|
|
return 0;
|
2007-10-15 20:19:22 +00:00
|
|
|
found_start = btrfs_header_bytenr(eb);
|
|
|
|
if (found_start != start) {
|
2008-01-09 20:55:33 +00:00
|
|
|
WARN_ON(1);
|
2012-03-07 21:20:05 +00:00
|
|
|
return 0;
|
2008-01-09 20:55:33 +00:00
|
|
|
}
|
|
|
|
if (!PageUptodate(page)) {
|
|
|
|
WARN_ON(1);
|
2012-03-07 21:20:05 +00:00
|
|
|
return 0;
|
2007-10-15 20:19:22 +00:00
|
|
|
}
|
|
|
|
csum_tree_block(root, eb, 0);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-11-18 02:11:30 +00:00
|
|
|
static int check_tree_block_fsid(struct btrfs_root *root,
|
|
|
|
struct extent_buffer *eb)
|
|
|
|
{
|
|
|
|
struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices;
|
|
|
|
u8 fsid[BTRFS_UUID_SIZE];
|
|
|
|
int ret = 1;
|
|
|
|
|
|
|
|
read_extent_buffer(eb, fsid, (unsigned long)btrfs_header_fsid(eb),
|
|
|
|
BTRFS_FSID_SIZE);
|
|
|
|
while (fs_devices) {
|
|
|
|
if (!memcmp(fsid, fs_devices->fsid, BTRFS_FSID_SIZE)) {
|
|
|
|
ret = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
fs_devices = fs_devices->seed;
|
|
|
|
}
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2011-03-16 17:42:43 +00:00
|
|
|
#define CORRUPT(reason, eb, root, slot) \
|
|
|
|
printk(KERN_CRIT "btrfs: corrupt leaf, %s: block=%llu," \
|
|
|
|
"root=%llu, slot=%d\n", reason, \
|
|
|
|
(unsigned long long)btrfs_header_bytenr(eb), \
|
|
|
|
(unsigned long long)root->objectid, slot)
|
|
|
|
|
|
|
|
static noinline int check_leaf(struct btrfs_root *root,
|
|
|
|
struct extent_buffer *leaf)
|
|
|
|
{
|
|
|
|
struct btrfs_key key;
|
|
|
|
struct btrfs_key leaf_key;
|
|
|
|
u32 nritems = btrfs_header_nritems(leaf);
|
|
|
|
int slot;
|
|
|
|
|
|
|
|
if (nritems == 0)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
/* Check the 0 item */
|
|
|
|
if (btrfs_item_offset_nr(leaf, 0) + btrfs_item_size_nr(leaf, 0) !=
|
|
|
|
BTRFS_LEAF_DATA_SIZE(root)) {
|
|
|
|
CORRUPT("invalid item offset size pair", leaf, root, 0);
|
|
|
|
return -EIO;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Check to make sure each items keys are in the correct order and their
|
|
|
|
* offsets make sense. We only have to loop through nritems-1 because
|
|
|
|
* we check the current slot against the next slot, which verifies the
|
|
|
|
* next slot's offset+size makes sense and that the current's slot
|
|
|
|
* offset is correct.
|
|
|
|
*/
|
|
|
|
for (slot = 0; slot < nritems - 1; slot++) {
|
|
|
|
btrfs_item_key_to_cpu(leaf, &leaf_key, slot);
|
|
|
|
btrfs_item_key_to_cpu(leaf, &key, slot + 1);
|
|
|
|
|
|
|
|
/* Make sure the keys are in the right order */
|
|
|
|
if (btrfs_comp_cpu_keys(&leaf_key, &key) >= 0) {
|
|
|
|
CORRUPT("bad key order", leaf, root, slot);
|
|
|
|
return -EIO;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Make sure the offset and ends are right, remember that the
|
|
|
|
* item data starts at the end of the leaf and grows towards the
|
|
|
|
* front.
|
|
|
|
*/
|
|
|
|
if (btrfs_item_offset_nr(leaf, slot) !=
|
|
|
|
btrfs_item_end_nr(leaf, slot + 1)) {
|
|
|
|
CORRUPT("slot offset bad", leaf, root, slot);
|
|
|
|
return -EIO;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Check to make sure that we don't point outside of the leaf,
|
|
|
|
* just incase all the items are consistent to eachother, but
|
|
|
|
* all point outside of the leaf.
|
|
|
|
*/
|
|
|
|
if (btrfs_item_end_nr(leaf, slot) >
|
|
|
|
BTRFS_LEAF_DATA_SIZE(root)) {
|
|
|
|
CORRUPT("slot end outside of leaf", leaf, root, slot);
|
|
|
|
return -EIO;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2010-08-06 17:21:20 +00:00
|
|
|
struct extent_buffer *find_eb_for_page(struct extent_io_tree *tree,
|
|
|
|
struct page *page, int max_walk)
|
|
|
|
{
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
u64 start = page_offset(page);
|
|
|
|
u64 target = start;
|
|
|
|
u64 min_start;
|
|
|
|
|
|
|
|
if (start < max_walk)
|
|
|
|
min_start = 0;
|
|
|
|
else
|
|
|
|
min_start = start - max_walk;
|
|
|
|
|
|
|
|
while (start >= min_start) {
|
|
|
|
eb = find_extent_buffer(tree, start, 0);
|
|
|
|
if (eb) {
|
|
|
|
/*
|
|
|
|
* we found an extent buffer and it contains our page
|
|
|
|
* horray!
|
|
|
|
*/
|
|
|
|
if (eb->start <= target &&
|
|
|
|
eb->start + eb->len > target)
|
|
|
|
return eb;
|
|
|
|
|
|
|
|
/* we found an extent buffer that wasn't for us */
|
|
|
|
free_extent_buffer(eb);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
if (start == 0)
|
|
|
|
break;
|
|
|
|
start -= PAGE_CACHE_SIZE;
|
|
|
|
}
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2008-12-02 14:54:17 +00:00
|
|
|
static int btree_readpage_end_io_hook(struct page *page, u64 start, u64 end,
|
2012-04-16 13:42:26 +00:00
|
|
|
struct extent_state *state, int mirror)
|
2008-04-09 20:28:12 +00:00
|
|
|
{
|
|
|
|
struct extent_io_tree *tree;
|
|
|
|
u64 found_start;
|
|
|
|
int found_level;
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
|
2008-04-09 20:28:12 +00:00
|
|
|
int ret = 0;
|
2010-08-06 17:21:20 +00:00
|
|
|
int reads_done;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
|
|
|
if (!page->private)
|
|
|
|
goto out;
|
2009-01-06 02:25:51 +00:00
|
|
|
|
2010-08-06 17:21:20 +00:00
|
|
|
tree = &BTRFS_I(page->mapping->host)->io_tree;
|
2012-03-07 21:20:05 +00:00
|
|
|
eb = (struct extent_buffer *)page->private;
|
2009-01-06 02:25:51 +00:00
|
|
|
|
2012-03-13 13:38:00 +00:00
|
|
|
/* the pending IO might have been the only thing that kept this buffer
|
|
|
|
* in memory. Make sure we have a ref for all this other checks
|
|
|
|
*/
|
|
|
|
extent_buffer_get(eb);
|
|
|
|
|
|
|
|
reads_done = atomic_dec_and_test(&eb->io_pages);
|
2010-08-06 17:21:20 +00:00
|
|
|
if (!reads_done)
|
|
|
|
goto err;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2012-04-16 13:42:26 +00:00
|
|
|
eb->read_mirror = mirror;
|
2012-03-27 01:57:36 +00:00
|
|
|
if (test_bit(EXTENT_BUFFER_IOERR, &eb->bflags)) {
|
|
|
|
ret = -EIO;
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2008-04-09 20:28:12 +00:00
|
|
|
found_start = btrfs_header_bytenr(eb);
|
2010-08-06 17:21:20 +00:00
|
|
|
if (found_start != eb->start) {
|
2011-05-06 13:33:15 +00:00
|
|
|
printk_ratelimited(KERN_INFO "btrfs bad tree block start "
|
2009-04-27 11:29:05 +00:00
|
|
|
"%llu %llu\n",
|
|
|
|
(unsigned long long)found_start,
|
|
|
|
(unsigned long long)eb->start);
|
2008-04-09 20:28:12 +00:00
|
|
|
ret = -EIO;
|
2008-04-09 20:28:12 +00:00
|
|
|
goto err;
|
|
|
|
}
|
2008-11-18 02:11:30 +00:00
|
|
|
if (check_tree_block_fsid(root, eb)) {
|
2011-05-06 13:33:15 +00:00
|
|
|
printk_ratelimited(KERN_INFO "btrfs bad fsid on block %llu\n",
|
2009-04-27 11:29:05 +00:00
|
|
|
(unsigned long long)eb->start);
|
2008-05-12 17:39:03 +00:00
|
|
|
ret = -EIO;
|
|
|
|
goto err;
|
|
|
|
}
|
2008-04-09 20:28:12 +00:00
|
|
|
found_level = btrfs_header_level(eb);
|
|
|
|
|
2011-07-26 20:11:19 +00:00
|
|
|
btrfs_set_buffer_lockdep_class(btrfs_header_owner(eb),
|
|
|
|
eb, found_level);
|
2009-02-12 19:09:45 +00:00
|
|
|
|
2008-04-09 20:28:12 +00:00
|
|
|
ret = csum_tree_block(root, eb, 1);
|
2011-03-16 17:42:43 +00:00
|
|
|
if (ret) {
|
2008-04-09 20:28:12 +00:00
|
|
|
ret = -EIO;
|
2011-03-16 17:42:43 +00:00
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If this is a leaf block and it is corrupt, set the corrupt bit so
|
|
|
|
* that we don't try and read the other copies of this block, just
|
|
|
|
* return -EIO.
|
|
|
|
*/
|
|
|
|
if (found_level == 0 && check_leaf(root, eb)) {
|
|
|
|
set_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags);
|
|
|
|
ret = -EIO;
|
|
|
|
}
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2012-03-13 13:38:00 +00:00
|
|
|
if (!ret)
|
|
|
|
set_extent_buffer_uptodate(eb);
|
2008-04-09 20:28:12 +00:00
|
|
|
err:
|
2011-06-10 11:55:54 +00:00
|
|
|
if (test_bit(EXTENT_BUFFER_READAHEAD, &eb->bflags)) {
|
|
|
|
clear_bit(EXTENT_BUFFER_READAHEAD, &eb->bflags);
|
|
|
|
btree_readahead_hook(root, eb, eb->start, ret);
|
|
|
|
}
|
|
|
|
|
2013-01-29 23:40:14 +00:00
|
|
|
if (ret) {
|
|
|
|
/*
|
|
|
|
* our io error hook is going to dec the io pages
|
|
|
|
* again, we have to make sure it has something
|
|
|
|
* to decrement
|
|
|
|
*/
|
|
|
|
atomic_inc(&eb->io_pages);
|
2012-03-13 13:38:00 +00:00
|
|
|
clear_extent_buffer_uptodate(eb);
|
2013-01-29 23:40:14 +00:00
|
|
|
}
|
2012-03-13 13:38:00 +00:00
|
|
|
free_extent_buffer(eb);
|
2008-04-09 20:28:12 +00:00
|
|
|
out:
|
2008-04-09 20:28:12 +00:00
|
|
|
return ret;
|
2008-04-09 20:28:12 +00:00
|
|
|
}
|
|
|
|
|
2012-03-27 01:57:36 +00:00
|
|
|
static int btree_io_failed_hook(struct page *page, int failed_mirror)
|
2011-06-10 11:55:54 +00:00
|
|
|
{
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
|
|
|
|
|
2012-03-07 21:20:05 +00:00
|
|
|
eb = (struct extent_buffer *)page->private;
|
2012-03-27 01:57:36 +00:00
|
|
|
set_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
|
2012-04-16 13:42:26 +00:00
|
|
|
eb->read_mirror = failed_mirror;
|
2013-01-29 23:40:14 +00:00
|
|
|
atomic_dec(&eb->io_pages);
|
2012-03-27 01:57:36 +00:00
|
|
|
if (test_and_clear_bit(EXTENT_BUFFER_READAHEAD, &eb->bflags))
|
2011-06-10 11:55:54 +00:00
|
|
|
btree_readahead_hook(root, eb, eb->start, -EIO);
|
|
|
|
return -EIO; /* we fixed nothing */
|
|
|
|
}
|
|
|
|
|
2008-04-09 20:28:12 +00:00
|
|
|
static void end_workqueue_bio(struct bio *bio, int err)
|
|
|
|
{
|
|
|
|
struct end_io_wq *end_io_wq = bio->bi_private;
|
|
|
|
struct btrfs_fs_info *fs_info;
|
|
|
|
|
|
|
|
fs_info = end_io_wq->info;
|
|
|
|
end_io_wq->error = err;
|
2008-06-11 20:50:36 +00:00
|
|
|
end_io_wq->work.func = end_workqueue_fn;
|
|
|
|
end_io_wq->work.flags = 0;
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
|
2010-08-07 16:20:39 +00:00
|
|
|
if (bio->bi_rw & REQ_WRITE) {
|
2013-01-29 23:40:14 +00:00
|
|
|
if (end_io_wq->metadata == BTRFS_WQ_ENDIO_METADATA)
|
2008-12-17 19:51:42 +00:00
|
|
|
btrfs_queue_worker(&fs_info->endio_meta_write_workers,
|
|
|
|
&end_io_wq->work);
|
2013-01-29 23:40:14 +00:00
|
|
|
else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_FREE_SPACE)
|
2010-07-02 16:14:14 +00:00
|
|
|
btrfs_queue_worker(&fs_info->endio_freespace_worker,
|
|
|
|
&end_io_wq->work);
|
2013-01-29 23:40:14 +00:00
|
|
|
else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_RAID56)
|
|
|
|
btrfs_queue_worker(&fs_info->endio_raid56_workers,
|
|
|
|
&end_io_wq->work);
|
2008-12-17 19:51:42 +00:00
|
|
|
else
|
|
|
|
btrfs_queue_worker(&fs_info->endio_write_workers,
|
|
|
|
&end_io_wq->work);
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
} else {
|
2013-01-29 23:40:14 +00:00
|
|
|
if (end_io_wq->metadata == BTRFS_WQ_ENDIO_RAID56)
|
|
|
|
btrfs_queue_worker(&fs_info->endio_raid56_workers,
|
|
|
|
&end_io_wq->work);
|
|
|
|
else if (end_io_wq->metadata)
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
btrfs_queue_worker(&fs_info->endio_meta_workers,
|
|
|
|
&end_io_wq->work);
|
|
|
|
else
|
|
|
|
btrfs_queue_worker(&fs_info->endio_workers,
|
|
|
|
&end_io_wq->work);
|
|
|
|
}
|
2008-04-09 20:28:12 +00:00
|
|
|
}
|
|
|
|
|
2010-07-02 16:14:14 +00:00
|
|
|
/*
|
|
|
|
* For the metadata arg you want
|
|
|
|
*
|
|
|
|
* 0 - if data
|
|
|
|
* 1 - if normal metadta
|
|
|
|
* 2 - if writing to the free space cache area
|
2013-01-29 23:40:14 +00:00
|
|
|
* 3 - raid parity work
|
2010-07-02 16:14:14 +00:00
|
|
|
*/
|
2008-04-09 20:28:12 +00:00
|
|
|
int btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio,
|
|
|
|
int metadata)
|
2008-03-24 19:01:56 +00:00
|
|
|
{
|
2008-04-09 20:28:12 +00:00
|
|
|
struct end_io_wq *end_io_wq;
|
|
|
|
end_io_wq = kmalloc(sizeof(*end_io_wq), GFP_NOFS);
|
|
|
|
if (!end_io_wq)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
end_io_wq->private = bio->bi_private;
|
|
|
|
end_io_wq->end_io = bio->bi_end_io;
|
2008-04-09 20:28:12 +00:00
|
|
|
end_io_wq->info = info;
|
2008-04-09 20:28:12 +00:00
|
|
|
end_io_wq->error = 0;
|
|
|
|
end_io_wq->bio = bio;
|
2008-04-09 20:28:12 +00:00
|
|
|
end_io_wq->metadata = metadata;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
|
|
|
bio->bi_private = end_io_wq;
|
|
|
|
bio->bi_end_io = end_workqueue_bio;
|
2008-04-09 20:28:12 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-08-20 17:39:41 +00:00
|
|
|
unsigned long btrfs_async_submit_limit(struct btrfs_fs_info *info)
|
2008-08-15 19:34:15 +00:00
|
|
|
{
|
2008-08-15 19:34:17 +00:00
|
|
|
unsigned long limit = min_t(unsigned long,
|
|
|
|
info->workers.max_workers,
|
|
|
|
info->fs_devices->open_devices);
|
|
|
|
return 256 * limit;
|
|
|
|
}
|
2008-08-15 19:34:15 +00:00
|
|
|
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
static void run_one_async_start(struct btrfs_work *work)
|
|
|
|
{
|
|
|
|
struct async_submit_bio *async;
|
2012-03-12 15:03:00 +00:00
|
|
|
int ret;
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
|
|
|
|
async = container_of(work, struct async_submit_bio, work);
|
2012-03-12 15:03:00 +00:00
|
|
|
ret = async->submit_bio_start(async->inode, async->rw, async->bio,
|
|
|
|
async->mirror_num, async->bio_flags,
|
|
|
|
async->bio_offset);
|
|
|
|
if (ret)
|
|
|
|
async->error = ret;
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void run_one_async_done(struct btrfs_work *work)
|
2008-06-11 20:50:36 +00:00
|
|
|
{
|
|
|
|
struct btrfs_fs_info *fs_info;
|
|
|
|
struct async_submit_bio *async;
|
2008-08-15 19:34:17 +00:00
|
|
|
int limit;
|
2008-06-11 20:50:36 +00:00
|
|
|
|
|
|
|
async = container_of(work, struct async_submit_bio, work);
|
|
|
|
fs_info = BTRFS_I(async->inode)->root->fs_info;
|
2008-08-15 19:34:17 +00:00
|
|
|
|
2008-08-20 17:39:41 +00:00
|
|
|
limit = btrfs_async_submit_limit(fs_info);
|
2008-08-15 19:34:17 +00:00
|
|
|
limit = limit * 2 / 3;
|
|
|
|
|
2012-08-01 19:36:24 +00:00
|
|
|
if (atomic_dec_return(&fs_info->nr_async_submits) < limit &&
|
2008-08-20 17:39:41 +00:00
|
|
|
waitqueue_active(&fs_info->async_submit_wait))
|
2008-08-15 19:34:17 +00:00
|
|
|
wake_up(&fs_info->async_submit_wait);
|
|
|
|
|
2012-03-12 15:03:00 +00:00
|
|
|
/* If an error occured we just want to clean up the bio and move on */
|
|
|
|
if (async->error) {
|
|
|
|
bio_endio(async->bio, async->error);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
async->submit_bio_done(async->inode, async->rw, async->bio,
|
2010-05-25 13:48:28 +00:00
|
|
|
async->mirror_num, async->bio_flags,
|
|
|
|
async->bio_offset);
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void run_one_async_free(struct btrfs_work *work)
|
|
|
|
{
|
|
|
|
struct async_submit_bio *async;
|
|
|
|
|
|
|
|
async = container_of(work, struct async_submit_bio, work);
|
2008-06-11 20:50:36 +00:00
|
|
|
kfree(async);
|
|
|
|
}
|
|
|
|
|
2008-04-16 15:14:51 +00:00
|
|
|
int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode,
|
|
|
|
int rw, struct bio *bio, int mirror_num,
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
unsigned long bio_flags,
|
2010-05-25 13:48:28 +00:00
|
|
|
u64 bio_offset,
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
extent_submit_bio_hook_t *submit_bio_start,
|
|
|
|
extent_submit_bio_hook_t *submit_bio_done)
|
2008-04-16 15:14:51 +00:00
|
|
|
{
|
|
|
|
struct async_submit_bio *async;
|
|
|
|
|
|
|
|
async = kmalloc(sizeof(*async), GFP_NOFS);
|
|
|
|
if (!async)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
async->inode = inode;
|
|
|
|
async->rw = rw;
|
|
|
|
async->bio = bio;
|
|
|
|
async->mirror_num = mirror_num;
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
async->submit_bio_start = submit_bio_start;
|
|
|
|
async->submit_bio_done = submit_bio_done;
|
|
|
|
|
|
|
|
async->work.func = run_one_async_start;
|
|
|
|
async->work.ordered_func = run_one_async_done;
|
|
|
|
async->work.ordered_free = run_one_async_free;
|
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
async->work.flags = 0;
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
async->bio_flags = bio_flags;
|
2010-05-25 13:48:28 +00:00
|
|
|
async->bio_offset = bio_offset;
|
2008-09-29 15:19:10 +00:00
|
|
|
|
2012-03-12 15:03:00 +00:00
|
|
|
async->error = 0;
|
|
|
|
|
2008-05-15 20:15:45 +00:00
|
|
|
atomic_inc(&fs_info->nr_async_submits);
|
2009-04-20 19:50:09 +00:00
|
|
|
|
2010-08-07 16:20:39 +00:00
|
|
|
if (rw & REQ_SYNC)
|
2009-04-20 19:50:09 +00:00
|
|
|
btrfs_set_work_high_prio(&async->work);
|
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
btrfs_queue_worker(&fs_info->workers, &async->work);
|
2008-08-28 10:15:24 +00:00
|
|
|
|
2009-01-06 02:25:51 +00:00
|
|
|
while (atomic_read(&fs_info->async_submit_draining) &&
|
2008-11-07 03:02:51 +00:00
|
|
|
atomic_read(&fs_info->nr_async_submits)) {
|
|
|
|
wait_event(fs_info->async_submit_wait,
|
|
|
|
(atomic_read(&fs_info->nr_async_submits) == 0));
|
|
|
|
}
|
|
|
|
|
2008-04-16 15:14:51 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-09-23 17:14:12 +00:00
|
|
|
static int btree_csum_one_bio(struct bio *bio)
|
|
|
|
{
|
|
|
|
struct bio_vec *bvec = bio->bi_io_vec;
|
|
|
|
int bio_index = 0;
|
|
|
|
struct btrfs_root *root;
|
2012-03-12 15:03:00 +00:00
|
|
|
int ret = 0;
|
2008-09-23 17:14:12 +00:00
|
|
|
|
|
|
|
WARN_ON(bio->bi_vcnt <= 0);
|
2009-01-06 02:25:51 +00:00
|
|
|
while (bio_index < bio->bi_vcnt) {
|
2008-09-23 17:14:12 +00:00
|
|
|
root = BTRFS_I(bvec->bv_page->mapping->host)->root;
|
2012-03-12 15:03:00 +00:00
|
|
|
ret = csum_dirty_buffer(root, bvec->bv_page);
|
|
|
|
if (ret)
|
|
|
|
break;
|
2008-09-23 17:14:12 +00:00
|
|
|
bio_index++;
|
|
|
|
bvec++;
|
|
|
|
}
|
2012-03-12 15:03:00 +00:00
|
|
|
return ret;
|
2008-09-23 17:14:12 +00:00
|
|
|
}
|
|
|
|
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
static int __btree_submit_bio_start(struct inode *inode, int rw,
|
|
|
|
struct bio *bio, int mirror_num,
|
2010-05-25 13:48:28 +00:00
|
|
|
unsigned long bio_flags,
|
|
|
|
u64 bio_offset)
|
2008-04-09 20:28:12 +00:00
|
|
|
{
|
2008-06-11 20:50:36 +00:00
|
|
|
/*
|
|
|
|
* when we're called for a write, we're already in the async
|
2008-08-15 19:34:16 +00:00
|
|
|
* submission context. Just jump into btrfs_map_bio
|
2008-06-11 20:50:36 +00:00
|
|
|
*/
|
2012-03-12 15:03:00 +00:00
|
|
|
return btree_csum_one_bio(bio);
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
}
|
2008-04-09 20:28:12 +00:00
|
|
|
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
static int __btree_submit_bio_done(struct inode *inode, int rw, struct bio *bio,
|
2010-05-25 13:48:28 +00:00
|
|
|
int mirror_num, unsigned long bio_flags,
|
|
|
|
u64 bio_offset)
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
{
|
2012-11-05 17:51:52 +00:00
|
|
|
int ret;
|
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
/*
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
* when we're called for a write, we're already in the async
|
|
|
|
* submission context. Just jump into btrfs_map_bio
|
2008-06-11 20:50:36 +00:00
|
|
|
*/
|
2012-11-05 17:51:52 +00:00
|
|
|
ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio, mirror_num, 1);
|
|
|
|
if (ret)
|
|
|
|
bio_endio(bio, ret);
|
|
|
|
return ret;
|
2008-03-24 19:01:56 +00:00
|
|
|
}
|
|
|
|
|
2012-09-25 18:25:58 +00:00
|
|
|
static int check_async_write(struct inode *inode, unsigned long bio_flags)
|
|
|
|
{
|
|
|
|
if (bio_flags & EXTENT_BIO_TREE_LOG)
|
|
|
|
return 0;
|
|
|
|
#ifdef CONFIG_X86
|
|
|
|
if (cpu_has_xmm4_2)
|
|
|
|
return 0;
|
|
|
|
#endif
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
2008-04-16 15:14:51 +00:00
|
|
|
static int btree_submit_bio_hook(struct inode *inode, int rw, struct bio *bio,
|
2010-05-25 13:48:28 +00:00
|
|
|
int mirror_num, unsigned long bio_flags,
|
|
|
|
u64 bio_offset)
|
2008-04-16 15:14:51 +00:00
|
|
|
{
|
2012-09-25 18:25:58 +00:00
|
|
|
int async = check_async_write(inode, bio_flags);
|
2008-12-17 19:51:42 +00:00
|
|
|
int ret;
|
|
|
|
|
2010-08-07 16:20:39 +00:00
|
|
|
if (!(rw & REQ_WRITE)) {
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
/*
|
|
|
|
* called for a read, do the setup so that checksum validation
|
|
|
|
* can happen in the async kernel threads
|
|
|
|
*/
|
2012-03-23 14:22:46 +00:00
|
|
|
ret = btrfs_bio_wq_end_io(BTRFS_I(inode)->root->fs_info,
|
|
|
|
bio, 1);
|
2012-03-29 00:31:37 +00:00
|
|
|
if (ret)
|
2012-11-05 17:51:52 +00:00
|
|
|
goto out_w_error;
|
|
|
|
ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio,
|
|
|
|
mirror_num, 0);
|
2012-09-25 18:25:58 +00:00
|
|
|
} else if (!async) {
|
|
|
|
ret = btree_csum_one_bio(bio);
|
|
|
|
if (ret)
|
2012-11-05 17:51:52 +00:00
|
|
|
goto out_w_error;
|
|
|
|
ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio,
|
|
|
|
mirror_num, 0);
|
|
|
|
} else {
|
|
|
|
/*
|
|
|
|
* kthread helpers are used to submit writes so that
|
|
|
|
* checksumming can happen in parallel across all CPUs
|
|
|
|
*/
|
|
|
|
ret = btrfs_wq_submit_bio(BTRFS_I(inode)->root->fs_info,
|
|
|
|
inode, rw, bio, mirror_num, 0,
|
|
|
|
bio_offset,
|
|
|
|
__btree_submit_bio_start,
|
|
|
|
__btree_submit_bio_done);
|
2008-04-16 15:14:51 +00:00
|
|
|
}
|
2009-04-20 19:50:09 +00:00
|
|
|
|
2012-11-05 17:51:52 +00:00
|
|
|
if (ret) {
|
|
|
|
out_w_error:
|
|
|
|
bio_endio(bio, ret);
|
|
|
|
}
|
|
|
|
return ret;
|
2008-04-16 15:14:51 +00:00
|
|
|
}
|
|
|
|
|
2010-12-07 14:54:09 +00:00
|
|
|
#ifdef CONFIG_MIGRATION
|
2010-11-22 03:20:49 +00:00
|
|
|
static int btree_migratepage(struct address_space *mapping,
|
2012-01-13 01:19:43 +00:00
|
|
|
struct page *newpage, struct page *page,
|
|
|
|
enum migrate_mode mode)
|
2010-11-22 03:20:49 +00:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* we can't safely write a btree page from here,
|
|
|
|
* we haven't done the locking hook
|
|
|
|
*/
|
|
|
|
if (PageDirty(page))
|
|
|
|
return -EAGAIN;
|
|
|
|
/*
|
|
|
|
* Buffers may be managed in a filesystem specific way.
|
|
|
|
* We must have no buffers or drop them.
|
|
|
|
*/
|
|
|
|
if (page_has_private(page) &&
|
|
|
|
!try_to_release_page(page, GFP_KERNEL))
|
|
|
|
return -EAGAIN;
|
2012-01-13 01:19:43 +00:00
|
|
|
return migrate_page(mapping, newpage, page, mode);
|
2010-11-22 03:20:49 +00:00
|
|
|
}
|
2010-12-07 14:54:09 +00:00
|
|
|
#endif
|
2010-11-22 03:20:49 +00:00
|
|
|
|
2007-11-08 02:08:01 +00:00
|
|
|
|
|
|
|
static int btree_writepages(struct address_space *mapping,
|
|
|
|
struct writeback_control *wbc)
|
|
|
|
{
|
2008-01-24 21:13:08 +00:00
|
|
|
struct extent_io_tree *tree;
|
2013-01-29 10:09:20 +00:00
|
|
|
struct btrfs_fs_info *fs_info;
|
|
|
|
int ret;
|
|
|
|
|
2008-01-24 21:13:08 +00:00
|
|
|
tree = &BTRFS_I(mapping->host)->io_tree;
|
2007-12-11 17:42:00 +00:00
|
|
|
if (wbc->sync_mode == WB_SYNC_NONE) {
|
2007-11-27 15:52:01 +00:00
|
|
|
|
|
|
|
if (wbc->for_kupdate)
|
|
|
|
return 0;
|
|
|
|
|
2013-01-29 10:09:20 +00:00
|
|
|
fs_info = BTRFS_I(mapping->host)->root->fs_info;
|
2009-03-13 15:00:37 +00:00
|
|
|
/* this is a bit racy, but that's ok */
|
2013-01-29 10:09:20 +00:00
|
|
|
ret = percpu_counter_compare(&fs_info->dirty_metadata_bytes,
|
|
|
|
BTRFS_DIRTY_METADATA_THRESH);
|
|
|
|
if (ret < 0)
|
2007-11-27 00:34:41 +00:00
|
|
|
return 0;
|
|
|
|
}
|
2012-03-13 13:38:00 +00:00
|
|
|
return btree_write_cache_pages(mapping, wbc);
|
2007-11-08 02:08:01 +00:00
|
|
|
}
|
|
|
|
|
2008-12-02 14:54:17 +00:00
|
|
|
static int btree_readpage(struct file *file, struct page *page)
|
2007-10-15 20:14:19 +00:00
|
|
|
{
|
2008-01-24 21:13:08 +00:00
|
|
|
struct extent_io_tree *tree;
|
|
|
|
tree = &BTRFS_I(page->mapping->host)->io_tree;
|
2011-06-13 18:02:58 +00:00
|
|
|
return extent_read_full_page(tree, page, btree_get_extent, 0);
|
2007-10-15 20:14:19 +00:00
|
|
|
}
|
2007-03-30 12:47:31 +00:00
|
|
|
|
2008-01-29 14:59:12 +00:00
|
|
|
static int btree_releasepage(struct page *page, gfp_t gfp_flags)
|
2007-10-15 20:14:19 +00:00
|
|
|
{
|
2008-09-11 19:51:43 +00:00
|
|
|
if (PageWriteback(page) || PageDirty(page))
|
2009-01-06 02:25:51 +00:00
|
|
|
return 0;
|
2012-01-26 20:01:12 +00:00
|
|
|
/*
|
|
|
|
* We need to mask out eg. __GFP_HIGHMEM and __GFP_DMA32 as we're doing
|
|
|
|
* slab allocation from alloc_extent_state down the callchain where
|
|
|
|
* it'd hit a BUG_ON as those flags are not allowed.
|
|
|
|
*/
|
|
|
|
gfp_flags &= ~GFP_SLAB_BUG_MASK;
|
|
|
|
|
2012-03-09 21:01:49 +00:00
|
|
|
return try_release_extent_buffer(page, gfp_flags);
|
2007-03-28 17:57:48 +00:00
|
|
|
}
|
|
|
|
|
2007-10-15 20:14:19 +00:00
|
|
|
static void btree_invalidatepage(struct page *page, unsigned long offset)
|
2007-03-28 17:57:48 +00:00
|
|
|
{
|
2008-01-24 21:13:08 +00:00
|
|
|
struct extent_io_tree *tree;
|
|
|
|
tree = &BTRFS_I(page->mapping->host)->io_tree;
|
2007-10-15 20:14:19 +00:00
|
|
|
extent_invalidatepage(tree, page, offset);
|
|
|
|
btree_releasepage(page, GFP_NOFS);
|
2008-04-18 20:11:30 +00:00
|
|
|
if (PagePrivate(page)) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_WARNING "btrfs warning page private not zero "
|
|
|
|
"on page %llu\n", (unsigned long long)page_offset(page));
|
2008-04-18 20:11:30 +00:00
|
|
|
ClearPagePrivate(page);
|
|
|
|
set_page_private(page, 0);
|
|
|
|
page_cache_release(page);
|
|
|
|
}
|
2007-03-28 17:57:48 +00:00
|
|
|
}
|
|
|
|
|
2012-03-13 13:38:00 +00:00
|
|
|
static int btree_set_page_dirty(struct page *page)
|
|
|
|
{
|
2012-10-15 17:30:43 +00:00
|
|
|
#ifdef DEBUG
|
2012-03-13 13:38:00 +00:00
|
|
|
struct extent_buffer *eb;
|
|
|
|
|
|
|
|
BUG_ON(!PagePrivate(page));
|
|
|
|
eb = (struct extent_buffer *)page->private;
|
|
|
|
BUG_ON(!eb);
|
|
|
|
BUG_ON(!test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags));
|
|
|
|
BUG_ON(!atomic_read(&eb->refs));
|
|
|
|
btrfs_assert_tree_locked(eb);
|
2012-10-15 17:30:43 +00:00
|
|
|
#endif
|
2012-03-13 13:38:00 +00:00
|
|
|
return __set_page_dirty_nobuffers(page);
|
|
|
|
}
|
|
|
|
|
2009-09-22 00:01:10 +00:00
|
|
|
static const struct address_space_operations btree_aops = {
|
2007-03-28 17:57:48 +00:00
|
|
|
.readpage = btree_readpage,
|
2007-11-08 02:08:01 +00:00
|
|
|
.writepages = btree_writepages,
|
2007-10-15 20:14:19 +00:00
|
|
|
.releasepage = btree_releasepage,
|
|
|
|
.invalidatepage = btree_invalidatepage,
|
2010-11-29 14:49:11 +00:00
|
|
|
#ifdef CONFIG_MIGRATION
|
2010-11-22 03:20:49 +00:00
|
|
|
.migratepage = btree_migratepage,
|
2010-11-29 14:49:11 +00:00
|
|
|
#endif
|
2012-03-13 13:38:00 +00:00
|
|
|
.set_page_dirty = btree_set_page_dirty,
|
2007-03-28 17:57:48 +00:00
|
|
|
};
|
|
|
|
|
2008-05-12 16:59:19 +00:00
|
|
|
int readahead_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
|
|
|
|
u64 parent_transid)
|
2007-05-01 12:53:32 +00:00
|
|
|
{
|
2007-10-15 20:14:19 +00:00
|
|
|
struct extent_buffer *buf = NULL;
|
|
|
|
struct inode *btree_inode = root->fs_info->btree_inode;
|
2007-05-18 17:28:27 +00:00
|
|
|
int ret = 0;
|
2007-05-01 12:53:32 +00:00
|
|
|
|
2007-10-15 20:15:53 +00:00
|
|
|
buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
|
2007-10-15 20:14:19 +00:00
|
|
|
if (!buf)
|
2007-05-01 12:53:32 +00:00
|
|
|
return 0;
|
2008-01-24 21:13:08 +00:00
|
|
|
read_extent_buffer_pages(&BTRFS_I(btree_inode)->io_tree,
|
2011-06-10 12:06:53 +00:00
|
|
|
buf, 0, WAIT_NONE, btree_get_extent, 0);
|
2007-10-15 20:14:19 +00:00
|
|
|
free_extent_buffer(buf);
|
2007-05-18 17:28:27 +00:00
|
|
|
return ret;
|
2007-05-01 12:53:32 +00:00
|
|
|
}
|
|
|
|
|
2011-05-23 12:25:41 +00:00
|
|
|
int reada_tree_block_flagged(struct btrfs_root *root, u64 bytenr, u32 blocksize,
|
|
|
|
int mirror_num, struct extent_buffer **eb)
|
|
|
|
{
|
|
|
|
struct extent_buffer *buf = NULL;
|
|
|
|
struct inode *btree_inode = root->fs_info->btree_inode;
|
|
|
|
struct extent_io_tree *io_tree = &BTRFS_I(btree_inode)->io_tree;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
|
|
|
|
if (!buf)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
set_bit(EXTENT_BUFFER_READAHEAD, &buf->bflags);
|
|
|
|
|
|
|
|
ret = read_extent_buffer_pages(io_tree, buf, 0, WAIT_PAGE_LOCK,
|
|
|
|
btree_get_extent, mirror_num);
|
|
|
|
if (ret) {
|
|
|
|
free_extent_buffer(buf);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (test_bit(EXTENT_BUFFER_CORRUPT, &buf->bflags)) {
|
|
|
|
free_extent_buffer(buf);
|
|
|
|
return -EIO;
|
2012-03-13 13:38:00 +00:00
|
|
|
} else if (extent_buffer_uptodate(buf)) {
|
2011-05-23 12:25:41 +00:00
|
|
|
*eb = buf;
|
|
|
|
} else {
|
|
|
|
free_extent_buffer(buf);
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-04-01 17:48:14 +00:00
|
|
|
struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
|
|
|
|
u64 bytenr, u32 blocksize)
|
|
|
|
{
|
|
|
|
struct inode *btree_inode = root->fs_info->btree_inode;
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
eb = find_extent_buffer(&BTRFS_I(btree_inode)->io_tree,
|
2011-04-20 23:08:01 +00:00
|
|
|
bytenr, blocksize);
|
2008-04-01 17:48:14 +00:00
|
|
|
return eb;
|
|
|
|
}
|
|
|
|
|
|
|
|
struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
|
|
|
|
u64 bytenr, u32 blocksize)
|
|
|
|
{
|
|
|
|
struct inode *btree_inode = root->fs_info->btree_inode;
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
|
|
|
|
eb = alloc_extent_buffer(&BTRFS_I(btree_inode)->io_tree,
|
2010-08-06 17:21:20 +00:00
|
|
|
bytenr, blocksize);
|
2008-04-01 17:48:14 +00:00
|
|
|
return eb;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2008-09-05 20:13:11 +00:00
|
|
|
int btrfs_write_tree_block(struct extent_buffer *buf)
|
|
|
|
{
|
2010-08-06 17:21:20 +00:00
|
|
|
return filemap_fdatawrite_range(buf->pages[0]->mapping, buf->start,
|
2009-10-01 16:58:30 +00:00
|
|
|
buf->start + buf->len - 1);
|
2008-09-05 20:13:11 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
int btrfs_wait_tree_block_writeback(struct extent_buffer *buf)
|
|
|
|
{
|
2010-08-06 17:21:20 +00:00
|
|
|
return filemap_fdatawait_range(buf->pages[0]->mapping,
|
2009-10-01 16:58:30 +00:00
|
|
|
buf->start, buf->start + buf->len - 1);
|
2008-09-05 20:13:11 +00:00
|
|
|
}
|
|
|
|
|
2008-04-01 17:48:14 +00:00
|
|
|
struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
|
2008-05-12 16:59:19 +00:00
|
|
|
u32 blocksize, u64 parent_transid)
|
2008-04-01 17:48:14 +00:00
|
|
|
{
|
|
|
|
struct extent_buffer *buf = NULL;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
|
|
|
|
if (!buf)
|
|
|
|
return NULL;
|
|
|
|
|
2008-05-12 16:59:19 +00:00
|
|
|
ret = btree_read_extent_buffer_pages(root, buf, 0, parent_transid);
|
2007-10-15 20:14:19 +00:00
|
|
|
return buf;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2007-02-02 14:18:22 +00:00
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:27 +00:00
|
|
|
void clean_tree_block(struct btrfs_trans_handle *trans, struct btrfs_root *root,
|
|
|
|
struct extent_buffer *buf)
|
2007-03-01 23:59:40 +00:00
|
|
|
{
|
2013-01-29 10:09:20 +00:00
|
|
|
struct btrfs_fs_info *fs_info = root->fs_info;
|
|
|
|
|
2008-01-09 20:55:33 +00:00
|
|
|
if (btrfs_header_generation(buf) ==
|
2013-01-29 10:09:20 +00:00
|
|
|
fs_info->running_transaction->transid) {
|
2009-03-09 15:45:38 +00:00
|
|
|
btrfs_assert_tree_locked(buf);
|
Btrfs: Change btree locking to use explicit blocking points
Most of the btrfs metadata operations can be protected by a spinlock,
but some operations still need to schedule.
So far, btrfs has been using a mutex along with a trylock loop,
most of the time it is able to avoid going for the full mutex, so
the trylock loop is a big performance gain.
This commit is step one for getting rid of the blocking locks entirely.
btrfs_tree_lock takes a spinlock, and the code explicitly switches
to a blocking lock when it starts an operation that can schedule.
We'll be able get rid of the blocking locks in smaller pieces over time.
Tracing allows us to find the most common cause of blocking, so we
can start with the hot spots first.
The basic idea is:
btrfs_tree_lock() returns with the spin lock held
btrfs_set_lock_blocking() sets the EXTENT_BUFFER_BLOCKING bit in
the extent buffer flags, and then drops the spin lock. The buffer is
still considered locked by all of the btrfs code.
If btrfs_tree_lock gets the spinlock but finds the blocking bit set, it drops
the spin lock and waits on a wait queue for the blocking bit to go away.
Much of the code that needs to set the blocking bit finishes without actually
blocking a good percentage of the time. So, an adaptive spin is still
used against the blocking bit to avoid very high context switch rates.
btrfs_clear_lock_blocking() clears the blocking bit and returns
with the spinlock held again.
btrfs_tree_unlock() can be called on either blocking or spinning locks,
it does the right thing based on the blocking bit.
ctree.c has a helper function to set/clear all the locked buffers in a
path as blocking.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-02-04 14:25:08 +00:00
|
|
|
|
2009-03-13 15:00:37 +00:00
|
|
|
if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &buf->bflags)) {
|
2013-01-29 10:09:20 +00:00
|
|
|
__percpu_counter_add(&fs_info->dirty_metadata_bytes,
|
|
|
|
-buf->len,
|
|
|
|
fs_info->dirty_metadata_batch);
|
2012-10-15 17:33:54 +00:00
|
|
|
/* ugh, clear_extent_buffer_dirty needs to lock the page */
|
|
|
|
btrfs_set_lock_blocking(buf);
|
|
|
|
clear_extent_buffer_dirty(buf);
|
|
|
|
}
|
2008-06-25 20:01:30 +00:00
|
|
|
}
|
2007-10-15 20:14:19 +00:00
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
|
|
|
|
u32 stripesize, struct btrfs_root *root,
|
|
|
|
struct btrfs_fs_info *fs_info,
|
|
|
|
u64 objectid)
|
2007-02-20 21:40:44 +00:00
|
|
|
{
|
2007-02-21 22:04:57 +00:00
|
|
|
root->node = NULL;
|
2007-03-07 01:08:01 +00:00
|
|
|
root->commit_root = NULL;
|
2007-10-15 20:15:53 +00:00
|
|
|
root->sectorsize = sectorsize;
|
|
|
|
root->nodesize = nodesize;
|
|
|
|
root->leafsize = leafsize;
|
2007-11-30 16:30:34 +00:00
|
|
|
root->stripesize = stripesize;
|
2007-03-14 18:14:43 +00:00
|
|
|
root->ref_cows = 0;
|
2008-03-24 19:01:56 +00:00
|
|
|
root->track_dirty = 0;
|
2009-11-12 09:34:40 +00:00
|
|
|
root->in_radix = 0;
|
2010-05-16 14:49:58 +00:00
|
|
|
root->orphan_item_inserted = 0;
|
|
|
|
root->orphan_cleanup_state = 0;
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2007-04-09 14:42:37 +00:00
|
|
|
root->objectid = objectid;
|
|
|
|
root->last_trans = 0;
|
2009-09-21 19:56:00 +00:00
|
|
|
root->highest_objectid = 0;
|
2007-08-29 19:47:34 +00:00
|
|
|
root->name = NULL;
|
2010-02-23 19:43:04 +00:00
|
|
|
root->inode_tree = RB_ROOT;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
INIT_RADIX_TREE(&root->delayed_nodes_tree, GFP_ATOMIC);
|
2010-05-16 14:46:25 +00:00
|
|
|
root->block_rsv = NULL;
|
2010-05-16 14:49:58 +00:00
|
|
|
root->orphan_block_rsv = NULL;
|
2008-03-24 19:01:56 +00:00
|
|
|
|
|
|
|
INIT_LIST_HEAD(&root->dirty_list);
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
INIT_LIST_HEAD(&root->root_list);
|
2012-10-12 19:27:49 +00:00
|
|
|
INIT_LIST_HEAD(&root->logged_list[0]);
|
|
|
|
INIT_LIST_HEAD(&root->logged_list[1]);
|
2010-05-16 14:49:58 +00:00
|
|
|
spin_lock_init(&root->orphan_lock);
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
spin_lock_init(&root->inode_lock);
|
2010-05-16 14:46:25 +00:00
|
|
|
spin_lock_init(&root->accounting_lock);
|
2012-10-12 19:27:49 +00:00
|
|
|
spin_lock_init(&root->log_extents_lock[0]);
|
|
|
|
spin_lock_init(&root->log_extents_lock[1]);
|
2008-06-25 20:01:30 +00:00
|
|
|
mutex_init(&root->objectid_mutex);
|
2008-09-05 20:13:11 +00:00
|
|
|
mutex_init(&root->log_mutex);
|
2009-01-21 17:54:03 +00:00
|
|
|
init_waitqueue_head(&root->log_writer_wait);
|
|
|
|
init_waitqueue_head(&root->log_commit_wait[0]);
|
|
|
|
init_waitqueue_head(&root->log_commit_wait[1]);
|
|
|
|
atomic_set(&root->log_commit[0], 0);
|
|
|
|
atomic_set(&root->log_commit[1], 0);
|
|
|
|
atomic_set(&root->log_writers, 0);
|
2012-09-06 10:04:27 +00:00
|
|
|
atomic_set(&root->log_batch, 0);
|
2012-05-23 18:26:42 +00:00
|
|
|
atomic_set(&root->orphan_inodes, 0);
|
2009-01-21 17:54:03 +00:00
|
|
|
root->log_transid = 0;
|
2009-10-13 17:21:08 +00:00
|
|
|
root->last_log_commit = 0;
|
2008-09-11 20:17:57 +00:00
|
|
|
extent_io_tree_init(&root->dirty_log_pages,
|
2011-04-20 21:35:57 +00:00
|
|
|
fs_info->btree_inode->i_mapping);
|
2008-07-28 19:32:51 +00:00
|
|
|
|
2007-03-13 20:47:54 +00:00
|
|
|
memset(&root->root_key, 0, sizeof(root->root_key));
|
|
|
|
memset(&root->root_item, 0, sizeof(root->root_item));
|
2007-08-07 20:15:09 +00:00
|
|
|
memset(&root->defrag_progress, 0, sizeof(root->defrag_progress));
|
2007-08-29 19:47:34 +00:00
|
|
|
memset(&root->root_kobj, 0, sizeof(root->root_kobj));
|
2008-06-25 20:01:31 +00:00
|
|
|
root->defrag_trans_start = fs_info->generation;
|
2007-08-29 19:47:34 +00:00
|
|
|
init_completion(&root->kobj_unregister);
|
2007-08-07 20:15:09 +00:00
|
|
|
root->defrag_running = 0;
|
2007-04-21 00:23:12 +00:00
|
|
|
root->root_key.objectid = objectid;
|
2011-07-07 19:44:25 +00:00
|
|
|
root->anon_dev = 0;
|
2012-07-25 15:35:53 +00:00
|
|
|
|
2012-12-07 09:28:54 +00:00
|
|
|
spin_lock_init(&root->root_item_lock);
|
2007-03-13 20:47:54 +00:00
|
|
|
}
|
|
|
|
|
2011-10-04 03:22:43 +00:00
|
|
|
static int __must_check find_and_setup_root(struct btrfs_root *tree_root,
|
|
|
|
struct btrfs_fs_info *fs_info,
|
|
|
|
u64 objectid,
|
|
|
|
struct btrfs_root *root)
|
2007-03-13 20:47:54 +00:00
|
|
|
{
|
|
|
|
int ret;
|
2007-10-15 20:15:53 +00:00
|
|
|
u32 blocksize;
|
2008-10-29 18:49:05 +00:00
|
|
|
u64 generation;
|
2007-03-13 20:47:54 +00:00
|
|
|
|
2007-10-15 20:15:53 +00:00
|
|
|
__setup_root(tree_root->nodesize, tree_root->leafsize,
|
2007-11-30 16:30:34 +00:00
|
|
|
tree_root->sectorsize, tree_root->stripesize,
|
|
|
|
root, fs_info, objectid);
|
2007-03-13 20:47:54 +00:00
|
|
|
ret = btrfs_find_last_root(tree_root, objectid,
|
|
|
|
&root->root_item, &root->root_key);
|
2009-09-21 19:56:00 +00:00
|
|
|
if (ret > 0)
|
|
|
|
return -ENOENT;
|
2011-10-04 03:22:43 +00:00
|
|
|
else if (ret < 0)
|
|
|
|
return ret;
|
2007-03-13 20:47:54 +00:00
|
|
|
|
2008-10-29 18:49:05 +00:00
|
|
|
generation = btrfs_root_generation(&root->root_item);
|
2007-10-15 20:15:53 +00:00
|
|
|
blocksize = btrfs_level_size(root, btrfs_root_level(&root->root_item));
|
2011-11-03 19:17:42 +00:00
|
|
|
root->commit_root = NULL;
|
2007-10-15 20:15:53 +00:00
|
|
|
root->node = read_tree_block(root, btrfs_root_bytenr(&root->root_item),
|
2008-10-29 18:49:05 +00:00
|
|
|
blocksize, generation);
|
2012-05-06 11:23:47 +00:00
|
|
|
if (!root->node || !btrfs_buffer_uptodate(root->node, generation, 0)) {
|
2010-12-13 19:47:58 +00:00
|
|
|
free_extent_buffer(root->node);
|
2011-11-03 19:17:42 +00:00
|
|
|
root->node = NULL;
|
2010-12-13 19:47:58 +00:00
|
|
|
return -EIO;
|
|
|
|
}
|
2009-09-21 19:56:00 +00:00
|
|
|
root->commit_root = btrfs_root_node(root);
|
2007-02-20 21:40:44 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2011-11-17 20:57:57 +00:00
|
|
|
static struct btrfs_root *btrfs_alloc_root(struct btrfs_fs_info *fs_info)
|
2011-11-17 05:46:16 +00:00
|
|
|
{
|
|
|
|
struct btrfs_root *root = kzalloc(sizeof(*root), GFP_NOFS);
|
|
|
|
if (root)
|
|
|
|
root->fs_info = fs_info;
|
|
|
|
return root;
|
|
|
|
}
|
|
|
|
|
2011-09-13 10:44:20 +00:00
|
|
|
struct btrfs_root *btrfs_create_tree(struct btrfs_trans_handle *trans,
|
|
|
|
struct btrfs_fs_info *fs_info,
|
|
|
|
u64 objectid)
|
|
|
|
{
|
|
|
|
struct extent_buffer *leaf;
|
|
|
|
struct btrfs_root *tree_root = fs_info->tree_root;
|
|
|
|
struct btrfs_root *root;
|
|
|
|
struct btrfs_key key;
|
|
|
|
int ret = 0;
|
|
|
|
u64 bytenr;
|
|
|
|
|
|
|
|
root = btrfs_alloc_root(fs_info);
|
|
|
|
if (!root)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
|
|
|
__setup_root(tree_root->nodesize, tree_root->leafsize,
|
|
|
|
tree_root->sectorsize, tree_root->stripesize,
|
|
|
|
root, fs_info, objectid);
|
|
|
|
root->root_key.objectid = objectid;
|
|
|
|
root->root_key.type = BTRFS_ROOT_ITEM_KEY;
|
|
|
|
root->root_key.offset = 0;
|
|
|
|
|
|
|
|
leaf = btrfs_alloc_free_block(trans, root, root->leafsize,
|
|
|
|
0, objectid, NULL, 0, 0, 0);
|
|
|
|
if (IS_ERR(leaf)) {
|
|
|
|
ret = PTR_ERR(leaf);
|
|
|
|
goto fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
bytenr = leaf->start;
|
|
|
|
memset_extent_buffer(leaf, 0, 0, sizeof(struct btrfs_header));
|
|
|
|
btrfs_set_header_bytenr(leaf, leaf->start);
|
|
|
|
btrfs_set_header_generation(leaf, trans->transid);
|
|
|
|
btrfs_set_header_backref_rev(leaf, BTRFS_MIXED_BACKREF_REV);
|
|
|
|
btrfs_set_header_owner(leaf, objectid);
|
|
|
|
root->node = leaf;
|
|
|
|
|
|
|
|
write_extent_buffer(leaf, fs_info->fsid,
|
|
|
|
(unsigned long)btrfs_header_fsid(leaf),
|
|
|
|
BTRFS_FSID_SIZE);
|
|
|
|
write_extent_buffer(leaf, fs_info->chunk_tree_uuid,
|
|
|
|
(unsigned long)btrfs_header_chunk_tree_uuid(leaf),
|
|
|
|
BTRFS_UUID_SIZE);
|
|
|
|
btrfs_mark_buffer_dirty(leaf);
|
|
|
|
|
|
|
|
root->commit_root = btrfs_root_node(root);
|
|
|
|
root->track_dirty = 1;
|
|
|
|
|
|
|
|
|
|
|
|
root->root_item.flags = 0;
|
|
|
|
root->root_item.byte_limit = 0;
|
|
|
|
btrfs_set_root_bytenr(&root->root_item, leaf->start);
|
|
|
|
btrfs_set_root_generation(&root->root_item, trans->transid);
|
|
|
|
btrfs_set_root_level(&root->root_item, 0);
|
|
|
|
btrfs_set_root_refs(&root->root_item, 1);
|
|
|
|
btrfs_set_root_used(&root->root_item, leaf->len);
|
|
|
|
btrfs_set_root_last_snapshot(&root->root_item, 0);
|
|
|
|
btrfs_set_root_dirid(&root->root_item, 0);
|
|
|
|
root->root_item.drop_level = 0;
|
|
|
|
|
|
|
|
key.objectid = objectid;
|
|
|
|
key.type = BTRFS_ROOT_ITEM_KEY;
|
|
|
|
key.offset = 0;
|
|
|
|
ret = btrfs_insert_root(trans, tree_root, &key, &root->root_item);
|
|
|
|
if (ret)
|
|
|
|
goto fail;
|
|
|
|
|
|
|
|
btrfs_tree_unlock(leaf);
|
|
|
|
|
|
|
|
fail:
|
|
|
|
if (ret)
|
|
|
|
return ERR_PTR(ret);
|
|
|
|
|
|
|
|
return root;
|
|
|
|
}
|
|
|
|
|
2009-01-21 17:54:03 +00:00
|
|
|
static struct btrfs_root *alloc_log_tree(struct btrfs_trans_handle *trans,
|
|
|
|
struct btrfs_fs_info *fs_info)
|
2007-04-09 14:42:37 +00:00
|
|
|
{
|
|
|
|
struct btrfs_root *root;
|
|
|
|
struct btrfs_root *tree_root = fs_info->tree_root;
|
2009-01-21 17:54:03 +00:00
|
|
|
struct extent_buffer *leaf;
|
2008-09-05 20:13:11 +00:00
|
|
|
|
2011-11-17 05:46:16 +00:00
|
|
|
root = btrfs_alloc_root(fs_info);
|
2008-09-05 20:13:11 +00:00
|
|
|
if (!root)
|
2009-01-21 17:54:03 +00:00
|
|
|
return ERR_PTR(-ENOMEM);
|
2008-09-05 20:13:11 +00:00
|
|
|
|
|
|
|
__setup_root(tree_root->nodesize, tree_root->leafsize,
|
|
|
|
tree_root->sectorsize, tree_root->stripesize,
|
|
|
|
root, fs_info, BTRFS_TREE_LOG_OBJECTID);
|
|
|
|
|
|
|
|
root->root_key.objectid = BTRFS_TREE_LOG_OBJECTID;
|
|
|
|
root->root_key.type = BTRFS_ROOT_ITEM_KEY;
|
|
|
|
root->root_key.offset = BTRFS_TREE_LOG_OBJECTID;
|
2009-01-21 17:54:03 +00:00
|
|
|
/*
|
|
|
|
* log trees do not get reference counted because they go away
|
|
|
|
* before a real commit is actually done. They do store pointers
|
|
|
|
* to file data extents, and those reference counts still get
|
|
|
|
* updated (along with back refs to the log tree).
|
|
|
|
*/
|
2008-09-05 20:13:11 +00:00
|
|
|
root->ref_cows = 0;
|
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
leaf = btrfs_alloc_free_block(trans, root, root->leafsize, 0,
|
2011-09-12 13:26:38 +00:00
|
|
|
BTRFS_TREE_LOG_OBJECTID, NULL,
|
2012-05-16 15:04:52 +00:00
|
|
|
0, 0, 0);
|
2009-01-21 17:54:03 +00:00
|
|
|
if (IS_ERR(leaf)) {
|
|
|
|
kfree(root);
|
|
|
|
return ERR_CAST(leaf);
|
|
|
|
}
|
2008-09-05 20:13:11 +00:00
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
memset_extent_buffer(leaf, 0, 0, sizeof(struct btrfs_header));
|
|
|
|
btrfs_set_header_bytenr(leaf, leaf->start);
|
|
|
|
btrfs_set_header_generation(leaf, trans->transid);
|
|
|
|
btrfs_set_header_backref_rev(leaf, BTRFS_MIXED_BACKREF_REV);
|
|
|
|
btrfs_set_header_owner(leaf, BTRFS_TREE_LOG_OBJECTID);
|
2009-01-21 17:54:03 +00:00
|
|
|
root->node = leaf;
|
2008-09-05 20:13:11 +00:00
|
|
|
|
|
|
|
write_extent_buffer(root->node, root->fs_info->fsid,
|
|
|
|
(unsigned long)btrfs_header_fsid(root->node),
|
|
|
|
BTRFS_FSID_SIZE);
|
|
|
|
btrfs_mark_buffer_dirty(root->node);
|
|
|
|
btrfs_tree_unlock(root->node);
|
2009-01-21 17:54:03 +00:00
|
|
|
return root;
|
|
|
|
}
|
|
|
|
|
|
|
|
int btrfs_init_log_root_tree(struct btrfs_trans_handle *trans,
|
|
|
|
struct btrfs_fs_info *fs_info)
|
|
|
|
{
|
|
|
|
struct btrfs_root *log_root;
|
|
|
|
|
|
|
|
log_root = alloc_log_tree(trans, fs_info);
|
|
|
|
if (IS_ERR(log_root))
|
|
|
|
return PTR_ERR(log_root);
|
|
|
|
WARN_ON(fs_info->log_root_tree);
|
|
|
|
fs_info->log_root_tree = log_root;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
int btrfs_add_log_tree(struct btrfs_trans_handle *trans,
|
|
|
|
struct btrfs_root *root)
|
|
|
|
{
|
|
|
|
struct btrfs_root *log_root;
|
|
|
|
struct btrfs_inode_item *inode_item;
|
|
|
|
|
|
|
|
log_root = alloc_log_tree(trans, root->fs_info);
|
|
|
|
if (IS_ERR(log_root))
|
|
|
|
return PTR_ERR(log_root);
|
|
|
|
|
|
|
|
log_root->last_trans = trans->transid;
|
|
|
|
log_root->root_key.offset = root->root_key.objectid;
|
|
|
|
|
|
|
|
inode_item = &log_root->root_item.inode;
|
|
|
|
inode_item->generation = cpu_to_le64(1);
|
|
|
|
inode_item->size = cpu_to_le64(3);
|
|
|
|
inode_item->nlink = cpu_to_le32(1);
|
|
|
|
inode_item->nbytes = cpu_to_le64(root->leafsize);
|
|
|
|
inode_item->mode = cpu_to_le32(S_IFDIR | 0755);
|
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
btrfs_set_root_node(&log_root->root_item, log_root->node);
|
2009-01-21 17:54:03 +00:00
|
|
|
|
|
|
|
WARN_ON(root->log_root);
|
|
|
|
root->log_root = log_root;
|
|
|
|
root->log_transid = 0;
|
2009-10-13 17:21:08 +00:00
|
|
|
root->last_log_commit = 0;
|
2008-09-05 20:13:11 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
|
|
|
|
struct btrfs_key *location)
|
|
|
|
{
|
|
|
|
struct btrfs_root *root;
|
|
|
|
struct btrfs_fs_info *fs_info = tree_root->fs_info;
|
2007-04-09 14:42:37 +00:00
|
|
|
struct btrfs_path *path;
|
2007-10-15 20:14:19 +00:00
|
|
|
struct extent_buffer *l;
|
2008-10-29 18:49:05 +00:00
|
|
|
u64 generation;
|
2007-10-15 20:15:53 +00:00
|
|
|
u32 blocksize;
|
2007-04-09 14:42:37 +00:00
|
|
|
int ret = 0;
|
2012-07-25 15:35:53 +00:00
|
|
|
int slot;
|
2007-04-09 14:42:37 +00:00
|
|
|
|
2011-11-17 05:46:16 +00:00
|
|
|
root = btrfs_alloc_root(fs_info);
|
2007-06-09 13:22:25 +00:00
|
|
|
if (!root)
|
2007-04-09 14:42:37 +00:00
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
if (location->offset == (u64)-1) {
|
2007-10-15 20:15:53 +00:00
|
|
|
ret = find_and_setup_root(tree_root, fs_info,
|
2007-04-09 14:42:37 +00:00
|
|
|
location->objectid, root);
|
|
|
|
if (ret) {
|
|
|
|
kfree(root);
|
|
|
|
return ERR_PTR(ret);
|
|
|
|
}
|
2009-09-21 19:56:00 +00:00
|
|
|
goto out;
|
2007-04-09 14:42:37 +00:00
|
|
|
}
|
|
|
|
|
2007-10-15 20:15:53 +00:00
|
|
|
__setup_root(tree_root->nodesize, tree_root->leafsize,
|
2007-11-30 16:30:34 +00:00
|
|
|
tree_root->sectorsize, tree_root->stripesize,
|
|
|
|
root, fs_info, location->objectid);
|
2007-04-09 14:42:37 +00:00
|
|
|
|
|
|
|
path = btrfs_alloc_path();
|
2011-03-23 08:14:16 +00:00
|
|
|
if (!path) {
|
|
|
|
kfree(root);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
2007-04-09 14:42:37 +00:00
|
|
|
ret = btrfs_search_slot(NULL, tree_root, location, path, 0, 0);
|
2009-09-21 19:56:00 +00:00
|
|
|
if (ret == 0) {
|
|
|
|
l = path->nodes[0];
|
2012-07-25 15:35:53 +00:00
|
|
|
slot = path->slots[0];
|
|
|
|
btrfs_read_root_item(tree_root, l, slot, &root->root_item);
|
2009-09-21 19:56:00 +00:00
|
|
|
memcpy(&root->root_key, location, sizeof(*location));
|
2007-04-09 14:42:37 +00:00
|
|
|
}
|
|
|
|
btrfs_free_path(path);
|
|
|
|
if (ret) {
|
2010-12-27 06:53:10 +00:00
|
|
|
kfree(root);
|
2009-09-21 19:56:00 +00:00
|
|
|
if (ret > 0)
|
|
|
|
ret = -ENOENT;
|
2007-04-09 14:42:37 +00:00
|
|
|
return ERR_PTR(ret);
|
|
|
|
}
|
2009-09-21 19:56:00 +00:00
|
|
|
|
2008-10-29 18:49:05 +00:00
|
|
|
generation = btrfs_root_generation(&root->root_item);
|
2007-10-15 20:15:53 +00:00
|
|
|
blocksize = btrfs_level_size(root, btrfs_root_level(&root->root_item));
|
|
|
|
root->node = read_tree_block(root, btrfs_root_bytenr(&root->root_item),
|
2008-10-29 18:49:05 +00:00
|
|
|
blocksize, generation);
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
root->commit_root = btrfs_root_node(root);
|
2012-03-12 15:03:00 +00:00
|
|
|
BUG_ON(!root->node); /* -ENOMEM */
|
2009-09-21 19:56:00 +00:00
|
|
|
out:
|
2011-03-28 02:01:25 +00:00
|
|
|
if (location->objectid != BTRFS_TREE_LOG_OBJECTID) {
|
2008-09-05 20:13:11 +00:00
|
|
|
root->ref_cows = 1;
|
2011-03-28 02:01:25 +00:00
|
|
|
btrfs_check_and_init_root_item(&root->root_item);
|
|
|
|
}
|
2009-09-21 19:56:00 +00:00
|
|
|
|
2007-06-22 18:16:25 +00:00
|
|
|
return root;
|
|
|
|
}
|
|
|
|
|
2007-12-21 21:27:24 +00:00
|
|
|
struct btrfs_root *btrfs_read_fs_root_no_name(struct btrfs_fs_info *fs_info,
|
|
|
|
struct btrfs_key *location)
|
2007-06-22 18:16:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_root *root;
|
|
|
|
int ret;
|
|
|
|
|
2007-12-21 21:27:24 +00:00
|
|
|
if (location->objectid == BTRFS_ROOT_TREE_OBJECTID)
|
|
|
|
return fs_info->tree_root;
|
|
|
|
if (location->objectid == BTRFS_EXTENT_TREE_OBJECTID)
|
|
|
|
return fs_info->extent_root;
|
2008-04-25 20:53:30 +00:00
|
|
|
if (location->objectid == BTRFS_CHUNK_TREE_OBJECTID)
|
|
|
|
return fs_info->chunk_root;
|
|
|
|
if (location->objectid == BTRFS_DEV_TREE_OBJECTID)
|
|
|
|
return fs_info->dev_root;
|
2008-12-11 01:32:51 +00:00
|
|
|
if (location->objectid == BTRFS_CSUM_TREE_OBJECTID)
|
|
|
|
return fs_info->csum_root;
|
2011-09-13 13:23:30 +00:00
|
|
|
if (location->objectid == BTRFS_QUOTA_TREE_OBJECTID)
|
|
|
|
return fs_info->quota_root ? fs_info->quota_root :
|
|
|
|
ERR_PTR(-ENOENT);
|
2009-09-21 19:56:00 +00:00
|
|
|
again:
|
|
|
|
spin_lock(&fs_info->fs_roots_radix_lock);
|
2007-06-22 18:16:25 +00:00
|
|
|
root = radix_tree_lookup(&fs_info->fs_roots_radix,
|
|
|
|
(unsigned long)location->objectid);
|
2009-09-21 19:56:00 +00:00
|
|
|
spin_unlock(&fs_info->fs_roots_radix_lock);
|
2007-06-22 18:16:25 +00:00
|
|
|
if (root)
|
|
|
|
return root;
|
|
|
|
|
2008-09-05 20:13:11 +00:00
|
|
|
root = btrfs_read_fs_root_no_radix(fs_info->tree_root, location);
|
2007-06-22 18:16:25 +00:00
|
|
|
if (IS_ERR(root))
|
|
|
|
return root;
|
2008-11-18 01:42:26 +00:00
|
|
|
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
root->free_ino_ctl = kzalloc(sizeof(*root->free_ino_ctl), GFP_NOFS);
|
|
|
|
root->free_ino_pinned = kzalloc(sizeof(*root->free_ino_pinned),
|
|
|
|
GFP_NOFS);
|
2011-06-13 15:18:23 +00:00
|
|
|
if (!root->free_ino_pinned || !root->free_ino_ctl) {
|
|
|
|
ret = -ENOMEM;
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
goto fail;
|
2011-06-13 15:18:23 +00:00
|
|
|
}
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
|
|
|
|
btrfs_init_free_ino_ctl(root);
|
|
|
|
mutex_init(&root->fs_commit_mutex);
|
|
|
|
spin_lock_init(&root->cache_lock);
|
|
|
|
init_waitqueue_head(&root->cache_wait);
|
|
|
|
|
2011-07-07 19:44:25 +00:00
|
|
|
ret = get_anon_bdev(&root->anon_dev);
|
2011-06-13 15:28:50 +00:00
|
|
|
if (ret)
|
|
|
|
goto fail;
|
2008-11-18 01:42:26 +00:00
|
|
|
|
2010-05-16 14:49:58 +00:00
|
|
|
if (btrfs_root_refs(&root->root_item) == 0) {
|
|
|
|
ret = -ENOENT;
|
|
|
|
goto fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
ret = btrfs_find_orphan_item(fs_info->tree_root, location->objectid);
|
|
|
|
if (ret < 0)
|
|
|
|
goto fail;
|
|
|
|
if (ret == 0)
|
|
|
|
root->orphan_item_inserted = 1;
|
|
|
|
|
2009-09-21 19:56:00 +00:00
|
|
|
ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
|
|
|
|
if (ret)
|
|
|
|
goto fail;
|
|
|
|
|
|
|
|
spin_lock(&fs_info->fs_roots_radix_lock);
|
2007-04-10 20:58:11 +00:00
|
|
|
ret = radix_tree_insert(&fs_info->fs_roots_radix,
|
|
|
|
(unsigned long)root->root_key.objectid,
|
2007-04-09 14:42:37 +00:00
|
|
|
root);
|
2010-05-16 14:49:58 +00:00
|
|
|
if (ret == 0)
|
2009-09-21 19:56:00 +00:00
|
|
|
root->in_radix = 1;
|
2010-05-16 14:49:58 +00:00
|
|
|
|
2009-09-21 19:56:00 +00:00
|
|
|
spin_unlock(&fs_info->fs_roots_radix_lock);
|
|
|
|
radix_tree_preload_end();
|
2007-04-09 14:42:37 +00:00
|
|
|
if (ret) {
|
2009-09-21 19:56:00 +00:00
|
|
|
if (ret == -EEXIST) {
|
|
|
|
free_fs_root(root);
|
|
|
|
goto again;
|
|
|
|
}
|
|
|
|
goto fail;
|
2007-04-09 14:42:37 +00:00
|
|
|
}
|
2009-09-21 19:56:00 +00:00
|
|
|
|
|
|
|
ret = btrfs_find_dead_roots(fs_info->tree_root,
|
|
|
|
root->root_key.objectid);
|
|
|
|
WARN_ON(ret);
|
2007-12-21 21:27:24 +00:00
|
|
|
return root;
|
2009-09-21 19:56:00 +00:00
|
|
|
fail:
|
|
|
|
free_fs_root(root);
|
|
|
|
return ERR_PTR(ret);
|
2007-12-21 21:27:24 +00:00
|
|
|
}
|
|
|
|
|
2008-03-26 14:28:07 +00:00
|
|
|
static int btrfs_congested_fn(void *congested_data, int bdi_bits)
|
|
|
|
{
|
|
|
|
struct btrfs_fs_info *info = (struct btrfs_fs_info *)congested_data;
|
|
|
|
int ret = 0;
|
|
|
|
struct btrfs_device *device;
|
|
|
|
struct backing_dev_info *bdi;
|
2009-04-27 11:29:04 +00:00
|
|
|
|
2011-04-20 10:09:16 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
list_for_each_entry_rcu(device, &info->fs_devices->devices, dev_list) {
|
2008-05-13 17:46:40 +00:00
|
|
|
if (!device->bdev)
|
|
|
|
continue;
|
2008-03-26 14:28:07 +00:00
|
|
|
bdi = blk_get_backing_dev_info(device->bdev);
|
|
|
|
if (bdi && bdi_congested(bdi, bdi_bits)) {
|
|
|
|
ret = 1;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
2011-04-20 10:09:16 +00:00
|
|
|
rcu_read_unlock();
|
2008-03-26 14:28:07 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2009-06-12 12:43:40 +00:00
|
|
|
/*
|
|
|
|
* If this fails, caller must call bdi_destroy() to get rid of the
|
|
|
|
* bdi again.
|
|
|
|
*/
|
2008-03-26 14:28:07 +00:00
|
|
|
static int setup_bdi(struct btrfs_fs_info *info, struct backing_dev_info *bdi)
|
|
|
|
{
|
2009-06-12 12:43:40 +00:00
|
|
|
int err;
|
|
|
|
|
|
|
|
bdi->capabilities = BDI_CAP_MAP_COPY;
|
2010-04-26 08:27:54 +00:00
|
|
|
err = bdi_setup_and_register(bdi, "btrfs", BDI_CAP_MAP_COPY);
|
2009-06-12 12:43:40 +00:00
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
2008-04-18 20:13:31 +00:00
|
|
|
bdi->ra_pages = default_backing_dev_info.ra_pages;
|
2008-03-26 14:28:07 +00:00
|
|
|
bdi->congested_fn = btrfs_congested_fn;
|
|
|
|
bdi->congested_data = info;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
/*
|
|
|
|
* called by the kthread helper functions to finally call the bio end_io
|
|
|
|
* functions. This is where read checksum verification actually happens
|
|
|
|
*/
|
|
|
|
static void end_workqueue_fn(struct btrfs_work *work)
|
2008-04-09 20:28:12 +00:00
|
|
|
{
|
|
|
|
struct bio *bio;
|
2008-06-11 20:50:36 +00:00
|
|
|
struct end_io_wq *end_io_wq;
|
|
|
|
struct btrfs_fs_info *fs_info;
|
2008-04-09 20:28:12 +00:00
|
|
|
int error;
|
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
end_io_wq = container_of(work, struct end_io_wq, work);
|
|
|
|
bio = end_io_wq->bio;
|
|
|
|
fs_info = end_io_wq->info;
|
2008-04-09 20:28:12 +00:00
|
|
|
|
2008-06-11 20:50:36 +00:00
|
|
|
error = end_io_wq->error;
|
|
|
|
bio->bi_private = end_io_wq->private;
|
|
|
|
bio->bi_end_io = end_io_wq->end_io;
|
|
|
|
kfree(end_io_wq);
|
|
|
|
bio_endio(bio, error);
|
2008-04-16 15:14:51 +00:00
|
|
|
}
|
|
|
|
|
2008-06-25 20:01:31 +00:00
|
|
|
static int cleaner_kthread(void *arg)
|
|
|
|
{
|
|
|
|
struct btrfs_root *root = arg;
|
|
|
|
|
|
|
|
do {
|
2009-09-21 20:00:26 +00:00
|
|
|
if (!(root->fs_info->sb->s_flags & MS_RDONLY) &&
|
|
|
|
mutex_trylock(&root->fs_info->cleaner_mutex)) {
|
2009-11-12 09:36:34 +00:00
|
|
|
btrfs_run_delayed_iputs(root);
|
2009-09-21 20:00:26 +00:00
|
|
|
btrfs_clean_old_snapshots(root);
|
|
|
|
mutex_unlock(&root->fs_info->cleaner_mutex);
|
2011-05-24 19:35:30 +00:00
|
|
|
btrfs_run_defrag_inodes(root->fs_info);
|
2009-09-21 20:00:26 +00:00
|
|
|
}
|
2008-06-25 20:01:31 +00:00
|
|
|
|
2011-11-21 20:32:22 +00:00
|
|
|
if (!try_to_freeze()) {
|
2008-06-25 20:01:31 +00:00
|
|
|
set_current_state(TASK_INTERRUPTIBLE);
|
2010-05-16 14:49:58 +00:00
|
|
|
if (!kthread_should_stop())
|
|
|
|
schedule();
|
2008-06-25 20:01:31 +00:00
|
|
|
__set_current_state(TASK_RUNNING);
|
|
|
|
}
|
|
|
|
} while (!kthread_should_stop());
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int transaction_kthread(void *arg)
|
|
|
|
{
|
|
|
|
struct btrfs_root *root = arg;
|
|
|
|
struct btrfs_trans_handle *trans;
|
|
|
|
struct btrfs_transaction *cur;
|
2010-05-16 14:49:58 +00:00
|
|
|
u64 transid;
|
2008-06-25 20:01:31 +00:00
|
|
|
unsigned long now;
|
|
|
|
unsigned long delay;
|
2012-03-12 15:05:50 +00:00
|
|
|
bool cannot_commit;
|
2008-06-25 20:01:31 +00:00
|
|
|
|
|
|
|
do {
|
2012-03-12 15:05:50 +00:00
|
|
|
cannot_commit = false;
|
2008-06-25 20:01:31 +00:00
|
|
|
delay = HZ * 30;
|
|
|
|
mutex_lock(&root->fs_info->transaction_kthread_mutex);
|
|
|
|
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_lock(&root->fs_info->trans_lock);
|
2008-06-25 20:01:31 +00:00
|
|
|
cur = root->fs_info->running_transaction;
|
|
|
|
if (!cur) {
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
2008-06-25 20:01:31 +00:00
|
|
|
goto sleep;
|
|
|
|
}
|
2008-07-28 19:32:19 +00:00
|
|
|
|
2008-06-25 20:01:31 +00:00
|
|
|
now = get_seconds();
|
2010-05-16 14:49:58 +00:00
|
|
|
if (!cur->blocked &&
|
|
|
|
(now < cur->start_time || now - cur->start_time < 30)) {
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
2008-06-25 20:01:31 +00:00
|
|
|
delay = HZ * 5;
|
|
|
|
goto sleep;
|
|
|
|
}
|
2010-05-16 14:49:58 +00:00
|
|
|
transid = cur->transid;
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
2009-03-13 14:10:06 +00:00
|
|
|
|
2012-03-12 15:03:00 +00:00
|
|
|
/* If the file system is aborted, this will always fail. */
|
Btrfs: fix orphan transaction on the freezed filesystem
With the following debug patch:
static int btrfs_freeze(struct super_block *sb)
{
+ struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+ struct btrfs_transaction *trans;
+
+ spin_lock(&fs_info->trans_lock);
+ trans = fs_info->running_transaction;
+ if (trans) {
+ printk("Transid %llu, use_count %d, num_writer %d\n",
+ trans->transid, atomic_read(&trans->use_count),
+ atomic_read(&trans->num_writers));
+ }
+ spin_unlock(&fs_info->trans_lock);
return 0;
}
I found there was a orphan transaction after the freeze operation was done.
It is because the transaction may not be committed when the transaction handle
end even though it is the last handle of the current transaction. This design
avoid committing the transaction frequently, but also introduce the above
problem.
So I add btrfs_attach_transaction() which can catch the current transaction
and commit it. If there is no transaction, it will return ENOENT, and do not
anything.
This function also can be used to instead of btrfs_join_transaction_freeze()
because it don't increase the writer counter and don't start a new transaction,
so it also can fix the deadlock between sync and freeze.
Besides that, it is used to instead of btrfs_join_transaction() in
transaction_kthread(), because if there is no transaction, the transaction
kthread needn't anything.
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
2012-09-20 07:54:00 +00:00
|
|
|
trans = btrfs_attach_transaction(root);
|
2012-03-12 15:05:50 +00:00
|
|
|
if (IS_ERR(trans)) {
|
Btrfs: fix orphan transaction on the freezed filesystem
With the following debug patch:
static int btrfs_freeze(struct super_block *sb)
{
+ struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+ struct btrfs_transaction *trans;
+
+ spin_lock(&fs_info->trans_lock);
+ trans = fs_info->running_transaction;
+ if (trans) {
+ printk("Transid %llu, use_count %d, num_writer %d\n",
+ trans->transid, atomic_read(&trans->use_count),
+ atomic_read(&trans->num_writers));
+ }
+ spin_unlock(&fs_info->trans_lock);
return 0;
}
I found there was a orphan transaction after the freeze operation was done.
It is because the transaction may not be committed when the transaction handle
end even though it is the last handle of the current transaction. This design
avoid committing the transaction frequently, but also introduce the above
problem.
So I add btrfs_attach_transaction() which can catch the current transaction
and commit it. If there is no transaction, it will return ENOENT, and do not
anything.
This function also can be used to instead of btrfs_join_transaction_freeze()
because it don't increase the writer counter and don't start a new transaction,
so it also can fix the deadlock between sync and freeze.
Besides that, it is used to instead of btrfs_join_transaction() in
transaction_kthread(), because if there is no transaction, the transaction
kthread needn't anything.
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
2012-09-20 07:54:00 +00:00
|
|
|
if (PTR_ERR(trans) != -ENOENT)
|
|
|
|
cannot_commit = true;
|
2012-03-12 15:03:00 +00:00
|
|
|
goto sleep;
|
2012-03-12 15:05:50 +00:00
|
|
|
}
|
2010-05-16 14:49:58 +00:00
|
|
|
if (transid == trans->transid) {
|
2012-03-12 15:03:00 +00:00
|
|
|
btrfs_commit_transaction(trans, root);
|
2010-05-16 14:49:58 +00:00
|
|
|
} else {
|
|
|
|
btrfs_end_transaction(trans, root);
|
|
|
|
}
|
2008-06-25 20:01:31 +00:00
|
|
|
sleep:
|
|
|
|
wake_up_process(root->fs_info->cleaner_kthread);
|
|
|
|
mutex_unlock(&root->fs_info->transaction_kthread_mutex);
|
|
|
|
|
2011-11-21 20:32:22 +00:00
|
|
|
if (!try_to_freeze()) {
|
2008-06-25 20:01:31 +00:00
|
|
|
set_current_state(TASK_INTERRUPTIBLE);
|
2010-05-16 14:49:58 +00:00
|
|
|
if (!kthread_should_stop() &&
|
2012-03-12 15:05:50 +00:00
|
|
|
(!btrfs_transaction_blocked(root->fs_info) ||
|
|
|
|
cannot_commit))
|
2010-05-16 14:49:58 +00:00
|
|
|
schedule_timeout(delay);
|
2008-06-25 20:01:31 +00:00
|
|
|
__set_current_state(TASK_RUNNING);
|
|
|
|
}
|
|
|
|
} while (!kthread_should_stop());
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2011-11-03 19:17:42 +00:00
|
|
|
/*
|
|
|
|
* this will find the highest generation in the array of
|
|
|
|
* root backups. The index of the highest array is returned,
|
|
|
|
* or -1 if we can't find anything.
|
|
|
|
*
|
|
|
|
* We check to make sure the array is valid by comparing the
|
|
|
|
* generation of the latest root in the array with the generation
|
|
|
|
* in the super block. If they don't match we pitch it.
|
|
|
|
*/
|
|
|
|
static int find_newest_super_backup(struct btrfs_fs_info *info, u64 newest_gen)
|
|
|
|
{
|
|
|
|
u64 cur;
|
|
|
|
int newest_index = -1;
|
|
|
|
struct btrfs_root_backup *root_backup;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < BTRFS_NUM_BACKUP_ROOTS; i++) {
|
|
|
|
root_backup = info->super_copy->super_roots + i;
|
|
|
|
cur = btrfs_backup_tree_root_gen(root_backup);
|
|
|
|
if (cur == newest_gen)
|
|
|
|
newest_index = i;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* check to see if we actually wrapped around */
|
|
|
|
if (newest_index == BTRFS_NUM_BACKUP_ROOTS - 1) {
|
|
|
|
root_backup = info->super_copy->super_roots;
|
|
|
|
cur = btrfs_backup_tree_root_gen(root_backup);
|
|
|
|
if (cur == newest_gen)
|
|
|
|
newest_index = 0;
|
|
|
|
}
|
|
|
|
return newest_index;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* find the oldest backup so we know where to store new entries
|
|
|
|
* in the backup array. This will set the backup_root_index
|
|
|
|
* field in the fs_info struct
|
|
|
|
*/
|
|
|
|
static void find_oldest_super_backup(struct btrfs_fs_info *info,
|
|
|
|
u64 newest_gen)
|
|
|
|
{
|
|
|
|
int newest_index = -1;
|
|
|
|
|
|
|
|
newest_index = find_newest_super_backup(info, newest_gen);
|
|
|
|
/* if there was garbage in there, just move along */
|
|
|
|
if (newest_index == -1) {
|
|
|
|
info->backup_root_index = 0;
|
|
|
|
} else {
|
|
|
|
info->backup_root_index = (newest_index + 1) % BTRFS_NUM_BACKUP_ROOTS;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* copy all the root pointers into the super backup array.
|
|
|
|
* this will bump the backup pointer by one when it is
|
|
|
|
* done
|
|
|
|
*/
|
|
|
|
static void backup_super_roots(struct btrfs_fs_info *info)
|
|
|
|
{
|
|
|
|
int next_backup;
|
|
|
|
struct btrfs_root_backup *root_backup;
|
|
|
|
int last_backup;
|
|
|
|
|
|
|
|
next_backup = info->backup_root_index;
|
|
|
|
last_backup = (next_backup + BTRFS_NUM_BACKUP_ROOTS - 1) %
|
|
|
|
BTRFS_NUM_BACKUP_ROOTS;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* just overwrite the last backup if we're at the same generation
|
|
|
|
* this happens only at umount
|
|
|
|
*/
|
|
|
|
root_backup = info->super_for_commit->super_roots + last_backup;
|
|
|
|
if (btrfs_backup_tree_root_gen(root_backup) ==
|
|
|
|
btrfs_header_generation(info->tree_root->node))
|
|
|
|
next_backup = last_backup;
|
|
|
|
|
|
|
|
root_backup = info->super_for_commit->super_roots + next_backup;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* make sure all of our padding and empty slots get zero filled
|
|
|
|
* regardless of which ones we use today
|
|
|
|
*/
|
|
|
|
memset(root_backup, 0, sizeof(*root_backup));
|
|
|
|
|
|
|
|
info->backup_root_index = (next_backup + 1) % BTRFS_NUM_BACKUP_ROOTS;
|
|
|
|
|
|
|
|
btrfs_set_backup_tree_root(root_backup, info->tree_root->node->start);
|
|
|
|
btrfs_set_backup_tree_root_gen(root_backup,
|
|
|
|
btrfs_header_generation(info->tree_root->node));
|
|
|
|
|
|
|
|
btrfs_set_backup_tree_root_level(root_backup,
|
|
|
|
btrfs_header_level(info->tree_root->node));
|
|
|
|
|
|
|
|
btrfs_set_backup_chunk_root(root_backup, info->chunk_root->node->start);
|
|
|
|
btrfs_set_backup_chunk_root_gen(root_backup,
|
|
|
|
btrfs_header_generation(info->chunk_root->node));
|
|
|
|
btrfs_set_backup_chunk_root_level(root_backup,
|
|
|
|
btrfs_header_level(info->chunk_root->node));
|
|
|
|
|
|
|
|
btrfs_set_backup_extent_root(root_backup, info->extent_root->node->start);
|
|
|
|
btrfs_set_backup_extent_root_gen(root_backup,
|
|
|
|
btrfs_header_generation(info->extent_root->node));
|
|
|
|
btrfs_set_backup_extent_root_level(root_backup,
|
|
|
|
btrfs_header_level(info->extent_root->node));
|
|
|
|
|
2011-11-06 23:50:56 +00:00
|
|
|
/*
|
|
|
|
* we might commit during log recovery, which happens before we set
|
|
|
|
* the fs_root. Make sure it is valid before we fill it in.
|
|
|
|
*/
|
|
|
|
if (info->fs_root && info->fs_root->node) {
|
|
|
|
btrfs_set_backup_fs_root(root_backup,
|
|
|
|
info->fs_root->node->start);
|
|
|
|
btrfs_set_backup_fs_root_gen(root_backup,
|
2011-11-03 19:17:42 +00:00
|
|
|
btrfs_header_generation(info->fs_root->node));
|
2011-11-06 23:50:56 +00:00
|
|
|
btrfs_set_backup_fs_root_level(root_backup,
|
2011-11-03 19:17:42 +00:00
|
|
|
btrfs_header_level(info->fs_root->node));
|
2011-11-06 23:50:56 +00:00
|
|
|
}
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
btrfs_set_backup_dev_root(root_backup, info->dev_root->node->start);
|
|
|
|
btrfs_set_backup_dev_root_gen(root_backup,
|
|
|
|
btrfs_header_generation(info->dev_root->node));
|
|
|
|
btrfs_set_backup_dev_root_level(root_backup,
|
|
|
|
btrfs_header_level(info->dev_root->node));
|
|
|
|
|
|
|
|
btrfs_set_backup_csum_root(root_backup, info->csum_root->node->start);
|
|
|
|
btrfs_set_backup_csum_root_gen(root_backup,
|
|
|
|
btrfs_header_generation(info->csum_root->node));
|
|
|
|
btrfs_set_backup_csum_root_level(root_backup,
|
|
|
|
btrfs_header_level(info->csum_root->node));
|
|
|
|
|
|
|
|
btrfs_set_backup_total_bytes(root_backup,
|
|
|
|
btrfs_super_total_bytes(info->super_copy));
|
|
|
|
btrfs_set_backup_bytes_used(root_backup,
|
|
|
|
btrfs_super_bytes_used(info->super_copy));
|
|
|
|
btrfs_set_backup_num_devices(root_backup,
|
|
|
|
btrfs_super_num_devices(info->super_copy));
|
|
|
|
|
|
|
|
/*
|
|
|
|
* if we don't copy this out to the super_copy, it won't get remembered
|
|
|
|
* for the next commit
|
|
|
|
*/
|
|
|
|
memcpy(&info->super_copy->super_roots,
|
|
|
|
&info->super_for_commit->super_roots,
|
|
|
|
sizeof(*root_backup) * BTRFS_NUM_BACKUP_ROOTS);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* this copies info out of the root backup array and back into
|
|
|
|
* the in-memory super block. It is meant to help iterate through
|
|
|
|
* the array, so you send it the number of backups you've already
|
|
|
|
* tried and the last backup index you used.
|
|
|
|
*
|
|
|
|
* this returns -1 when it has tried all the backups
|
|
|
|
*/
|
|
|
|
static noinline int next_root_backup(struct btrfs_fs_info *info,
|
|
|
|
struct btrfs_super_block *super,
|
|
|
|
int *num_backups_tried, int *backup_index)
|
|
|
|
{
|
|
|
|
struct btrfs_root_backup *root_backup;
|
|
|
|
int newest = *backup_index;
|
|
|
|
|
|
|
|
if (*num_backups_tried == 0) {
|
|
|
|
u64 gen = btrfs_super_generation(super);
|
|
|
|
|
|
|
|
newest = find_newest_super_backup(info, gen);
|
|
|
|
if (newest == -1)
|
|
|
|
return -1;
|
|
|
|
|
|
|
|
*backup_index = newest;
|
|
|
|
*num_backups_tried = 1;
|
|
|
|
} else if (*num_backups_tried == BTRFS_NUM_BACKUP_ROOTS) {
|
|
|
|
/* we've tried all the backups, all done */
|
|
|
|
return -1;
|
|
|
|
} else {
|
|
|
|
/* jump to the next oldest backup */
|
|
|
|
newest = (*backup_index + BTRFS_NUM_BACKUP_ROOTS - 1) %
|
|
|
|
BTRFS_NUM_BACKUP_ROOTS;
|
|
|
|
*backup_index = newest;
|
|
|
|
*num_backups_tried += 1;
|
|
|
|
}
|
|
|
|
root_backup = super->super_roots + newest;
|
|
|
|
|
|
|
|
btrfs_set_super_generation(super,
|
|
|
|
btrfs_backup_tree_root_gen(root_backup));
|
|
|
|
btrfs_set_super_root(super, btrfs_backup_tree_root(root_backup));
|
|
|
|
btrfs_set_super_root_level(super,
|
|
|
|
btrfs_backup_tree_root_level(root_backup));
|
|
|
|
btrfs_set_super_bytes_used(super, btrfs_backup_bytes_used(root_backup));
|
|
|
|
|
|
|
|
/*
|
|
|
|
* fixme: the total bytes and num_devices need to match or we should
|
|
|
|
* need a fsck
|
|
|
|
*/
|
|
|
|
btrfs_set_super_total_bytes(super, btrfs_backup_total_bytes(root_backup));
|
|
|
|
btrfs_set_super_num_devices(super, btrfs_backup_num_devices(root_backup));
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* helper to cleanup tree roots */
|
|
|
|
static void free_root_pointers(struct btrfs_fs_info *info, int chunk_root)
|
|
|
|
{
|
|
|
|
free_extent_buffer(info->tree_root->node);
|
|
|
|
free_extent_buffer(info->tree_root->commit_root);
|
|
|
|
free_extent_buffer(info->dev_root->node);
|
|
|
|
free_extent_buffer(info->dev_root->commit_root);
|
|
|
|
free_extent_buffer(info->extent_root->node);
|
|
|
|
free_extent_buffer(info->extent_root->commit_root);
|
|
|
|
free_extent_buffer(info->csum_root->node);
|
|
|
|
free_extent_buffer(info->csum_root->commit_root);
|
2011-09-13 13:23:30 +00:00
|
|
|
if (info->quota_root) {
|
|
|
|
free_extent_buffer(info->quota_root->node);
|
|
|
|
free_extent_buffer(info->quota_root->commit_root);
|
|
|
|
}
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
info->tree_root->node = NULL;
|
|
|
|
info->tree_root->commit_root = NULL;
|
|
|
|
info->dev_root->node = NULL;
|
|
|
|
info->dev_root->commit_root = NULL;
|
|
|
|
info->extent_root->node = NULL;
|
|
|
|
info->extent_root->commit_root = NULL;
|
|
|
|
info->csum_root->node = NULL;
|
|
|
|
info->csum_root->commit_root = NULL;
|
2011-09-13 13:23:30 +00:00
|
|
|
if (info->quota_root) {
|
|
|
|
info->quota_root->node = NULL;
|
|
|
|
info->quota_root->commit_root = NULL;
|
|
|
|
}
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
if (chunk_root) {
|
|
|
|
free_extent_buffer(info->chunk_root->node);
|
|
|
|
free_extent_buffer(info->chunk_root->commit_root);
|
|
|
|
info->chunk_root->node = NULL;
|
|
|
|
info->chunk_root->commit_root = NULL;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2011-11-17 06:10:02 +00:00
|
|
|
int open_ctree(struct super_block *sb,
|
|
|
|
struct btrfs_fs_devices *fs_devices,
|
|
|
|
char *options)
|
2007-03-21 15:12:56 +00:00
|
|
|
{
|
2007-10-15 20:15:53 +00:00
|
|
|
u32 sectorsize;
|
|
|
|
u32 nodesize;
|
|
|
|
u32 leafsize;
|
|
|
|
u32 blocksize;
|
2007-11-30 16:30:34 +00:00
|
|
|
u32 stripesize;
|
2008-10-29 18:49:05 +00:00
|
|
|
u64 generation;
|
2008-12-02 11:36:08 +00:00
|
|
|
u64 features;
|
2008-11-18 02:02:50 +00:00
|
|
|
struct btrfs_key location;
|
2008-05-07 15:43:44 +00:00
|
|
|
struct buffer_head *bh;
|
2011-11-08 22:08:15 +00:00
|
|
|
struct btrfs_super_block *disk_super;
|
2011-11-17 20:40:49 +00:00
|
|
|
struct btrfs_fs_info *fs_info = btrfs_sb(sb);
|
2011-11-17 20:57:57 +00:00
|
|
|
struct btrfs_root *tree_root;
|
2011-11-08 22:08:15 +00:00
|
|
|
struct btrfs_root *extent_root;
|
|
|
|
struct btrfs_root *csum_root;
|
|
|
|
struct btrfs_root *chunk_root;
|
|
|
|
struct btrfs_root *dev_root;
|
2011-09-13 13:23:30 +00:00
|
|
|
struct btrfs_root *quota_root;
|
2008-09-05 20:13:11 +00:00
|
|
|
struct btrfs_root *log_tree_root;
|
2007-02-02 14:18:22 +00:00
|
|
|
int ret;
|
2008-04-01 15:21:34 +00:00
|
|
|
int err = -EINVAL;
|
2011-11-03 19:17:42 +00:00
|
|
|
int num_backups_tried = 0;
|
|
|
|
int backup_index = 0;
|
2008-06-12 01:47:56 +00:00
|
|
|
|
2011-11-17 20:57:57 +00:00
|
|
|
tree_root = fs_info->tree_root = btrfs_alloc_root(fs_info);
|
2011-11-17 05:46:16 +00:00
|
|
|
extent_root = fs_info->extent_root = btrfs_alloc_root(fs_info);
|
|
|
|
csum_root = fs_info->csum_root = btrfs_alloc_root(fs_info);
|
|
|
|
chunk_root = fs_info->chunk_root = btrfs_alloc_root(fs_info);
|
|
|
|
dev_root = fs_info->dev_root = btrfs_alloc_root(fs_info);
|
2011-09-13 13:23:30 +00:00
|
|
|
quota_root = fs_info->quota_root = btrfs_alloc_root(fs_info);
|
2008-04-03 20:29:03 +00:00
|
|
|
|
2011-11-17 20:57:57 +00:00
|
|
|
if (!tree_root || !extent_root || !csum_root ||
|
2011-09-13 13:23:30 +00:00
|
|
|
!chunk_root || !dev_root || !quota_root) {
|
2007-06-12 10:35:45 +00:00
|
|
|
err = -ENOMEM;
|
|
|
|
goto fail;
|
|
|
|
}
|
2009-09-21 20:00:26 +00:00
|
|
|
|
|
|
|
ret = init_srcu_struct(&fs_info->subvol_srcu);
|
|
|
|
if (ret) {
|
|
|
|
err = ret;
|
|
|
|
goto fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
ret = setup_bdi(fs_info, &fs_info->bdi);
|
|
|
|
if (ret) {
|
|
|
|
err = ret;
|
|
|
|
goto fail_srcu;
|
|
|
|
}
|
|
|
|
|
2013-01-29 10:09:20 +00:00
|
|
|
ret = percpu_counter_init(&fs_info->dirty_metadata_bytes, 0);
|
|
|
|
if (ret) {
|
|
|
|
err = ret;
|
|
|
|
goto fail_bdi;
|
|
|
|
}
|
|
|
|
fs_info->dirty_metadata_batch = PAGE_CACHE_SIZE *
|
|
|
|
(1 + ilog2(nr_cpu_ids));
|
|
|
|
|
2013-01-29 10:10:51 +00:00
|
|
|
ret = percpu_counter_init(&fs_info->delalloc_bytes, 0);
|
|
|
|
if (ret) {
|
|
|
|
err = ret;
|
|
|
|
goto fail_dirty_metadata_bytes;
|
|
|
|
}
|
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
fs_info->btree_inode = new_inode(sb);
|
|
|
|
if (!fs_info->btree_inode) {
|
|
|
|
err = -ENOMEM;
|
2013-01-29 10:10:51 +00:00
|
|
|
goto fail_delalloc_bytes;
|
2009-09-21 20:00:26 +00:00
|
|
|
}
|
|
|
|
|
2011-07-19 16:04:14 +00:00
|
|
|
mapping_set_gfp_mask(fs_info->btree_inode->i_mapping, GFP_NOFS);
|
2011-03-27 08:07:36 +00:00
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
INIT_RADIX_TREE(&fs_info->fs_roots_radix, GFP_ATOMIC);
|
2007-04-20 01:01:03 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->trans_list);
|
2007-06-08 22:11:48 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->dead_roots);
|
2009-11-12 09:36:34 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->delayed_iputs);
|
2008-08-05 03:17:27 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->delalloc_inodes);
|
2009-09-11 20:11:19 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->caching_block_groups);
|
2007-12-21 21:27:21 +00:00
|
|
|
spin_lock_init(&fs_info->delalloc_lock);
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_lock_init(&fs_info->trans_lock);
|
2009-09-21 20:00:26 +00:00
|
|
|
spin_lock_init(&fs_info->fs_roots_radix_lock);
|
2009-11-12 09:36:34 +00:00
|
|
|
spin_lock_init(&fs_info->delayed_iput_lock);
|
2011-05-24 19:35:30 +00:00
|
|
|
spin_lock_init(&fs_info->defrag_inodes_lock);
|
2011-09-26 21:12:22 +00:00
|
|
|
spin_lock_init(&fs_info->free_chunk_lock);
|
2012-05-16 15:55:38 +00:00
|
|
|
spin_lock_init(&fs_info->tree_mod_seq_lock);
|
|
|
|
rwlock_init(&fs_info->tree_mod_log_lock);
|
2011-06-14 00:00:16 +00:00
|
|
|
mutex_init(&fs_info->reloc_mutex);
|
2013-01-29 10:13:12 +00:00
|
|
|
seqlock_init(&fs_info->profiles_lock);
|
2007-10-15 20:19:22 +00:00
|
|
|
|
2007-08-29 19:47:34 +00:00
|
|
|
init_completion(&fs_info->kobj_unregister);
|
2008-03-24 19:01:56 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->dirty_cowonly_roots);
|
2008-03-24 19:01:59 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->space_info);
|
2012-05-16 15:55:38 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->tree_mod_seq_list);
|
2008-03-24 19:01:56 +00:00
|
|
|
btrfs_mapping_init(&fs_info->mapping_tree);
|
2012-09-06 10:02:28 +00:00
|
|
|
btrfs_init_block_rsv(&fs_info->global_block_rsv,
|
|
|
|
BTRFS_BLOCK_RSV_GLOBAL);
|
|
|
|
btrfs_init_block_rsv(&fs_info->delalloc_block_rsv,
|
|
|
|
BTRFS_BLOCK_RSV_DELALLOC);
|
|
|
|
btrfs_init_block_rsv(&fs_info->trans_block_rsv, BTRFS_BLOCK_RSV_TRANS);
|
|
|
|
btrfs_init_block_rsv(&fs_info->chunk_block_rsv, BTRFS_BLOCK_RSV_CHUNK);
|
|
|
|
btrfs_init_block_rsv(&fs_info->empty_block_rsv, BTRFS_BLOCK_RSV_EMPTY);
|
|
|
|
btrfs_init_block_rsv(&fs_info->delayed_block_rsv,
|
|
|
|
BTRFS_BLOCK_RSV_DELOPS);
|
2008-05-15 20:15:45 +00:00
|
|
|
atomic_set(&fs_info->nr_async_submits, 0);
|
2008-11-07 03:02:51 +00:00
|
|
|
atomic_set(&fs_info->async_delalloc_pages, 0);
|
2008-09-29 15:19:10 +00:00
|
|
|
atomic_set(&fs_info->async_submit_draining, 0);
|
2008-08-15 19:34:15 +00:00
|
|
|
atomic_set(&fs_info->nr_async_bios, 0);
|
2011-05-24 19:35:30 +00:00
|
|
|
atomic_set(&fs_info->defrag_running, 0);
|
2012-05-16 15:55:38 +00:00
|
|
|
atomic_set(&fs_info->tree_mod_seq, 0);
|
2007-03-22 16:13:20 +00:00
|
|
|
fs_info->sb = sb;
|
2008-01-29 21:03:38 +00:00
|
|
|
fs_info->max_inline = 8192 * 1024;
|
2009-09-11 20:12:44 +00:00
|
|
|
fs_info->metadata_ratio = 0;
|
2011-05-24 19:35:30 +00:00
|
|
|
fs_info->defrag_inodes = RB_ROOT;
|
2011-04-11 21:25:13 +00:00
|
|
|
fs_info->trans_no_join = 0;
|
2011-09-26 21:12:22 +00:00
|
|
|
fs_info->free_chunk_space = 0;
|
2012-05-16 15:55:38 +00:00
|
|
|
fs_info->tree_mod_log = RB_ROOT;
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
|
2011-05-23 12:30:00 +00:00
|
|
|
/* readahead state */
|
|
|
|
INIT_RADIX_TREE(&fs_info->reada_tree, GFP_NOFS & ~__GFP_WAIT);
|
|
|
|
spin_lock_init(&fs_info->reada_lock);
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
|
2008-12-19 20:43:22 +00:00
|
|
|
fs_info->thread_pool_size = min_t(unsigned long,
|
|
|
|
num_online_cpus() + 2, 8);
|
2008-04-18 18:17:20 +00:00
|
|
|
|
2008-07-24 15:57:52 +00:00
|
|
|
INIT_LIST_HEAD(&fs_info->ordered_extents);
|
|
|
|
spin_lock_init(&fs_info->ordered_extent_lock);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
fs_info->delayed_root = kmalloc(sizeof(struct btrfs_delayed_root),
|
|
|
|
GFP_NOFS);
|
|
|
|
if (!fs_info->delayed_root) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
goto fail_iput;
|
|
|
|
}
|
|
|
|
btrfs_init_delayed_root(fs_info->delayed_root);
|
2008-07-24 15:57:52 +00:00
|
|
|
|
2011-03-08 13:14:00 +00:00
|
|
|
mutex_init(&fs_info->scrub_lock);
|
|
|
|
atomic_set(&fs_info->scrubs_running, 0);
|
|
|
|
atomic_set(&fs_info->scrub_pause_req, 0);
|
|
|
|
atomic_set(&fs_info->scrubs_paused, 0);
|
|
|
|
atomic_set(&fs_info->scrub_cancel_req, 0);
|
|
|
|
init_waitqueue_head(&fs_info->scrub_pause_wait);
|
|
|
|
init_rwsem(&fs_info->scrub_super_lock);
|
|
|
|
fs_info->scrub_workers_refcnt = 0;
|
2011-11-09 12:44:05 +00:00
|
|
|
#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY
|
|
|
|
fs_info->check_integrity_print_mask = 0;
|
|
|
|
#endif
|
2011-03-08 13:14:00 +00:00
|
|
|
|
2012-01-16 20:04:47 +00:00
|
|
|
spin_lock_init(&fs_info->balance_lock);
|
|
|
|
mutex_init(&fs_info->balance_mutex);
|
2012-01-16 20:04:49 +00:00
|
|
|
atomic_set(&fs_info->balance_running, 0);
|
|
|
|
atomic_set(&fs_info->balance_pause_req, 0);
|
2012-01-16 20:04:49 +00:00
|
|
|
atomic_set(&fs_info->balance_cancel_req, 0);
|
2012-01-16 20:04:47 +00:00
|
|
|
fs_info->balance_ctl = NULL;
|
2012-01-16 20:04:49 +00:00
|
|
|
init_waitqueue_head(&fs_info->balance_wait_q);
|
2011-03-08 13:14:00 +00:00
|
|
|
|
2008-05-07 15:43:44 +00:00
|
|
|
sb->s_blocksize = 4096;
|
|
|
|
sb->s_blocksize_bits = blksize_bits(4096);
|
2009-09-16 13:02:33 +00:00
|
|
|
sb->s_bdi = &fs_info->bdi;
|
2008-05-07 15:43:44 +00:00
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
fs_info->btree_inode->i_ino = BTRFS_BTREE_INODE_OBJECTID;
|
2011-10-28 12:13:29 +00:00
|
|
|
set_nlink(fs_info->btree_inode, 1);
|
2008-04-18 18:17:20 +00:00
|
|
|
/*
|
|
|
|
* we set the i_size on the btree inode to the max possible int.
|
|
|
|
* the real end of the address space is determined by all of
|
|
|
|
* the devices in the system
|
|
|
|
*/
|
|
|
|
fs_info->btree_inode->i_size = OFFSET_MAX;
|
2007-03-28 17:57:48 +00:00
|
|
|
fs_info->btree_inode->i_mapping->a_ops = &btree_aops;
|
2008-03-26 14:28:07 +00:00
|
|
|
fs_info->btree_inode->i_mapping->backing_dev_info = &fs_info->bdi;
|
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
RB_CLEAR_NODE(&BTRFS_I(fs_info->btree_inode)->rb_node);
|
2008-01-24 21:13:08 +00:00
|
|
|
extent_io_tree_init(&BTRFS_I(fs_info->btree_inode)->io_tree,
|
2011-04-20 21:35:57 +00:00
|
|
|
fs_info->btree_inode->i_mapping);
|
2012-03-13 13:38:00 +00:00
|
|
|
BTRFS_I(fs_info->btree_inode)->io_tree.track_uptodate = 0;
|
2011-04-20 22:34:43 +00:00
|
|
|
extent_map_tree_init(&BTRFS_I(fs_info->btree_inode)->extent_tree);
|
2008-01-24 21:13:08 +00:00
|
|
|
|
|
|
|
BTRFS_I(fs_info->btree_inode)->io_tree.ops = &btree_extent_io_ops;
|
2007-11-08 02:08:01 +00:00
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
BTRFS_I(fs_info->btree_inode)->root = tree_root;
|
|
|
|
memset(&BTRFS_I(fs_info->btree_inode)->location, 0,
|
|
|
|
sizeof(struct btrfs_key));
|
2012-05-23 18:13:11 +00:00
|
|
|
set_bit(BTRFS_INODE_DUMMY,
|
|
|
|
&BTRFS_I(fs_info->btree_inode)->runtime_flags);
|
2009-09-24 13:24:43 +00:00
|
|
|
insert_inode_hash(fs_info->btree_inode);
|
2009-09-21 20:00:26 +00:00
|
|
|
|
Btrfs: free space accounting redo
1) replace the per fs_info extent_io_tree that tracked free space with two
rb-trees per block group to track free space areas via offset and size. The
reason to do this is because most allocations come with a hint byte where to
start, so we can usually find a chunk of free space at that hint byte to satisfy
the allocation and get good space packing. If we cannot find free space at or
after the given offset we fall back on looking for a chunk of the given size as
close to that given offset as possible. When we fall back on the size search we
also try to find a slot as close to the size we want as possible, to avoid
breaking small chunks off of huge areas if possible.
2) remove the extent_io_tree that tracked the block group cache from fs_info and
replaced it with an rb-tree thats tracks block group cache via offset. also
added a per space_info list that tracks the block group cache for the particular
space so we can lookup related block groups easily.
3) cleaned up the allocation code to make it a little easier to read and a
little less complicated. Basically there are 3 steps, first look from our
provided hint. If we couldn't find from that given hint, start back at our
original search start and look for space from there. If that fails try to
allocate space if we can and start looking again. If not we're screwed and need
to start over again.
4) small fixes. there were some issues in volumes.c where we wouldn't allocate
the rest of the disk. fixed cow_file_range to actually pass the alloc_hint,
which has helped a good bit in making the fs_mark test I run have semi-normal
results as we run out of space. Generally with data allocations we don't track
where we last allocated from, so everytime we did a data allocation we'd search
through every block group that we have looking for free space. Now searching a
block group with no free space isn't terribly time consuming, it was causing a
slight degradation as we got more data block groups. The alloc_hint has fixed
this slight degredation and made things semi-normal.
There is still one nagging problem I'm working on where we will get ENOSPC when
there is definitely plenty of space. This only happens with metadata
allocations, and only when we are almost full. So you generally hit the 85%
mark first, but sometimes you'll hit the BUG before you hit the 85% wall. I'm
still tracking it down, but until then this seems to be pretty stable and make a
significant performance gain.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-09-23 17:14:11 +00:00
|
|
|
spin_lock_init(&fs_info->block_group_cache_lock);
|
2010-02-23 19:43:04 +00:00
|
|
|
fs_info->block_group_cache_tree = RB_ROOT;
|
2012-12-27 09:01:23 +00:00
|
|
|
fs_info->first_logical_byte = (u64)-1;
|
Btrfs: free space accounting redo
1) replace the per fs_info extent_io_tree that tracked free space with two
rb-trees per block group to track free space areas via offset and size. The
reason to do this is because most allocations come with a hint byte where to
start, so we can usually find a chunk of free space at that hint byte to satisfy
the allocation and get good space packing. If we cannot find free space at or
after the given offset we fall back on looking for a chunk of the given size as
close to that given offset as possible. When we fall back on the size search we
also try to find a slot as close to the size we want as possible, to avoid
breaking small chunks off of huge areas if possible.
2) remove the extent_io_tree that tracked the block group cache from fs_info and
replaced it with an rb-tree thats tracks block group cache via offset. also
added a per space_info list that tracks the block group cache for the particular
space so we can lookup related block groups easily.
3) cleaned up the allocation code to make it a little easier to read and a
little less complicated. Basically there are 3 steps, first look from our
provided hint. If we couldn't find from that given hint, start back at our
original search start and look for space from there. If that fails try to
allocate space if we can and start looking again. If not we're screwed and need
to start over again.
4) small fixes. there were some issues in volumes.c where we wouldn't allocate
the rest of the disk. fixed cow_file_range to actually pass the alloc_hint,
which has helped a good bit in making the fs_mark test I run have semi-normal
results as we run out of space. Generally with data allocations we don't track
where we last allocated from, so everytime we did a data allocation we'd search
through every block group that we have looking for free space. Now searching a
block group with no free space isn't terribly time consuming, it was causing a
slight degradation as we got more data block groups. The alloc_hint has fixed
this slight degredation and made things semi-normal.
There is still one nagging problem I'm working on where we will get ENOSPC when
there is definitely plenty of space. This only happens with metadata
allocations, and only when we are almost full. So you generally hit the 85%
mark first, but sometimes you'll hit the BUG before you hit the 85% wall. I'm
still tracking it down, but until then this seems to be pretty stable and make a
significant performance gain.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-09-23 17:14:11 +00:00
|
|
|
|
2009-09-11 20:11:19 +00:00
|
|
|
extent_io_tree_init(&fs_info->freed_extents[0],
|
2011-04-20 21:35:57 +00:00
|
|
|
fs_info->btree_inode->i_mapping);
|
2009-09-11 20:11:19 +00:00
|
|
|
extent_io_tree_init(&fs_info->freed_extents[1],
|
2011-04-20 21:35:57 +00:00
|
|
|
fs_info->btree_inode->i_mapping);
|
2009-09-11 20:11:19 +00:00
|
|
|
fs_info->pinned_extents = &fs_info->freed_extents[0];
|
2007-04-20 17:16:02 +00:00
|
|
|
fs_info->do_barriers = 1;
|
2008-01-18 15:54:22 +00:00
|
|
|
|
2007-06-12 10:35:45 +00:00
|
|
|
|
2009-03-31 17:27:11 +00:00
|
|
|
mutex_init(&fs_info->ordered_operations_mutex);
|
2008-09-05 20:13:11 +00:00
|
|
|
mutex_init(&fs_info->tree_log_mutex);
|
2008-06-25 20:01:30 +00:00
|
|
|
mutex_init(&fs_info->chunk_mutex);
|
2008-06-25 20:01:31 +00:00
|
|
|
mutex_init(&fs_info->transaction_kthread_mutex);
|
|
|
|
mutex_init(&fs_info->cleaner_mutex);
|
2008-07-08 18:19:17 +00:00
|
|
|
mutex_init(&fs_info->volume_mutex);
|
2009-07-30 13:40:40 +00:00
|
|
|
init_rwsem(&fs_info->extent_commit_sem);
|
2009-11-12 09:34:40 +00:00
|
|
|
init_rwsem(&fs_info->cleanup_work_sem);
|
2009-09-21 20:00:26 +00:00
|
|
|
init_rwsem(&fs_info->subvol_sem);
|
2012-11-05 16:26:40 +00:00
|
|
|
fs_info->dev_replace.lock_owner = 0;
|
|
|
|
atomic_set(&fs_info->dev_replace.nesting_level, 0);
|
|
|
|
mutex_init(&fs_info->dev_replace.lock_finishing_cancel_unmount);
|
|
|
|
mutex_init(&fs_info->dev_replace.lock_management_lock);
|
|
|
|
mutex_init(&fs_info->dev_replace.lock);
|
2009-04-03 13:47:43 +00:00
|
|
|
|
2011-09-13 10:56:09 +00:00
|
|
|
spin_lock_init(&fs_info->qgroup_lock);
|
|
|
|
fs_info->qgroup_tree = RB_ROOT;
|
|
|
|
INIT_LIST_HEAD(&fs_info->dirty_qgroups);
|
|
|
|
fs_info->qgroup_seq = 1;
|
|
|
|
fs_info->quota_enabled = 0;
|
|
|
|
fs_info->pending_quota_state = 0;
|
|
|
|
|
2009-04-03 13:47:43 +00:00
|
|
|
btrfs_init_free_cluster(&fs_info->meta_alloc_cluster);
|
|
|
|
btrfs_init_free_cluster(&fs_info->data_alloc_cluster);
|
|
|
|
|
2008-07-17 16:53:50 +00:00
|
|
|
init_waitqueue_head(&fs_info->transaction_throttle);
|
2008-07-17 16:54:14 +00:00
|
|
|
init_waitqueue_head(&fs_info->transaction_wait);
|
2010-10-29 19:37:34 +00:00
|
|
|
init_waitqueue_head(&fs_info->transaction_blocked_wait);
|
2008-08-15 19:34:17 +00:00
|
|
|
init_waitqueue_head(&fs_info->async_submit_wait);
|
2007-03-13 20:47:54 +00:00
|
|
|
|
2013-01-29 23:40:14 +00:00
|
|
|
ret = btrfs_alloc_stripe_hash_table(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
goto fail_alloc;
|
|
|
|
}
|
|
|
|
|
2008-03-24 19:01:56 +00:00
|
|
|
__setup_root(4096, 4096, 4096, 4096, tree_root,
|
2007-04-02 14:50:19 +00:00
|
|
|
fs_info, BTRFS_ROOT_TREE_OBJECTID);
|
2007-04-11 19:53:25 +00:00
|
|
|
|
2012-03-27 22:56:56 +00:00
|
|
|
invalidate_bdev(fs_devices->latest_bdev);
|
2008-12-08 21:46:26 +00:00
|
|
|
bh = btrfs_read_dev_super(fs_devices->latest_bdev);
|
2011-01-08 10:09:13 +00:00
|
|
|
if (!bh) {
|
|
|
|
err = -EINVAL;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
goto fail_alloc;
|
2011-01-08 10:09:13 +00:00
|
|
|
}
|
2007-06-12 10:35:45 +00:00
|
|
|
|
2011-04-13 13:41:04 +00:00
|
|
|
memcpy(fs_info->super_copy, bh->b_data, sizeof(*fs_info->super_copy));
|
|
|
|
memcpy(fs_info->super_for_commit, fs_info->super_copy,
|
|
|
|
sizeof(*fs_info->super_for_commit));
|
2008-05-07 15:43:44 +00:00
|
|
|
brelse(bh);
|
2007-10-15 20:14:19 +00:00
|
|
|
|
2011-04-13 13:41:04 +00:00
|
|
|
memcpy(fs_info->fsid, fs_info->super_copy->fsid, BTRFS_FSID_SIZE);
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2011-04-13 13:41:04 +00:00
|
|
|
disk_super = fs_info->super_copy;
|
2007-04-09 14:42:37 +00:00
|
|
|
if (!btrfs_super_root(disk_super))
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
goto fail_alloc;
|
2007-04-09 14:42:37 +00:00
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
/* check FS state, whether FS is broken. */
|
2013-01-29 10:14:48 +00:00
|
|
|
if (btrfs_super_flags(disk_super) & BTRFS_SUPER_FLAG_ERROR)
|
|
|
|
set_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
2012-03-05 23:06:18 +00:00
|
|
|
ret = btrfs_check_super_valid(fs_info, sb->s_flags & MS_RDONLY);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_ERR "btrfs: superblock contains fatal errors\n");
|
|
|
|
err = ret;
|
|
|
|
goto fail_alloc;
|
|
|
|
}
|
2011-01-06 11:30:25 +00:00
|
|
|
|
2011-11-03 19:17:42 +00:00
|
|
|
/*
|
|
|
|
* run through our array of backup supers and setup
|
|
|
|
* our ring pointer to the oldest one
|
|
|
|
*/
|
|
|
|
generation = btrfs_super_generation(disk_super);
|
|
|
|
find_oldest_super_backup(fs_info, generation);
|
|
|
|
|
Btrfs: Per file/directory controls for COW and compression
Data compression and data cow are controlled across the entire FS by mount
options right now. ioctls are needed to set this on a per file or per
directory basis. This has been proposed previously, but VFS developers
wanted us to use generic ioctls rather than btrfs-specific ones.
According to Chris's comment, there should be just one true compression
method(probably LZO) stored in the super. However, before this, we would
wait for that one method is stable enough to be adopted into the super.
So I list it as a long term goal, and just store it in ram today.
After applying this patch, we can use the generic "FS_IOC_SETFLAGS" ioctl to
control file and directory's datacow and compression attribute.
NOTE:
- The compression type is selected by such rules:
If we mount btrfs with compress options, ie, zlib/lzo, the type is it.
Otherwise, we'll use the default compress type (zlib today).
v1->v2:
- rebase to the latest btrfs.
v2->v3:
- fix a problem, i.e. when a file is set NOCOW via mount option, then this NOCOW
will be screwed by inheritance from parent directory.
Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-03-22 10:12:20 +00:00
|
|
|
/*
|
|
|
|
* In the long term, we'll store the compression type in the super
|
|
|
|
* block, and it'll be used for per file compression control.
|
|
|
|
*/
|
|
|
|
fs_info->compress_type = BTRFS_COMPRESS_ZLIB;
|
|
|
|
|
2008-11-18 02:11:30 +00:00
|
|
|
ret = btrfs_parse_options(tree_root, options);
|
|
|
|
if (ret) {
|
|
|
|
err = ret;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
goto fail_alloc;
|
2008-11-18 02:11:30 +00:00
|
|
|
}
|
2008-05-13 17:46:40 +00:00
|
|
|
|
2008-12-02 11:36:08 +00:00
|
|
|
features = btrfs_super_incompat_flags(disk_super) &
|
|
|
|
~BTRFS_FEATURE_INCOMPAT_SUPP;
|
|
|
|
if (features) {
|
|
|
|
printk(KERN_ERR "BTRFS: couldn't mount because of "
|
|
|
|
"unsupported optional features (%Lx).\n",
|
2009-04-21 19:38:29 +00:00
|
|
|
(unsigned long long)features);
|
2008-12-02 11:36:08 +00:00
|
|
|
err = -EINVAL;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
goto fail_alloc;
|
2008-12-02 11:36:08 +00:00
|
|
|
}
|
|
|
|
|
2010-08-06 17:21:20 +00:00
|
|
|
if (btrfs_super_leafsize(disk_super) !=
|
|
|
|
btrfs_super_nodesize(disk_super)) {
|
|
|
|
printk(KERN_ERR "BTRFS: couldn't mount because metadata "
|
|
|
|
"blocksizes don't match. node %d leaf %d\n",
|
|
|
|
btrfs_super_nodesize(disk_super),
|
|
|
|
btrfs_super_leafsize(disk_super));
|
|
|
|
err = -EINVAL;
|
|
|
|
goto fail_alloc;
|
|
|
|
}
|
|
|
|
if (btrfs_super_leafsize(disk_super) > BTRFS_MAX_METADATA_BLOCKSIZE) {
|
|
|
|
printk(KERN_ERR "BTRFS: couldn't mount because metadata "
|
|
|
|
"blocksize (%d) was too large\n",
|
|
|
|
btrfs_super_leafsize(disk_super));
|
|
|
|
err = -EINVAL;
|
|
|
|
goto fail_alloc;
|
|
|
|
}
|
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
features = btrfs_super_incompat_flags(disk_super);
|
2010-10-25 07:12:26 +00:00
|
|
|
features |= BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF;
|
2012-06-11 09:10:51 +00:00
|
|
|
if (tree_root->fs_info->compress_type == BTRFS_COMPRESS_LZO)
|
2010-10-25 07:12:26 +00:00
|
|
|
features |= BTRFS_FEATURE_INCOMPAT_COMPRESS_LZO;
|
2010-08-06 17:21:20 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* flag our filesystem as having big metadata blocks if
|
|
|
|
* they are bigger than the page size
|
|
|
|
*/
|
|
|
|
if (btrfs_super_leafsize(disk_super) > PAGE_CACHE_SIZE) {
|
|
|
|
if (!(features & BTRFS_FEATURE_INCOMPAT_BIG_METADATA))
|
|
|
|
printk(KERN_INFO "btrfs flagging fs with big metadata feature\n");
|
|
|
|
features |= BTRFS_FEATURE_INCOMPAT_BIG_METADATA;
|
|
|
|
}
|
|
|
|
|
2012-03-29 21:02:47 +00:00
|
|
|
nodesize = btrfs_super_nodesize(disk_super);
|
|
|
|
leafsize = btrfs_super_leafsize(disk_super);
|
|
|
|
sectorsize = btrfs_super_sectorsize(disk_super);
|
|
|
|
stripesize = btrfs_super_stripesize(disk_super);
|
2013-01-29 10:09:20 +00:00
|
|
|
fs_info->dirty_metadata_batch = leafsize * (1 + ilog2(nr_cpu_ids));
|
2013-01-29 10:10:51 +00:00
|
|
|
fs_info->delalloc_batch = sectorsize * 512 * (1 + ilog2(nr_cpu_ids));
|
2012-03-29 21:02:47 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* mixed block groups end up with duplicate but slightly offset
|
|
|
|
* extent buffers for the same range. It leads to corruptions
|
|
|
|
*/
|
|
|
|
if ((features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS) &&
|
|
|
|
(sectorsize != leafsize)) {
|
|
|
|
printk(KERN_WARNING "btrfs: unequal leaf/node/sector sizes "
|
|
|
|
"are not allowed for mixed block groups on %s\n",
|
|
|
|
sb->s_id);
|
|
|
|
goto fail_alloc;
|
|
|
|
}
|
|
|
|
|
2010-10-25 07:12:26 +00:00
|
|
|
btrfs_set_super_incompat_flags(disk_super, features);
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
|
2008-12-02 11:36:08 +00:00
|
|
|
features = btrfs_super_compat_ro_flags(disk_super) &
|
|
|
|
~BTRFS_FEATURE_COMPAT_RO_SUPP;
|
|
|
|
if (!(sb->s_flags & MS_RDONLY) && features) {
|
|
|
|
printk(KERN_ERR "BTRFS: couldn't mount RDWR because of "
|
|
|
|
"unsupported option features (%Lx).\n",
|
2009-04-21 19:38:29 +00:00
|
|
|
(unsigned long long)features);
|
2008-12-02 11:36:08 +00:00
|
|
|
err = -EINVAL;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
goto fail_alloc;
|
2008-12-02 11:36:08 +00:00
|
|
|
}
|
2009-10-02 23:11:56 +00:00
|
|
|
|
|
|
|
btrfs_init_workers(&fs_info->generic_worker,
|
|
|
|
"genwork", 1, NULL);
|
|
|
|
|
2008-08-15 19:34:16 +00:00
|
|
|
btrfs_init_workers(&fs_info->workers, "worker",
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
|
2008-11-07 03:02:51 +00:00
|
|
|
btrfs_init_workers(&fs_info->delalloc_workers, "delalloc",
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2008-11-07 03:02:51 +00:00
|
|
|
|
2012-10-25 09:28:04 +00:00
|
|
|
btrfs_init_workers(&fs_info->flush_workers, "flush_delalloc",
|
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
|
|
|
|
2008-08-15 19:34:16 +00:00
|
|
|
btrfs_init_workers(&fs_info->submit_workers, "submit",
|
2008-08-15 19:34:14 +00:00
|
|
|
min_t(u64, fs_devices->num_devices,
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size),
|
|
|
|
&fs_info->generic_worker);
|
2008-07-31 19:42:53 +00:00
|
|
|
|
2011-06-30 18:42:28 +00:00
|
|
|
btrfs_init_workers(&fs_info->caching_workers, "cache",
|
|
|
|
2, &fs_info->generic_worker);
|
|
|
|
|
2008-07-31 19:42:53 +00:00
|
|
|
/* a higher idle thresh on the submit workers makes it much more
|
|
|
|
* likely that bios will be send down in a sane order to the
|
|
|
|
* devices
|
|
|
|
*/
|
|
|
|
fs_info->submit_workers.idle_thresh = 64;
|
2008-08-15 19:34:18 +00:00
|
|
|
|
2008-11-07 03:02:51 +00:00
|
|
|
fs_info->workers.idle_thresh = 16;
|
Btrfs: Add ordered async work queues
Btrfs uses kernel threads to create async work queues for cpu intensive
operations such as checksumming and decompression. These work well,
but they make it difficult to keep IO order intact.
A single writepages call from pdflush or fsync will turn into a number
of bios, and each bio is checksummed in parallel. Once the checksum is
computed, the bio is sent down to the disk, and since we don't control
the order in which the parallel operations happen, they might go down to
the disk in almost any order.
The code deals with this somewhat by having deep work queues for a single
kernel thread, making it very likely that a single thread will process all
the bios for a single inode.
This patch introduces an explicitly ordered work queue. As work structs
are placed into the queue they are put onto the tail of a list. They have
three callbacks:
->func (cpu intensive processing here)
->ordered_func (order sensitive processing here)
->ordered_free (free the work struct, all processing is done)
The work struct has three callbacks. The func callback does the cpu intensive
work, and when it completes the work struct is marked as done.
Every time a work struct completes, the list is checked to see if the head
is marked as done. If so the ordered_func callback is used to do the
order sensitive processing and the ordered_free callback is used to do
any cleanup. Then we loop back and check the head of the list again.
This patch also changes the checksumming code to use the ordered workqueues.
One a 4 drive array, it increases streaming writes from 280MB/s to 350MB/s.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-11-07 03:03:00 +00:00
|
|
|
fs_info->workers.ordered = 1;
|
2008-07-31 19:42:53 +00:00
|
|
|
|
2008-11-07 03:02:51 +00:00
|
|
|
fs_info->delalloc_workers.idle_thresh = 2;
|
|
|
|
fs_info->delalloc_workers.ordered = 1;
|
|
|
|
|
2009-10-02 23:11:56 +00:00
|
|
|
btrfs_init_workers(&fs_info->fixup_workers, "fixup", 1,
|
|
|
|
&fs_info->generic_worker);
|
2008-08-15 19:34:16 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_workers, "endio",
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_meta_workers, "endio-meta",
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2008-12-17 19:51:42 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_meta_write_workers,
|
2009-10-02 23:11:56 +00:00
|
|
|
"endio-meta-write", fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2013-01-29 23:40:14 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_raid56_workers,
|
|
|
|
"endio-raid56", fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
|
|
|
btrfs_init_workers(&fs_info->rmw_workers,
|
|
|
|
"rmw", fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2008-08-15 19:34:16 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_write_workers, "endio-write",
|
2009-10-02 23:11:56 +00:00
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2010-07-02 16:14:14 +00:00
|
|
|
btrfs_init_workers(&fs_info->endio_freespace_worker, "freespace-write",
|
|
|
|
1, &fs_info->generic_worker);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
btrfs_init_workers(&fs_info->delayed_workers, "delayed-meta",
|
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2011-05-23 12:30:00 +00:00
|
|
|
btrfs_init_workers(&fs_info->readahead_workers, "readahead",
|
|
|
|
fs_info->thread_pool_size,
|
|
|
|
&fs_info->generic_worker);
|
2008-07-31 19:42:53 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* endios are largely parallel and should have a very
|
|
|
|
* low idle thresh
|
|
|
|
*/
|
|
|
|
fs_info->endio_workers.idle_thresh = 4;
|
2009-02-04 14:23:24 +00:00
|
|
|
fs_info->endio_meta_workers.idle_thresh = 4;
|
2013-01-29 23:40:14 +00:00
|
|
|
fs_info->endio_raid56_workers.idle_thresh = 4;
|
|
|
|
fs_info->rmw_workers.idle_thresh = 2;
|
2009-02-04 14:23:24 +00:00
|
|
|
|
2009-08-04 20:56:34 +00:00
|
|
|
fs_info->endio_write_workers.idle_thresh = 2;
|
|
|
|
fs_info->endio_meta_write_workers.idle_thresh = 2;
|
2011-05-23 12:30:00 +00:00
|
|
|
fs_info->readahead_workers.idle_thresh = 2;
|
2009-08-04 20:56:34 +00:00
|
|
|
|
2011-11-18 19:37:27 +00:00
|
|
|
/*
|
|
|
|
* btrfs_start_workers can really only fail because of ENOMEM so just
|
|
|
|
* return -ENOMEM if any of these fail.
|
|
|
|
*/
|
|
|
|
ret = btrfs_start_workers(&fs_info->workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->generic_worker);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->submit_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->delalloc_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->fixup_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_meta_workers);
|
2013-01-29 23:40:14 +00:00
|
|
|
ret |= btrfs_start_workers(&fs_info->rmw_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_raid56_workers);
|
2011-11-18 19:37:27 +00:00
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_meta_write_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_write_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->endio_freespace_worker);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->delayed_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->caching_workers);
|
|
|
|
ret |= btrfs_start_workers(&fs_info->readahead_workers);
|
2012-10-25 09:28:04 +00:00
|
|
|
ret |= btrfs_start_workers(&fs_info->flush_workers);
|
2011-11-18 19:37:27 +00:00
|
|
|
if (ret) {
|
2012-06-22 18:13:01 +00:00
|
|
|
err = -ENOMEM;
|
2011-11-18 19:37:27 +00:00
|
|
|
goto fail_sb_buffer;
|
|
|
|
}
|
2008-06-12 01:47:56 +00:00
|
|
|
|
2008-04-18 20:13:31 +00:00
|
|
|
fs_info->bdi.ra_pages *= btrfs_super_num_devices(disk_super);
|
Btrfs: Add zlib compression support
This is a large change for adding compression on reading and writing,
both for inline and regular extents. It does some fairly large
surgery to the writeback paths.
Compression is off by default and enabled by mount -o compress. Even
when the -o compress mount option is not used, it is possible to read
compressed extents off the disk.
If compression for a given set of pages fails to make them smaller, the
file is flagged to avoid future compression attempts later.
* While finding delalloc extents, the pages are locked before being sent down
to the delalloc handler. This allows the delalloc handler to do complex things
such as cleaning the pages, marking them writeback and starting IO on their
behalf.
* Inline extents are inserted at delalloc time now. This allows us to compress
the data before inserting the inline extent, and it allows us to insert
an inline extent that spans multiple pages.
* All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
are changed to record both an in-memory size and an on disk size, as well
as a flag for compression.
From a disk format point of view, the extent pointers in the file are changed
to record the on disk size of a given extent and some encoding flags.
Space in the disk format is allocated for compression encoding, as well
as encryption and a generic 'other' field. Neither the encryption or the
'other' field are currently used.
In order to limit the amount of data read for a single random read in the
file, the size of a compressed extent is limited to 128k. This is a
software only limit, the disk format supports u64 sized compressed extents.
In order to limit the ram consumed while processing extents, the uncompressed
size of a compressed extent is limited to 256k. This is a software only limit
and will be subject to tuning later.
Checksumming is still done on compressed extents, and it is done on the
uncompressed version of the data. This way additional encodings can be
layered on without having to figure out which encoding to checksum.
Compression happens at delalloc time, which is basically singled threaded because
it is usually done by a single pdflush thread. This makes it tricky to
spread the compression load across all the cpus on the box. We'll have to
look at parallel pdflush walks of dirty inodes at a later time.
Decompression is hooked into readpages and it does spread across CPUs nicely.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 18:49:59 +00:00
|
|
|
fs_info->bdi.ra_pages = max(fs_info->bdi.ra_pages,
|
|
|
|
4 * 1024 * 1024 / PAGE_CACHE_SIZE);
|
2008-04-18 20:13:31 +00:00
|
|
|
|
2007-10-15 20:15:53 +00:00
|
|
|
tree_root->nodesize = nodesize;
|
|
|
|
tree_root->leafsize = leafsize;
|
|
|
|
tree_root->sectorsize = sectorsize;
|
2007-11-30 16:30:34 +00:00
|
|
|
tree_root->stripesize = stripesize;
|
2008-05-07 15:43:44 +00:00
|
|
|
|
|
|
|
sb->s_blocksize = sectorsize;
|
|
|
|
sb->s_blocksize_bits = blksize_bits(sectorsize);
|
2007-10-15 20:15:53 +00:00
|
|
|
|
2013-02-20 00:55:13 +00:00
|
|
|
if (disk_super->magic != cpu_to_le64(BTRFS_MAGIC)) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_INFO "btrfs: valid FS not found on %s\n", sb->s_id);
|
2007-06-12 10:35:45 +00:00
|
|
|
goto fail_sb_buffer;
|
|
|
|
}
|
2007-10-15 20:19:22 +00:00
|
|
|
|
2012-04-03 01:56:53 +00:00
|
|
|
if (sectorsize != PAGE_SIZE) {
|
|
|
|
printk(KERN_WARNING "btrfs: Incompatible sector size(%lu) "
|
|
|
|
"found on %s\n", (unsigned long)sectorsize, sb->s_id);
|
2011-11-30 01:44:12 +00:00
|
|
|
goto fail_sb_buffer;
|
|
|
|
}
|
|
|
|
|
2008-06-25 20:01:30 +00:00
|
|
|
mutex_lock(&fs_info->chunk_mutex);
|
2008-12-12 15:03:26 +00:00
|
|
|
ret = btrfs_read_sys_array(tree_root);
|
2008-06-25 20:01:30 +00:00
|
|
|
mutex_unlock(&fs_info->chunk_mutex);
|
2008-04-25 13:04:37 +00:00
|
|
|
if (ret) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_WARNING "btrfs: failed to read the system "
|
|
|
|
"array on %s\n", sb->s_id);
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
goto fail_sb_buffer;
|
2008-04-25 13:04:37 +00:00
|
|
|
}
|
2008-03-24 19:01:56 +00:00
|
|
|
|
|
|
|
blocksize = btrfs_level_size(tree_root,
|
|
|
|
btrfs_super_chunk_root_level(disk_super));
|
2008-10-29 18:49:05 +00:00
|
|
|
generation = btrfs_super_chunk_root_generation(disk_super);
|
2008-03-24 19:01:56 +00:00
|
|
|
|
|
|
|
__setup_root(nodesize, leafsize, sectorsize, stripesize,
|
|
|
|
chunk_root, fs_info, BTRFS_CHUNK_TREE_OBJECTID);
|
|
|
|
|
|
|
|
chunk_root->node = read_tree_block(chunk_root,
|
|
|
|
btrfs_super_chunk_root(disk_super),
|
2008-10-29 18:49:05 +00:00
|
|
|
blocksize, generation);
|
2012-03-12 15:03:00 +00:00
|
|
|
BUG_ON(!chunk_root->node); /* -ENOMEM */
|
2009-07-22 20:52:13 +00:00
|
|
|
if (!test_bit(EXTENT_BUFFER_UPTODATE, &chunk_root->node->bflags)) {
|
|
|
|
printk(KERN_WARNING "btrfs: failed to read chunk root on %s\n",
|
|
|
|
sb->s_id);
|
2011-11-03 19:17:42 +00:00
|
|
|
goto fail_tree_roots;
|
2009-07-22 20:52:13 +00:00
|
|
|
}
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
btrfs_set_root_node(&chunk_root->root_item, chunk_root->node);
|
|
|
|
chunk_root->commit_root = btrfs_root_node(chunk_root);
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2008-04-15 19:41:47 +00:00
|
|
|
read_extent_buffer(chunk_root->node, fs_info->chunk_tree_uuid,
|
2009-01-06 02:25:51 +00:00
|
|
|
(unsigned long)btrfs_header_chunk_tree_uuid(chunk_root->node),
|
|
|
|
BTRFS_UUID_SIZE);
|
2008-04-15 19:41:47 +00:00
|
|
|
|
2008-03-24 19:01:56 +00:00
|
|
|
ret = btrfs_read_chunk_tree(chunk_root);
|
2008-11-18 02:11:30 +00:00
|
|
|
if (ret) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_WARNING "btrfs: failed to read chunk tree on %s\n",
|
|
|
|
sb->s_id);
|
2011-11-03 19:17:42 +00:00
|
|
|
goto fail_tree_roots;
|
2008-11-18 02:11:30 +00:00
|
|
|
}
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2012-11-06 12:15:27 +00:00
|
|
|
/*
|
|
|
|
* keep the device that is marked to be the target device for the
|
|
|
|
* dev_replace procedure
|
|
|
|
*/
|
|
|
|
btrfs_close_extra_devices(fs_info, fs_devices, 0);
|
2008-05-13 17:46:40 +00:00
|
|
|
|
2012-02-21 01:53:43 +00:00
|
|
|
if (!fs_devices->latest_bdev) {
|
|
|
|
printk(KERN_CRIT "btrfs: failed to read devices on %s\n",
|
|
|
|
sb->s_id);
|
|
|
|
goto fail_tree_roots;
|
|
|
|
}
|
|
|
|
|
2011-11-03 19:17:42 +00:00
|
|
|
retry_root_backup:
|
2007-10-15 20:15:53 +00:00
|
|
|
blocksize = btrfs_level_size(tree_root,
|
|
|
|
btrfs_super_root_level(disk_super));
|
2008-10-29 18:49:05 +00:00
|
|
|
generation = btrfs_super_generation(disk_super);
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2007-03-22 16:13:20 +00:00
|
|
|
tree_root->node = read_tree_block(tree_root,
|
2007-10-15 20:15:53 +00:00
|
|
|
btrfs_super_root(disk_super),
|
2008-10-29 18:49:05 +00:00
|
|
|
blocksize, generation);
|
2011-11-03 19:17:42 +00:00
|
|
|
if (!tree_root->node ||
|
|
|
|
!test_bit(EXTENT_BUFFER_UPTODATE, &tree_root->node->bflags)) {
|
2009-07-22 20:52:13 +00:00
|
|
|
printk(KERN_WARNING "btrfs: failed to read tree root on %s\n",
|
|
|
|
sb->s_id);
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
goto recovery_tree_root;
|
2009-07-22 20:52:13 +00:00
|
|
|
}
|
2011-11-03 19:17:42 +00:00
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
btrfs_set_root_node(&tree_root->root_item, tree_root->node);
|
|
|
|
tree_root->commit_root = btrfs_root_node(tree_root);
|
2007-10-15 20:15:53 +00:00
|
|
|
|
|
|
|
ret = find_and_setup_root(tree_root, fs_info,
|
2007-03-22 16:13:20 +00:00
|
|
|
BTRFS_EXTENT_TREE_OBJECTID, extent_root);
|
2008-03-24 19:01:56 +00:00
|
|
|
if (ret)
|
2011-11-03 19:17:42 +00:00
|
|
|
goto recovery_tree_root;
|
2008-03-24 19:01:56 +00:00
|
|
|
extent_root->track_dirty = 1;
|
|
|
|
|
|
|
|
ret = find_and_setup_root(tree_root, fs_info,
|
|
|
|
BTRFS_DEV_TREE_OBJECTID, dev_root);
|
|
|
|
if (ret)
|
2011-11-03 19:17:42 +00:00
|
|
|
goto recovery_tree_root;
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
dev_root->track_dirty = 1;
|
2007-03-13 20:47:54 +00:00
|
|
|
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
ret = find_and_setup_root(tree_root, fs_info,
|
|
|
|
BTRFS_CSUM_TREE_OBJECTID, csum_root);
|
|
|
|
if (ret)
|
2011-11-03 19:17:42 +00:00
|
|
|
goto recovery_tree_root;
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
csum_root->track_dirty = 1;
|
|
|
|
|
2011-09-13 13:23:30 +00:00
|
|
|
ret = find_and_setup_root(tree_root, fs_info,
|
|
|
|
BTRFS_QUOTA_TREE_OBJECTID, quota_root);
|
|
|
|
if (ret) {
|
|
|
|
kfree(quota_root);
|
|
|
|
quota_root = fs_info->quota_root = NULL;
|
|
|
|
} else {
|
|
|
|
quota_root->track_dirty = 1;
|
|
|
|
fs_info->quota_enabled = 1;
|
|
|
|
fs_info->pending_quota_state = 1;
|
|
|
|
}
|
|
|
|
|
2010-05-16 14:49:58 +00:00
|
|
|
fs_info->generation = generation;
|
|
|
|
fs_info->last_trans_committed = generation;
|
|
|
|
|
2012-06-22 18:24:12 +00:00
|
|
|
ret = btrfs_recover_balance(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_WARNING "btrfs: failed to recover balance\n");
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
|
|
|
|
2012-05-25 14:06:10 +00:00
|
|
|
ret = btrfs_init_dev_stats(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_ERR "btrfs: failed to init dev_stats: %d\n",
|
|
|
|
ret);
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
|
|
|
|
2012-11-06 12:15:27 +00:00
|
|
|
ret = btrfs_init_dev_replace(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
pr_err("btrfs: failed to init dev_replace: %d\n", ret);
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
|
|
|
|
|
|
|
btrfs_close_extra_devices(fs_info, fs_devices, 1);
|
|
|
|
|
2011-03-07 02:13:14 +00:00
|
|
|
ret = btrfs_init_space_info(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_ERR "Failed to initial space info: %d\n", ret);
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
|
|
|
|
2010-03-19 20:49:55 +00:00
|
|
|
ret = btrfs_read_block_groups(extent_root);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_ERR "Failed to read block groups: %d\n", ret);
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
2012-08-01 16:56:49 +00:00
|
|
|
fs_info->num_tolerated_disk_barrier_failures =
|
|
|
|
btrfs_calc_num_tolerated_disk_barrier_failures(fs_info);
|
2012-10-30 17:16:16 +00:00
|
|
|
if (fs_info->fs_devices->missing_devices >
|
|
|
|
fs_info->num_tolerated_disk_barrier_failures &&
|
|
|
|
!(sb->s_flags & MS_RDONLY)) {
|
|
|
|
printk(KERN_WARNING
|
|
|
|
"Btrfs: too many missing devices, writeable mount is not allowed\n");
|
|
|
|
goto fail_block_groups;
|
|
|
|
}
|
2007-04-26 20:46:15 +00:00
|
|
|
|
2008-06-25 20:01:31 +00:00
|
|
|
fs_info->cleaner_kthread = kthread_run(cleaner_kthread, tree_root,
|
|
|
|
"btrfs-cleaner");
|
2009-01-21 15:49:16 +00:00
|
|
|
if (IS_ERR(fs_info->cleaner_kthread))
|
2010-03-19 20:49:55 +00:00
|
|
|
goto fail_block_groups;
|
2008-06-25 20:01:31 +00:00
|
|
|
|
|
|
|
fs_info->transaction_kthread = kthread_run(transaction_kthread,
|
|
|
|
tree_root,
|
|
|
|
"btrfs-transaction");
|
2009-01-21 15:49:16 +00:00
|
|
|
if (IS_ERR(fs_info->transaction_kthread))
|
2008-06-25 20:01:31 +00:00
|
|
|
goto fail_cleaner;
|
2008-06-25 20:01:31 +00:00
|
|
|
|
2009-06-10 13:51:32 +00:00
|
|
|
if (!btrfs_test_opt(tree_root, SSD) &&
|
|
|
|
!btrfs_test_opt(tree_root, NOSSD) &&
|
|
|
|
!fs_info->fs_devices->rotating) {
|
|
|
|
printk(KERN_INFO "Btrfs detected SSD devices, enabling SSD "
|
|
|
|
"mode\n");
|
|
|
|
btrfs_set_opt(fs_info->mount_opt, SSD);
|
|
|
|
}
|
|
|
|
|
2011-11-09 12:44:05 +00:00
|
|
|
#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY
|
|
|
|
if (btrfs_test_opt(tree_root, CHECK_INTEGRITY)) {
|
|
|
|
ret = btrfsic_mount(tree_root, fs_devices,
|
|
|
|
btrfs_test_opt(tree_root,
|
|
|
|
CHECK_INTEGRITY_INCLUDING_EXTENT_DATA) ?
|
|
|
|
1 : 0,
|
|
|
|
fs_info->check_integrity_print_mask);
|
|
|
|
if (ret)
|
|
|
|
printk(KERN_WARNING "btrfs: failed to initialize"
|
|
|
|
" integrity check module %s\n", sb->s_id);
|
|
|
|
}
|
|
|
|
#endif
|
2011-09-13 13:23:30 +00:00
|
|
|
ret = btrfs_read_qgroup_config(fs_info);
|
|
|
|
if (ret)
|
|
|
|
goto fail_trans_kthread;
|
2011-11-09 12:44:05 +00:00
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
/* do not make disk changes in broken FS */
|
2012-08-01 11:45:52 +00:00
|
|
|
if (btrfs_super_log_root(disk_super) != 0) {
|
2008-09-05 20:13:11 +00:00
|
|
|
u64 bytenr = btrfs_super_log_root(disk_super);
|
|
|
|
|
2008-11-19 20:13:35 +00:00
|
|
|
if (fs_devices->rw_devices == 0) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_WARNING "Btrfs log replay required "
|
|
|
|
"on RO media\n");
|
2008-11-19 20:13:35 +00:00
|
|
|
err = -EIO;
|
2011-09-13 13:23:30 +00:00
|
|
|
goto fail_qgroup;
|
2008-11-19 20:13:35 +00:00
|
|
|
}
|
2008-09-05 20:13:11 +00:00
|
|
|
blocksize =
|
|
|
|
btrfs_level_size(tree_root,
|
|
|
|
btrfs_super_log_root_level(disk_super));
|
2008-04-04 19:40:00 +00:00
|
|
|
|
2011-11-17 05:46:16 +00:00
|
|
|
log_tree_root = btrfs_alloc_root(fs_info);
|
2010-05-29 09:43:07 +00:00
|
|
|
if (!log_tree_root) {
|
|
|
|
err = -ENOMEM;
|
2011-09-13 13:23:30 +00:00
|
|
|
goto fail_qgroup;
|
2010-05-29 09:43:07 +00:00
|
|
|
}
|
2008-09-05 20:13:11 +00:00
|
|
|
|
|
|
|
__setup_root(nodesize, leafsize, sectorsize, stripesize,
|
|
|
|
log_tree_root, fs_info, BTRFS_TREE_LOG_OBJECTID);
|
|
|
|
|
|
|
|
log_tree_root->node = read_tree_block(tree_root, bytenr,
|
2008-10-29 18:49:05 +00:00
|
|
|
blocksize,
|
|
|
|
generation + 1);
|
2012-03-12 15:03:00 +00:00
|
|
|
/* returns with log_tree_root freed on success */
|
2008-09-05 20:13:11 +00:00
|
|
|
ret = btrfs_recover_log_trees(log_tree_root);
|
2012-03-12 15:03:00 +00:00
|
|
|
if (ret) {
|
|
|
|
btrfs_error(tree_root->fs_info, ret,
|
|
|
|
"Failed to recover log tree");
|
|
|
|
free_extent_buffer(log_tree_root->node);
|
|
|
|
kfree(log_tree_root);
|
|
|
|
goto fail_trans_kthread;
|
|
|
|
}
|
2008-11-20 15:25:19 +00:00
|
|
|
|
|
|
|
if (sb->s_flags & MS_RDONLY) {
|
2012-03-12 15:03:00 +00:00
|
|
|
ret = btrfs_commit_super(tree_root);
|
|
|
|
if (ret)
|
|
|
|
goto fail_trans_kthread;
|
2008-11-20 15:25:19 +00:00
|
|
|
}
|
2008-09-05 20:13:11 +00:00
|
|
|
}
|
Btrfs: update space balancing code
This patch updates the space balancing code to utilize the new
backref format. Before, btrfs-vol -b would break any COW links
on data blocks or metadata. This was slow and caused the amount
of space used to explode if a large number of snapshots were present.
The new code can keeps the sharing of all data extents and
most of the tree blocks.
To maintain the sharing of data extents, the space balance code uses
a seperate inode hold data extent pointers, then updates the references
to point to the new location.
To maintain the sharing of tree blocks, the space balance code uses
reloc trees to relocate tree blocks in reference counted roots.
There is one reloc tree for each subvol, and all reloc trees share
same root key objectid. Reloc trees are snapshots of the latest
committed roots of subvols (root->commit_root).
To relocate a tree block referenced by a subvol, there are two steps.
COW the block through subvol's reloc tree, then update block pointer in
the subvol to point to the new block. Since all reloc trees share
same root key objectid, doing special handing for tree blocks
owned by them is easy. Once a tree block has been COWed in one
reloc tree, we can use the resulting new block directly when the
same block is required to COW again through other reloc trees.
In this way, relocated tree blocks are shared between reloc trees,
so they are also shared between subvols.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-09-26 14:09:34 +00:00
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
ret = btrfs_find_orphan_roots(tree_root);
|
2012-03-12 15:03:00 +00:00
|
|
|
if (ret)
|
|
|
|
goto fail_trans_kthread;
|
2009-09-21 20:00:26 +00:00
|
|
|
|
2008-11-19 20:13:35 +00:00
|
|
|
if (!(sb->s_flags & MS_RDONLY)) {
|
2010-05-16 14:49:58 +00:00
|
|
|
ret = btrfs_cleanup_fs_roots(fs_info);
|
2012-06-22 18:14:13 +00:00
|
|
|
if (ret)
|
|
|
|
goto fail_trans_kthread;
|
2010-05-16 14:49:58 +00:00
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
ret = btrfs_recover_relocation(tree_root);
|
2010-02-02 08:46:44 +00:00
|
|
|
if (ret < 0) {
|
|
|
|
printk(KERN_WARNING
|
|
|
|
"btrfs: failed to recover relocation\n");
|
|
|
|
err = -EINVAL;
|
2011-09-13 13:23:30 +00:00
|
|
|
goto fail_qgroup;
|
2010-02-02 08:46:44 +00:00
|
|
|
}
|
2008-11-19 20:13:35 +00:00
|
|
|
}
|
Btrfs: update space balancing code
This patch updates the space balancing code to utilize the new
backref format. Before, btrfs-vol -b would break any COW links
on data blocks or metadata. This was slow and caused the amount
of space used to explode if a large number of snapshots were present.
The new code can keeps the sharing of all data extents and
most of the tree blocks.
To maintain the sharing of data extents, the space balance code uses
a seperate inode hold data extent pointers, then updates the references
to point to the new location.
To maintain the sharing of tree blocks, the space balance code uses
reloc trees to relocate tree blocks in reference counted roots.
There is one reloc tree for each subvol, and all reloc trees share
same root key objectid. Reloc trees are snapshots of the latest
committed roots of subvols (root->commit_root).
To relocate a tree block referenced by a subvol, there are two steps.
COW the block through subvol's reloc tree, then update block pointer in
the subvol to point to the new block. Since all reloc trees share
same root key objectid, doing special handing for tree blocks
owned by them is easy. Once a tree block has been COWed in one
reloc tree, we can use the resulting new block directly when the
same block is required to COW again through other reloc trees.
In this way, relocated tree blocks are shared between reloc trees,
so they are also shared between subvols.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-09-26 14:09:34 +00:00
|
|
|
|
2008-11-18 02:02:50 +00:00
|
|
|
location.objectid = BTRFS_FS_TREE_OBJECTID;
|
|
|
|
location.type = BTRFS_ROOT_ITEM_KEY;
|
|
|
|
location.offset = (u64)-1;
|
|
|
|
|
|
|
|
fs_info->fs_root = btrfs_read_fs_root_no_name(fs_info, &location);
|
|
|
|
if (!fs_info->fs_root)
|
2011-09-13 13:23:30 +00:00
|
|
|
goto fail_qgroup;
|
2010-05-29 09:44:10 +00:00
|
|
|
if (IS_ERR(fs_info->fs_root)) {
|
|
|
|
err = PTR_ERR(fs_info->fs_root);
|
2011-09-13 13:23:30 +00:00
|
|
|
goto fail_qgroup;
|
2010-05-29 09:44:10 +00:00
|
|
|
}
|
2009-06-10 13:51:32 +00:00
|
|
|
|
2012-06-22 18:24:13 +00:00
|
|
|
if (sb->s_flags & MS_RDONLY)
|
|
|
|
return 0;
|
2012-01-16 20:04:48 +00:00
|
|
|
|
2012-06-22 18:24:13 +00:00
|
|
|
down_read(&fs_info->cleanup_work_sem);
|
|
|
|
if ((ret = btrfs_orphan_cleanup(fs_info->fs_root)) ||
|
|
|
|
(ret = btrfs_orphan_cleanup(fs_info->tree_root))) {
|
2010-01-26 14:30:53 +00:00
|
|
|
up_read(&fs_info->cleanup_work_sem);
|
2012-06-22 18:24:13 +00:00
|
|
|
close_ctree(tree_root);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
up_read(&fs_info->cleanup_work_sem);
|
2012-01-16 20:04:48 +00:00
|
|
|
|
2012-06-22 18:24:13 +00:00
|
|
|
ret = btrfs_resume_balance_async(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_WARNING "btrfs: failed to resume balance\n");
|
|
|
|
close_ctree(tree_root);
|
|
|
|
return ret;
|
2010-01-26 14:30:53 +00:00
|
|
|
}
|
|
|
|
|
2012-11-06 12:15:27 +00:00
|
|
|
ret = btrfs_resume_dev_replace_async(fs_info);
|
|
|
|
if (ret) {
|
|
|
|
pr_warn("btrfs: failed to resume dev_replace\n");
|
|
|
|
close_ctree(tree_root);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2011-11-17 06:10:02 +00:00
|
|
|
return 0;
|
2007-06-12 10:35:45 +00:00
|
|
|
|
2011-09-13 13:23:30 +00:00
|
|
|
fail_qgroup:
|
|
|
|
btrfs_free_qgroup_config(fs_info);
|
2008-11-19 20:13:35 +00:00
|
|
|
fail_trans_kthread:
|
|
|
|
kthread_stop(fs_info->transaction_kthread);
|
2008-06-25 20:01:31 +00:00
|
|
|
fail_cleaner:
|
2008-06-25 20:01:31 +00:00
|
|
|
kthread_stop(fs_info->cleaner_kthread);
|
2008-11-19 20:13:35 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* make sure we're done with the btree inode before we stop our
|
|
|
|
* kthreads
|
|
|
|
*/
|
|
|
|
filemap_write_and_wait(fs_info->btree_inode->i_mapping);
|
|
|
|
|
2010-03-19 20:49:55 +00:00
|
|
|
fail_block_groups:
|
|
|
|
btrfs_free_block_groups(fs_info);
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
fail_tree_roots:
|
|
|
|
free_root_pointers(fs_info, 1);
|
2013-02-07 06:01:35 +00:00
|
|
|
invalidate_inode_pages2(fs_info->btree_inode->i_mapping);
|
2011-11-03 19:17:42 +00:00
|
|
|
|
2007-06-12 10:35:45 +00:00
|
|
|
fail_sb_buffer:
|
2009-10-02 23:11:56 +00:00
|
|
|
btrfs_stop_workers(&fs_info->generic_worker);
|
2011-11-03 19:21:39 +00:00
|
|
|
btrfs_stop_workers(&fs_info->readahead_workers);
|
2008-07-17 16:53:51 +00:00
|
|
|
btrfs_stop_workers(&fs_info->fixup_workers);
|
2008-11-07 03:02:51 +00:00
|
|
|
btrfs_stop_workers(&fs_info->delalloc_workers);
|
2008-06-11 20:50:36 +00:00
|
|
|
btrfs_stop_workers(&fs_info->workers);
|
|
|
|
btrfs_stop_workers(&fs_info->endio_workers);
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_meta_workers);
|
2013-01-29 23:40:14 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_raid56_workers);
|
|
|
|
btrfs_stop_workers(&fs_info->rmw_workers);
|
2008-12-17 19:51:42 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_meta_write_workers);
|
2008-07-17 16:53:50 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_write_workers);
|
2010-07-02 16:14:14 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_freespace_worker);
|
2008-06-12 18:46:17 +00:00
|
|
|
btrfs_stop_workers(&fs_info->submit_workers);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
btrfs_stop_workers(&fs_info->delayed_workers);
|
2011-06-30 18:42:28 +00:00
|
|
|
btrfs_stop_workers(&fs_info->caching_workers);
|
2012-10-25 09:28:04 +00:00
|
|
|
btrfs_stop_workers(&fs_info->flush_workers);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
fail_alloc:
|
2008-06-12 01:47:56 +00:00
|
|
|
fail_iput:
|
2011-11-09 11:26:37 +00:00
|
|
|
btrfs_mapping_tree_free(&fs_info->mapping_tree);
|
|
|
|
|
2008-06-12 01:47:56 +00:00
|
|
|
iput(fs_info->btree_inode);
|
2013-01-29 10:10:51 +00:00
|
|
|
fail_delalloc_bytes:
|
|
|
|
percpu_counter_destroy(&fs_info->delalloc_bytes);
|
2013-01-29 10:09:20 +00:00
|
|
|
fail_dirty_metadata_bytes:
|
|
|
|
percpu_counter_destroy(&fs_info->dirty_metadata_bytes);
|
2009-06-12 12:43:40 +00:00
|
|
|
fail_bdi:
|
2009-01-21 15:49:16 +00:00
|
|
|
bdi_destroy(&fs_info->bdi);
|
2009-09-21 20:00:26 +00:00
|
|
|
fail_srcu:
|
|
|
|
cleanup_srcu_struct(&fs_info->subvol_srcu);
|
2009-01-21 15:49:16 +00:00
|
|
|
fail:
|
2013-01-29 23:40:14 +00:00
|
|
|
btrfs_free_stripe_hash_table(fs_info);
|
2011-11-09 11:26:37 +00:00
|
|
|
btrfs_close_devices(fs_info->fs_devices);
|
2011-11-17 06:10:02 +00:00
|
|
|
return err;
|
2011-11-03 19:17:42 +00:00
|
|
|
|
|
|
|
recovery_tree_root:
|
|
|
|
if (!btrfs_test_opt(tree_root, RECOVERY))
|
|
|
|
goto fail_tree_roots;
|
|
|
|
|
|
|
|
free_root_pointers(fs_info, 0);
|
|
|
|
|
|
|
|
/* don't use the log in recovery mode, it won't be valid */
|
|
|
|
btrfs_set_super_log_root(disk_super, 0);
|
|
|
|
|
|
|
|
/* we can't trust the free space cache either */
|
|
|
|
btrfs_set_opt(fs_info->mount_opt, CLEAR_CACHE);
|
|
|
|
|
|
|
|
ret = next_root_backup(fs_info, fs_info->super_copy,
|
|
|
|
&num_backups_tried, &backup_index);
|
|
|
|
if (ret == -1)
|
|
|
|
goto fail_block_groups;
|
|
|
|
goto retry_root_backup;
|
2007-02-02 14:18:22 +00:00
|
|
|
}
|
|
|
|
|
2008-04-10 20:19:33 +00:00
|
|
|
static void btrfs_end_buffer_write_sync(struct buffer_head *bh, int uptodate)
|
|
|
|
{
|
|
|
|
if (uptodate) {
|
|
|
|
set_buffer_uptodate(bh);
|
|
|
|
} else {
|
2012-05-25 14:06:08 +00:00
|
|
|
struct btrfs_device *device = (struct btrfs_device *)
|
|
|
|
bh->b_private;
|
|
|
|
|
2012-06-04 18:03:51 +00:00
|
|
|
printk_ratelimited_in_rcu(KERN_WARNING "lost page write due to "
|
|
|
|
"I/O error on %s\n",
|
|
|
|
rcu_str_deref(device->name));
|
2008-05-12 17:39:03 +00:00
|
|
|
/* note, we dont' set_buffer_write_io_error because we have
|
|
|
|
* our own ways of dealing with the IO errors
|
|
|
|
*/
|
2008-04-10 20:19:33 +00:00
|
|
|
clear_buffer_uptodate(bh);
|
2012-05-25 14:06:08 +00:00
|
|
|
btrfs_dev_stat_inc_and_print(device, BTRFS_DEV_STAT_WRITE_ERRS);
|
2008-04-10 20:19:33 +00:00
|
|
|
}
|
|
|
|
unlock_buffer(bh);
|
|
|
|
put_bh(bh);
|
|
|
|
}
|
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
struct buffer_head *btrfs_read_dev_super(struct block_device *bdev)
|
|
|
|
{
|
|
|
|
struct buffer_head *bh;
|
|
|
|
struct buffer_head *latest = NULL;
|
|
|
|
struct btrfs_super_block *super;
|
|
|
|
int i;
|
|
|
|
u64 transid = 0;
|
|
|
|
u64 bytenr;
|
|
|
|
|
|
|
|
/* we would like to check all the supers, but that would make
|
|
|
|
* a btrfs mount succeed after a mkfs from a different FS.
|
|
|
|
* So, we need to add a special mount option to scan for
|
|
|
|
* later supers, using BTRFS_SUPER_MIRROR_MAX instead
|
|
|
|
*/
|
|
|
|
for (i = 0; i < 1; i++) {
|
|
|
|
bytenr = btrfs_sb_offset(i);
|
|
|
|
if (bytenr + 4096 >= i_size_read(bdev->bd_inode))
|
|
|
|
break;
|
|
|
|
bh = __bread(bdev, bytenr / 4096, 4096);
|
|
|
|
if (!bh)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
super = (struct btrfs_super_block *)bh->b_data;
|
|
|
|
if (btrfs_super_bytenr(super) != bytenr ||
|
2013-02-20 00:55:13 +00:00
|
|
|
super->magic != cpu_to_le64(BTRFS_MAGIC)) {
|
2008-12-08 21:46:26 +00:00
|
|
|
brelse(bh);
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!latest || btrfs_super_generation(super) > transid) {
|
|
|
|
brelse(latest);
|
|
|
|
latest = bh;
|
|
|
|
transid = btrfs_super_generation(super);
|
|
|
|
} else {
|
|
|
|
brelse(bh);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return latest;
|
|
|
|
}
|
|
|
|
|
2009-06-10 19:28:55 +00:00
|
|
|
/*
|
|
|
|
* this should be called twice, once with wait == 0 and
|
|
|
|
* once with wait == 1. When wait == 0 is done, all the buffer heads
|
|
|
|
* we write are pinned.
|
|
|
|
*
|
|
|
|
* They are released when wait == 1 is done.
|
|
|
|
* max_mirrors must be the same for both runs, and it indicates how
|
|
|
|
* many supers on this one device should be written.
|
|
|
|
*
|
|
|
|
* max_mirrors == 0 means to write them all.
|
|
|
|
*/
|
2008-12-08 21:46:26 +00:00
|
|
|
static int write_dev_supers(struct btrfs_device *device,
|
|
|
|
struct btrfs_super_block *sb,
|
|
|
|
int do_barriers, int wait, int max_mirrors)
|
|
|
|
{
|
|
|
|
struct buffer_head *bh;
|
|
|
|
int i;
|
|
|
|
int ret;
|
|
|
|
int errors = 0;
|
|
|
|
u32 crc;
|
|
|
|
u64 bytenr;
|
|
|
|
|
|
|
|
if (max_mirrors == 0)
|
|
|
|
max_mirrors = BTRFS_SUPER_MIRROR_MAX;
|
|
|
|
|
|
|
|
for (i = 0; i < max_mirrors; i++) {
|
|
|
|
bytenr = btrfs_sb_offset(i);
|
|
|
|
if (bytenr + BTRFS_SUPER_INFO_SIZE >= device->total_bytes)
|
|
|
|
break;
|
|
|
|
|
|
|
|
if (wait) {
|
|
|
|
bh = __find_get_block(device->bdev, bytenr / 4096,
|
|
|
|
BTRFS_SUPER_INFO_SIZE);
|
|
|
|
BUG_ON(!bh);
|
|
|
|
wait_on_buffer(bh);
|
2009-06-10 19:28:55 +00:00
|
|
|
if (!buffer_uptodate(bh))
|
|
|
|
errors++;
|
|
|
|
|
|
|
|
/* drop our reference */
|
|
|
|
brelse(bh);
|
|
|
|
|
|
|
|
/* drop the reference from the wait == 0 run */
|
|
|
|
brelse(bh);
|
|
|
|
continue;
|
2008-12-08 21:46:26 +00:00
|
|
|
} else {
|
|
|
|
btrfs_set_super_bytenr(sb, bytenr);
|
|
|
|
|
|
|
|
crc = ~(u32)0;
|
|
|
|
crc = btrfs_csum_data(NULL, (char *)sb +
|
|
|
|
BTRFS_CSUM_SIZE, crc,
|
|
|
|
BTRFS_SUPER_INFO_SIZE -
|
|
|
|
BTRFS_CSUM_SIZE);
|
|
|
|
btrfs_csum_final(crc, sb->csum);
|
|
|
|
|
2009-06-10 19:28:55 +00:00
|
|
|
/*
|
|
|
|
* one reference for us, and we leave it for the
|
|
|
|
* caller
|
|
|
|
*/
|
2008-12-08 21:46:26 +00:00
|
|
|
bh = __getblk(device->bdev, bytenr / 4096,
|
|
|
|
BTRFS_SUPER_INFO_SIZE);
|
|
|
|
memcpy(bh->b_data, sb, BTRFS_SUPER_INFO_SIZE);
|
|
|
|
|
2009-06-10 19:28:55 +00:00
|
|
|
/* one reference for submit_bh */
|
2008-12-08 21:46:26 +00:00
|
|
|
get_bh(bh);
|
2009-06-10 19:28:55 +00:00
|
|
|
|
|
|
|
set_buffer_uptodate(bh);
|
2008-12-08 21:46:26 +00:00
|
|
|
lock_buffer(bh);
|
|
|
|
bh->b_end_io = btrfs_end_buffer_write_sync;
|
2012-05-25 14:06:08 +00:00
|
|
|
bh->b_private = device;
|
2008-12-08 21:46:26 +00:00
|
|
|
}
|
|
|
|
|
2011-11-18 20:07:51 +00:00
|
|
|
/*
|
|
|
|
* we fua the first super. The others we allow
|
|
|
|
* to go down lazy.
|
|
|
|
*/
|
2011-11-09 12:44:05 +00:00
|
|
|
ret = btrfsic_submit_bh(WRITE_FUA, bh);
|
2009-06-10 19:28:55 +00:00
|
|
|
if (ret)
|
2008-12-08 21:46:26 +00:00
|
|
|
errors++;
|
|
|
|
}
|
|
|
|
return errors < i ? 0 : -1;
|
|
|
|
}
|
|
|
|
|
2011-11-18 20:07:51 +00:00
|
|
|
/*
|
|
|
|
* endio for the write_dev_flush, this will wake anyone waiting
|
|
|
|
* for the barrier when it is done
|
|
|
|
*/
|
|
|
|
static void btrfs_end_empty_barrier(struct bio *bio, int err)
|
|
|
|
{
|
|
|
|
if (err) {
|
|
|
|
if (err == -EOPNOTSUPP)
|
|
|
|
set_bit(BIO_EOPNOTSUPP, &bio->bi_flags);
|
|
|
|
clear_bit(BIO_UPTODATE, &bio->bi_flags);
|
|
|
|
}
|
|
|
|
if (bio->bi_private)
|
|
|
|
complete(bio->bi_private);
|
|
|
|
bio_put(bio);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* trigger flushes for one the devices. If you pass wait == 0, the flushes are
|
|
|
|
* sent down. With wait == 1, it waits for the previous flush.
|
|
|
|
*
|
|
|
|
* any device where the flush fails with eopnotsupp are flagged as not-barrier
|
|
|
|
* capable
|
|
|
|
*/
|
|
|
|
static int write_dev_flush(struct btrfs_device *device, int wait)
|
|
|
|
{
|
|
|
|
struct bio *bio;
|
|
|
|
int ret = 0;
|
|
|
|
|
|
|
|
if (device->nobarriers)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
if (wait) {
|
|
|
|
bio = device->flush_bio;
|
|
|
|
if (!bio)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
wait_for_completion(&device->flush_wait);
|
|
|
|
|
|
|
|
if (bio_flagged(bio, BIO_EOPNOTSUPP)) {
|
2012-06-04 18:03:51 +00:00
|
|
|
printk_in_rcu("btrfs: disabling barriers on dev %s\n",
|
|
|
|
rcu_str_deref(device->name));
|
2011-11-18 20:07:51 +00:00
|
|
|
device->nobarriers = 1;
|
2012-08-01 16:56:49 +00:00
|
|
|
} else if (!bio_flagged(bio, BIO_UPTODATE)) {
|
2011-11-18 20:07:51 +00:00
|
|
|
ret = -EIO;
|
2012-08-01 16:56:49 +00:00
|
|
|
btrfs_dev_stat_inc_and_print(device,
|
|
|
|
BTRFS_DEV_STAT_FLUSH_ERRS);
|
2011-11-18 20:07:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* drop the reference from the wait == 0 run */
|
|
|
|
bio_put(bio);
|
|
|
|
device->flush_bio = NULL;
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* one reference for us, and we leave it for the
|
|
|
|
* caller
|
|
|
|
*/
|
2012-02-26 23:04:56 +00:00
|
|
|
device->flush_bio = NULL;
|
2011-11-18 20:07:51 +00:00
|
|
|
bio = bio_alloc(GFP_NOFS, 0);
|
|
|
|
if (!bio)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
bio->bi_end_io = btrfs_end_empty_barrier;
|
|
|
|
bio->bi_bdev = device->bdev;
|
|
|
|
init_completion(&device->flush_wait);
|
|
|
|
bio->bi_private = &device->flush_wait;
|
|
|
|
device->flush_bio = bio;
|
|
|
|
|
|
|
|
bio_get(bio);
|
2011-11-09 12:44:05 +00:00
|
|
|
btrfsic_submit_bio(WRITE_FLUSH, bio);
|
2011-11-18 20:07:51 +00:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* send an empty flush down to each device in parallel,
|
|
|
|
* then wait for them
|
|
|
|
*/
|
|
|
|
static int barrier_all_devices(struct btrfs_fs_info *info)
|
|
|
|
{
|
|
|
|
struct list_head *head;
|
|
|
|
struct btrfs_device *dev;
|
2012-08-01 16:56:49 +00:00
|
|
|
int errors_send = 0;
|
|
|
|
int errors_wait = 0;
|
2011-11-18 20:07:51 +00:00
|
|
|
int ret;
|
|
|
|
|
|
|
|
/* send down all the barriers */
|
|
|
|
head = &info->fs_devices->devices;
|
|
|
|
list_for_each_entry_rcu(dev, head, dev_list) {
|
|
|
|
if (!dev->bdev) {
|
2012-08-01 16:56:49 +00:00
|
|
|
errors_send++;
|
2011-11-18 20:07:51 +00:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
if (!dev->in_fs_metadata || !dev->writeable)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
ret = write_dev_flush(dev, 0);
|
|
|
|
if (ret)
|
2012-08-01 16:56:49 +00:00
|
|
|
errors_send++;
|
2011-11-18 20:07:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* wait for all the barriers */
|
|
|
|
list_for_each_entry_rcu(dev, head, dev_list) {
|
|
|
|
if (!dev->bdev) {
|
2012-08-01 16:56:49 +00:00
|
|
|
errors_wait++;
|
2011-11-18 20:07:51 +00:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
if (!dev->in_fs_metadata || !dev->writeable)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
ret = write_dev_flush(dev, 1);
|
|
|
|
if (ret)
|
2012-08-01 16:56:49 +00:00
|
|
|
errors_wait++;
|
2011-11-18 20:07:51 +00:00
|
|
|
}
|
2012-08-01 16:56:49 +00:00
|
|
|
if (errors_send > info->num_tolerated_disk_barrier_failures ||
|
|
|
|
errors_wait > info->num_tolerated_disk_barrier_failures)
|
2011-11-18 20:07:51 +00:00
|
|
|
return -EIO;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-08-01 16:56:49 +00:00
|
|
|
int btrfs_calc_num_tolerated_disk_barrier_failures(
|
|
|
|
struct btrfs_fs_info *fs_info)
|
|
|
|
{
|
|
|
|
struct btrfs_ioctl_space_info space;
|
|
|
|
struct btrfs_space_info *sinfo;
|
|
|
|
u64 types[] = {BTRFS_BLOCK_GROUP_DATA,
|
|
|
|
BTRFS_BLOCK_GROUP_SYSTEM,
|
|
|
|
BTRFS_BLOCK_GROUP_METADATA,
|
|
|
|
BTRFS_BLOCK_GROUP_DATA | BTRFS_BLOCK_GROUP_METADATA};
|
|
|
|
int num_types = 4;
|
|
|
|
int i;
|
|
|
|
int c;
|
|
|
|
int num_tolerated_disk_barrier_failures =
|
|
|
|
(int)fs_info->fs_devices->num_devices;
|
|
|
|
|
|
|
|
for (i = 0; i < num_types; i++) {
|
|
|
|
struct btrfs_space_info *tmp;
|
|
|
|
|
|
|
|
sinfo = NULL;
|
|
|
|
rcu_read_lock();
|
|
|
|
list_for_each_entry_rcu(tmp, &fs_info->space_info, list) {
|
|
|
|
if (tmp->flags == types[i]) {
|
|
|
|
sinfo = tmp;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
if (!sinfo)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
down_read(&sinfo->groups_sem);
|
|
|
|
for (c = 0; c < BTRFS_NR_RAID_TYPES; c++) {
|
|
|
|
if (!list_empty(&sinfo->block_groups[c])) {
|
|
|
|
u64 flags;
|
|
|
|
|
|
|
|
btrfs_get_block_group_info(
|
|
|
|
&sinfo->block_groups[c], &space);
|
|
|
|
if (space.total_bytes == 0 ||
|
|
|
|
space.used_bytes == 0)
|
|
|
|
continue;
|
|
|
|
flags = space.flags;
|
|
|
|
/*
|
|
|
|
* return
|
|
|
|
* 0: if dup, single or RAID0 is configured for
|
|
|
|
* any of metadata, system or data, else
|
|
|
|
* 1: if RAID5 is configured, or if RAID1 or
|
|
|
|
* RAID10 is configured and only two mirrors
|
|
|
|
* are used, else
|
|
|
|
* 2: if RAID6 is configured, else
|
|
|
|
* num_mirrors - 1: if RAID1 or RAID10 is
|
|
|
|
* configured and more than
|
|
|
|
* 2 mirrors are used.
|
|
|
|
*/
|
|
|
|
if (num_tolerated_disk_barrier_failures > 0 &&
|
|
|
|
((flags & (BTRFS_BLOCK_GROUP_DUP |
|
|
|
|
BTRFS_BLOCK_GROUP_RAID0)) ||
|
|
|
|
((flags & BTRFS_BLOCK_GROUP_PROFILE_MASK)
|
|
|
|
== 0)))
|
|
|
|
num_tolerated_disk_barrier_failures = 0;
|
2013-01-29 23:40:14 +00:00
|
|
|
else if (num_tolerated_disk_barrier_failures > 1) {
|
|
|
|
if (flags & (BTRFS_BLOCK_GROUP_RAID1 |
|
|
|
|
BTRFS_BLOCK_GROUP_RAID5 |
|
|
|
|
BTRFS_BLOCK_GROUP_RAID10)) {
|
|
|
|
num_tolerated_disk_barrier_failures = 1;
|
|
|
|
} else if (flags &
|
|
|
|
BTRFS_BLOCK_GROUP_RAID5) {
|
|
|
|
num_tolerated_disk_barrier_failures = 2;
|
|
|
|
}
|
|
|
|
}
|
2012-08-01 16:56:49 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
up_read(&sinfo->groups_sem);
|
|
|
|
}
|
|
|
|
|
|
|
|
return num_tolerated_disk_barrier_failures;
|
|
|
|
}
|
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
int write_all_supers(struct btrfs_root *root, int max_mirrors)
|
2008-04-10 20:19:33 +00:00
|
|
|
{
|
2009-06-10 19:17:02 +00:00
|
|
|
struct list_head *head;
|
2008-04-10 20:19:33 +00:00
|
|
|
struct btrfs_device *dev;
|
2008-05-07 15:43:44 +00:00
|
|
|
struct btrfs_super_block *sb;
|
2008-04-10 20:19:33 +00:00
|
|
|
struct btrfs_dev_item *dev_item;
|
|
|
|
int ret;
|
|
|
|
int do_barriers;
|
2008-04-29 13:38:00 +00:00
|
|
|
int max_errors;
|
|
|
|
int total_errors = 0;
|
2008-05-07 15:43:44 +00:00
|
|
|
u64 flags;
|
2008-04-10 20:19:33 +00:00
|
|
|
|
2011-04-13 13:41:04 +00:00
|
|
|
max_errors = btrfs_super_num_devices(root->fs_info->super_copy) - 1;
|
2008-04-10 20:19:33 +00:00
|
|
|
do_barriers = !btrfs_test_opt(root, NOBARRIER);
|
2011-11-03 19:17:42 +00:00
|
|
|
backup_super_roots(root->fs_info);
|
2008-04-10 20:19:33 +00:00
|
|
|
|
2011-04-13 13:41:04 +00:00
|
|
|
sb = root->fs_info->super_for_commit;
|
2008-05-07 15:43:44 +00:00
|
|
|
dev_item = &sb->dev_item;
|
2009-06-10 19:17:02 +00:00
|
|
|
|
2011-05-27 14:03:58 +00:00
|
|
|
mutex_lock(&root->fs_info->fs_devices->device_list_mutex);
|
2009-06-10 19:17:02 +00:00
|
|
|
head = &root->fs_info->fs_devices->devices;
|
2011-11-18 20:07:51 +00:00
|
|
|
|
2012-08-01 16:56:49 +00:00
|
|
|
if (do_barriers) {
|
|
|
|
ret = barrier_all_devices(root->fs_info);
|
|
|
|
if (ret) {
|
|
|
|
mutex_unlock(
|
|
|
|
&root->fs_info->fs_devices->device_list_mutex);
|
|
|
|
btrfs_error(root->fs_info, ret,
|
|
|
|
"errors while submitting device barriers.");
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
}
|
2011-11-18 20:07:51 +00:00
|
|
|
|
2011-04-20 10:09:16 +00:00
|
|
|
list_for_each_entry_rcu(dev, head, dev_list) {
|
2008-05-13 17:46:40 +00:00
|
|
|
if (!dev->bdev) {
|
|
|
|
total_errors++;
|
|
|
|
continue;
|
|
|
|
}
|
2008-11-18 02:11:30 +00:00
|
|
|
if (!dev->in_fs_metadata || !dev->writeable)
|
2008-05-13 17:46:40 +00:00
|
|
|
continue;
|
|
|
|
|
2008-11-18 02:11:30 +00:00
|
|
|
btrfs_set_stack_device_generation(dev_item, 0);
|
2008-05-07 15:43:44 +00:00
|
|
|
btrfs_set_stack_device_type(dev_item, dev->type);
|
|
|
|
btrfs_set_stack_device_id(dev_item, dev->devid);
|
|
|
|
btrfs_set_stack_device_total_bytes(dev_item, dev->total_bytes);
|
|
|
|
btrfs_set_stack_device_bytes_used(dev_item, dev->bytes_used);
|
|
|
|
btrfs_set_stack_device_io_align(dev_item, dev->io_align);
|
|
|
|
btrfs_set_stack_device_io_width(dev_item, dev->io_width);
|
|
|
|
btrfs_set_stack_device_sector_size(dev_item, dev->sector_size);
|
|
|
|
memcpy(dev_item->uuid, dev->uuid, BTRFS_UUID_SIZE);
|
2008-11-18 02:11:30 +00:00
|
|
|
memcpy(dev_item->fsid, dev->fs_devices->fsid, BTRFS_UUID_SIZE);
|
2008-12-08 21:46:26 +00:00
|
|
|
|
2008-05-07 15:43:44 +00:00
|
|
|
flags = btrfs_super_flags(sb);
|
|
|
|
btrfs_set_super_flags(sb, flags | BTRFS_HEADER_FLAG_WRITTEN);
|
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
ret = write_dev_supers(dev, sb, do_barriers, 0, max_mirrors);
|
2008-04-29 13:38:00 +00:00
|
|
|
if (ret)
|
|
|
|
total_errors++;
|
2008-04-10 20:19:33 +00:00
|
|
|
}
|
2008-04-29 13:38:00 +00:00
|
|
|
if (total_errors > max_errors) {
|
2009-01-06 02:25:51 +00:00
|
|
|
printk(KERN_ERR "btrfs: %d errors while writing supers\n",
|
|
|
|
total_errors);
|
2012-03-12 15:03:00 +00:00
|
|
|
|
|
|
|
/* This shouldn't happen. FUA is masked off if unsupported */
|
2008-04-29 13:38:00 +00:00
|
|
|
BUG();
|
|
|
|
}
|
2008-04-10 20:19:33 +00:00
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
total_errors = 0;
|
2011-04-20 10:09:16 +00:00
|
|
|
list_for_each_entry_rcu(dev, head, dev_list) {
|
2008-05-13 17:46:40 +00:00
|
|
|
if (!dev->bdev)
|
|
|
|
continue;
|
2008-11-18 02:11:30 +00:00
|
|
|
if (!dev->in_fs_metadata || !dev->writeable)
|
2008-05-13 17:46:40 +00:00
|
|
|
continue;
|
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
ret = write_dev_supers(dev, sb, do_barriers, 1, max_mirrors);
|
|
|
|
if (ret)
|
|
|
|
total_errors++;
|
2008-04-10 20:19:33 +00:00
|
|
|
}
|
2011-05-27 14:03:58 +00:00
|
|
|
mutex_unlock(&root->fs_info->fs_devices->device_list_mutex);
|
2008-04-29 13:38:00 +00:00
|
|
|
if (total_errors > max_errors) {
|
2012-03-12 15:03:00 +00:00
|
|
|
btrfs_error(root->fs_info, -EIO,
|
|
|
|
"%d errors while writing supers", total_errors);
|
|
|
|
return -EIO;
|
2008-04-29 13:38:00 +00:00
|
|
|
}
|
2008-04-10 20:19:33 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
int write_ctree_super(struct btrfs_trans_handle *trans,
|
|
|
|
struct btrfs_root *root, int max_mirrors)
|
2007-02-02 14:18:22 +00:00
|
|
|
{
|
2007-04-20 17:16:02 +00:00
|
|
|
int ret;
|
2007-10-15 20:14:19 +00:00
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
ret = write_all_supers(root, max_mirrors);
|
2007-10-15 20:14:19 +00:00
|
|
|
return ret;
|
2007-02-21 22:04:57 +00:00
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
void btrfs_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root *root)
|
2007-04-10 20:58:11 +00:00
|
|
|
{
|
2009-09-21 19:56:00 +00:00
|
|
|
spin_lock(&fs_info->fs_roots_radix_lock);
|
2007-04-10 20:58:11 +00:00
|
|
|
radix_tree_delete(&fs_info->fs_roots_radix,
|
|
|
|
(unsigned long)root->root_key.objectid);
|
2009-09-21 19:56:00 +00:00
|
|
|
spin_unlock(&fs_info->fs_roots_radix_lock);
|
2009-09-21 20:00:26 +00:00
|
|
|
|
|
|
|
if (btrfs_root_refs(&root->root_item) == 0)
|
|
|
|
synchronize_srcu(&fs_info->subvol_srcu);
|
|
|
|
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
__btrfs_remove_free_space_cache(root->free_ino_pinned);
|
|
|
|
__btrfs_remove_free_space_cache(root->free_ino_ctl);
|
2009-09-21 19:56:00 +00:00
|
|
|
free_fs_root(root);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void free_fs_root(struct btrfs_root *root)
|
|
|
|
{
|
2011-04-20 02:33:24 +00:00
|
|
|
iput(root->cache_inode);
|
2009-09-21 19:56:00 +00:00
|
|
|
WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
|
2011-07-07 19:44:25 +00:00
|
|
|
if (root->anon_dev)
|
|
|
|
free_anon_bdev(root->anon_dev);
|
2009-09-21 19:56:00 +00:00
|
|
|
free_extent_buffer(root->node);
|
|
|
|
free_extent_buffer(root->commit_root);
|
Btrfs: Cache free inode numbers in memory
Currently btrfs stores the highest objectid of the fs tree, and it always
returns (highest+1) inode number when we create a file, so inode numbers
won't be reclaimed when we delete files, so we'll run out of inode numbers
as we keep create/delete files in 32bits machines.
This fixes it, and it works similarly to how we cache free space in block
cgroups.
We start a kernel thread to read the file tree. By scanning inode items,
we know which chunks of inode numbers are free, and we cache them in
an rb-tree.
Because we are searching the commit root, we have to carefully handle the
cross-transaction case.
The rb-tree is a hybrid extent+bitmap tree, so if we have too many small
chunks of inode numbers, we'll use bitmaps. Initially we allow 16K ram
of extents, and a bitmap will be used if we exceed this threshold. The
extents threshold is adjusted in runtime.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2011-04-20 02:06:11 +00:00
|
|
|
kfree(root->free_ino_ctl);
|
|
|
|
kfree(root->free_ino_pinned);
|
2009-01-06 02:25:51 +00:00
|
|
|
kfree(root->name);
|
2007-04-10 20:58:11 +00:00
|
|
|
kfree(root);
|
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
static void del_fs_roots(struct btrfs_fs_info *fs_info)
|
2007-04-09 14:42:37 +00:00
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
struct btrfs_root *gang[8];
|
|
|
|
int i;
|
|
|
|
|
2009-09-21 20:00:26 +00:00
|
|
|
while (!list_empty(&fs_info->dead_roots)) {
|
|
|
|
gang[0] = list_entry(fs_info->dead_roots.next,
|
|
|
|
struct btrfs_root, root_list);
|
|
|
|
list_del(&gang[0]->root_list);
|
|
|
|
|
|
|
|
if (gang[0]->in_radix) {
|
|
|
|
btrfs_free_fs_root(fs_info, gang[0]);
|
|
|
|
} else {
|
|
|
|
free_extent_buffer(gang[0]->node);
|
|
|
|
free_extent_buffer(gang[0]->commit_root);
|
|
|
|
kfree(gang[0]);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-01-06 02:25:51 +00:00
|
|
|
while (1) {
|
2007-04-09 14:42:37 +00:00
|
|
|
ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
|
|
|
|
(void **)gang, 0,
|
|
|
|
ARRAY_SIZE(gang));
|
|
|
|
if (!ret)
|
|
|
|
break;
|
2007-04-10 20:58:11 +00:00
|
|
|
for (i = 0; i < ret; i++)
|
2007-06-22 18:16:25 +00:00
|
|
|
btrfs_free_fs_root(fs_info, gang[i]);
|
2007-04-09 14:42:37 +00:00
|
|
|
}
|
|
|
|
}
|
2007-04-12 16:14:00 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info)
|
2007-02-21 22:04:57 +00:00
|
|
|
{
|
2008-11-12 19:34:12 +00:00
|
|
|
u64 root_objectid = 0;
|
|
|
|
struct btrfs_root *gang[8];
|
|
|
|
int i;
|
2007-03-13 20:47:54 +00:00
|
|
|
int ret;
|
2007-03-16 20:20:31 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
while (1) {
|
|
|
|
ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
|
|
|
|
(void **)gang, root_objectid,
|
|
|
|
ARRAY_SIZE(gang));
|
|
|
|
if (!ret)
|
|
|
|
break;
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
|
|
|
|
root_objectid = gang[ret - 1]->root_key.objectid + 1;
|
2008-11-12 19:34:12 +00:00
|
|
|
for (i = 0; i < ret; i++) {
|
2011-01-31 21:22:42 +00:00
|
|
|
int err;
|
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
root_objectid = gang[i]->root_key.objectid;
|
2011-01-31 21:22:42 +00:00
|
|
|
err = btrfs_orphan_cleanup(gang[i]);
|
|
|
|
if (err)
|
|
|
|
return err;
|
2008-11-12 19:34:12 +00:00
|
|
|
}
|
|
|
|
root_objectid++;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
2008-06-25 20:01:30 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
int btrfs_commit_super(struct btrfs_root *root)
|
|
|
|
{
|
|
|
|
struct btrfs_trans_handle *trans;
|
|
|
|
int ret;
|
2008-06-25 20:01:31 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
mutex_lock(&root->fs_info->cleaner_mutex);
|
2009-11-12 09:36:34 +00:00
|
|
|
btrfs_run_delayed_iputs(root);
|
2008-06-25 20:01:31 +00:00
|
|
|
btrfs_clean_old_snapshots(root);
|
2008-11-12 19:34:12 +00:00
|
|
|
mutex_unlock(&root->fs_info->cleaner_mutex);
|
2009-11-12 09:34:40 +00:00
|
|
|
|
|
|
|
/* wait until ongoing cleanup work done */
|
|
|
|
down_write(&root->fs_info->cleanup_work_sem);
|
|
|
|
up_write(&root->fs_info->cleanup_work_sem);
|
|
|
|
|
2011-04-13 16:54:33 +00:00
|
|
|
trans = btrfs_join_transaction(root);
|
2011-01-25 02:51:38 +00:00
|
|
|
if (IS_ERR(trans))
|
|
|
|
return PTR_ERR(trans);
|
2007-06-22 18:16:25 +00:00
|
|
|
ret = btrfs_commit_transaction(trans, root);
|
2012-03-12 15:03:00 +00:00
|
|
|
if (ret)
|
|
|
|
return ret;
|
2008-11-12 19:34:12 +00:00
|
|
|
/* run commit again to drop the original snapshot */
|
2011-04-13 16:54:33 +00:00
|
|
|
trans = btrfs_join_transaction(root);
|
2011-01-25 02:51:38 +00:00
|
|
|
if (IS_ERR(trans))
|
|
|
|
return PTR_ERR(trans);
|
2012-03-12 15:03:00 +00:00
|
|
|
ret = btrfs_commit_transaction(trans, root);
|
|
|
|
if (ret)
|
|
|
|
return ret;
|
2007-03-22 19:59:16 +00:00
|
|
|
ret = btrfs_write_and_wait_transaction(NULL, root);
|
2012-03-12 15:03:00 +00:00
|
|
|
if (ret) {
|
|
|
|
btrfs_error(root->fs_info, ret,
|
|
|
|
"Failed to sync btree inode to disk.");
|
|
|
|
return ret;
|
|
|
|
}
|
2008-04-30 17:59:35 +00:00
|
|
|
|
2008-12-08 21:46:26 +00:00
|
|
|
ret = write_ctree_super(NULL, root, 0);
|
2008-11-12 19:34:12 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
int close_ctree(struct btrfs_root *root)
|
|
|
|
{
|
|
|
|
struct btrfs_fs_info *fs_info = root->fs_info;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
fs_info->closing = 1;
|
|
|
|
smp_mb();
|
|
|
|
|
2012-01-16 20:04:49 +00:00
|
|
|
/* pause restriper - we want to resume on mount */
|
2012-11-05 16:03:39 +00:00
|
|
|
btrfs_pause_balance(fs_info);
|
2012-01-16 20:04:49 +00:00
|
|
|
|
2012-11-06 12:15:27 +00:00
|
|
|
btrfs_dev_replace_suspend_for_unmount(fs_info);
|
|
|
|
|
2012-11-05 16:03:39 +00:00
|
|
|
btrfs_scrub_cancel(fs_info);
|
2011-05-24 19:35:30 +00:00
|
|
|
|
|
|
|
/* wait for any defraggers to finish */
|
|
|
|
wait_event(fs_info->transaction_wait,
|
|
|
|
(atomic_read(&fs_info->defrag_running) == 0));
|
|
|
|
|
|
|
|
/* clear out the rbtree of defraggable inodes */
|
2012-11-26 09:26:20 +00:00
|
|
|
btrfs_cleanup_defrag_inodes(fs_info);
|
2011-05-24 19:35:30 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
if (!(fs_info->sb->s_flags & MS_RDONLY)) {
|
2011-01-06 11:30:25 +00:00
|
|
|
ret = btrfs_commit_super(root);
|
|
|
|
if (ret)
|
|
|
|
printk(KERN_ERR "btrfs: commit super ret %d\n", ret);
|
|
|
|
}
|
|
|
|
|
2013-01-29 10:14:48 +00:00
|
|
|
if (test_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state))
|
2012-08-01 11:45:52 +00:00
|
|
|
btrfs_error_commit_super(root);
|
2007-04-09 14:42:37 +00:00
|
|
|
|
2011-08-29 18:06:00 +00:00
|
|
|
btrfs_put_block_group_cache(fs_info);
|
|
|
|
|
2011-11-17 05:56:18 +00:00
|
|
|
kthread_stop(fs_info->transaction_kthread);
|
|
|
|
kthread_stop(fs_info->cleaner_kthread);
|
2010-05-16 14:49:58 +00:00
|
|
|
|
2009-07-28 12:41:57 +00:00
|
|
|
fs_info->closing = 2;
|
|
|
|
smp_mb();
|
|
|
|
|
2011-09-13 13:23:30 +00:00
|
|
|
btrfs_free_qgroup_config(root->fs_info);
|
|
|
|
|
2013-01-29 10:10:51 +00:00
|
|
|
if (percpu_counter_sum(&fs_info->delalloc_bytes)) {
|
|
|
|
printk(KERN_INFO "btrfs: at unmount delalloc count %lld\n",
|
|
|
|
percpu_counter_sum(&fs_info->delalloc_bytes));
|
2008-01-31 16:05:37 +00:00
|
|
|
}
|
2008-07-30 20:29:20 +00:00
|
|
|
|
Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
When a tree block in subvolume tree is cow'd, the reference counts of all
extents it points to are increased by one. At transaction commit time,
the old root of the subvolume is recorded in a "dead root" data structure,
and the btree it points to is later walked, dropping reference counts
and freeing any blocks where the reference count goes to 0.
The increments done during cow and decrements done after commit cancel out,
and the walk is a very expensive way to go about freeing the blocks that
are no longer referenced by the new btree root. This commit reduces the
transaction overhead by avoiding the need for dead root records.
When a non-shared tree block is cow'd, we free the old block at once, and the
new block inherits old block's references. When a tree block with reference
count > 1 is cow'd, we increase the reference counts of all extents
the new block points to by one, and decrease the old block's reference count by
one.
This dead tree avoidance code removes the need to modify the reference
counts of lower level extents when a non-shared tree block is cow'd.
But we still need to update back ref for all pointers in the block.
This is because the location of the block is recorded in the back ref
item.
We can solve this by introducing a new type of back ref. The new
back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer
by searching the tree. The shortcoming of the new back ref is that it
only works for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these
fuzzy back references would be O(number_of_snapshots) and quite slow.
The solution used here is to use the fuzzy back references in the common
case where a given tree block is only referenced by one root,
and use the full back references when multiple roots have a reference
on a given block.
This commit adds per subvolume red-black tree to keep trace of cached
inodes. The red-black tree helps the balancing code to find cached
inodes whose inode numbers within a given range.
This commit improves the balancing code by introducing several data
structures to keep the state of balancing. The most important one
is the back ref cache. It caches how the upper level tree blocks are
referenced. This greatly reduce the overhead of checking back ref.
The improved balancing code scales significantly better with a large
number of snapshots.
This is a very large commit and was written in a number of
pieces. But, they depend heavily on the disk format change and were
squashed together to make sure git bisect didn't end up in a
bad state wrt space balancing or the format change.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-06-10 14:45:14 +00:00
|
|
|
free_extent_buffer(fs_info->extent_root->node);
|
|
|
|
free_extent_buffer(fs_info->extent_root->commit_root);
|
|
|
|
free_extent_buffer(fs_info->tree_root->node);
|
|
|
|
free_extent_buffer(fs_info->tree_root->commit_root);
|
2011-11-17 05:56:18 +00:00
|
|
|
free_extent_buffer(fs_info->chunk_root->node);
|
|
|
|
free_extent_buffer(fs_info->chunk_root->commit_root);
|
|
|
|
free_extent_buffer(fs_info->dev_root->node);
|
|
|
|
free_extent_buffer(fs_info->dev_root->commit_root);
|
|
|
|
free_extent_buffer(fs_info->csum_root->node);
|
|
|
|
free_extent_buffer(fs_info->csum_root->commit_root);
|
2011-09-13 13:23:30 +00:00
|
|
|
if (fs_info->quota_root) {
|
|
|
|
free_extent_buffer(fs_info->quota_root->node);
|
|
|
|
free_extent_buffer(fs_info->quota_root->commit_root);
|
|
|
|
}
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
|
2011-11-17 05:56:18 +00:00
|
|
|
btrfs_free_block_groups(fs_info);
|
2007-12-18 01:14:04 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
del_fs_roots(fs_info);
|
2007-12-18 01:14:04 +00:00
|
|
|
|
2008-11-12 19:34:12 +00:00
|
|
|
iput(fs_info->btree_inode);
|
2008-04-18 20:11:30 +00:00
|
|
|
|
2009-10-02 23:11:56 +00:00
|
|
|
btrfs_stop_workers(&fs_info->generic_worker);
|
2008-07-17 16:53:51 +00:00
|
|
|
btrfs_stop_workers(&fs_info->fixup_workers);
|
2008-11-07 03:02:51 +00:00
|
|
|
btrfs_stop_workers(&fs_info->delalloc_workers);
|
2008-06-11 20:50:36 +00:00
|
|
|
btrfs_stop_workers(&fs_info->workers);
|
|
|
|
btrfs_stop_workers(&fs_info->endio_workers);
|
Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 21:58:54 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_meta_workers);
|
2013-01-29 23:40:14 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_raid56_workers);
|
|
|
|
btrfs_stop_workers(&fs_info->rmw_workers);
|
2008-12-17 19:51:42 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_meta_write_workers);
|
2008-07-17 16:53:50 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_write_workers);
|
2010-07-02 16:14:14 +00:00
|
|
|
btrfs_stop_workers(&fs_info->endio_freespace_worker);
|
2008-06-12 18:46:17 +00:00
|
|
|
btrfs_stop_workers(&fs_info->submit_workers);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
btrfs_stop_workers(&fs_info->delayed_workers);
|
2011-06-30 18:42:28 +00:00
|
|
|
btrfs_stop_workers(&fs_info->caching_workers);
|
2011-05-23 12:30:00 +00:00
|
|
|
btrfs_stop_workers(&fs_info->readahead_workers);
|
2012-10-25 09:28:04 +00:00
|
|
|
btrfs_stop_workers(&fs_info->flush_workers);
|
2008-04-30 17:59:35 +00:00
|
|
|
|
2011-11-09 12:44:05 +00:00
|
|
|
#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY
|
|
|
|
if (btrfs_test_opt(root, CHECK_INTEGRITY))
|
|
|
|
btrfsic_unmount(root, fs_info->fs_devices);
|
|
|
|
#endif
|
|
|
|
|
2008-05-13 17:46:40 +00:00
|
|
|
btrfs_close_devices(fs_info->fs_devices);
|
2008-03-24 19:01:56 +00:00
|
|
|
btrfs_mapping_tree_free(&fs_info->mapping_tree);
|
2008-04-14 13:48:18 +00:00
|
|
|
|
2013-01-29 10:09:20 +00:00
|
|
|
percpu_counter_destroy(&fs_info->dirty_metadata_bytes);
|
2013-01-29 10:10:51 +00:00
|
|
|
percpu_counter_destroy(&fs_info->delalloc_bytes);
|
2008-03-26 14:28:07 +00:00
|
|
|
bdi_destroy(&fs_info->bdi);
|
2009-09-21 20:00:26 +00:00
|
|
|
cleanup_srcu_struct(&fs_info->subvol_srcu);
|
2008-03-24 19:01:56 +00:00
|
|
|
|
2013-01-29 23:40:14 +00:00
|
|
|
btrfs_free_stripe_hash_table(fs_info);
|
|
|
|
|
2007-02-02 14:18:22 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-05-06 11:23:47 +00:00
|
|
|
int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
|
|
|
|
int atomic)
|
2007-10-15 20:14:19 +00:00
|
|
|
{
|
2008-05-12 17:39:03 +00:00
|
|
|
int ret;
|
2010-08-06 17:21:20 +00:00
|
|
|
struct inode *btree_inode = buf->pages[0]->mapping->host;
|
2008-05-12 17:39:03 +00:00
|
|
|
|
2012-03-13 13:38:00 +00:00
|
|
|
ret = extent_buffer_uptodate(buf);
|
2008-05-12 17:39:03 +00:00
|
|
|
if (!ret)
|
|
|
|
return ret;
|
|
|
|
|
|
|
|
ret = verify_parent_transid(&BTRFS_I(btree_inode)->io_tree, buf,
|
2012-05-06 11:23:47 +00:00
|
|
|
parent_transid, atomic);
|
|
|
|
if (ret == -EAGAIN)
|
|
|
|
return ret;
|
2008-05-12 17:39:03 +00:00
|
|
|
return !ret;
|
2007-10-15 20:14:19 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
int btrfs_set_buffer_uptodate(struct extent_buffer *buf)
|
2007-06-28 19:57:36 +00:00
|
|
|
{
|
2012-03-13 13:38:00 +00:00
|
|
|
return set_extent_buffer_uptodate(buf);
|
2007-10-15 20:14:19 +00:00
|
|
|
}
|
2007-08-07 20:15:09 +00:00
|
|
|
|
2007-10-15 20:14:19 +00:00
|
|
|
void btrfs_mark_buffer_dirty(struct extent_buffer *buf)
|
|
|
|
{
|
2010-08-06 17:21:20 +00:00
|
|
|
struct btrfs_root *root = BTRFS_I(buf->pages[0]->mapping->host)->root;
|
2007-10-15 20:14:19 +00:00
|
|
|
u64 transid = btrfs_header_generation(buf);
|
2009-03-13 15:00:37 +00:00
|
|
|
int was_dirty;
|
Btrfs: Change btree locking to use explicit blocking points
Most of the btrfs metadata operations can be protected by a spinlock,
but some operations still need to schedule.
So far, btrfs has been using a mutex along with a trylock loop,
most of the time it is able to avoid going for the full mutex, so
the trylock loop is a big performance gain.
This commit is step one for getting rid of the blocking locks entirely.
btrfs_tree_lock takes a spinlock, and the code explicitly switches
to a blocking lock when it starts an operation that can schedule.
We'll be able get rid of the blocking locks in smaller pieces over time.
Tracing allows us to find the most common cause of blocking, so we
can start with the hot spots first.
The basic idea is:
btrfs_tree_lock() returns with the spin lock held
btrfs_set_lock_blocking() sets the EXTENT_BUFFER_BLOCKING bit in
the extent buffer flags, and then drops the spin lock. The buffer is
still considered locked by all of the btrfs code.
If btrfs_tree_lock gets the spinlock but finds the blocking bit set, it drops
the spin lock and waits on a wait queue for the blocking bit to go away.
Much of the code that needs to set the blocking bit finishes without actually
blocking a good percentage of the time. So, an adaptive spin is still
used against the blocking bit to avoid very high context switch rates.
btrfs_clear_lock_blocking() clears the blocking bit and returns
with the spinlock held again.
btrfs_tree_unlock() can be called on either blocking or spinning locks,
it does the right thing based on the blocking bit.
ctree.c has a helper function to set/clear all the locked buffers in a
path as blocking.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2009-02-04 14:25:08 +00:00
|
|
|
|
2009-03-09 15:45:38 +00:00
|
|
|
btrfs_assert_tree_locked(buf);
|
2012-11-03 10:58:34 +00:00
|
|
|
if (transid != root->fs_info->generation)
|
|
|
|
WARN(1, KERN_CRIT "btrfs transid mismatch buffer %llu, "
|
2009-01-06 02:25:51 +00:00
|
|
|
"found %llu running %llu\n",
|
2007-10-15 20:15:53 +00:00
|
|
|
(unsigned long long)buf->start,
|
2009-01-06 02:25:51 +00:00
|
|
|
(unsigned long long)transid,
|
|
|
|
(unsigned long long)root->fs_info->generation);
|
2012-03-13 13:38:00 +00:00
|
|
|
was_dirty = set_extent_buffer_dirty(buf);
|
2013-01-29 10:09:20 +00:00
|
|
|
if (!was_dirty)
|
|
|
|
__percpu_counter_add(&root->fs_info->dirty_metadata_bytes,
|
|
|
|
buf->len,
|
|
|
|
root->fs_info->dirty_metadata_batch);
|
2007-02-02 14:18:22 +00:00
|
|
|
}
|
|
|
|
|
2012-11-14 14:34:34 +00:00
|
|
|
static void __btrfs_btree_balance_dirty(struct btrfs_root *root,
|
|
|
|
int flush_delayed)
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* looks as though older kernels can get into trouble with
|
|
|
|
* this code, they end up stuck in balance_dirty_pages forever
|
|
|
|
*/
|
2013-01-29 10:09:20 +00:00
|
|
|
int ret;
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
|
|
|
|
if (current->flags & PF_MEMALLOC)
|
|
|
|
return;
|
|
|
|
|
2012-11-14 14:34:34 +00:00
|
|
|
if (flush_delayed)
|
|
|
|
btrfs_balance_delayed_items(root);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
|
2013-01-29 10:09:20 +00:00
|
|
|
ret = percpu_counter_compare(&root->fs_info->dirty_metadata_bytes,
|
|
|
|
BTRFS_DIRTY_METADATA_THRESH);
|
|
|
|
if (ret > 0) {
|
2012-12-12 00:00:21 +00:00
|
|
|
balance_dirty_pages_ratelimited(
|
|
|
|
root->fs_info->btree_inode->i_mapping);
|
btrfs: implement delayed inode items operation
Changelog V5 -> V6:
- Fix oom when the memory load is high, by storing the delayed nodes into the
root's radix tree, and letting btrfs inodes go.
Changelog V4 -> V5:
- Fix the race on adding the delayed node to the inode, which is spotted by
Chris Mason.
- Merge Chris Mason's incremental patch into this patch.
- Fix deadlock between readdir() and memory fault, which is reported by
Itaru Kitayama.
Changelog V3 -> V4:
- Fix nested lock, which is reported by Itaru Kitayama, by updating space cache
inode in time.
Changelog V2 -> V3:
- Fix the race between the delayed worker and the task which does delayed items
balance, which is reported by Tsutomu Itoh.
- Modify the patch address David Sterba's comment.
- Fix the bug of the cpu recursion spinlock, reported by Chris Mason
Changelog V1 -> V2:
- break up the global rb-tree, use a list to manage the delayed nodes,
which is created for every directory and file, and used to manage the
delayed directory name index items and the delayed inode item.
- introduce a worker to deal with the delayed nodes.
Compare with Ext3/4, the performance of file creation and deletion on btrfs
is very poor. the reason is that btrfs must do a lot of b+ tree insertions,
such as inode item, directory name item, directory name index and so on.
If we can do some delayed b+ tree insertion or deletion, we can improve the
performance, so we made this patch which implemented delayed directory name
index insertion/deletion and delayed inode update.
Implementation:
- introduce a delayed root object into the filesystem, that use two lists to
manage the delayed nodes which are created for every file/directory.
One is used to manage all the delayed nodes that have delayed items. And the
other is used to manage the delayed nodes which is waiting to be dealt with
by the work thread.
- Every delayed node has two rb-tree, one is used to manage the directory name
index which is going to be inserted into b+ tree, and the other is used to
manage the directory name index which is going to be deleted from b+ tree.
- introduce a worker to deal with the delayed operation. This worker is used
to deal with the works of the delayed directory name index items insertion
and deletion and the delayed inode update.
When the delayed items is beyond the lower limit, we create works for some
delayed nodes and insert them into the work queue of the worker, and then
go back.
When the delayed items is beyond the upper bound, we create works for all
the delayed nodes that haven't been dealt with, and insert them into the work
queue of the worker, and then wait for that the untreated items is below some
threshold value.
- When we want to insert a directory name index into b+ tree, we just add the
information into the delayed inserting rb-tree.
And then we check the number of the delayed items and do delayed items
balance. (The balance policy is above.)
- When we want to delete a directory name index from the b+ tree, we search it
in the inserting rb-tree at first. If we look it up, just drop it. If not,
add the key of it into the delayed deleting rb-tree.
Similar to the delayed inserting rb-tree, we also check the number of the
delayed items and do delayed items balance.
(The same to inserting manipulation)
- When we want to update the metadata of some inode, we cached the data of the
inode into the delayed node. the worker will flush it into the b+ tree after
dealing with the delayed insertion and deletion.
- We will move the delayed node to the tail of the list after we access the
delayed node, By this way, we can cache more delayed items and merge more
inode updates.
- If we want to commit transaction, we will deal with all the delayed node.
- the delayed node will be freed when we free the btrfs inode.
- Before we log the inode items, we commit all the directory name index items
and the delayed inode update.
I did a quick test by the benchmark tool[1] and found we can improve the
performance of file creation by ~15%, and file deletion by ~20%.
Before applying this patch:
Create files:
Total files: 50000
Total time: 1.096108
Average time: 0.000022
Delete files:
Total files: 50000
Total time: 1.510403
Average time: 0.000030
After applying this patch:
Create files:
Total files: 50000
Total time: 0.932899
Average time: 0.000019
Delete files:
Total files: 50000
Total time: 1.215732
Average time: 0.000024
[1] http://marc.info/?l=linux-btrfs&m=128212635122920&q=p3
Many thanks for Kitayama-san's help!
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Itaru Kitayama <kitayama@cl.bb4u.ne.jp>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-04-22 10:12:22 +00:00
|
|
|
}
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2012-11-14 14:34:34 +00:00
|
|
|
void btrfs_btree_balance_dirty(struct btrfs_root *root)
|
2007-05-02 19:53:43 +00:00
|
|
|
{
|
2012-11-14 14:34:34 +00:00
|
|
|
__btrfs_btree_balance_dirty(root, 1);
|
|
|
|
}
|
2009-05-18 14:41:58 +00:00
|
|
|
|
2012-11-14 14:34:34 +00:00
|
|
|
void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root)
|
|
|
|
{
|
|
|
|
__btrfs_btree_balance_dirty(root, 0);
|
2007-05-02 19:53:43 +00:00
|
|
|
}
|
2007-10-15 20:17:34 +00:00
|
|
|
|
2008-05-12 16:59:19 +00:00
|
|
|
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid)
|
2007-10-15 20:17:34 +00:00
|
|
|
{
|
2010-08-06 17:21:20 +00:00
|
|
|
struct btrfs_root *root = BTRFS_I(buf->pages[0]->mapping->host)->root;
|
2012-03-13 13:38:00 +00:00
|
|
|
return btree_read_extent_buffer_pages(root, buf, 0, parent_transid);
|
2007-10-15 20:17:34 +00:00
|
|
|
}
|
2007-11-08 02:08:01 +00:00
|
|
|
|
2012-03-05 23:06:18 +00:00
|
|
|
static int btrfs_check_super_valid(struct btrfs_fs_info *fs_info,
|
2011-01-06 11:30:25 +00:00
|
|
|
int read_only)
|
|
|
|
{
|
2012-03-05 23:06:18 +00:00
|
|
|
if (btrfs_super_csum_type(fs_info->super_copy) >= ARRAY_SIZE(btrfs_csum_sizes)) {
|
|
|
|
printk(KERN_ERR "btrfs: unsupported checksum algorithm\n");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
if (read_only)
|
2012-03-05 23:06:18 +00:00
|
|
|
return 0;
|
2011-01-06 11:30:25 +00:00
|
|
|
|
2012-03-05 23:06:18 +00:00
|
|
|
return 0;
|
2011-01-06 11:30:25 +00:00
|
|
|
}
|
|
|
|
|
2012-08-01 11:45:52 +00:00
|
|
|
void btrfs_error_commit_super(struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
mutex_lock(&root->fs_info->cleaner_mutex);
|
|
|
|
btrfs_run_delayed_iputs(root);
|
|
|
|
mutex_unlock(&root->fs_info->cleaner_mutex);
|
|
|
|
|
|
|
|
down_write(&root->fs_info->cleanup_work_sem);
|
|
|
|
up_write(&root->fs_info->cleanup_work_sem);
|
|
|
|
|
|
|
|
/* cleanup FS via transaction */
|
|
|
|
btrfs_cleanup_transaction(root);
|
|
|
|
}
|
|
|
|
|
2013-02-13 16:09:14 +00:00
|
|
|
static void btrfs_destroy_ordered_operations(struct btrfs_transaction *t,
|
|
|
|
struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_inode *btrfs_inode;
|
|
|
|
struct list_head splice;
|
|
|
|
|
|
|
|
INIT_LIST_HEAD(&splice);
|
|
|
|
|
|
|
|
mutex_lock(&root->fs_info->ordered_operations_mutex);
|
|
|
|
spin_lock(&root->fs_info->ordered_extent_lock);
|
|
|
|
|
2013-02-13 16:09:14 +00:00
|
|
|
list_splice_init(&t->ordered_operations, &splice);
|
2011-01-06 11:30:25 +00:00
|
|
|
while (!list_empty(&splice)) {
|
|
|
|
btrfs_inode = list_entry(splice.next, struct btrfs_inode,
|
|
|
|
ordered_operations);
|
|
|
|
|
|
|
|
list_del_init(&btrfs_inode->ordered_operations);
|
|
|
|
|
|
|
|
btrfs_invalidate_inodes(btrfs_inode->root);
|
|
|
|
}
|
|
|
|
|
|
|
|
spin_unlock(&root->fs_info->ordered_extent_lock);
|
|
|
|
mutex_unlock(&root->fs_info->ordered_operations_mutex);
|
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
static void btrfs_destroy_ordered_extents(struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_ordered_extent *ordered;
|
|
|
|
|
|
|
|
spin_lock(&root->fs_info->ordered_extent_lock);
|
2013-01-31 19:30:08 +00:00
|
|
|
/*
|
|
|
|
* This will just short circuit the ordered completion stuff which will
|
|
|
|
* make sure the ordered extent gets properly cleaned up.
|
|
|
|
*/
|
|
|
|
list_for_each_entry(ordered, &root->fs_info->ordered_extents,
|
|
|
|
root_extent_list)
|
|
|
|
set_bit(BTRFS_ORDERED_IOERR, &ordered->flags);
|
2011-01-06 11:30:25 +00:00
|
|
|
spin_unlock(&root->fs_info->ordered_extent_lock);
|
|
|
|
}
|
|
|
|
|
2012-03-12 15:03:00 +00:00
|
|
|
int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,
|
|
|
|
struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct rb_node *node;
|
|
|
|
struct btrfs_delayed_ref_root *delayed_refs;
|
|
|
|
struct btrfs_delayed_ref_node *ref;
|
|
|
|
int ret = 0;
|
|
|
|
|
|
|
|
delayed_refs = &trans->delayed_refs;
|
|
|
|
|
|
|
|
spin_lock(&delayed_refs->lock);
|
|
|
|
if (delayed_refs->num_entries == 0) {
|
2011-04-25 23:43:52 +00:00
|
|
|
spin_unlock(&delayed_refs->lock);
|
2011-01-06 11:30:25 +00:00
|
|
|
printk(KERN_INFO "delayed_refs has NO entry\n");
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2012-05-31 15:06:33 +00:00
|
|
|
while ((node = rb_first(&delayed_refs->root)) != NULL) {
|
2013-01-30 21:03:59 +00:00
|
|
|
struct btrfs_delayed_ref_head *head = NULL;
|
2011-01-06 11:30:25 +00:00
|
|
|
|
2013-01-30 21:03:59 +00:00
|
|
|
ref = rb_entry(node, struct btrfs_delayed_ref_node, rb_node);
|
2011-01-06 11:30:25 +00:00
|
|
|
atomic_set(&ref->refs, 1);
|
|
|
|
if (btrfs_delayed_ref_is_head(ref)) {
|
|
|
|
|
|
|
|
head = btrfs_delayed_node_to_head(ref);
|
2012-05-31 15:06:33 +00:00
|
|
|
if (!mutex_trylock(&head->mutex)) {
|
|
|
|
atomic_inc(&ref->refs);
|
|
|
|
spin_unlock(&delayed_refs->lock);
|
|
|
|
|
|
|
|
/* Need to wait for the delayed ref to run */
|
|
|
|
mutex_lock(&head->mutex);
|
|
|
|
mutex_unlock(&head->mutex);
|
|
|
|
btrfs_put_delayed_ref(ref);
|
|
|
|
|
2012-06-18 13:23:18 +00:00
|
|
|
spin_lock(&delayed_refs->lock);
|
2012-05-31 15:06:33 +00:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2012-11-21 02:21:28 +00:00
|
|
|
btrfs_free_delayed_extent_op(head->extent_op);
|
2011-01-06 11:30:25 +00:00
|
|
|
delayed_refs->num_heads--;
|
|
|
|
if (list_empty(&head->cluster))
|
|
|
|
delayed_refs->num_heads_ready--;
|
|
|
|
list_del_init(&head->cluster);
|
|
|
|
}
|
2013-01-30 21:03:59 +00:00
|
|
|
|
2012-05-31 15:06:33 +00:00
|
|
|
ref->in_tree = 0;
|
|
|
|
rb_erase(&ref->rb_node, &delayed_refs->root);
|
|
|
|
delayed_refs->num_entries--;
|
2013-01-30 21:03:59 +00:00
|
|
|
if (head)
|
|
|
|
mutex_unlock(&head->mutex);
|
2011-01-06 11:30:25 +00:00
|
|
|
spin_unlock(&delayed_refs->lock);
|
|
|
|
btrfs_put_delayed_ref(ref);
|
|
|
|
|
|
|
|
cond_resched();
|
|
|
|
spin_lock(&delayed_refs->lock);
|
|
|
|
}
|
|
|
|
|
|
|
|
spin_unlock(&delayed_refs->lock);
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
static void btrfs_destroy_pending_snapshots(struct btrfs_transaction *t)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_pending_snapshot *snapshot;
|
|
|
|
struct list_head splice;
|
|
|
|
|
|
|
|
INIT_LIST_HEAD(&splice);
|
|
|
|
|
|
|
|
list_splice_init(&t->pending_snapshots, &splice);
|
|
|
|
|
|
|
|
while (!list_empty(&splice)) {
|
|
|
|
snapshot = list_entry(splice.next,
|
|
|
|
struct btrfs_pending_snapshot,
|
|
|
|
list);
|
|
|
|
|
|
|
|
list_del_init(&snapshot->list);
|
|
|
|
|
|
|
|
kfree(snapshot);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-03-01 13:56:26 +00:00
|
|
|
static void btrfs_destroy_delalloc_inodes(struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_inode *btrfs_inode;
|
|
|
|
struct list_head splice;
|
|
|
|
|
|
|
|
INIT_LIST_HEAD(&splice);
|
|
|
|
|
|
|
|
spin_lock(&root->fs_info->delalloc_lock);
|
2011-06-09 10:02:51 +00:00
|
|
|
list_splice_init(&root->fs_info->delalloc_inodes, &splice);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
while (!list_empty(&splice)) {
|
|
|
|
btrfs_inode = list_entry(splice.next, struct btrfs_inode,
|
|
|
|
delalloc_inodes);
|
|
|
|
|
|
|
|
list_del_init(&btrfs_inode->delalloc_inodes);
|
2013-01-29 10:11:59 +00:00
|
|
|
clear_bit(BTRFS_INODE_IN_DELALLOC_LIST,
|
|
|
|
&btrfs_inode->runtime_flags);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
btrfs_invalidate_inodes(btrfs_inode->root);
|
|
|
|
}
|
|
|
|
|
|
|
|
spin_unlock(&root->fs_info->delalloc_lock);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int btrfs_destroy_marked_extents(struct btrfs_root *root,
|
|
|
|
struct extent_io_tree *dirty_pages,
|
|
|
|
int mark)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
struct page *page;
|
|
|
|
struct inode *btree_inode = root->fs_info->btree_inode;
|
|
|
|
struct extent_buffer *eb;
|
|
|
|
u64 start = 0;
|
|
|
|
u64 end;
|
|
|
|
u64 offset;
|
|
|
|
unsigned long index;
|
|
|
|
|
|
|
|
while (1) {
|
|
|
|
ret = find_first_extent_bit(dirty_pages, start, &start, &end,
|
2012-09-27 21:07:30 +00:00
|
|
|
mark, NULL);
|
2011-01-06 11:30:25 +00:00
|
|
|
if (ret)
|
|
|
|
break;
|
|
|
|
|
|
|
|
clear_extent_bits(dirty_pages, start, end, mark, GFP_NOFS);
|
|
|
|
while (start <= end) {
|
|
|
|
index = start >> PAGE_CACHE_SHIFT;
|
|
|
|
start = (u64)(index + 1) << PAGE_CACHE_SHIFT;
|
|
|
|
page = find_get_page(btree_inode->i_mapping, index);
|
|
|
|
if (!page)
|
|
|
|
continue;
|
|
|
|
offset = page_offset(page);
|
|
|
|
|
|
|
|
spin_lock(&dirty_pages->buffer_lock);
|
|
|
|
eb = radix_tree_lookup(
|
|
|
|
&(&BTRFS_I(page->mapping->host)->io_tree)->buffer,
|
|
|
|
offset >> PAGE_CACHE_SHIFT);
|
|
|
|
spin_unlock(&dirty_pages->buffer_lock);
|
2012-05-31 19:54:30 +00:00
|
|
|
if (eb)
|
2011-01-06 11:30:25 +00:00
|
|
|
ret = test_and_clear_bit(EXTENT_BUFFER_DIRTY,
|
|
|
|
&eb->bflags);
|
|
|
|
if (PageWriteback(page))
|
|
|
|
end_page_writeback(page);
|
|
|
|
|
|
|
|
lock_page(page);
|
|
|
|
if (PageDirty(page)) {
|
|
|
|
clear_page_dirty_for_io(page);
|
|
|
|
spin_lock_irq(&page->mapping->tree_lock);
|
|
|
|
radix_tree_tag_clear(&page->mapping->page_tree,
|
|
|
|
page_index(page),
|
|
|
|
PAGECACHE_TAG_DIRTY);
|
|
|
|
spin_unlock_irq(&page->mapping->tree_lock);
|
|
|
|
}
|
|
|
|
|
|
|
|
unlock_page(page);
|
2012-05-31 19:54:30 +00:00
|
|
|
page_cache_release(page);
|
2011-01-06 11:30:25 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int btrfs_destroy_pinned_extent(struct btrfs_root *root,
|
|
|
|
struct extent_io_tree *pinned_extents)
|
|
|
|
{
|
|
|
|
struct extent_io_tree *unpin;
|
|
|
|
u64 start;
|
|
|
|
u64 end;
|
|
|
|
int ret;
|
2012-06-14 08:23:21 +00:00
|
|
|
bool loop = true;
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
unpin = pinned_extents;
|
2012-06-14 08:23:21 +00:00
|
|
|
again:
|
2011-01-06 11:30:25 +00:00
|
|
|
while (1) {
|
|
|
|
ret = find_first_extent_bit(unpin, 0, &start, &end,
|
2012-09-27 21:07:30 +00:00
|
|
|
EXTENT_DIRTY, NULL);
|
2011-01-06 11:30:25 +00:00
|
|
|
if (ret)
|
|
|
|
break;
|
|
|
|
|
|
|
|
/* opt_discard */
|
2011-03-24 10:24:27 +00:00
|
|
|
if (btrfs_test_opt(root, DISCARD))
|
|
|
|
ret = btrfs_error_discard_extent(root, start,
|
|
|
|
end + 1 - start,
|
|
|
|
NULL);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
clear_extent_dirty(unpin, start, end, GFP_NOFS);
|
|
|
|
btrfs_error_unpin_extent_range(root, start, end);
|
|
|
|
cond_resched();
|
|
|
|
}
|
|
|
|
|
2012-06-14 08:23:21 +00:00
|
|
|
if (loop) {
|
|
|
|
if (unpin == &root->fs_info->freed_extents[0])
|
|
|
|
unpin = &root->fs_info->freed_extents[1];
|
|
|
|
else
|
|
|
|
unpin = &root->fs_info->freed_extents[0];
|
|
|
|
loop = false;
|
|
|
|
goto again;
|
|
|
|
}
|
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-03-01 16:24:58 +00:00
|
|
|
void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans,
|
|
|
|
struct btrfs_root *root)
|
|
|
|
{
|
|
|
|
btrfs_destroy_delayed_refs(cur_trans, root);
|
|
|
|
btrfs_block_rsv_release(root, &root->fs_info->trans_block_rsv,
|
|
|
|
cur_trans->dirty_pages.dirty_bytes);
|
|
|
|
|
|
|
|
/* FIXME: cleanup wait for commit */
|
|
|
|
cur_trans->in_commit = 1;
|
|
|
|
cur_trans->blocked = 1;
|
2012-05-31 19:49:57 +00:00
|
|
|
wake_up(&root->fs_info->transaction_blocked_wait);
|
2012-03-01 16:24:58 +00:00
|
|
|
|
|
|
|
cur_trans->blocked = 0;
|
2012-05-31 19:49:57 +00:00
|
|
|
wake_up(&root->fs_info->transaction_wait);
|
2012-03-01 16:24:58 +00:00
|
|
|
|
|
|
|
cur_trans->commit_done = 1;
|
2012-05-31 19:49:57 +00:00
|
|
|
wake_up(&cur_trans->commit_wait);
|
2012-03-01 16:24:58 +00:00
|
|
|
|
2012-06-14 08:23:22 +00:00
|
|
|
btrfs_destroy_delayed_inodes(root);
|
|
|
|
btrfs_assert_delayed_root_empty(root);
|
2012-03-01 16:24:58 +00:00
|
|
|
|
|
|
|
btrfs_destroy_pending_snapshots(cur_trans);
|
|
|
|
|
|
|
|
btrfs_destroy_marked_extents(root, &cur_trans->dirty_pages,
|
|
|
|
EXTENT_DIRTY);
|
2012-06-14 08:23:20 +00:00
|
|
|
btrfs_destroy_pinned_extent(root,
|
|
|
|
root->fs_info->pinned_extents);
|
2012-03-01 16:24:58 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
memset(cur_trans, 0, sizeof(*cur_trans));
|
|
|
|
kmem_cache_free(btrfs_transaction_cachep, cur_trans);
|
|
|
|
*/
|
|
|
|
}
|
|
|
|
|
|
|
|
int btrfs_cleanup_transaction(struct btrfs_root *root)
|
2011-01-06 11:30:25 +00:00
|
|
|
{
|
|
|
|
struct btrfs_transaction *t;
|
|
|
|
LIST_HEAD(list);
|
|
|
|
|
|
|
|
mutex_lock(&root->fs_info->transaction_kthread_mutex);
|
|
|
|
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_lock(&root->fs_info->trans_lock);
|
2011-01-06 11:30:25 +00:00
|
|
|
list_splice_init(&root->fs_info->trans_list, &list);
|
2011-04-11 21:25:13 +00:00
|
|
|
root->fs_info->trans_no_join = 1;
|
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
while (!list_empty(&list)) {
|
|
|
|
t = list_entry(list.next, struct btrfs_transaction, list);
|
|
|
|
|
2013-02-13 16:09:14 +00:00
|
|
|
btrfs_destroy_ordered_operations(t, root);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
btrfs_destroy_ordered_extents(root);
|
|
|
|
|
|
|
|
btrfs_destroy_delayed_refs(t, root);
|
|
|
|
|
|
|
|
btrfs_block_rsv_release(root,
|
|
|
|
&root->fs_info->trans_block_rsv,
|
|
|
|
t->dirty_pages.dirty_bytes);
|
|
|
|
|
|
|
|
/* FIXME: cleanup wait for commit */
|
|
|
|
t->in_commit = 1;
|
|
|
|
t->blocked = 1;
|
2012-08-01 19:36:24 +00:00
|
|
|
smp_mb();
|
2011-01-06 11:30:25 +00:00
|
|
|
if (waitqueue_active(&root->fs_info->transaction_blocked_wait))
|
|
|
|
wake_up(&root->fs_info->transaction_blocked_wait);
|
|
|
|
|
|
|
|
t->blocked = 0;
|
2012-08-01 19:36:24 +00:00
|
|
|
smp_mb();
|
2011-01-06 11:30:25 +00:00
|
|
|
if (waitqueue_active(&root->fs_info->transaction_wait))
|
|
|
|
wake_up(&root->fs_info->transaction_wait);
|
|
|
|
|
|
|
|
t->commit_done = 1;
|
2012-08-01 19:36:24 +00:00
|
|
|
smp_mb();
|
2011-01-06 11:30:25 +00:00
|
|
|
if (waitqueue_active(&t->commit_wait))
|
|
|
|
wake_up(&t->commit_wait);
|
|
|
|
|
2012-06-14 08:23:22 +00:00
|
|
|
btrfs_destroy_delayed_inodes(root);
|
|
|
|
btrfs_assert_delayed_root_empty(root);
|
|
|
|
|
2011-01-06 11:30:25 +00:00
|
|
|
btrfs_destroy_pending_snapshots(t);
|
|
|
|
|
|
|
|
btrfs_destroy_delalloc_inodes(root);
|
|
|
|
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_lock(&root->fs_info->trans_lock);
|
2011-01-06 11:30:25 +00:00
|
|
|
root->fs_info->running_transaction = NULL;
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
2011-01-06 11:30:25 +00:00
|
|
|
|
|
|
|
btrfs_destroy_marked_extents(root, &t->dirty_pages,
|
|
|
|
EXTENT_DIRTY);
|
|
|
|
|
|
|
|
btrfs_destroy_pinned_extent(root,
|
|
|
|
root->fs_info->pinned_extents);
|
|
|
|
|
2011-04-11 19:45:29 +00:00
|
|
|
atomic_set(&t->use_count, 0);
|
2011-01-06 11:30:25 +00:00
|
|
|
list_del_init(&t->list);
|
|
|
|
memset(t, 0, sizeof(*t));
|
|
|
|
kmem_cache_free(btrfs_transaction_cachep, t);
|
|
|
|
}
|
|
|
|
|
2011-04-11 21:25:13 +00:00
|
|
|
spin_lock(&root->fs_info->trans_lock);
|
|
|
|
root->fs_info->trans_no_join = 0;
|
|
|
|
spin_unlock(&root->fs_info->trans_lock);
|
2011-01-06 11:30:25 +00:00
|
|
|
mutex_unlock(&root->fs_info->transaction_kthread_mutex);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-01-24 21:13:08 +00:00
|
|
|
static struct extent_io_ops btree_extent_io_ops = {
|
2008-04-09 20:28:12 +00:00
|
|
|
.readpage_end_io_hook = btree_readpage_end_io_hook,
|
2011-06-10 11:55:54 +00:00
|
|
|
.readpage_io_failed_hook = btree_io_failed_hook,
|
2008-03-24 19:01:56 +00:00
|
|
|
.submit_bio_hook = btree_submit_bio_hook,
|
2008-03-24 19:02:07 +00:00
|
|
|
/* note we're sharing with inode.c for the merge bio hook */
|
|
|
|
.merge_bio_hook = btrfs_merge_bio_hook,
|
2007-11-08 02:08:01 +00:00
|
|
|
};
|