Commit Graph

31 Commits

Author SHA1 Message Date
Daniel Rosenberg 471fbbea7f ext4: handle casefolding with encryption
This adds support for encryption with casefolding.

Since the name on disk is case preserving, and also encrypted, we can no
longer just recompute the hash on the fly. Additionally, to avoid
leaking extra information from the hash of the unencrypted name, we use
siphash via an fscrypt v2 policy.

The hash is stored at the end of the directory entry for all entries
inside of an encrypted and casefolded directory apart from those that
deal with '.' and '..'. This way, the change is backwards compatible
with existing ext4 filesystems.

[ Changed to advertise this feature via the file:
  /sys/fs/ext4/features/encrypted_casefold -- TYT ]

Signed-off-by: Daniel Rosenberg <drosen@google.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Link: https://lore.kernel.org/r/20210319073414.1381041-2-drosen@google.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-04-05 22:04:20 -04:00
Harshad Shirwadkar b1b7dce3f0 ext4: add docs about fast commit idempotence
Fast commit on-disk format is designed such that the replay of these
tags can be idempotent. This patch adds documentation in the code in
form of comments and in form kernel docs that describes these
characteristics. This patch also adds a TODO item needed to ensure
kernel fast commit replay idempotence.

Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Link: https://lore.kernel.org/r/20201119232822.1860882-1-harshadshirwadkar@gmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-12-17 13:30:44 -05:00
Harshad Shirwadkar a44ad6835d ext4: describe fast_commit feature flags
Fast commit feature has flags in the file system as well in JBD2. The
meaning of fast commit feature flags can get confusing. Update docs
and code to add more documentation about it.

Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20201106035911.1942128-2-harshadshirwadkar@gmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-11-06 23:01:01 -05:00
Harshad Shirwadkar f5b8b297b0 doc: update ext4 and journalling docs to include fast commit feature
This patch adds necessary documentation for fast commits.

Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Link: https://lore.kernel.org/r/20201015203802.3597742-2-harshadshirwadkar@gmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-10-21 23:19:43 -04:00
Alexander A. Klimov e65bf6e468 ext4: replace HTTP links with HTTPS ones
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.

Deterministic algorithm:
For each file:
  If not .svg:
    For each line:
      If doesn't contain `\bxmlns\b`:
        For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
          If both the HTTP and HTTPS versions
          return 200 OK and serve the same content:
            Replace HTTP with HTTPS.

Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Link: https://lore.kernel.org/r/20200706190339.20709-1-grandmaster@al2klimov.de
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-08-06 01:01:57 -04:00
Ira Weiny 6c0d077ff8 fs/ext4: Disallow verity if inode is DAX
Verity and DAX are incompatible.  Changing the DAX mode due to a verity
flag change is wrong without a corresponding address_space_operations
update.

Make the 2 options mutually exclusive by returning an error if DAX was
set first.

(Setting DAX is already disabled if Verity is set first.)

Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>

Link: https://lore.kernel.org/r/20200528150003.828793-3-ira.weiny@intel.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-05-28 22:09:47 -04:00
Linus Torvalds 70cb0d02b5 Added new ext4 debugging ioctls to allow userspace to get information
about the state of the extent status cache.
 
 Dropped workaround for pre-1970 dates which were encoded incorrectly
 in pre-4.4 kernels.  Since both the kernel correctly generates, and
 e2fsck detects and fixes this issue for the past four years, it'e time
 to drop the workaround.  (Also, it's not like files with dates in the
 distant past were all that common in the first place.)
 
 A lot of miscellaneous bug fixes and cleanups, including some ext4
 Documentation fixes.  Also included are two minor bug fixes in
 fs/unicode.
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEK2m5VNv+CHkogTfJ8vlZVpUNgaMFAl2D5ZIACgkQ8vlZVpUN
 gaO8NQf+ONLK5nu8KUk14uh8MOXMisiT+g1iqhynZcqtuZzTr4nKqUbHLmPDHrCC
 RiD/gkLhp6u+UlzYRJq6nudunid1be2/1bjoUm6lddE4XLtbeGHhZsGn1+9K/wy+
 l8UFMXd8fCOlXNzajS85Hb0KSuzlrGYEjSrNecSa3KLxrv1kM1+FyKFcqQ7Ejs5/
 VZYNtWo69R4wSEIawGkEZuNu/wFeLOzqJgxFJLo6zFxTAp449bbEduz12ssmkUhl
 QbXH9cXLR4pAZykzMRqHC8UFFTKmpLnc5EiT1Ajxzu4EAzB1SzqRJvbz/3CF3d/Z
 gBKDrDlasv75VJqVtqw4mCxmEoEYjw==
 =Iwrf
 -----END PGP SIGNATURE-----

Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4

Pull ext4 updates from Ted Ts'o:
 "Added new ext4 debugging ioctls to allow userspace to get information
  about the state of the extent status cache.

  Dropped workaround for pre-1970 dates which were encoded incorrectly
  in pre-4.4 kernels. Since both the kernel correctly generates, and
  e2fsck detects and fixes this issue for the past four years, it'e time
  to drop the workaround. (Also, it's not like files with dates in the
  distant past were all that common in the first place.)

  A lot of miscellaneous bug fixes and cleanups, including some ext4
  Documentation fixes. Also included are two minor bug fixes in
  fs/unicode"

* tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4: (21 commits)
  unicode: make array 'token' static const, makes object smaller
  unicode: Move static keyword to the front of declarations
  ext4: add missing bigalloc documentation.
  ext4: fix kernel oops caused by spurious casefold flag
  ext4: fix integer overflow when calculating commit interval
  ext4: use percpu_counters for extent_status cache hits/misses
  ext4: fix potential use after free after remounting with noblock_validity
  jbd2: add missing tracepoint for reserved handle
  ext4: fix punch hole for inline_data file systems
  ext4: rework reserved cluster accounting when invalidating pages
  ext4: documentation fixes
  ext4: treat buffers with write errors as containing valid data
  ext4: fix warning inside ext4_convert_unwritten_extents_endio
  ext4: set error return correctly when ext4_htree_store_dirent fails
  ext4: drop legacy pre-1970 encoding workaround
  ext4: add new ioctl EXT4_IOC_GET_ES_CACHE
  ext4: add a new ioctl EXT4_IOC_GETSTATE
  ext4: add a new ioctl EXT4_IOC_CLEAR_ES_CACHE
  jbd2: flush_descriptor(): Do not decrease buffer head's ref count
  ext4: remove unnecessary error check
  ...
2019-09-21 13:37:39 -07:00
Ayush Ranjan e85526404c ext4: add missing bigalloc documentation.
There was a broken link for bigalloc.  The page
https://ext4.wiki.kernel.org/index.php/Bigalloc was not migrated into
the current documentation sources.  This patch adds the contents of
that missing page into the section for Bigalloc itself.

Signed-off-by: Ayush Ranjan <ayushr2@illinois.edu>
Link: https://lore.kernel.org/r/20190831154419.GA30357@fa19-cs241-404.cs.illinois.edu
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-07 11:56:47 -04:00
Ayush Ranjan 219db95bbe ext4: documentation fixes
This commit aims to fix the following issues in ext4 documentation:
- Flexible block group docs said that the aim was to group block
  metadata together instead of block group metadata.
- The documentation consistly uses "location" instead of "block number".
  It is easy to confuse location to be an absolute offset on disk. Added
  a line to clarify all location values are in terms of block numbers.
- Dirent2 docs said that the rec_len field is shortened instead of the
  name_len field.
- Typo in bg_checksum description.
- Inode size is 160 bytes now, and hence i_extra_isize is now 32.
- Cluster size formula was incorrect, it did not include the +10 to
  s_log_cluster_size value.
- Typo: there were two s_wtime_hi in the superblock struct.
- Superblock struct was outdated, added the new fields which were part
  of s_reserved earlier.
- Multiple mount protection seems to be implemented in fs/ext4/mmp.c.

Signed-off-by: Ayush Ranjan <ayushr2@illinois.edu>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
2019-08-22 23:18:33 -04:00
Eric Biggers 84fb7ca4b3 ext4: update on-disk format documentation for fs-verity
Document the format of verity files on ext4, and the corresponding inode
and superblock flags.

Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Eric Biggers <ebiggers@google.com>
2019-08-12 19:33:51 -07:00
Mauro Carvalho Chehab 7c116d22ad docs: filesystems: Remove uneeded .rst extension on toctables
There's no need to use a .rst on Sphinx toc tables. As most of
the Documentation don't use, remove the remaing occurrences.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-06-26 16:08:22 -06:00
Darrick J. Wong 8a98ec7c7b docs: promote the ext4 data structures book to top level
Move the ext4 data structures book to Documentation/filesystems/ext4/
since the administrative information moved elsewhere.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-10-05 19:20:08 -04:00
Darrick J. Wong d309121592 docs: move ext4 administrative docs to admin-guide/
Move the ext4 mount option and other administrative stuff to the Linux
administrator's guide.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-10-05 19:11:59 -04:00
Darrick J. Wong c0e3e0406a docs: make ext4 readme tables readable
The tables in the ext4 readme are not particularly space efficient in
the text or html outputs, and they're totally broken in the pdf output.
Convert them into titled paragraphs so that they render more nicely.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-10-02 22:45:25 -04:00
Darrick J. Wong de7abd7bbb docs: fix ext4 documentation table formatting problems
It turns out that the latex table formatters lay out table columns with
the exact proportional widths given in the table metadata, even if text
overflows outside the box.  This was not caught during the initial
import because the HTML renderers are smart enough to fudge the table.

Fix the table column width formatting problems in the data structures
and algorithms documentation so that we don't have squashed columns.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-10-02 22:43:40 -04:00
Theodore Ts'o 0694f8c39f docs: fix up the obviously obsolete bits in the new ext4 documentation
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 16:35:23 -04:00
Darrick J. Wong f5cb282d8b docs: add new ext4 superblock time extension fields
The superblock timestamp fields were enlarged by u8 to be 40 bits wide.
Update the documentation to reflect this.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 16:16:21 -04:00
Darrick J. Wong 6684874af0 docs: create filesystem internal section
Create a new top-level section for documentation of filesystem usage,
on-disk format information, and anything else.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 16:14:02 -04:00
Darrick J. Wong 66d3239a4d ext4: import extended attributes chapter from wiki page
Import the chapter about extended attributes from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:47:00 -04:00
Darrick J. Wong 60edae3a04 ext4: import directory layout chapter from wiki page
Import the chapter about directory layout from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:46:00 -04:00
Darrick J. Wong b4becd48b7 ext4: import inode data fork chapter from wiki page
Import the chapter about inode data fork from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:45:00 -04:00
Darrick J. Wong 46180558f1 ext4: import inodes chapter from wiki page
Import the chapter about inodes from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:44:00 -04:00
Darrick J. Wong 567d118a98 ext4: import journal chapter from wiki page
Import the chapter about the journal from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:43:00 -04:00
Darrick J. Wong 18ba5a45ce ext4: import multi-mount protection chapter from wiki page
Import the chapter about multi-mount protection from the on-disk format
wiki page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:42:00 -04:00
Darrick J. Wong 33dfadc747 ext4: import bitmaps chapter from wiki page
Import the chapter about bitmaps from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:41:00 -04:00
Darrick J. Wong 3c6ba09dd3 ext4: import group descriptors chapter from wiki page
Import the chapter about group descriptors from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:40:00 -04:00
Darrick J. Wong 3db42a2139 ext4: import superblocks chapter from wiki page
Import the chapter about superblocks from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:39:00 -04:00
Darrick J. Wong c09f3bac6d ext4: import high level design chapter from wiki page
Import the chapter about high level design from the on-disk format wiki
page into the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:38:00 -04:00
Darrick J. Wong b2e60723c1 ext4: import on-disk layout book from wiki page
Create the basic structure of the "new" data structures & algorithms
book to be ported over from the on-disk format wiki, and then start by
pulling in the introductory information.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:37:00 -04:00
Darrick J. Wong 489fcb9124 ext4: convert ext4.rst to restructuredtext format
Convert the existing ext4 documentation into rst format and link it in
with the rest of the kernel documentation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:36:00 -04:00
Darrick J. Wong a801e56997 ext4: move ext4.txt into its own directory
Move Documentation/filesystems/ext4.txt into
Documentation/filesystems/ext4/ext4.rst in preparation for adding more
ext4 documentation.

Note that the documentation isn't in rst format yet, but as it's not
linked from anywhere it won't cause build errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-07-29 15:35:00 -04:00