The catkey->name could be unaligned since the address of 'void* record'
is calculated as offset in bytes to a malloc buffer.
The fix is using aligned buffer allocated by grub_malloc for holding
the UTF16 string copied from catkey->name. And use that buffer as
argument for grub_utf16_to_utf8 to convert to UTF8 strings.
In addition, using a new copy of buffer rather than catkey->name itself
for processing the endianess conversion, we can also get rid of the hunk
restoring byte order of catkey->name to what it was previously.
[ 59s] ../grub-core/fs/hfsplus.c: In function 'list_nodes':
[ 59s] ../grub-core/fs/hfsplus.c:738:57: error: taking address of packed member of 'struct grub_hfsplus_catkey' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 738 | *grub_utf16_to_utf8 ((grub_uint8_t *) filename, catkey->name,
[ 59s] | ~~~~~~^~~~~~
[ 59s] ../grub-core/fs/hfsplus.c: In function 'grub_hfsplus_label':
[ 59s] ../grub-core/fs/hfsplus.c:1019:57: error: taking address of packed member of 'struct grub_hfsplus_catkey' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 1019 | *grub_utf16_to_utf8 ((grub_uint8_t *) (*label), catkey->name,
[ 59s] | ~~~~~~^~~~~~
Signed-off-by: Michael Chang <mchang@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Disable the -Wadress-of-packaed-member diagnostic for the
grub_jfs_getent function since the result is found to be false postive.
The leaf is read into memory as continous chunks in size of 32 bytes and
the pointer to its base is aligned, which also guarentee its member
leaf->namepart is aligned.
[ 60s] ../grub-core/fs/jfs.c: In function 'grub_jfs_getent':
[ 60s] ../grub-core/fs/jfs.c:557:44: error: taking address of packed member of 'struct grub_jfs_leaf_dirent' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 60s] 557 | le_to_cpu16_copy (filename + strpos, leaf->namepart, len < diro->data->namecomponentlen ? len
[ 60s] | ~~~~^~~~~~~~~~
[ 60s] ../grub-core/fs/jfs.c:570:48: error: taking address of packed member of 'struct grub_jfs_leaf_next_dirent' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 60s] 570 | le_to_cpu16_copy (filename + strpos, next_leaf->namepart, len < 15 ? len : 15);
[ 60s] | ~~~~~~~~~^~~~~~~~~~
[ 60s] cc1: all warnings being treated as errors
Signed-off-by: Michael Chang <mchang@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Disable the -Wadress-of-packaed-member diagnostic for the
grub_cpio_find_file function since the result is found to be false
postive. Any pointers to member of the 'struct head hd' is aligned even
if the structure is packed without paddings.
[ 59s] In file included from ../grub-core/fs/cpio.c:51:
[ 59s] ../grub-core/fs/cpio_common.c: In function 'grub_cpio_find_file':
[ 59s] ../grub-core/fs/cpio_common.c:58:31: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 58 | data->size = read_number (hd.filesize, ARRAY_SIZE (hd.filesize));
[ 59s] | ~~^~~~~~~~~
[ 59s] ../grub-core/fs/cpio_common.c:60:29: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 60 | *mtime = read_number (hd.mtime, ARRAY_SIZE (hd.mtime));
[ 59s] | ~~^~~~~~
[ 59s] ../grub-core/fs/cpio_common.c:61:28: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 61 | modeval = read_number (hd.mode, ARRAY_SIZE (hd.mode));
[ 59s] | ~~^~~~~
[ 59s] ../grub-core/fs/cpio_common.c:62:29: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 62 | namesize = read_number (hd.namesize, ARRAY_SIZE (hd.namesize));
[ 59s] | ~~^~~~~~~~~
[ 59s] In file included from ../grub-core/fs/cpio_be.c:51:
[ 59s] ../grub-core/fs/cpio_common.c: In function 'grub_cpio_find_file':
[ 59s] ../grub-core/fs/cpio_common.c:58:31: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 58 | data->size = read_number (hd.filesize, ARRAY_SIZE (hd.filesize));
[ 59s] | ~~^~~~~~~~~
[ 59s] ../grub-core/fs/cpio_common.c:60:29: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 60 | *mtime = read_number (hd.mtime, ARRAY_SIZE (hd.mtime));
[ 59s] | ~~^~~~~~
[ 59s] ../grub-core/fs/cpio_common.c:61:28: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 61 | modeval = read_number (hd.mode, ARRAY_SIZE (hd.mode));
[ 59s] | ~~^~~~~
[ 59s] ../grub-core/fs/cpio_common.c:62:29: error: taking address of packed member of 'struct head' may result in an unaligned pointer value [-Werror=address-of-packed-member]
[ 59s] 62 | namesize = read_number (hd.namesize, ARRAY_SIZE (hd.namesize));
[ 59s] | ~~^~~~~~~~~
Signed-off-by: Michael Chang <mchang@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
- Adds zstd support to the btrfs module.
- Adds a test case for btrfs zstd support.
- Changes top_srcdir to srcdir in the btrfs module's lzo include
following comments from Daniel Kiper about the zstd include.
Tested on Ubuntu-18.04 with a btrfs /boot partition with and without zstd
compression. A test case was also added to the test suite that fails before
the patch, and passes after.
Signed-off-by: Nick Terrell <terrelln@fb.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Let's provide file type info to the I/O layer. This way verifiers
framework and its users will be able to differentiate files and verify
only required ones.
This is preparatory patch.
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Add the RAID 6 recovery, in order to use a RAID 6 filesystem even if some
disks (up to two) are missing. This code use the md RAID 6 code already
present in grub.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Add support for recovery for a RAID 5 btrfs profile. In addition
it is added some code as preparatory work for RAID 6 recovery code.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Move the code in charge to read the data from disk into a separate
function. This helps to separate the error handling logic (which
depends on the different raid profiles) from the read from disk
logic. Refactoring this code increases the general readability too.
This is a preparatory patch, to help the adding of the RAID 5/6 recovery code.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
A portion of the logging code is moved outside of internal for(;;). The part
that is left inside is the one which depends on the internal for(;;) index.
This is a preparatory patch. The next one will refactor the code inside
the for(;;) into an another function.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Currently read from missing device triggers rescan. However, it is never
recorded that the device is missing. So, each read of a missing device
triggers rescan again and again. This behavior causes a lot of unneeded
rescans leading to huge slowdowns.
This patch fixes above mentioned issue. Information about missing devices
is stored in the data->devices_attached[] array as NULL value in dev
member. Rescan is triggered only if no information is found for a given
device. This means that only first time read triggers rescan.
The patch drops premature return. This way data->devices_attached[] is
filled even when a given device is missing.
Signed-off-by: Goffredo Baroncelli <kreikack@inwind.it>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The caller knows better if this error is fatal or not, i.e. another disk is
available or not.
This is a preparatory patch.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This helper is used in a few places to help the debugging. As
conservative approach the error is only logged.
This does not impact the error handling.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The sparse inode metadata format became a mkfs.xfs default in
xfsprogs-4.16.0, and such filesystems are now rejected by grub as
containing an incompatible feature.
In essence, this feature allows xfs to allocate inodes into fragmented
freespace. (Without this feature, if xfs could not allocate contiguous
space for 64 new inodes, inode creation would fail.)
In practice, the disk format change is restricted to the inode btree,
which as far as I can tell is not used by grub. If all you're doing
today is parsing a directory, reading an inode number, and converting
that inode number to a disk location, then ignoring this feature
should be fine, so I've added it to XFS_SB_FEAT_INCOMPAT_SUPPORTED
I did some brief testing of this patch by hacking up the regression
tests to completely fragment freespace on the test xfs filesystem, and
then write a large-ish number of inodes to consume any existing
contiguous 64-inode chunk. This way any files the grub tests add and
traverse would be in such a fragmented inode allocation. Tests passed,
but I'm not sure how to cleanly integrate that into the test harness.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Chris Murphy <lists@colorremedies.com>
"F2FS (Flash-Friendly File System) is flash-friendly file system which was merged
into Linux kernel v3.8 in 2013.
The motive for F2FS was to build a file system that from the start, takes into
account the characteristics of NAND flash memory-based storage devices (such as
solid-state disks, eMMC, and SD cards).
F2FS was designed on a basis of a log-structured file system approach, which
remedies some known issues of the older log structured file systems, such as
the snowball effect of wandering trees and high cleaning overhead. In addition,
since a NAND-based storage device shows different characteristics according to
its internal geometry or flash memory management scheme (such as the Flash
Translation Layer or FTL), it supports various parameters not only for
configuring on-disk layout, but also for selecting allocation and cleaning
algorithm.", quote by https://en.wikipedia.org/wiki/F2FS.
The source codes for F2FS are available from:
http://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs.githttp://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git
This patch has been integrated in OpenMandriva Lx 3.
https://www.openmandriva.org/
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Pete Batard <pete@akeo.ie>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Use same algorithm as in libblkid from util-linux v2.30.
1. Take first 16 bytes from UTF-8 encoded string of VolumeSetIdentifier
2. If all bytes are hexadecimal digits, convert to lowercase and use as UUID
3. If first 8 bytes are not all hexadecimal digits, convert those 8 bytes
to their hexadecimal representation, resulting in 16 bytes for UUID
4. Otherwise, compose UUID from two parts:
1. part: converted first 8 bytes (which are hexadecimal digits) to lowercase
2. part: encoded following 4 bytes to their hexadecimal representation (16 bytes)
So UUID would always have 16 hexadecimal digits in lowercase variant.
According to UDF specification, first 16 Unicode characters of
VolumeSetIdentifier should be unique value and first 8 should be
hexadecimal characters.
In most cases all 16 characters are hexadecimal, but e.g. MS Windows
format.exe set only first 8 as hexadecimal and remaining as fixed
(non-unique) which violates specification.
UDF dstring has stored length in the last byte of buffer. Therefore last
byte is not part of recorded characters. And empty string in dstring is
encoded as empty buffer, including first byte (compression id).
On such a filesystem, inodes may have EXT4_ENCRYPT_FLAG set.
For a regular file, this means its contents are encrypted; for a
directory, this means the filenames in its directory entries are
encrypted; and for a symlink, this means its target is encrypted. Since
GRUB cannot decrypt encrypted contents or filenames, just issue an error
if it would need to do so. This is sufficient to allow unencrypted boot
files to co-exist with encrypted files elsewhere on the filesystem.
(Note that encrypted regular files and symlinks will not normally be
encountered outside an encrypted directory; however, it's possible via
hard links, so they still need to be handled.)
Tested by booting from an ext4 /boot partition on which I had run
'tune2fs -O encrypt'. I also verified that the expected error messages
are printed when trying to access encrypted directories, files, and
symlinks from the GRUB command line. Also ran 'sudo ./grub-fs-tester
ext4_encrypt'; note that this requires e2fsprogs v1.43+ and Linux v4.1+.
Signed-off-by: Eric Biggers <ebiggers@google.com>
When building with GCC 8, there are several errors regarding packed-not-aligned.
./include/grub/gpt_partition.h:79:1: error: alignment 1 of ‘struct grub_gpt_partentry’ is less than 8 [-Werror=packed-not-aligned]
This patch fixes the build error by cleaning up the ambiguity of placing
aligned structure in a packed one. In "struct grub_btrfs_time" and "struct
grub_gpt_part_type", the aligned attribute seems to be superfluous, and also
has to be packed, to ensure the structure is bit-to-bit mapped to the format
laid on disk. I think we could blame to copy and paste error here for the
mistake. In "struct efi_variable", we have to use grub_efi_packed_guid_t, as
the name suggests. :)
Signed-off-by: Michael Chang <mchang@suse.com>
Tested-by: Michael Chang <mchang@suse.com>
Tested-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
(cherry picked from commit 563b1da6e6)
1. Do not assume block list and fragment are mutually exclusive. Squash
can pack file tail as fragment (unless -no-fragments is specified); so
check read offset and read either from block list or from fragments as
appropriate.
2. Support sparse files with zero blocks.
3. Fix fragment read - frag.offset is absolute fragment position,
not offset relative to ino.chunk.
Reported and tested by Carlo Caione <carlo@endlessm.com>
XFS V5 stores UUID in metadata and compares them with superblock UUID.
To allow changing of user-visible UUID it stores original value in new
superblock field (meta_uuid) and sets incompatible flag to indicate that
new field must be used to verify metadata. Our driver currently does not
check metadata UUID so simply accept such filesystem.
Reported-By: Marcos Mello <marcosfrm@outlook.com>
Reviewd by Jan Kara <jack@suse.cz>
Historically this variable hold previous value of filename that
had to be freed if allocated previously. Currently this branch
is entered only if filename was not allocated previously so it
became redundant. It did not cause real problems because grub_free
was not called, but code is confusing and causes compilation error
in some cases.
grub_xfs_iterate_dir did not restore first character after inline
name when match was found. Dependning on XFS format this character
could be inode number and we could return to the same node later in
find_file if processing cycled symlinks.
CID: 86724
Recent tests have discovered that many of our filesystems have flawed
handling of "." and "..". Rather than attempting to fix it in filesystems
themselves, make the common code fshelp aware of "." and ".." and handle
them in this layer. Add grub_fshelp_find_file_lookup for easy conversion
of BFS, HFS and exFAT which have the same problem and don't use fshelp.
