- Adds zstd support to the btrfs module.
- Adds a test case for btrfs zstd support.
- Changes top_srcdir to srcdir in the btrfs module's lzo include
following comments from Daniel Kiper about the zstd include.
Tested on Ubuntu-18.04 with a btrfs /boot partition with and without zstd
compression. A test case was also added to the test suite that fails before
the patch, and passes after.
Signed-off-by: Nick Terrell <terrelln@fb.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
- Import zstd-1.3.6 from upstream
- Add zstd's module.c file
- Add the zstd module to Makefile.core.def
Import zstd-1.3.6 from upstream [1]. Only the files need for decompression
are imported. I used the latest zstd release, which includes patches [2] to
build cleanly in GRUB.
I included the script used to import zstd-1.3.6 below at the bottom of the
commit message.
Upstream zstd commit hash: 4fa456d7f12f8b27bd3b2f5dfd4f46898cb31c24
Upstream zstd commit name: Merge pull request #1354 from facebook/dev
Zstd requires some posix headers, which it gets from posix_wrap.
This can be checked by inspecting the .Po files generated by automake,
which contain the header dependencies. After building run the command
`cat grub-core/lib/zstd/.deps-core/*.Po` to see the dependencies [3].
The only OS dependencies are:
- stddef.h, which is already a dependency in posix_wrap, and used for size_t
by lzo and xz.
- stdarg.h, which comes from the grub/misc.h header, and we don't use in zstd.
All the types like uint64_t are typedefed to grub_uint64_t under the hood.
The only exception is size_t, which comes from stddef.h. This is already the
case for lzo and xz. I don't think there are any cross-compilation concerns,
because cross-compilers provide their own system headers (and it would already
be broken).
[1] https://github.com/facebook/zstd/releases/tag/v1.3.6
[2] https://github.com/facebook/zstd/pull/1344
[3] https://gist.github.com/terrelln/7a16b92f5a1b3aecf980f944b4a966c4
```
curl -L -O https://github.com/facebook/zstd/releases/download/v1.3.6/zstd-1.3.6.tar.gz
curl -L -O https://github.com/facebook/zstd/releases/download/v1.3.6/zstd-1.3.6.tar.gz.sha256
sha256sum --check zstd-1.3.6.tar.gz.sha256
tar xzf zstd-1.3.6.tar.gz
SRC_LIB="zstd-1.3.6/lib"
DST_LIB="grub-core/lib/zstd"
rm -rf $DST_LIB
mkdir -p $DST_LIB
cp $SRC_LIB/zstd.h $DST_LIB/
cp $SRC_LIB/common/*.[hc] $DST_LIB/
cp $SRC_LIB/decompress/*.[hc] $DST_LIB/
rm $DST_LIB/{pool.[hc],threading.[hc]}
rm -rf zstd-1.3.6*
echo SUCCESS!
```
Signed-off-by: Nick Terrell <terrelln@fb.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
An error emerged as when I was testing the verifiers branch, so instead
of putting it in pgp prefix, the verifiers is used to reflect what the
patch is based on.
While running verify_detached, grub aborts with error.
verify_detached /@/.snapshots/1/snapshot/boot/grub/grub.cfg
/@/.snapshots/1/snapshot/boot/grub/grub.cfg.sig
alloc magic is broken at 0x7beea660: 0
Aborted. Press any key to exit.
The error is caused by sig file descriptor been closed twice, first time
in grub_verify_signature() to which it is passed as parameter. Second in
grub_cmd_verify_signature() or in whichever opens the sig file
descriptor. The second close is not consider as bug to me either, as in
common rule of what opens a file has to close it to avoid file
descriptor leakage.
After all the design of grub_verify_signature() makes it difficult to keep
a good trace on opened file descriptor from it's caller. Let's refine
the application interface to accept file path rather than descriptor, in
this way the caller doesn't have to care about closing the descriptor by
delegating it to grub_verify_signature() with full tracing to opened
file descriptor by itself.
Also making it clear that sig descriptor is not referenced in error
returning path of grub_verify_signature_init(), so it can be closed
directly by it's caller. This also makes delegating it to
grub_pubkey_close() infeasible to help in relieving file descriptor
leakage as it has to depend on uncertainty of ctxt fields in error
returning path.
Signed-off-by: Michael Chang <mchang@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
grub_file_filter_disable_compression() no longer exists.
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Without this fix, building xen_boot.c omits:
loader/arm64/xen_boot.c: In function ‘xen_boot_binary_load’:
loader/arm64/xen_boot.c:370:7: error: too few arguments to function ‘grub_create_loader_cmdline’
grub_create_loader_cmdline (argc - 1, argv + 1, binary->cmdline,
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from loader/arm64/xen_boot.c:36:0:
../include/grub/lib/cmdline.h:29:12: note: declared here
grub_err_t grub_create_loader_cmdline (int argc, char *argv[], char *buf,
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The verifiers framework changed the grub_file_open() interface, breaking all
non-x86 linux loaders. Add file types to the grub_file_open() calls to make
them build again.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The verifiers framework changed the API of grub_file_open(), but did not
fix up all users. Add the file type GRUB_FILE_TYPE_DEVICE_TREE_IMAGE
to the "devicetree" command handler call.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The API change of grub_file_open() for adding verifiers did not include
a type for device tree blobs. Add GRUB_FILE_TYPE_DEVICE_TREE_IMAGE to
the grub_file_type enum.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
verify.h was added without include guards. This means compiling anything
including both include/grub/verify.h and include/grub/lib/cmdline.h fails
(at least grub-core/loader/arm64/linux.c.
Add the necessary include guard.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Device tree (DTB) lengths are being padded to a multiple of 4 bytes
rather than the target-specific pointer size. This causes objects
following OBJ_TYPE_DTB objects to be incorrectly parsed during GRUB
execution on arm64.
Fix by using ALIGN_ADDR(), not ALIGN_UP().
Signed-by-off: Matthew Daley <mattd@bugfuzz.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
If / is on the root dataset in a ZFS pool, then ${bootfs} will be set to
"/" (whereas if it is on a non-root dataset, there will be no trailing
slash). Passing "root=ZFS=${rpool}/" will fail to boot, but
"root=ZFS=${rpool}" works fine, so strip the trailing slash.
Fixes: https://savannah.gnu.org/bugs/?52746
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Tested-by: Fejes József <jozsef.fejes@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
On a UEFI system, were no boot entry *grub* is present, currently,
`grub-install` fails with an error.
$ efibootmgr
BootCurrent: 0000
Timeout: 0 seconds
BootOrder: 0001,0006,0003,0004,0005
Boot0001 Diskette Drive
Boot0003* USB Storage Device
Boot0004* CD/DVD/CD-RW Drive
Boot0005 Onboard NIC
Boot0006* WDC WD2500AAKX-75U6AA0
$ sudo grub-install /dev/sda
Installing for x86_64-efi platform.
grub-install: error: efibootmgr failed to register the boot entry: Unknown error 22020.
The error code is always different, and the error message (incorrectly)
points to efibootmgr.
But, the error is in GRUB’s function
`grub_install_remove_efi_entries_by_distributor()`, where the variable
`rc` for the return value, is uninitialized and never set, when no boot
entry for the distributor is found.
The content of that uninitialized variable is then returned as the error
code of efibootmgr.
Set the variable to 0, so that success is returned, when no entry needs
to be deleted.
Tested on Dell OptiPlex 7010 with firmware A28.
$ sudo ./grub-install /dev/sda
Installing for x86_64-efi platform.
Installation finished. No error reported.
[1]: https://github.com/rhboot/efibootmgr/issues/100
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This module provides shim lock verification for various kernels
if UEFI secure boot is enabled on a machine.
It is recommended to put this module into GRUB2 standalone image
(avoid putting iorw and memrw modules into it; they are disallowed
if UEFI secure boot is enabled). However, it is also possible to use
it as a normal module. Though such configurations are more fragile
and less secure due to various limitations.
If the module is loaded and UEFI secure boot is enabled then:
- module itself cannot be unloaded (persistent module),
- the iorw and memrw modules cannot be loaded,
- if the iorw and memrw modules are loaded then
machine boot is disabled,
- GRUB2 defers modules and ACPI tables verification to
other verifiers.
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
This type of modules cannot be unloaded. This is useful if a given
functionality, e.g. UEFI secure boot shim signature verification, should
not be disabled if it was enabled at some point in time. Somebody may
say that we can use standalone GRUB2 here. That is true. However, the
code is not so big nor complicated hence it make sense to support
modularized configs too.
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
This way if a verifier requires verification of a given file it can defer task
to another verifier (another authority) if it is not able to do it itself. E.g.
shim_lock verifier, posted as a subsequent patch, is able to verify only PE
files. This means that it is not able to verify any of GRUB2 modules which have
to be trusted on UEFI systems with secure boot enabled. So, it can defer
verification to other verifier, e.g. PGP one.
I silently assume that other verifiers are trusted and will do good job for us.
Or at least they will not do any harm.
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Verifiers framework provides core file verification functionality which
can be used by various security mechanisms, e.g., UEFI secure boot, TPM,
PGP signature verification, etc.
The patch contains PGP code changes and probably they should be extracted
to separate patch for the sake of clarity.
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Let's provide file type info to the I/O layer. This way verifiers
framework and its users will be able to differentiate files and verify
only required ones.
This is preparatory patch.
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Signed-off-by: Vladimir Serbinenko <phcoder@gmail.com>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
Add the RAID 6 recovery, in order to use a RAID 6 filesystem even if some
disks (up to two) are missing. This code use the md RAID 6 code already
present in grub.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The original code which handles the recovery of a RAID 6 disks array
assumes that all reads are multiple of 1 << GRUB_DISK_SECTOR_BITS and it
assumes that all the I/O is done via the struct grub_diskfilter_segment.
This is not true for the btrfs code. In order to reuse the native
grub_raid6_recover() code, it is modified to not call
grub_diskfilter_read_node() directly, but to call an handler passed
as an argument.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Add support for recovery for a RAID 5 btrfs profile. In addition
it is added some code as preparatory work for RAID 6 recovery code.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Move the code in charge to read the data from disk into a separate
function. This helps to separate the error handling logic (which
depends on the different raid profiles) from the read from disk
logic. Refactoring this code increases the general readability too.
This is a preparatory patch, to help the adding of the RAID 5/6 recovery code.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
A portion of the logging code is moved outside of internal for(;;). The part
that is left inside is the one which depends on the internal for(;;) index.
This is a preparatory patch. The next one will refactor the code inside
the for(;;) into an another function.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Currently read from missing device triggers rescan. However, it is never
recorded that the device is missing. So, each read of a missing device
triggers rescan again and again. This behavior causes a lot of unneeded
rescans leading to huge slowdowns.
This patch fixes above mentioned issue. Information about missing devices
is stored in the data->devices_attached[] array as NULL value in dev
member. Rescan is triggered only if no information is found for a given
device. This means that only first time read triggers rescan.
The patch drops premature return. This way data->devices_attached[] is
filled even when a given device is missing.
Signed-off-by: Goffredo Baroncelli <kreikack@inwind.it>
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The caller knows better if this error is fatal or not, i.e. another disk is
available or not.
This is a preparatory patch.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This helper is used in a few places to help the debugging. As
conservative approach the error is only logged.
This does not impact the error handling.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When booting from NVME SSD with 4k sector size, it fails with the message.
error: attempt to read or write outside of partition.
This patch fixes the problem by fixing overflow in converting partition start
and length into 512B blocks.
Signed-off-by: Michael Chang <mchang@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When reading data off a disk, sector values are based on the disk sector
length.
Within grub_util_fd_open_device(), the start of the partition was taken
directly from grub's partition information structure, which uses the
internal sector length (currently 512b), but never transformed to the
disk's sector length.
Subsequent calculations were all wrong for devices that have a diverging
sector length and the functions eventually skipped to the wrong stream
location, reading invalid data.
Signed-off-by: Mihai Moldovan <ionic@ionic.de>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
gentpl.py is python2/3-agnostic, but there's no way to cause it
to be run with any interpreter other than 'python', it's just
hard-coded into Makefile.common that way. Adjust that to use
AM_PATH_PYTHON (provided by automake) to find an interpreter
and run gentpl.py with that instead. This makes grub buildable
when `python` does not exist (but rather `python3` or `python2`
or `python2.7`, etc.) Minimum version is set to 2.6 as this is
the first version with `__future__.print_function` available.
Note, AM_PATH_PYTHON respects the PYTHON environment variable
and will treat its value as the *only* candidate for a valid
interpreter if it is set - when PYTHON is set, AM_PATH_PYTHON
will not try to find any alternative interpreter, it will only
check whether the interpreter set as the value of PYTHON meets
the requirements and use it if so or fail if not. This means
that when using grub's `autogen.sh`, as it too uses the value
of the PYTHON environment variable (and if it is not set, just
sets it to 'python') you cannot rely on AM_PATH_PYTHON
interpreter discovery. If your desired Python interpreter is
not just 'python', you must set the PYTHON environment variable,
e.g. 'PYTHON=/usr/local/bin/python3 ./autogen.sh'. The specified
interpreter will then be used both by autogen.sh itself and by
the autotools-driven build scripts.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
pkg-config is apparently preferred over freetype-config these days (see
the BUGS section of freetype-config(1)). pkg-config support was added
to FreeType in version 2.1.5, which was released in 2003, so it should
comfortably be available everywhere by now.
We no longer need to explicitly substitute FREETYPE_CFLAGS and
FREETYPE_LIBS, since PKG_CHECK_MODULES does that automatically.
Fixes Debian bug #887721.
Reported-by: Hugh McMaster <hugh.mcmaster@outlook.com>
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Using FREETYPE_CFLAGS and FREETYPE_LIBS is more in line with the naming
scheme used by pkg-config macros.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Code later on checks if variables inside the struct are
0 to see if they have been set, like if there were addresses
in the bootpath.
The variables were not initialized however, so the check
might succeed with uninitialized data, and a new interface
with random addresses and the same name is added. This causes
$net_default_mac to point to the random one, so, for example,
using that variable to load per-mac config files fails.
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1785859
Signed-off-by: Julian Andres Klode <julian.klode@canonical.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The "for the next boot only" property of grub-reboot is dependent upon
GRUB being able to clear the next_entry variable in the environment
block. However, GRUB cannot write to devices using the diskfilter
and lvm abstractions.
Ref: https://lists.gnu.org/archive/html/grub-devel/2009-12/msg00276.html
Ref: https://bugs.launchpad.net/bugs/788298
Signed-off-by: dann frazier <dann.frazier@canonical.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This is a cryptographically signed message in MIME format.
Date: Thu, 9 Aug 2018 07:27:35 +0200
Currently, the GRUB payload for coreboot does not detect the Western
Digital hard disk WDC WD20EARS-60M AB51 connected to the ASRock E350M1,
as that takes over ten seconds to spin up.
```
disk/ahci.c:533: port 0, err: 0
disk/ahci.c:539: port 0, err: 0
disk/ahci.c:543: port 0, err: 0
disk/ahci.c:549: port 0, offset: 120, tfd:80, CMD: 6016
disk/ahci.c:552: port 0, err: 0
disk/ahci.c:563: port 0, offset: 120, tfd:80, CMD: 6016
disk/ahci.c:566: port: 0, err: 0
disk/ahci.c:593: port 0 is busy
disk/ahci.c:621: cleaning up failed devs
```
GRUB detects the drive, when either unloading the module *ahci*, and
then loading it again, or when doing a warm reset.
As the ten second time-out is too short, increase it to 32 seconds,
used by SeaBIOS. which detects the drive successfully.
The AHCI driver in libpayload uses 30 seconds, and that time-out was
added in commit 354066e1 (libpayload: ahci: Increase timeout for
signature reading) with the description below.
> We can't read the drives signature before it's ready, i.e. spun up.
> So set the timeout to the standard 30s. Also put a notice on the
> console, so the user knows why the signature reading failed.
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
1. move relocator related code more close to each other
2. use variable "len" since it has correct assignment, and keep coding
style with upper code
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
qemu 2.12 removed the -usbdevice option. Use a more modern spelling
instead, in line with other USB-related tests.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
SeaBIOS 1.11.0 added support for VGA emulation over a serial port, which
interferes with grub-shell. Turn it off.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Make it so that when grub-module-verifier complains of an issue, it tells you
which module the issue was with.
Signed-off-by: Peter Jones <pjones@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Delete local copy of function to determine required buffer size for the
UEFI memory map, use helper in kern/efi/mm.c.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Delete local copy of function to determine required buffer size for the
UEFI memory map, use helper in kern/efi/mm.c.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Delete local copy of function to determine required buffer size for the
UEFI memory map, use helper in kern/efi/mm.c.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Commit 0ba90a7f01 ("efi: Move grub_reboot() into kernel") broke
the build on i386-efi - genmoddep.awk bails out with message
grub_reboot in reboot is duplicated in kernel
This is because both lib/i386/reset.c and kern/efi/efi.c now provide
this function.
Rather than explicitly list each i386 platform variant in
Makefile.core.def, include the contents of lib/i386/reset.c only when
GRUB_MACHINE_EFI is not set.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The 32-bit arm Linux kernel is built as a zImage, which self-decompresses
down to near start of RAM. In order for an initrd/initramfs to be
accessible, it needs to be placed within the first ~768MB of RAM.
The initrd loader built into the kernel EFI stub restricts this down to
512MB for simplicity - so enable the same restriction in grub.
For arm64, the requirement is within a 1GB aligned 32GB window also
covering the (runtime) kernel image. Since the EFI stub loader itself
will attempt to relocate to near start of RAM, force initrd to be loaded
completely within the first 32GB of RAM.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
