License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 14:07:57 +00:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2009-04-29 07:47:18 +00:00
|
|
|
/*
|
|
|
|
|
* ld script for the x86 kernel
|
|
|
|
|
*
|
|
|
|
|
* Historic 32-bit version written by Martin Mares <mj@atrey.karlin.mff.cuni.cz>
|
|
|
|
|
*
|
2009-04-29 08:58:38 +00:00
|
|
|
* Modernisation, unification and other changes and fixes:
|
|
|
|
|
* Copyright (C) 2007-2009 Sam Ravnborg <sam@ravnborg.org>
|
2009-04-29 07:47:18 +00:00
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* Don't define absolute symbols until and unless you know that symbol
|
|
|
|
|
* value is should remain constant even if kernel image is relocated
|
|
|
|
|
* at run time. Absolute symbols are not relocated. If symbol value should
|
|
|
|
|
* change if kernel is relocated, make the symbol section relative and
|
|
|
|
|
* put it inside the section definition.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifdef CONFIG_X86_32
|
|
|
|
|
#define LOAD_OFFSET __PAGE_OFFSET
|
|
|
|
|
#else
|
|
|
|
|
#define LOAD_OFFSET __START_KERNEL_map
|
|
|
|
|
#endif
|
|
|
|
|
|
2020-03-26 19:30:20 +00:00
|
|
|
#define RUNTIME_DISCARD_EXIT
|
2019-10-29 21:13:30 +00:00
|
|
|
#define EMITS_PT_NOTE
|
2019-10-29 21:13:38 +00:00
|
|
|
#define RO_EXCEPTION_TABLE_ALIGN 16
|
2019-10-29 21:13:30 +00:00
|
|
|
|
2009-04-29 07:47:18 +00:00
|
|
|
#include <asm-generic/vmlinux.lds.h>
|
|
|
|
|
#include <asm/asm-offsets.h>
|
|
|
|
|
#include <asm/thread_info.h>
|
|
|
|
|
#include <asm/page_types.h>
|
2017-07-24 23:36:57 +00:00
|
|
|
#include <asm/orc_lookup.h>
|
2009-04-29 07:47:18 +00:00
|
|
|
#include <asm/cache.h>
|
|
|
|
|
#include <asm/boot.h>
|
|
|
|
|
|
|
|
|
|
#undef i386 /* in case the preprocessor is a 32bit one */
|
|
|
|
|
|
2019-01-09 16:32:10 +00:00
|
|
|
OUTPUT_FORMAT(CONFIG_OUTPUT_FORMAT)
|
2009-04-29 07:47:18 +00:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_X86_32
|
|
|
|
|
OUTPUT_ARCH(i386)
|
|
|
|
|
ENTRY(phys_startup_32)
|
|
|
|
|
#else
|
|
|
|
|
OUTPUT_ARCH(i386:x86-64)
|
|
|
|
|
ENTRY(phys_startup_64)
|
|
|
|
|
#endif
|
|
|
|
|
|
x86_64: Fix jiffies ODR violation
'jiffies' and 'jiffies_64' are meant to alias (two different symbols that
share the same address). Most architectures make the symbols alias to the
same address via a linker script assignment in their
arch/<arch>/kernel/vmlinux.lds.S:
jiffies = jiffies_64;
which is effectively a definition of jiffies.
jiffies and jiffies_64 are both forward declared for all architectures in
include/linux/jiffies.h. jiffies_64 is defined in kernel/time/timer.c.
x86_64 was peculiar in that it wasn't doing the above linker script
assignment, but rather was:
1. defining jiffies in arch/x86/kernel/time.c instead via the linker script.
2. overriding the symbol jiffies_64 from kernel/time/timer.c in
arch/x86/kernel/vmlinux.lds.s via 'jiffies_64 = jiffies;'.
As Fangrui notes:
In LLD, symbol assignments in linker scripts override definitions in
object files. GNU ld appears to have the same behavior. It would
probably make sense for LLD to error "duplicate symbol" but GNU ld
is unlikely to adopt for compatibility reasons.
This results in an ODR violation (UB), which seems to have survived
thus far. Where it becomes harmful is when;
1. -fno-semantic-interposition is used:
As Fangrui notes:
Clang after LLVM commit 5b22bcc2b70d
("[X86][ELF] Prefer to lower MC_GlobalAddress operands to .Lfoo$local")
defaults to -fno-semantic-interposition similar semantics which help
-fpic/-fPIC code avoid GOT/PLT when the referenced symbol is defined
within the same translation unit. Unlike GCC
-fno-semantic-interposition, Clang emits such relocations referencing
local symbols for non-pic code as well.
This causes references to jiffies to refer to '.Ljiffies$local' when
jiffies is defined in the same translation unit. Likewise, references to
jiffies_64 become references to '.Ljiffies_64$local' in translation units
that define jiffies_64. Because these differ from the names used in the
linker script, they will not be rewritten to alias one another.
2. Full LTO
Full LTO effectively treats all source files as one translation
unit, causing these local references to be produced everywhere. When
the linker processes the linker script, there are no longer any
references to jiffies_64' anywhere to replace with 'jiffies'. And
thus '.Ljiffies$local' and '.Ljiffies_64$local' no longer alias
at all.
In the process of porting patches enabling Full LTO from arm64 to x86_64,
spooky bugs have been observed where the kernel appeared to boot, but init
doesn't get scheduled.
Avoid the ODR violation by matching other architectures and define jiffies
only by linker script. For -fno-semantic-interposition + Full LTO, there
is no longer a global definition of jiffies for the compiler to produce a
local symbol which the linker script won't ensure aliases to jiffies_64.
Fixes: 40747ffa5aa8 ("asmlinkage: Make jiffies visible")
Reported-by: Nathan Chancellor <natechancellor@gmail.com>
Reported-by: Alistair Delva <adelva@google.com>
Debugged-by: Nick Desaulniers <ndesaulniers@google.com>
Debugged-by: Sami Tolvanen <samitolvanen@google.com>
Suggested-by: Fangrui Song <maskray@google.com>
Signed-off-by: Bob Haarman <inglorion@google.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # build+boot on
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: stable@vger.kernel.org
Link: https://github.com/ClangBuiltLinux/linux/issues/852
Link: https://lkml.kernel.org/r/20200602193100.229287-1-inglorion@google.com
2020-06-02 19:30:59 +00:00
|
|
|
jiffies = jiffies_64;
|
|
|
|
|
|
2016-02-17 22:41:14 +00:00
|
|
|
#if defined(CONFIG_X86_64)
|
2009-10-19 13:12:04 +00:00
|
|
|
/*
|
2016-02-17 22:41:14 +00:00
|
|
|
* On 64-bit, align RODATA to 2MB so we retain large page mappings for
|
|
|
|
|
* boundaries spanning kernel text, rodata and data sections.
|
2009-10-19 13:12:04 +00:00
|
|
|
*
|
|
|
|
|
* However, kernel identity mappings will have different RWX permissions
|
|
|
|
|
* to the pages mapping to text and to the pages padding (which are freed) the
|
|
|
|
|
* text section. Hence kernel identity mappings will be broken to smaller
|
|
|
|
|
* pages. For 64-bit, kernel text and kernel identity mappings are different,
|
2016-02-17 22:41:14 +00:00
|
|
|
* so we can enable protection checks as well as retain 2MB large page
|
|
|
|
|
* mappings for kernel text.
|
2009-10-19 13:12:04 +00:00
|
|
|
*/
|
2018-07-18 09:41:04 +00:00
|
|
|
#define X86_ALIGN_RODATA_BEGIN . = ALIGN(HPAGE_SIZE);
|
2009-10-14 21:46:56 +00:00
|
|
|
|
2018-07-18 09:41:04 +00:00
|
|
|
#define X86_ALIGN_RODATA_END \
|
2009-10-14 21:46:56 +00:00
|
|
|
. = ALIGN(HPAGE_SIZE); \
|
2018-07-18 09:41:04 +00:00
|
|
|
__end_rodata_hpage_align = .; \
|
|
|
|
|
__end_rodata_aligned = .;
|
2009-10-14 21:46:56 +00:00
|
|
|
|
2017-12-04 14:07:46 +00:00
|
|
|
#define ALIGN_ENTRY_TEXT_BEGIN . = ALIGN(PMD_SIZE);
|
|
|
|
|
#define ALIGN_ENTRY_TEXT_END . = ALIGN(PMD_SIZE);
|
|
|
|
|
|
2018-09-14 13:45:58 +00:00
|
|
|
/*
|
|
|
|
|
* This section contains data which will be mapped as decrypted. Memory
|
|
|
|
|
* encryption operates on a page basis. Make this section PMD-aligned
|
|
|
|
|
* to avoid splitting the pages while mapping the section early.
|
|
|
|
|
*
|
|
|
|
|
* Note: We use a separate section so that only this section gets
|
|
|
|
|
* decrypted to avoid exposing more than we wish.
|
|
|
|
|
*/
|
|
|
|
|
#define BSS_DECRYPTED \
|
|
|
|
|
. = ALIGN(PMD_SIZE); \
|
|
|
|
|
__start_bss_decrypted = .; \
|
|
|
|
|
*(.bss..decrypted); \
|
|
|
|
|
. = ALIGN(PAGE_SIZE); \
|
|
|
|
|
__start_bss_decrypted_unused = .; \
|
|
|
|
|
. = ALIGN(PMD_SIZE); \
|
|
|
|
|
__end_bss_decrypted = .; \
|
|
|
|
|
|
2009-10-14 21:46:56 +00:00
|
|
|
#else
|
|
|
|
|
|
2018-07-18 09:41:04 +00:00
|
|
|
#define X86_ALIGN_RODATA_BEGIN
|
|
|
|
|
#define X86_ALIGN_RODATA_END \
|
|
|
|
|
. = ALIGN(PAGE_SIZE); \
|
|
|
|
|
__end_rodata_aligned = .;
|
2009-10-14 21:46:56 +00:00
|
|
|
|
2017-12-04 14:07:46 +00:00
|
|
|
#define ALIGN_ENTRY_TEXT_BEGIN
|
|
|
|
|
#define ALIGN_ENTRY_TEXT_END
|
2018-09-14 13:45:58 +00:00
|
|
|
#define BSS_DECRYPTED
|
2017-12-04 14:07:46 +00:00
|
|
|
|
2009-10-14 21:46:56 +00:00
|
|
|
#endif
|
|
|
|
|
|
2009-04-29 07:47:19 +00:00
|
|
|
PHDRS {
|
|
|
|
|
text PT_LOAD FLAGS(5); /* R_E */
|
2010-11-16 21:31:26 +00:00
|
|
|
data PT_LOAD FLAGS(6); /* RW_ */
|
2009-04-29 07:47:19 +00:00
|
|
|
#ifdef CONFIG_X86_64
|
|
|
|
|
#ifdef CONFIG_SMP
|
2009-09-04 08:18:07 +00:00
|
|
|
percpu PT_LOAD FLAGS(6); /* RW_ */
|
2009-04-29 07:47:19 +00:00
|
|
|
#endif
|
2009-08-25 13:50:53 +00:00
|
|
|
init PT_LOAD FLAGS(7); /* RWE */
|
2009-04-29 07:47:19 +00:00
|
|
|
#endif
|
|
|
|
|
note PT_NOTE FLAGS(0); /* ___ */
|
|
|
|
|
}
|
2009-04-29 07:47:18 +00:00
|
|
|
|
2009-04-29 07:47:20 +00:00
|
|
|
SECTIONS
|
|
|
|
|
{
|
|
|
|
|
#ifdef CONFIG_X86_32
|
x86/kallsyms: fix GOLD link failure with new relative kallsyms table format
Commit 2213e9a66bb8 ("kallsyms: add support for relative offsets in
kallsyms address table") changed the default kallsyms symbol table
format to use relative references rather than absolute addresses.
This reduces the size of the kallsyms symbol table by 50% on 64-bit
architectures, and further reduces the size of the relocation tables
used by relocatable kernels. Since the memory footprint of the static
kernel image is always much smaller than 4 GB, these relative references
are assumed to be representable in 32 bits, even when the native word
size is 64 bits.
On 64-bit architectures, this obviously only works if the distance
between each relative reference and the chosen anchor point is
representable in 32 bits, and so the table generation code in
scripts/kallsyms.c scans the table for the lowest value that is covered
by the kernel text, and selects it as the anchor point.
However, when using the GOLD linker rather than the default BFD linker
to build the x86_64 kernel, the symbol phys_offset_64, which is the
result of arithmetic defined in the linker script, is emitted as a 'T'
rather than an 'A' type symbol, resulting in scripts/kallsyms.c to
mistake it for a suitable anchor point, even though it is far away from
the actual kernel image in the virtual address space. This results in
out-of-range warnings from scripts/kallsyms.c and a broken build.
So let's align with the BFD linker, and emit the phys_offset_[32|64]
symbols as absolute symbols explicitly. Note that the out of range
issue does not exist on 32-bit x86, but this patch changes both symbols
for symmetry.
Reported-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-18 09:04:37 +00:00
|
|
|
. = LOAD_OFFSET + LOAD_PHYSICAL_ADDR;
|
|
|
|
|
phys_startup_32 = ABSOLUTE(startup_32 - LOAD_OFFSET);
|
2009-04-29 07:47:20 +00:00
|
|
|
#else
|
x86/kallsyms: fix GOLD link failure with new relative kallsyms table format
Commit 2213e9a66bb8 ("kallsyms: add support for relative offsets in
kallsyms address table") changed the default kallsyms symbol table
format to use relative references rather than absolute addresses.
This reduces the size of the kallsyms symbol table by 50% on 64-bit
architectures, and further reduces the size of the relocation tables
used by relocatable kernels. Since the memory footprint of the static
kernel image is always much smaller than 4 GB, these relative references
are assumed to be representable in 32 bits, even when the native word
size is 64 bits.
On 64-bit architectures, this obviously only works if the distance
between each relative reference and the chosen anchor point is
representable in 32 bits, and so the table generation code in
scripts/kallsyms.c scans the table for the lowest value that is covered
by the kernel text, and selects it as the anchor point.
However, when using the GOLD linker rather than the default BFD linker
to build the x86_64 kernel, the symbol phys_offset_64, which is the
result of arithmetic defined in the linker script, is emitted as a 'T'
rather than an 'A' type symbol, resulting in scripts/kallsyms.c to
mistake it for a suitable anchor point, even though it is far away from
the actual kernel image in the virtual address space. This results in
out-of-range warnings from scripts/kallsyms.c and a broken build.
So let's align with the BFD linker, and emit the phys_offset_[32|64]
symbols as absolute symbols explicitly. Note that the out of range
issue does not exist on 32-bit x86, but this patch changes both symbols
for symmetry.
Reported-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-18 09:04:37 +00:00
|
|
|
. = __START_KERNEL;
|
|
|
|
|
phys_startup_64 = ABSOLUTE(startup_64 - LOAD_OFFSET);
|
2009-04-29 07:47:20 +00:00
|
|
|
#endif
|
|
|
|
|
|
2009-04-29 07:47:21 +00:00
|
|
|
/* Text and read-only data */
|
|
|
|
|
.text : AT(ADDR(.text) - LOAD_OFFSET) {
|
2009-09-16 20:44:28 +00:00
|
|
|
_text = .;
|
2016-09-21 21:04:07 +00:00
|
|
|
_stext = .;
|
2009-09-16 20:44:28 +00:00
|
|
|
/* bootstrapping code */
|
|
|
|
|
HEAD_TEXT
|
2009-04-29 07:47:21 +00:00
|
|
|
TEXT_TEXT
|
|
|
|
|
SCHED_TEXT
|
2016-10-08 00:02:55 +00:00
|
|
|
CPUIDLE_TEXT
|
2009-04-29 07:47:21 +00:00
|
|
|
LOCK_TEXT
|
|
|
|
|
KPROBES_TEXT
|
2017-12-04 14:07:46 +00:00
|
|
|
ALIGN_ENTRY_TEXT_BEGIN
|
2011-03-07 18:10:39 +00:00
|
|
|
ENTRY_TEXT
|
2017-12-04 14:07:46 +00:00
|
|
|
ALIGN_ENTRY_TEXT_END
|
2016-03-25 21:22:05 +00:00
|
|
|
SOFTIRQENTRY_TEXT
|
2020-08-18 13:57:45 +00:00
|
|
|
STATIC_CALL_TEXT
|
2009-04-29 07:47:21 +00:00
|
|
|
*(.fixup)
|
|
|
|
|
*(.gnu.warning)
|
x86/entry/64: Create a per-CPU SYSCALL entry trampoline
Handling SYSCALL is tricky: the SYSCALL handler is entered with every
single register (except FLAGS), including RSP, live. It somehow needs
to set RSP to point to a valid stack, which means it needs to save the
user RSP somewhere and find its own stack pointer. The canonical way
to do this is with SWAPGS, which lets us access percpu data using the
%gs prefix.
With PAGE_TABLE_ISOLATION-like pagetable switching, this is
problematic. Without a scratch register, switching CR3 is impossible, so
%gs-based percpu memory would need to be mapped in the user pagetables.
Doing that without information leaks is difficult or impossible.
Instead, use a different sneaky trick. Map a copy of the first part
of the SYSCALL asm at a different address for each CPU. Now RIP
varies depending on the CPU, so we can use RIP-relative memory access
to access percpu memory. By putting the relevant information (one
scratch slot and the stack address) at a constant offset relative to
RIP, we can make SYSCALL work without relying on %gs.
A nice thing about this approach is that we can easily switch it on
and off if we want pagetable switching to be configurable.
The compat variant of SYSCALL doesn't have this problem in the first
place -- there are plenty of scratch registers, since we don't care
about preserving r8-r15. This patch therefore doesn't touch SYSCALL32
at all.
This patch actually seems to be a small speedup. With this patch,
SYSCALL touches an extra cache line and an extra virtual page, but
the pipeline no longer stalls waiting for SWAPGS. It seems that, at
least in a tight loop, the latter outweights the former.
Thanks to David Laight for an optimization tip.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Borislav Petkov <bpetkov@suse.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Laight <David.Laight@aculab.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Eduardo Valentin <eduval@amazon.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: aliguori@amazon.com
Cc: daniel.gruss@iaik.tugraz.at
Cc: hughd@google.com
Cc: keescook@google.com
Link: https://lkml.kernel.org/r/20171204150606.403607157@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-12-04 14:07:25 +00:00
|
|
|
|
2018-01-18 16:14:21 +00:00
|
|
|
#ifdef CONFIG_RETPOLINE
|
|
|
|
|
__indirect_thunk_start = .;
|
|
|
|
|
*(.text.__x86.indirect_thunk)
|
|
|
|
|
__indirect_thunk_end = .;
|
|
|
|
|
#endif
|
2019-10-29 21:13:51 +00:00
|
|
|
} :text =0xcccc
|
2019-04-23 18:38:27 +00:00
|
|
|
|
2019-10-29 21:13:37 +00:00
|
|
|
/* End of text section, which should occupy whole number of pages */
|
|
|
|
|
_etext = .;
|
2010-11-16 21:31:26 +00:00
|
|
|
. = ALIGN(PAGE_SIZE);
|
2019-10-29 21:13:37 +00:00
|
|
|
|
2018-07-18 09:41:04 +00:00
|
|
|
X86_ALIGN_RODATA_BEGIN
|
2009-08-25 13:50:53 +00:00
|
|
|
RO_DATA(PAGE_SIZE)
|
2018-07-18 09:41:04 +00:00
|
|
|
X86_ALIGN_RODATA_END
|
2009-04-29 07:47:22 +00:00
|
|
|
|
2009-04-29 07:47:23 +00:00
|
|
|
/* Data */
|
|
|
|
|
.data : AT(ADDR(.data) - LOAD_OFFSET) {
|
2009-05-11 12:22:00 +00:00
|
|
|
/* Start of data section */
|
|
|
|
|
_sdata = .;
|
2009-08-25 13:50:53 +00:00
|
|
|
|
|
|
|
|
/* init_task */
|
|
|
|
|
INIT_TASK_DATA(THREAD_SIZE)
|
2009-04-29 07:47:23 +00:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_X86_32
|
2009-08-25 13:50:53 +00:00
|
|
|
/* 32 bit has nosave before _edata */
|
|
|
|
|
NOSAVE_DATA
|
2009-04-29 07:47:23 +00:00
|
|
|
#endif
|
|
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
PAGE_ALIGNED_DATA(PAGE_SIZE)
|
2009-04-29 07:47:23 +00:00
|
|
|
|
2009-11-13 11:54:40 +00:00
|
|
|
CACHELINE_ALIGNED_DATA(L1_CACHE_BYTES)
|
2009-04-29 07:47:23 +00:00
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
DATA_DATA
|
|
|
|
|
CONSTRUCTORS
|
|
|
|
|
|
|
|
|
|
/* rarely changed data like cpu maps */
|
2009-11-13 11:54:40 +00:00
|
|
|
READ_MOSTLY_DATA(INTERNODE_CACHE_BYTES)
|
2009-04-29 07:47:23 +00:00
|
|
|
|
|
|
|
|
/* End of data section */
|
|
|
|
|
_edata = .;
|
2009-08-25 13:50:53 +00:00
|
|
|
} :data
|
2009-04-29 07:47:23 +00:00
|
|
|
|
2017-03-30 15:49:27 +00:00
|
|
|
BUG_TABLE
|
2009-04-29 07:47:24 +00:00
|
|
|
|
2017-07-24 23:36:57 +00:00
|
|
|
ORC_UNWIND_TABLE
|
|
|
|
|
|
2011-08-03 13:31:50 +00:00
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
__vvar_page = .;
|
|
|
|
|
|
|
|
|
|
.vvar : AT(ADDR(.vvar) - LOAD_OFFSET) {
|
2011-08-03 13:31:51 +00:00
|
|
|
/* work around gold bug 13023 */
|
|
|
|
|
__vvar_beginning_hack = .;
|
2011-08-03 13:31:50 +00:00
|
|
|
|
2011-08-03 13:31:51 +00:00
|
|
|
/* Place all vvars at the offsets in asm/vvar.h. */
|
2019-11-12 01:27:10 +00:00
|
|
|
#define EMIT_VVAR(name, offset) \
|
2011-08-03 13:31:51 +00:00
|
|
|
. = __vvar_beginning_hack + offset; \
|
2011-08-03 13:31:50 +00:00
|
|
|
*(.vvar_ ## name)
|
|
|
|
|
#include <asm/vvar.h>
|
|
|
|
|
#undef EMIT_VVAR
|
|
|
|
|
|
2014-03-17 22:22:11 +00:00
|
|
|
/*
|
|
|
|
|
* Pad the rest of the page with zeros. Otherwise the loader
|
|
|
|
|
* can leave garbage here.
|
|
|
|
|
*/
|
|
|
|
|
. = __vvar_beginning_hack + PAGE_SIZE;
|
2011-08-03 13:31:50 +00:00
|
|
|
} :data
|
|
|
|
|
|
2018-02-08 06:38:57 +00:00
|
|
|
. = ALIGN(__vvar_page + PAGE_SIZE, PAGE_SIZE);
|
2011-08-03 13:31:50 +00:00
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
/* Init code and data - will be freed after init */
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
.init.begin : AT(ADDR(.init.begin) - LOAD_OFFSET) {
|
|
|
|
|
__init_begin = .; /* paired with __init_end */
|
2009-04-29 07:47:25 +00:00
|
|
|
}
|
|
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
#if defined(CONFIG_X86_64) && defined(CONFIG_SMP)
|
2009-04-29 07:47:25 +00:00
|
|
|
/*
|
2009-08-25 13:50:53 +00:00
|
|
|
* percpu offsets are zero-based on SMP. PERCPU_VADDR() changes the
|
|
|
|
|
* output PHDR, so the next output section - .init.text - should
|
|
|
|
|
* start another segment - init.
|
2009-04-29 07:47:25 +00:00
|
|
|
*/
|
2011-01-25 13:26:50 +00:00
|
|
|
PERCPU_VADDR(INTERNODE_CACHE_BYTES, 0, :percpu)
|
2014-11-04 08:50:48 +00:00
|
|
|
ASSERT(SIZEOF(.data..percpu) < CONFIG_PHYSICAL_START,
|
|
|
|
|
"per-CPU data too large - increase CONFIG_PHYSICAL_START")
|
2009-08-25 13:50:53 +00:00
|
|
|
#endif
|
2009-04-29 07:47:25 +00:00
|
|
|
|
2009-09-16 20:44:30 +00:00
|
|
|
INIT_TEXT_SECTION(PAGE_SIZE)
|
2009-08-25 13:50:53 +00:00
|
|
|
#ifdef CONFIG_X86_64
|
|
|
|
|
:init
|
|
|
|
|
#endif
|
2009-04-29 07:47:25 +00:00
|
|
|
|
2016-01-26 21:12:07 +00:00
|
|
|
/*
|
|
|
|
|
* Section for code used exclusively before alternatives are run. All
|
|
|
|
|
* references to such code must be patched out by alternatives, normally
|
|
|
|
|
* by using X86_FEATURE_ALWAYS CPU feature bit.
|
|
|
|
|
*
|
|
|
|
|
* See static_cpu_has() for an example.
|
|
|
|
|
*/
|
|
|
|
|
.altinstr_aux : AT(ADDR(.altinstr_aux) - LOAD_OFFSET) {
|
|
|
|
|
*(.altinstr_aux)
|
|
|
|
|
}
|
|
|
|
|
|
2009-09-16 20:44:30 +00:00
|
|
|
INIT_DATA_SECTION(16)
|
2009-04-29 07:47:25 +00:00
|
|
|
|
|
|
|
|
.x86_cpu_dev.init : AT(ADDR(.x86_cpu_dev.init) - LOAD_OFFSET) {
|
|
|
|
|
__x86_cpu_dev_start = .;
|
|
|
|
|
*(.x86_cpu_dev.init)
|
|
|
|
|
__x86_cpu_dev_end = .;
|
|
|
|
|
}
|
|
|
|
|
|
2013-10-17 22:35:35 +00:00
|
|
|
#ifdef CONFIG_X86_INTEL_MID
|
|
|
|
|
.x86_intel_mid_dev.init : AT(ADDR(.x86_intel_mid_dev.init) - \
|
|
|
|
|
LOAD_OFFSET) {
|
|
|
|
|
__x86_intel_mid_dev_start = .;
|
|
|
|
|
*(.x86_intel_mid_dev.init)
|
|
|
|
|
__x86_intel_mid_dev_end = .;
|
|
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
2010-08-27 18:19:33 +00:00
|
|
|
/*
|
|
|
|
|
* start address and size of operations which during runtime
|
|
|
|
|
* can be patched with virtualization friendly instructions or
|
|
|
|
|
* baremetal native ones. Think page table operations.
|
|
|
|
|
* Details in paravirt_types.h
|
|
|
|
|
*/
|
2009-04-29 07:47:26 +00:00
|
|
|
. = ALIGN(8);
|
|
|
|
|
.parainstructions : AT(ADDR(.parainstructions) - LOAD_OFFSET) {
|
|
|
|
|
__parainstructions = .;
|
|
|
|
|
*(.parainstructions)
|
|
|
|
|
__parainstructions_end = .;
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 18:19:33 +00:00
|
|
|
/*
|
|
|
|
|
* struct alt_inst entries. From the header (alternative.h):
|
|
|
|
|
* "Alternative instructions for different CPU types or capabilities"
|
|
|
|
|
* Think locking instructions on spinlocks.
|
|
|
|
|
*/
|
2009-04-29 07:47:26 +00:00
|
|
|
. = ALIGN(8);
|
|
|
|
|
.altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) {
|
|
|
|
|
__alt_instructions = .;
|
|
|
|
|
*(.altinstructions)
|
|
|
|
|
__alt_instructions_end = .;
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 18:19:33 +00:00
|
|
|
/*
|
|
|
|
|
* And here are the replacement instructions. The linker sticks
|
|
|
|
|
* them as binary blobs. The .altinstructions has enough data to
|
|
|
|
|
* get the address and the length of them to patch the kernel safely.
|
|
|
|
|
*/
|
2009-04-29 07:47:26 +00:00
|
|
|
.altinstr_replacement : AT(ADDR(.altinstr_replacement) - LOAD_OFFSET) {
|
|
|
|
|
*(.altinstr_replacement)
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 18:19:33 +00:00
|
|
|
/*
|
|
|
|
|
* struct iommu_table_entry entries are injected in this section.
|
|
|
|
|
* It is an array of IOMMUs which during run time gets sorted depending
|
|
|
|
|
* on its dependency order. After rootfs_initcall is complete
|
|
|
|
|
* this section can be safely removed.
|
|
|
|
|
*/
|
x86, iommu: Add IOMMU_INIT macros, .iommu_table section, and iommu_table_entry structure
This patch set adds a mechanism to "modularize" the IOMMUs we have
on X86. Currently the count of IOMMUs is up to six and they have a complex
relationship that requires careful execution order. 'pci_iommu_alloc'
does that today, but most folks are unhappy with how it does it.
This patch set addresses this and also paves a mechanism to jettison
unused IOMMUs during run-time. For details that sparked this, please
refer to: http://lkml.org/lkml/2010/8/2/282
The first solution that comes to mind is to convert wholesale
the IOMMU detection routines to be called during initcall
time frame. Unfortunately that misses the dependency relationship
that some of the IOMMUs have (for example: for AMD-Vi IOMMU to work,
GART detection MUST run first, and before all of that SWIOTLB MUST run).
The second solution would be to introduce a registration call wherein
the IOMMU would provide its detection/init routines and as well on what
MUST run before it. That would work, except that the 'pci_iommu_alloc'
which would run through this list, is called during mem_init. This means we
don't have any memory allocator, and it is so early that we haven't yet
started running through the initcall_t list.
This solution borrows concepts from the 2nd idea and from how
MODULE_INIT works. A macro is provided that each IOMMU uses to define
it's detect function and early_init (before the memory allocate is
active), and as well what other IOMMU MUST run before us. Since most IOMMUs
depend on having SWIOTLB run first ("pci_swiotlb_detect") a convenience macro
to depends on that is also provided.
This macro is similar in design to MODULE_PARAM macro wherein
we setup a .iommu_table section in which we populate it with the values
that match a struct iommu_table_entry. During bootup we will sort
through the array so that the IOMMUs that MUST run before us are first
elements in the array. And then we just iterate through them calling the
detection routine and if appropiate, the init routines.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
LKML-Reference: <1282845485-8991-2-git-send-email-konrad.wilk@oracle.com>
CC: H. Peter Anvin <hpa@zytor.com>
CC: Fujita Tomonori <fujita.tomonori@lab.ntt.co.jp>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Ingo Molnar <mingo@redhat.com>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2010-08-26 17:57:56 +00:00
|
|
|
.iommu_table : AT(ADDR(.iommu_table) - LOAD_OFFSET) {
|
|
|
|
|
__iommu_table = .;
|
|
|
|
|
*(.iommu_table)
|
|
|
|
|
__iommu_table_end = .;
|
|
|
|
|
}
|
2011-02-14 23:34:57 +00:00
|
|
|
|
2011-05-21 00:51:17 +00:00
|
|
|
. = ALIGN(8);
|
|
|
|
|
.apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) {
|
|
|
|
|
__apicdrivers = .;
|
|
|
|
|
*(.apicdrivers);
|
|
|
|
|
__apicdrivers_end = .;
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-30 18:10:02 +00:00
|
|
|
. = ALIGN(8);
|
2009-04-29 07:47:27 +00:00
|
|
|
/*
|
2020-02-24 23:21:29 +00:00
|
|
|
* .exit.text is discarded at runtime, not link time, to deal with
|
|
|
|
|
* references from .altinstructions
|
2009-04-29 07:47:27 +00:00
|
|
|
*/
|
|
|
|
|
.exit.text : AT(ADDR(.exit.text) - LOAD_OFFSET) {
|
|
|
|
|
EXIT_TEXT
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
.exit.data : AT(ADDR(.exit.data) - LOAD_OFFSET) {
|
|
|
|
|
EXIT_DATA
|
|
|
|
|
}
|
|
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
#if !defined(CONFIG_X86_64) || !defined(CONFIG_SMP)
|
2011-03-24 17:50:09 +00:00
|
|
|
PERCPU_SECTION(INTERNODE_CACHE_BYTES)
|
2009-04-29 07:47:28 +00:00
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
2009-04-29 10:56:58 +00:00
|
|
|
|
2009-04-29 07:47:28 +00:00
|
|
|
/* freed after init ends here */
|
2009-04-29 10:56:58 +00:00
|
|
|
.init.end : AT(ADDR(.init.end) - LOAD_OFFSET) {
|
|
|
|
|
__init_end = .;
|
|
|
|
|
}
|
2009-04-29 07:47:28 +00:00
|
|
|
|
2009-08-25 13:50:53 +00:00
|
|
|
/*
|
|
|
|
|
* smp_locks might be freed after init
|
|
|
|
|
* start/end must be page aligned
|
|
|
|
|
*/
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
.smp_locks : AT(ADDR(.smp_locks) - LOAD_OFFSET) {
|
|
|
|
|
__smp_locks = .;
|
|
|
|
|
*(.smp_locks)
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
2010-03-29 02:42:54 +00:00
|
|
|
__smp_locks_end = .;
|
2009-08-25 13:50:53 +00:00
|
|
|
}
|
|
|
|
|
|
2009-04-29 07:47:28 +00:00
|
|
|
#ifdef CONFIG_X86_64
|
|
|
|
|
.data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) {
|
2009-08-25 13:50:53 +00:00
|
|
|
NOSAVE_DATA
|
|
|
|
|
}
|
2009-04-29 07:47:28 +00:00
|
|
|
#endif
|
|
|
|
|
|
2009-04-29 07:47:29 +00:00
|
|
|
/* BSS */
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
.bss : AT(ADDR(.bss) - LOAD_OFFSET) {
|
|
|
|
|
__bss_start = .;
|
2010-02-20 00:03:38 +00:00
|
|
|
*(.bss..page_aligned)
|
2020-07-21 09:34:48 +00:00
|
|
|
. = ALIGN(PAGE_SIZE);
|
2019-04-15 16:49:56 +00:00
|
|
|
*(BSS_MAIN)
|
2018-09-14 13:45:58 +00:00
|
|
|
BSS_DECRYPTED
|
2010-11-16 21:31:26 +00:00
|
|
|
. = ALIGN(PAGE_SIZE);
|
2009-04-29 07:47:29 +00:00
|
|
|
__bss_stop = .;
|
|
|
|
|
}
|
2009-04-29 07:47:28 +00:00
|
|
|
|
2019-06-19 18:40:57 +00:00
|
|
|
/*
|
|
|
|
|
* The memory occupied from _text to here, __end_of_kernel_reserve, is
|
|
|
|
|
* automatically reserved in setup_arch(). Anything after here must be
|
|
|
|
|
* explicitly reserved using memblock_reserve() or it will be discarded
|
|
|
|
|
* and treated as available memory.
|
|
|
|
|
*/
|
|
|
|
|
__end_of_kernel_reserve = .;
|
|
|
|
|
|
2009-04-29 07:47:29 +00:00
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
.brk : AT(ADDR(.brk) - LOAD_OFFSET) {
|
|
|
|
|
__brk_base = .;
|
|
|
|
|
. += 64 * 1024; /* 64k alignment slop space */
|
|
|
|
|
*(.brk_reservation) /* areas brk users have reserved */
|
|
|
|
|
__brk_limit = .;
|
|
|
|
|
}
|
|
|
|
|
|
x86/boot: Move compressed kernel to the end of the decompression buffer
This change makes later calculations about where the kernel is located
easier to reason about. To better understand this change, we must first
clarify what 'VO' and 'ZO' are. These values were introduced in commits
by hpa:
77d1a4999502 ("x86, boot: make symbols from the main vmlinux available")
37ba7ab5e33c ("x86, boot: make kernel_alignment adjustable; new bzImage fields")
Specifically:
All names prefixed with 'VO_':
- relate to the uncompressed kernel image
- the size of the VO image is: VO__end-VO__text ("VO_INIT_SIZE" define)
All names prefixed with 'ZO_':
- relate to the bootable compressed kernel image (boot/compressed/vmlinux),
which is composed of the following memory areas:
- head text
- compressed kernel (VO image and relocs table)
- decompressor code
- the size of the ZO image is: ZO__end - ZO_startup_32 ("ZO_INIT_SIZE" define, though see below)
The 'INIT_SIZE' value is used to find the larger of the two image sizes:
#define ZO_INIT_SIZE (ZO__end - ZO_startup_32 + ZO_z_extract_offset)
#define VO_INIT_SIZE (VO__end - VO__text)
#if ZO_INIT_SIZE > VO_INIT_SIZE
# define INIT_SIZE ZO_INIT_SIZE
#else
# define INIT_SIZE VO_INIT_SIZE
#endif
The current code uses extract_offset to decide where to position the
copied ZO (i.e. ZO starts at extract_offset). (This is why ZO_INIT_SIZE
currently includes the extract_offset.)
Why does z_extract_offset exist? It's needed because we are trying to minimize
the amount of RAM used for the whole act of creating an uncompressed, executable,
properly relocation-linked kernel image in system memory. We do this so that
kernels can be booted on even very small systems.
To achieve the goal of minimal memory consumption we have implemented an in-place
decompression strategy: instead of cleanly separating the VO and ZO images and
also allocating some memory for the decompression code's runtime needs, we instead
create this elaborate layout of memory buffers where the output (decompressed)
stream, as it progresses, overlaps with and destroys the input (compressed)
stream. This can only be done safely if the ZO image is placed to the end of the
VO range, plus a certain amount of safety distance to make sure that when the last
bytes of the VO range are decompressed, the compressed stream pointer is safely
beyond the end of the VO range.
z_extract_offset is calculated in arch/x86/boot/compressed/mkpiggy.c during
the build process, at a point when we know the exact compressed and
uncompressed size of the kernel images and can calculate this safe minimum
offset value. (Note that the mkpiggy.c calculation is not perfect, because
we don't know the decompressor used at that stage, so the z_extract_offset
calculation is necessarily imprecise and is mostly based on gzip internals -
we'll improve that in the next patch.)
When INIT_SIZE is bigger than VO_INIT_SIZE (uncommon but possible),
the copied ZO occupies the memory from extract_offset to the end of
decompression buffer. It overlaps with the soon-to-be-uncompressed kernel
like this:
|-----compressed kernel image------|
V V
0 extract_offset +INIT_SIZE
|-----------|---------------|-------------------------|--------|
| | | |
VO__text startup_32 of ZO VO__end ZO__end
^ ^
|-------uncompressed kernel image---------|
When INIT_SIZE is equal to VO_INIT_SIZE (likely) there's still space
left from end of ZO to the end of decompressing buffer, like below.
|-compressed kernel image-|
V V
0 extract_offset +INIT_SIZE
|-----------|---------------|-------------------------|--------|
| | | |
VO__text startup_32 of ZO ZO__end VO__end
^ ^
|------------uncompressed kernel image-------------|
To simplify calculations and avoid special cases, it is cleaner to
always place the compressed kernel image in memory so that ZO__end
is at the end of the decompression buffer, instead of placing t at
the start of extract_offset as is currently done.
This patch adds BP_init_size (which is the INIT_SIZE as passed in from
the boot_params) into asm-offsets.c to make it visible to the assembly
code.
Then when moving the ZO, it calculates the starting position of
the copied ZO (via BP_init_size and the ZO run size) so that the VO__end
will be at the end of the decompression buffer. To make the position
calculation safe, the end of ZO is page aligned (and a comment is added
to the existing VO alignment for good measure).
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
[ Rewrote changelog and comments. ]
Signed-off-by: Kees Cook <keescook@chromium.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: lasse.collin@tukaani.org
Link: http://lkml.kernel.org/r/1461888548-32439-3-git-send-email-keescook@chromium.org
[ Rewrote the changelog some more. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-04-29 00:09:04 +00:00
|
|
|
. = ALIGN(PAGE_SIZE); /* keep VO_INIT_SIZE page aligned */
|
2009-12-14 21:55:20 +00:00
|
|
|
_end = .;
|
2009-04-29 07:47:29 +00:00
|
|
|
|
x86/mm: Create a workarea in the kernel for SME early encryption
In order for the kernel to be encrypted "in place" during boot, a workarea
outside of the kernel must be used. This SME workarea used during early
encryption of the kernel is situated on a 2MB boundary after the end of
the kernel text, data, etc. sections (_end).
This works well during initial boot of a compressed kernel because of
the relocation used for decompression of the kernel. But when performing
a kexec boot, there's a chance that the SME workarea may not be mapped
by the kexec pagetables or that some of the other data used by kexec
could exist in this range.
Create a section for SME in vmlinux.lds.S. Position it after "_end", which
is after "__end_of_kernel_reserve", so that the memory will be reclaimed
during boot and since this area is all zeroes, it compresses well. This
new section will be part of the kernel image, so kexec will account for it
in pagetable mappings and placement of data after the kernel.
Here's an example of a kernel size without and with the SME section:
without:
vmlinux: 36,501,616
bzImage: 6,497,344
100000000-47f37ffff : System RAM
1e4000000-1e47677d4 : Kernel code (0x7677d4)
1e47677d5-1e4e2e0bf : Kernel data (0x6c68ea)
1e5074000-1e5372fff : Kernel bss (0x2fefff)
with:
vmlinux: 44,419,408
bzImage: 6,503,136
880000000-c7ff7ffff : System RAM
8cf000000-8cf7677d4 : Kernel code (0x7677d4)
8cf7677d5-8cfe2e0bf : Kernel data (0x6c68ea)
8d0074000-8d0372fff : Kernel bss (0x2fefff)
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Dave Hansen <dave.hansen@intel.com>
Tested-by: Lianbo Jiang <lijiang@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Kees Cook <keescook@chromium.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Rafael Ávila de Espíndola" <rafael@espindo.la>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "x86@kernel.org" <x86@kernel.org>
Link: https://lkml.kernel.org/r/3c483262eb4077b1654b2052bd14a8d011bffde3.1560969363.git.thomas.lendacky@amd.com
2019-06-19 18:40:59 +00:00
|
|
|
#ifdef CONFIG_AMD_MEM_ENCRYPT
|
|
|
|
|
/*
|
|
|
|
|
* Early scratch/workarea section: Lives outside of the kernel proper
|
|
|
|
|
* (_text - _end).
|
|
|
|
|
*
|
|
|
|
|
* Resides after _end because even though the .brk section is after
|
|
|
|
|
* __end_of_kernel_reserve, the .brk section is later reserved as a
|
|
|
|
|
* part of the kernel. Since it is located after __end_of_kernel_reserve
|
|
|
|
|
* it will be discarded and become part of the available memory. As
|
|
|
|
|
* such, it can only be used by very early boot code and must not be
|
|
|
|
|
* needed afterwards.
|
|
|
|
|
*
|
|
|
|
|
* Currently used by SME for performing in-place encryption of the
|
|
|
|
|
* kernel during boot. Resides on a 2MB boundary to simplify the
|
|
|
|
|
* pagetable setup used for SME in-place encryption.
|
|
|
|
|
*/
|
|
|
|
|
. = ALIGN(HPAGE_SIZE);
|
|
|
|
|
.init.scratch : AT(ADDR(.init.scratch) - LOAD_OFFSET) {
|
|
|
|
|
__init_scratch_begin = .;
|
|
|
|
|
*(.init.scratch)
|
|
|
|
|
. = ALIGN(HPAGE_SIZE);
|
|
|
|
|
__init_scratch_end = .;
|
|
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
2018-02-08 06:38:57 +00:00
|
|
|
STABS_DEBUG
|
|
|
|
|
DWARF_DEBUG
|
2020-08-21 19:42:45 +00:00
|
|
|
ELF_DETAILS
|
linker script: unify usage of discard definition
Discarded sections in different archs share some commonality but have
considerable differences. This led to linker script for each arch
implementing its own /DISCARD/ definition, which makes maintaining
tedious and adding new entries error-prone.
This patch makes all linker scripts to move discard definitions to the
end of the linker script and use the common DISCARDS macro. As ld
uses the first matching section definition, archs can include default
discarded sections by including them earlier in the linker script.
ia64 is notable because it first throws away some ia64 specific
subsections and then include the rest of the sections into the final
image, so those sections must be discarded before the inclusion.
defconfig compile tested for x86, x86-64, powerpc, powerpc64, ia64,
alpha, sparc, sparc64 and s390. Michal Simek tested microblaze.
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Paul Mundt <lethal@linux-sh.org>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Tested-by: Michal Simek <monstr@monstr.eu>
Cc: linux-arch@vger.kernel.org
Cc: Michal Simek <monstr@monstr.eu>
Cc: microblaze-uclinux@itee.uq.edu.au
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Tony Luck <tony.luck@intel.com>
2009-07-09 02:27:40 +00:00
|
|
|
|
|
|
|
|
DISCARDS
|
2009-04-29 07:47:20 +00:00
|
|
|
|
2020-08-21 19:43:04 +00:00
|
|
|
/*
|
|
|
|
|
* Make sure that the .got.plt is either completely empty or it
|
|
|
|
|
* contains only the lazy dispatch entries.
|
|
|
|
|
*/
|
|
|
|
|
.got.plt (INFO) : { *(.got.plt) }
|
|
|
|
|
ASSERT(SIZEOF(.got.plt) == 0 ||
|
|
|
|
|
#ifdef CONFIG_X86_64
|
|
|
|
|
SIZEOF(.got.plt) == 0x18,
|
|
|
|
|
#else
|
|
|
|
|
SIZEOF(.got.plt) == 0xc,
|
|
|
|
|
#endif
|
|
|
|
|
"Unexpected GOT/PLT entries detected!")
|
2020-08-21 19:43:05 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Sections that should stay zero sized, which is safer to
|
|
|
|
|
* explicitly check instead of blindly discarding.
|
|
|
|
|
*/
|
|
|
|
|
.got : {
|
|
|
|
|
*(.got) *(.igot.*)
|
|
|
|
|
}
|
|
|
|
|
ASSERT(SIZEOF(.got) == 0, "Unexpected GOT entries detected!")
|
|
|
|
|
|
|
|
|
|
.plt : {
|
|
|
|
|
*(.plt) *(.plt.*) *(.iplt)
|
|
|
|
|
}
|
|
|
|
|
ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
|
|
|
|
|
|
|
|
|
|
.rel.dyn : {
|
|
|
|
|
*(.rel.*) *(.rel_*)
|
|
|
|
|
}
|
|
|
|
|
ASSERT(SIZEOF(.rel.dyn) == 0, "Unexpected run-time relocations (.rel) detected!")
|
|
|
|
|
|
|
|
|
|
.rela.dyn : {
|
|
|
|
|
*(.rela.*) *(.rela_*)
|
|
|
|
|
}
|
|
|
|
|
ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!")
|
2020-08-21 19:43:04 +00:00
|
|
|
}
|
2009-04-29 07:47:18 +00:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_X86_32
|
2009-10-16 05:18:46 +00:00
|
|
|
/*
|
|
|
|
|
* The ASSERT() sink to . is intentional, for binutils 2.14 compatibility:
|
|
|
|
|
*/
|
2009-08-03 21:44:54 +00:00
|
|
|
. = ASSERT((_end - LOAD_OFFSET <= KERNEL_IMAGE_SIZE),
|
|
|
|
|
"kernel image bigger than KERNEL_IMAGE_SIZE");
|
2009-04-29 07:47:18 +00:00
|
|
|
#else
|
|
|
|
|
/*
|
|
|
|
|
* Per-cpu symbols which need to be offset from __per_cpu_load
|
|
|
|
|
* for the boot processor.
|
|
|
|
|
*/
|
2018-12-19 19:01:43 +00:00
|
|
|
#define INIT_PER_CPU(x) init_per_cpu__##x = ABSOLUTE(x) + __per_cpu_load
|
2009-04-29 07:47:18 +00:00
|
|
|
INIT_PER_CPU(gdt_page);
|
2019-04-14 16:00:06 +00:00
|
|
|
INIT_PER_CPU(fixed_percpu_data);
|
|
|
|
|
INIT_PER_CPU(irq_stack_backing_store);
|
2009-04-29 07:47:18 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Build-time check on the image size:
|
|
|
|
|
*/
|
2009-08-03 21:44:54 +00:00
|
|
|
. = ASSERT((_end - _text <= KERNEL_IMAGE_SIZE),
|
|
|
|
|
"kernel image bigger than KERNEL_IMAGE_SIZE");
|
2009-04-29 07:47:18 +00:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_SMP
|
2019-04-14 16:00:06 +00:00
|
|
|
. = ASSERT((fixed_percpu_data == 0),
|
|
|
|
|
"fixed_percpu_data is not at start of per-cpu area");
|
2009-04-29 07:47:18 +00:00
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#endif /* CONFIG_X86_32 */
|
|
|
|
|
|
2015-09-09 22:38:55 +00:00
|
|
|
#ifdef CONFIG_KEXEC_CORE
|
2009-04-29 07:47:18 +00:00
|
|
|
#include <asm/kexec.h>
|
|
|
|
|
|
2009-08-03 21:44:54 +00:00
|
|
|
. = ASSERT(kexec_control_code_size <= KEXEC_CONTROL_CODE_MAX_SIZE,
|
|
|
|
|
"kexec control code size is too big");
|
2009-04-29 07:47:18 +00:00
|
|
|
#endif
|
|
|
|
|
|