License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 14:07:57 +00:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2008-10-23 05:26:29 +00:00
|
|
|
#ifndef _ASM_X86_ALTERNATIVE_H
|
|
|
|
|
#define _ASM_X86_ALTERNATIVE_H
|
2008-01-30 12:30:30 +00:00
|
|
|
|
|
|
|
|
#include <linux/types.h>
|
2009-04-28 15:13:46 +00:00
|
|
|
#include <linux/stringify.h>
|
2008-01-30 12:30:30 +00:00
|
|
|
#include <asm/asm.h>
|
|
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALT_FLAGS_SHIFT 16
|
|
|
|
|
|
2023-04-18 06:42:28 +00:00
|
|
|
#define ALT_FLAG_NOT (1 << 0)
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALT_NOT(feature) ((ALT_FLAG_NOT << ALT_FLAGS_SHIFT) | (feature))
|
2023-12-10 06:21:36 +00:00
|
|
|
#define ALT_FLAG_DIRECT_CALL (1 << 1)
|
|
|
|
|
#define ALT_DIRECT_CALL(feature) ((ALT_FLAG_DIRECT_CALL << ALT_FLAGS_SHIFT) | (feature))
|
|
|
|
|
#define ALT_CALL_ALWAYS ALT_DIRECT_CALL(X86_FEATURE_ALWAYS)
|
2021-03-11 14:23:10 +00:00
|
|
|
|
2021-03-11 14:23:06 +00:00
|
|
|
#ifndef __ASSEMBLY__
|
|
|
|
|
|
|
|
|
|
#include <linux/stddef.h>
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
/*
|
|
|
|
|
* Alternative inline assembly for SMP.
|
|
|
|
|
*
|
|
|
|
|
* The LOCK_PREFIX macro defined here replaces the LOCK and
|
|
|
|
|
* LOCK_PREFIX macros used everywhere in the source tree.
|
|
|
|
|
*
|
|
|
|
|
* SMP alternatives use the same data structures as the other
|
|
|
|
|
* alternatives and the X86_FEATURE_UP flag to indicate the case of a
|
|
|
|
|
* UP system running a SMP kernel. The existing apply_alternatives()
|
|
|
|
|
* works fine for patching a SMP kernel for UP.
|
|
|
|
|
*
|
|
|
|
|
* The SMP alternative tables can be kept after boot and contain both
|
|
|
|
|
* UP and SMP versions of the instructions to allow switching back to
|
|
|
|
|
* SMP at runtime, when hotplugging in a new CPU, which is especially
|
|
|
|
|
* useful in virtualized environments.
|
|
|
|
|
*
|
|
|
|
|
* The very common lock prefix is handled as special case in a
|
|
|
|
|
* separate table which is a pure address list without replacement ptr
|
|
|
|
|
* and size information. That keeps the table sizes small.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifdef CONFIG_SMP
|
2018-12-19 10:20:57 +00:00
|
|
|
#define LOCK_PREFIX_HERE \
|
|
|
|
|
".pushsection .smp_locks,\"a\"\n" \
|
|
|
|
|
".balign 4\n" \
|
|
|
|
|
".long 671f - .\n" /* offset */ \
|
|
|
|
|
".popsection\n" \
|
|
|
|
|
"671:"
|
|
|
|
|
|
|
|
|
|
#define LOCK_PREFIX LOCK_PREFIX_HERE "\n\tlock; "
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
#else /* ! CONFIG_SMP */
|
2010-04-29 23:03:57 +00:00
|
|
|
#define LOCK_PREFIX_HERE ""
|
2008-01-30 12:30:30 +00:00
|
|
|
#define LOCK_PREFIX ""
|
|
|
|
|
#endif
|
|
|
|
|
|
2019-03-18 13:33:07 +00:00
|
|
|
/*
|
|
|
|
|
* objtool annotation to ignore the alternatives and only consider the original
|
|
|
|
|
* instruction(s).
|
|
|
|
|
*/
|
|
|
|
|
#define ANNOTATE_IGNORE_ALTERNATIVE \
|
|
|
|
|
"999:\n\t" \
|
|
|
|
|
".pushsection .discard.ignore_alts\n\t" \
|
x86/speculation, objtool: Use absolute relocations for annotations
.discard.retpoline_safe sections do not have the SHF_ALLOC flag. These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable. Newer
LLD will report warnings for REL relocations even for relocatable links [1]:
ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''
Switch to absolute relocations instead, which indicate link-time
addresses. In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c. When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.
Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.
Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized. That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).
[0] commit 1c0c1faf5692 ("objtool: Use relative pointers for annotations")
[1] https://github.com/ClangBuiltLinux/linux/issues/1937
Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
2023-09-20 00:17:28 +00:00
|
|
|
".long 999b\n\t" \
|
2019-03-18 13:33:07 +00:00
|
|
|
".popsection\n\t"
|
|
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
/*
|
|
|
|
|
* The patching flags are part of the upper bits of the @ft_flags parameter when
|
|
|
|
|
* specifying them. The split is currently like this:
|
|
|
|
|
*
|
|
|
|
|
* [31... flags ...16][15... CPUID feature bit ...0]
|
|
|
|
|
*
|
|
|
|
|
* but since this is all hidden in the macros argument being split, those fields can be
|
|
|
|
|
* extended in the future to fit in a u64 or however the need arises.
|
|
|
|
|
*/
|
2008-01-30 12:30:30 +00:00
|
|
|
struct alt_instr {
|
2011-07-13 13:24:10 +00:00
|
|
|
s32 instr_offset; /* original instruction */
|
|
|
|
|
s32 repl_offset; /* offset to replacement instruction */
|
2022-12-22 11:42:24 +00:00
|
|
|
|
|
|
|
|
union {
|
|
|
|
|
struct {
|
|
|
|
|
u32 cpuid: 16; /* CPUID bit set for replacement */
|
|
|
|
|
u32 flags: 16; /* patching control flags */
|
|
|
|
|
};
|
|
|
|
|
u32 ft_flags;
|
|
|
|
|
};
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
u8 instrlen; /* length of original instruction */
|
2014-12-27 09:41:52 +00:00
|
|
|
u8 replacementlen; /* length of new instruction */
|
|
|
|
|
} __packed;
|
2008-01-30 12:30:30 +00:00
|
|
|
|
2023-12-10 06:21:37 +00:00
|
|
|
extern struct alt_instr __alt_instructions[], __alt_instructions_end[];
|
|
|
|
|
|
2015-04-30 07:09:26 +00:00
|
|
|
/*
|
|
|
|
|
* Debug flag that can be tested to see whether alternative
|
|
|
|
|
* instructions were patched in already:
|
|
|
|
|
*/
|
|
|
|
|
extern int alternatives_patched;
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
extern void alternative_instructions(void);
|
|
|
|
|
extern void apply_alternatives(struct alt_instr *start, struct alt_instr *end);
|
2021-10-26 12:01:42 +00:00
|
|
|
extern void apply_retpolines(s32 *start, s32 *end);
|
2022-06-14 21:15:37 +00:00
|
|
|
extern void apply_returns(s32 *start, s32 *end);
|
2023-06-22 13:36:50 +00:00
|
|
|
extern void apply_seal_endbr(s32 *start, s32 *end);
|
2022-10-27 09:28:14 +00:00
|
|
|
extern void apply_fineibt(s32 *start_retpoline, s32 *end_retpoine,
|
|
|
|
|
s32 *start_cfi, s32 *end_cfi);
|
2008-01-30 12:30:30 +00:00
|
|
|
|
|
|
|
|
struct module;
|
2022-09-15 11:11:23 +00:00
|
|
|
|
|
|
|
|
struct callthunk_sites {
|
|
|
|
|
s32 *call_start, *call_end;
|
2023-12-10 06:21:37 +00:00
|
|
|
struct alt_instr *alt_start, *alt_end;
|
2022-09-15 11:11:23 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
#ifdef CONFIG_CALL_THUNKS
|
|
|
|
|
extern void callthunks_patch_builtin_calls(void);
|
2022-09-15 11:11:24 +00:00
|
|
|
extern void callthunks_patch_module_calls(struct callthunk_sites *sites,
|
|
|
|
|
struct module *mod);
|
2022-09-15 11:11:31 +00:00
|
|
|
extern void *callthunks_translate_call_dest(void *dest);
|
2024-04-01 18:55:30 +00:00
|
|
|
extern int x86_call_depth_emit_accounting(u8 **pprog, void *func, void *ip);
|
2022-09-15 11:11:23 +00:00
|
|
|
#else
|
|
|
|
|
static __always_inline void callthunks_patch_builtin_calls(void) {}
|
2022-09-15 11:11:24 +00:00
|
|
|
static __always_inline void
|
|
|
|
|
callthunks_patch_module_calls(struct callthunk_sites *sites,
|
|
|
|
|
struct module *mod) {}
|
2022-09-15 11:11:31 +00:00
|
|
|
static __always_inline void *callthunks_translate_call_dest(void *dest)
|
|
|
|
|
{
|
|
|
|
|
return dest;
|
|
|
|
|
}
|
2022-09-15 11:11:34 +00:00
|
|
|
static __always_inline int x86_call_depth_emit_accounting(u8 **pprog,
|
2024-04-01 18:55:30 +00:00
|
|
|
void *func, void *ip)
|
2022-09-15 11:11:34 +00:00
|
|
|
{
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
2022-09-15 11:11:23 +00:00
|
|
|
#endif
|
2008-01-30 12:30:30 +00:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_SMP
|
|
|
|
|
extern void alternatives_smp_module_add(struct module *mod, char *name,
|
|
|
|
|
void *locks, void *locks_end,
|
|
|
|
|
void *text, void *text_end);
|
|
|
|
|
extern void alternatives_smp_module_del(struct module *mod);
|
2012-08-06 07:59:49 +00:00
|
|
|
extern void alternatives_enable_smp(void);
|
2010-02-02 21:49:11 +00:00
|
|
|
extern int alternatives_text_reserved(void *start, void *end);
|
2010-11-24 00:11:40 +00:00
|
|
|
extern bool skip_smp_alternatives;
|
2008-01-30 12:30:30 +00:00
|
|
|
#else
|
|
|
|
|
static inline void alternatives_smp_module_add(struct module *mod, char *name,
|
2008-03-23 08:01:37 +00:00
|
|
|
void *locks, void *locks_end,
|
|
|
|
|
void *text, void *text_end) {}
|
2008-01-30 12:30:30 +00:00
|
|
|
static inline void alternatives_smp_module_del(struct module *mod) {}
|
2012-08-06 07:59:49 +00:00
|
|
|
static inline void alternatives_enable_smp(void) {}
|
2010-02-02 21:49:11 +00:00
|
|
|
static inline int alternatives_text_reserved(void *start, void *end)
|
|
|
|
|
{
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
2008-01-30 12:30:30 +00:00
|
|
|
#endif /* CONFIG_SMP */
|
|
|
|
|
|
2023-12-10 06:21:36 +00:00
|
|
|
#define ALT_CALL_INSTR "call BUG_func"
|
|
|
|
|
|
2014-12-27 09:41:52 +00:00
|
|
|
#define b_replacement(num) "664"#num
|
|
|
|
|
#define e_replacement(num) "665"#num
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2014-12-27 09:41:52 +00:00
|
|
|
#define alt_end_marker "663"
|
|
|
|
|
#define alt_slen "662b-661b"
|
|
|
|
|
#define alt_total_slen alt_end_marker"b-661b"
|
|
|
|
|
#define alt_rlen(num) e_replacement(num)"f-"b_replacement(num)"f"
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2018-12-10 11:21:48 +00:00
|
|
|
#define OLDINSTR(oldinstr, num) \
|
|
|
|
|
"# ALT: oldnstr\n" \
|
2014-12-27 09:41:52 +00:00
|
|
|
"661:\n\t" oldinstr "\n662:\n" \
|
2018-12-10 11:21:48 +00:00
|
|
|
"# ALT: padding\n" \
|
2014-12-27 09:41:52 +00:00
|
|
|
".skip -(((" alt_rlen(num) ")-(" alt_slen ")) > 0) * " \
|
2018-12-10 11:21:48 +00:00
|
|
|
"((" alt_rlen(num) ")-(" alt_slen ")),0x90\n" \
|
2014-12-27 09:41:52 +00:00
|
|
|
alt_end_marker ":\n"
|
|
|
|
|
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
/*
|
x86/alternatives: Fix alt_max_short macro to really be a max()
The alt_max_short() macro in asm/alternative.h does not work as
intended, leading to nasty bugs. E.g. alt_max_short("1", "3")
evaluates to 3, but alt_max_short("3", "1") evaluates to 1 -- not
exactly the maximum of 1 and 3.
In fact, I had to learn it the hard way by crashing my kernel in not
so funny ways by attempting to make use of the ALTENATIVE_2 macro
with alternatives where the first one was larger than the second
one.
According to [1] and commit dbe4058a6a44 ("x86/alternatives: Fix
ALTERNATIVE_2 padding generation properly") the right handed side
should read "-(-(a < b))" not "-(-(a - b))". Fix that, to make the
macro work as intended.
While at it, fix up the comments regarding the additional "-", too.
It's not about gas' usage of s32 but brain dead logic of having a
"true" value of -1 for the < operator ... *sigh*
Btw., the one in asm/alternative-asm.h is correct. And, apparently,
all current users of ALTERNATIVE_2() pass same sized alternatives,
avoiding to hit the bug.
[1] http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
Reviewed-and-tested-by: Borislav Petkov <bp@suse.de>
Fixes: dbe4058a6a44 ("x86/alternatives: Fix ALTERNATIVE_2 padding generation properly")
Signed-off-by: Mathias Krause <minipli@googlemail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/1507228213-13095-1-git-send-email-minipli@googlemail.com
2017-10-05 18:30:12 +00:00
|
|
|
* gas compatible max based on the idea from:
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
* http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
|
|
|
|
|
*
|
x86/alternatives: Fix alt_max_short macro to really be a max()
The alt_max_short() macro in asm/alternative.h does not work as
intended, leading to nasty bugs. E.g. alt_max_short("1", "3")
evaluates to 3, but alt_max_short("3", "1") evaluates to 1 -- not
exactly the maximum of 1 and 3.
In fact, I had to learn it the hard way by crashing my kernel in not
so funny ways by attempting to make use of the ALTENATIVE_2 macro
with alternatives where the first one was larger than the second
one.
According to [1] and commit dbe4058a6a44 ("x86/alternatives: Fix
ALTERNATIVE_2 padding generation properly") the right handed side
should read "-(-(a < b))" not "-(-(a - b))". Fix that, to make the
macro work as intended.
While at it, fix up the comments regarding the additional "-", too.
It's not about gas' usage of s32 but brain dead logic of having a
"true" value of -1 for the < operator ... *sigh*
Btw., the one in asm/alternative-asm.h is correct. And, apparently,
all current users of ALTERNATIVE_2() pass same sized alternatives,
avoiding to hit the bug.
[1] http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
Reviewed-and-tested-by: Borislav Petkov <bp@suse.de>
Fixes: dbe4058a6a44 ("x86/alternatives: Fix ALTERNATIVE_2 padding generation properly")
Signed-off-by: Mathias Krause <minipli@googlemail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/1507228213-13095-1-git-send-email-minipli@googlemail.com
2017-10-05 18:30:12 +00:00
|
|
|
* The additional "-" is needed because gas uses a "true" value of -1.
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
*/
|
x86/alternatives: Fix alt_max_short macro to really be a max()
The alt_max_short() macro in asm/alternative.h does not work as
intended, leading to nasty bugs. E.g. alt_max_short("1", "3")
evaluates to 3, but alt_max_short("3", "1") evaluates to 1 -- not
exactly the maximum of 1 and 3.
In fact, I had to learn it the hard way by crashing my kernel in not
so funny ways by attempting to make use of the ALTENATIVE_2 macro
with alternatives where the first one was larger than the second
one.
According to [1] and commit dbe4058a6a44 ("x86/alternatives: Fix
ALTERNATIVE_2 padding generation properly") the right handed side
should read "-(-(a < b))" not "-(-(a - b))". Fix that, to make the
macro work as intended.
While at it, fix up the comments regarding the additional "-", too.
It's not about gas' usage of s32 but brain dead logic of having a
"true" value of -1 for the < operator ... *sigh*
Btw., the one in asm/alternative-asm.h is correct. And, apparently,
all current users of ALTERNATIVE_2() pass same sized alternatives,
avoiding to hit the bug.
[1] http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
Reviewed-and-tested-by: Borislav Petkov <bp@suse.de>
Fixes: dbe4058a6a44 ("x86/alternatives: Fix ALTERNATIVE_2 padding generation properly")
Signed-off-by: Mathias Krause <minipli@googlemail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/1507228213-13095-1-git-send-email-minipli@googlemail.com
2017-10-05 18:30:12 +00:00
|
|
|
#define alt_max_short(a, b) "((" a ") ^ (((" a ") ^ (" b ")) & -(-((" a ") < (" b ")))))"
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
|
2014-12-27 09:41:52 +00:00
|
|
|
/*
|
|
|
|
|
* Pad the second replacement alternative with additional NOPs if it is
|
|
|
|
|
* additionally longer than the first replacement alternative.
|
|
|
|
|
*/
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
#define OLDINSTR_2(oldinstr, num1, num2) \
|
2018-12-10 11:21:48 +00:00
|
|
|
"# ALT: oldinstr2\n" \
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
"661:\n\t" oldinstr "\n662:\n" \
|
2018-12-10 11:21:48 +00:00
|
|
|
"# ALT: padding2\n" \
|
x86/alternatives: Fix ALTERNATIVE_2 padding generation properly
Quentin caught a corner case with the generation of instruction
padding in the ALTERNATIVE_2 macro: if len(orig_insn) <
len(alt1) < len(alt2), then not enough padding gets added and
that is not good(tm) as we could overwrite the beginning of the
next instruction.
Luckily, at the time of this writing, we don't have
ALTERNATIVE_2() invocations which have that problem and even if
we did, a simple fix would be to prepend the instructions with
enough prefixes so that that corner case doesn't happen.
However, best it would be if we fixed it properly. See below for
a simple, abstracted example of what we're doing.
So what we ended up doing is, we compute the
max(len(alt1), len(alt2)) - len(orig_insn)
and feed that value to the .skip gas directive. The max() cannot
have conditionals due to gas limitations, thus the fancy integer
math.
With this patch, all ALTERNATIVE_2 sites get padded correctly;
generating obscure test cases pass too:
#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
#define gen_skip(orig, alt1, alt2, marker) \
.skip -((alt_max_short(alt1, alt2) - (orig)) > 0) * \
(alt_max_short(alt1, alt2) - (orig)),marker
.pushsection .text, "ax"
.globl main
main:
gen_skip(1, 2, 4, 0x09)
gen_skip(4, 1, 2, 0x10)
...
.popsection
Thanks to Quentin for catching it and double-checking the fix!
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150404133443.GE21152@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-04 13:34:43 +00:00
|
|
|
".skip -((" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")) > 0) * " \
|
|
|
|
|
"(" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")), 0x90\n" \
|
2014-12-27 09:41:52 +00:00
|
|
|
alt_end_marker ":\n"
|
|
|
|
|
|
2018-12-10 15:17:23 +00:00
|
|
|
#define OLDINSTR_3(oldinsn, n1, n2, n3) \
|
|
|
|
|
"# ALT: oldinstr3\n" \
|
|
|
|
|
"661:\n\t" oldinsn "\n662:\n" \
|
|
|
|
|
"# ALT: padding3\n" \
|
|
|
|
|
".skip -((" alt_max_short(alt_max_short(alt_rlen(n1), alt_rlen(n2)), alt_rlen(n3)) \
|
|
|
|
|
" - (" alt_slen ")) > 0) * " \
|
|
|
|
|
"(" alt_max_short(alt_max_short(alt_rlen(n1), alt_rlen(n2)), alt_rlen(n3)) \
|
|
|
|
|
" - (" alt_slen ")), 0x90\n" \
|
|
|
|
|
alt_end_marker ":\n"
|
|
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALTINSTR_ENTRY(ft_flags, num) \
|
2012-05-25 01:19:45 +00:00
|
|
|
" .long 661b - .\n" /* label */ \
|
2014-12-27 09:41:52 +00:00
|
|
|
" .long " b_replacement(num)"f - .\n" /* new instruction */ \
|
2022-12-22 11:42:24 +00:00
|
|
|
" .4byte " __stringify(ft_flags) "\n" /* feature + flags */ \
|
2014-12-27 09:41:52 +00:00
|
|
|
" .byte " alt_total_slen "\n" /* source len */ \
|
2021-03-26 15:12:01 +00:00
|
|
|
" .byte " alt_rlen(num) "\n" /* replacement len */
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2021-03-09 13:48:04 +00:00
|
|
|
#define ALTINSTR_REPLACEMENT(newinstr, num) /* replacement */ \
|
2018-12-10 11:21:48 +00:00
|
|
|
"# ALT: replacement " #num "\n" \
|
|
|
|
|
b_replacement(num)":\n\t" newinstr "\n" e_replacement(num) ":\n"
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2009-04-28 15:13:46 +00:00
|
|
|
/* alternative assembly primitive: */
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALTERNATIVE(oldinstr, newinstr, ft_flags) \
|
2014-12-27 09:41:52 +00:00
|
|
|
OLDINSTR(oldinstr, 1) \
|
2012-09-21 19:43:08 +00:00
|
|
|
".pushsection .altinstructions,\"a\"\n" \
|
2022-12-22 11:42:24 +00:00
|
|
|
ALTINSTR_ENTRY(ft_flags, 1) \
|
2012-09-21 19:43:08 +00:00
|
|
|
".popsection\n" \
|
|
|
|
|
".pushsection .altinstr_replacement, \"ax\"\n" \
|
2021-03-09 13:48:04 +00:00
|
|
|
ALTINSTR_REPLACEMENT(newinstr, 1) \
|
2018-01-04 14:37:05 +00:00
|
|
|
".popsection\n"
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) \
|
2014-12-27 09:41:52 +00:00
|
|
|
OLDINSTR_2(oldinstr, 1, 2) \
|
2012-09-21 19:43:08 +00:00
|
|
|
".pushsection .altinstructions,\"a\"\n" \
|
2022-12-22 11:42:24 +00:00
|
|
|
ALTINSTR_ENTRY(ft_flags1, 1) \
|
|
|
|
|
ALTINSTR_ENTRY(ft_flags2, 2) \
|
2012-09-21 19:43:08 +00:00
|
|
|
".popsection\n" \
|
|
|
|
|
".pushsection .altinstr_replacement, \"ax\"\n" \
|
2021-03-09 13:48:04 +00:00
|
|
|
ALTINSTR_REPLACEMENT(newinstr1, 1) \
|
|
|
|
|
ALTINSTR_REPLACEMENT(newinstr2, 2) \
|
2018-01-04 14:37:05 +00:00
|
|
|
".popsection\n"
|
2009-04-28 15:13:46 +00:00
|
|
|
|
2021-03-11 14:23:11 +00:00
|
|
|
/* If @feature is set, patch in @newinstr_yes, otherwise @newinstr_no. */
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
|
2021-03-11 14:23:11 +00:00
|
|
|
ALTERNATIVE_2(oldinstr, newinstr_no, X86_FEATURE_ALWAYS, \
|
2022-12-22 11:42:24 +00:00
|
|
|
newinstr_yes, ft_flags)
|
|
|
|
|
|
|
|
|
|
#define ALTERNATIVE_3(oldinsn, newinsn1, ft_flags1, newinsn2, ft_flags2, \
|
|
|
|
|
newinsn3, ft_flags3) \
|
|
|
|
|
OLDINSTR_3(oldinsn, 1, 2, 3) \
|
|
|
|
|
".pushsection .altinstructions,\"a\"\n" \
|
|
|
|
|
ALTINSTR_ENTRY(ft_flags1, 1) \
|
|
|
|
|
ALTINSTR_ENTRY(ft_flags2, 2) \
|
|
|
|
|
ALTINSTR_ENTRY(ft_flags3, 3) \
|
|
|
|
|
".popsection\n" \
|
|
|
|
|
".pushsection .altinstr_replacement, \"ax\"\n" \
|
|
|
|
|
ALTINSTR_REPLACEMENT(newinsn1, 1) \
|
|
|
|
|
ALTINSTR_REPLACEMENT(newinsn2, 2) \
|
|
|
|
|
ALTINSTR_REPLACEMENT(newinsn3, 3) \
|
2018-12-10 15:17:23 +00:00
|
|
|
".popsection\n"
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
/*
|
|
|
|
|
* Alternative instructions for different CPU types or capabilities.
|
|
|
|
|
*
|
|
|
|
|
* This allows to use optimized instructions even on generic binary
|
|
|
|
|
* kernels.
|
|
|
|
|
*
|
|
|
|
|
* length of oldinstr must be longer or equal the length of newinstr
|
|
|
|
|
* It can be padded with nops as needed.
|
|
|
|
|
*
|
|
|
|
|
* For non barrier like inlines please define new variants
|
|
|
|
|
* without volatile and memory clobber.
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative(oldinstr, newinstr, ft_flags) \
|
|
|
|
|
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) : : : "memory")
|
2008-01-30 12:30:30 +00:00
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) \
|
|
|
|
|
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) ::: "memory")
|
2014-12-27 09:41:52 +00:00
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_ternary(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
|
|
|
|
|
asm_inline volatile(ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) ::: "memory")
|
2021-03-11 14:23:11 +00:00
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
/*
|
|
|
|
|
* Alternative inline assembly with input.
|
|
|
|
|
*
|
2018-12-03 09:47:34 +00:00
|
|
|
* Peculiarities:
|
2008-01-30 12:30:30 +00:00
|
|
|
* No memory clobber here.
|
|
|
|
|
* Argument numbers start with 1.
|
2009-04-28 15:13:46 +00:00
|
|
|
* Leaving an unused argument 0 to keep API compatibility.
|
2008-01-30 12:30:30 +00:00
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_input(oldinstr, newinstr, ft_flags, input...) \
|
|
|
|
|
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) \
|
2009-04-28 15:13:46 +00:00
|
|
|
: : "i" (0), ## input)
|
2008-01-30 12:30:30 +00:00
|
|
|
|
x86/alternative: Add alternative_input_2 to support alternative with two features and input
alternative_input_2() replaces old instruction with new instructions with
input based on two features.
In alternative_input_2(oldinstr, newinstr1, feature1, newinstr2, feature2,
input...),
feature2 has higher priority to replace oldinstr than feature1.
If CPU has feature2, newinstr2 replaces oldinstr and newinstr2 is
executed during run time.
If CPU doesn't have feature2, but it has feature1, newinstr1 replaces oldinstr
and newinstr1 is executed during run time.
If CPU doesn't have feature2 and feature1, oldinstr is executed during run
time.
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Link: http://lkml.kernel.org/r/1401387164-43416-5-git-send-email-fenghua.yu@intel.com
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2014-05-29 18:12:32 +00:00
|
|
|
/*
|
|
|
|
|
* This is similar to alternative_input. But it has two features and
|
|
|
|
|
* respective instructions.
|
|
|
|
|
*
|
|
|
|
|
* If CPU has feature2, newinstr2 is used.
|
|
|
|
|
* Otherwise, if CPU has feature1, newinstr1 is used.
|
|
|
|
|
* Otherwise, oldinstr is used.
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_input_2(oldinstr, newinstr1, ft_flags1, newinstr2, \
|
|
|
|
|
ft_flags2, input...) \
|
|
|
|
|
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, \
|
|
|
|
|
newinstr2, ft_flags2) \
|
x86/alternative: Add alternative_input_2 to support alternative with two features and input
alternative_input_2() replaces old instruction with new instructions with
input based on two features.
In alternative_input_2(oldinstr, newinstr1, feature1, newinstr2, feature2,
input...),
feature2 has higher priority to replace oldinstr than feature1.
If CPU has feature2, newinstr2 replaces oldinstr and newinstr2 is
executed during run time.
If CPU doesn't have feature2, but it has feature1, newinstr1 replaces oldinstr
and newinstr1 is executed during run time.
If CPU doesn't have feature2 and feature1, oldinstr is executed during run
time.
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Link: http://lkml.kernel.org/r/1401387164-43416-5-git-send-email-fenghua.yu@intel.com
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2014-05-29 18:12:32 +00:00
|
|
|
: : "i" (0), ## input)
|
|
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
/* Like alternative_input, but with a single output argument */
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_io(oldinstr, newinstr, ft_flags, output, input...) \
|
|
|
|
|
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) \
|
2009-04-28 15:13:46 +00:00
|
|
|
: output : "i" (0), ## input)
|
2008-01-30 12:30:30 +00:00
|
|
|
|
2009-12-18 16:12:56 +00:00
|
|
|
/* Like alternative_io, but for replacing a direct call with another one. */
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_call(oldfunc, newfunc, ft_flags, output, input...) \
|
|
|
|
|
asm_inline volatile (ALTERNATIVE("call %P[old]", "call %P[new]", ft_flags) \
|
2009-12-18 16:12:56 +00:00
|
|
|
: output : [old] "i" (oldfunc), [new] "i" (newfunc), ## input)
|
|
|
|
|
|
2012-05-25 01:19:45 +00:00
|
|
|
/*
|
|
|
|
|
* Like alternative_call, but there are two features and respective functions.
|
|
|
|
|
* If CPU has feature2, function2 is used.
|
|
|
|
|
* Otherwise, if CPU has feature1, function1 is used.
|
|
|
|
|
* Otherwise, old function is used.
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
#define alternative_call_2(oldfunc, newfunc1, ft_flags1, newfunc2, ft_flags2, \
|
2012-05-25 01:19:45 +00:00
|
|
|
output, input...) \
|
2022-12-22 11:42:24 +00:00
|
|
|
asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", ft_flags1,\
|
|
|
|
|
"call %P[new2]", ft_flags2) \
|
x86/asm: Fix inline asm call constraints for Clang
For inline asm statements which have a CALL instruction, we list the
stack pointer as a constraint to convince GCC to ensure the frame
pointer is set up first:
static inline void foo()
{
register void *__sp asm(_ASM_SP);
asm("call bar" : "+r" (__sp))
}
Unfortunately, that pattern causes Clang to corrupt the stack pointer.
The fix is easy: convert the stack pointer register variable to a global
variable.
It should be noted that the end result is different based on the GCC
version. With GCC 6.4, this patch has exactly the same result as
before:
defconfig defconfig-nofp distro distro-nofp
before 9820389 9491555 8816046 8516940
after 9820389 9491555 8816046 8516940
With GCC 7.2, however, GCC's behavior has changed. It now changes its
behavior based on the conversion of the register variable to a global.
That somehow convinces it to *always* set up the frame pointer before
inserting *any* inline asm. (Therefore, listing the variable as an
output constraint is a no-op and is no longer necessary.) It's a bit
overkill, but the performance impact should be negligible. And in fact,
there's a nice improvement with frame pointers disabled:
defconfig defconfig-nofp distro distro-nofp
before 9796316 9468236 9076191 8790305
after 9796957 9464267 9076381 8785949
So in summary, while listing the stack pointer as an output constraint
is no longer necessary for newer versions of GCC, it's still needed for
older versions.
Suggested-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Reported-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Dmitriy Vyukov <dvyukov@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Miguel Bernal Marin <miguel.bernal.marin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/3db862e970c432ae823cf515c52b54fec8270e0e.1505942196.git.jpoimboe@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-09-20 21:24:33 +00:00
|
|
|
: output, ASM_CALL_CONSTRAINT \
|
2016-09-23 21:49:39 +00:00
|
|
|
: [old] "i" (oldfunc), [new1] "i" (newfunc1), \
|
2018-01-14 12:05:04 +00:00
|
|
|
[new2] "i" (newfunc2), ## input)
|
2012-05-25 01:19:45 +00:00
|
|
|
|
2008-01-30 12:30:30 +00:00
|
|
|
/*
|
|
|
|
|
* use this macro(s) if you need more than one output parameter
|
|
|
|
|
* in alternative_io
|
|
|
|
|
*/
|
2009-12-18 16:12:56 +00:00
|
|
|
#define ASM_OUTPUT2(a...) a
|
2008-01-30 12:30:30 +00:00
|
|
|
|
2012-01-20 16:21:41 +00:00
|
|
|
/*
|
|
|
|
|
* use this macro if you need clobbers but no inputs in
|
|
|
|
|
* alternative_{input,io,call}()
|
|
|
|
|
*/
|
|
|
|
|
#define ASM_NO_INPUT_CLOBBER(clbr...) "i" (0) : clbr
|
|
|
|
|
|
2023-11-29 13:33:29 +00:00
|
|
|
/* Macro for creating assembler functions avoiding any C magic. */
|
|
|
|
|
#define DEFINE_ASM_FUNC(func, instr, sec) \
|
|
|
|
|
asm (".pushsection " #sec ", \"ax\"\n" \
|
|
|
|
|
".global " #func "\n\t" \
|
|
|
|
|
".type " #func ", @function\n\t" \
|
|
|
|
|
ASM_FUNC_ALIGN "\n" \
|
|
|
|
|
#func ":\n\t" \
|
|
|
|
|
ASM_ENDBR \
|
|
|
|
|
instr "\n\t" \
|
|
|
|
|
ASM_RET \
|
|
|
|
|
".size " #func ", . - " #func "\n\t" \
|
|
|
|
|
".popsection")
|
|
|
|
|
|
|
|
|
|
void BUG_func(void);
|
|
|
|
|
void nop_func(void);
|
|
|
|
|
|
2021-03-11 14:23:06 +00:00
|
|
|
#else /* __ASSEMBLY__ */
|
|
|
|
|
|
|
|
|
|
#ifdef CONFIG_SMP
|
|
|
|
|
.macro LOCK_PREFIX
|
|
|
|
|
672: lock
|
|
|
|
|
.pushsection .smp_locks,"a"
|
|
|
|
|
.balign 4
|
|
|
|
|
.long 672b - .
|
|
|
|
|
.popsection
|
|
|
|
|
.endm
|
|
|
|
|
#else
|
|
|
|
|
.macro LOCK_PREFIX
|
|
|
|
|
.endm
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* objtool annotation to ignore the alternatives and only consider the original
|
|
|
|
|
* instruction(s).
|
|
|
|
|
*/
|
|
|
|
|
.macro ANNOTATE_IGNORE_ALTERNATIVE
|
|
|
|
|
.Lannotate_\@:
|
|
|
|
|
.pushsection .discard.ignore_alts
|
x86/speculation, objtool: Use absolute relocations for annotations
.discard.retpoline_safe sections do not have the SHF_ALLOC flag. These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable. Newer
LLD will report warnings for REL relocations even for relocatable links [1]:
ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''
Switch to absolute relocations instead, which indicate link-time
addresses. In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c. When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.
Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.
Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized. That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).
[0] commit 1c0c1faf5692 ("objtool: Use relative pointers for annotations")
[1] https://github.com/ClangBuiltLinux/linux/issues/1937
Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
2023-09-20 00:17:28 +00:00
|
|
|
.long .Lannotate_\@
|
2021-03-11 14:23:06 +00:00
|
|
|
.popsection
|
|
|
|
|
.endm
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Issue one struct alt_instr descriptor entry (need to put it into
|
|
|
|
|
* the section .altinstructions, see below). This entry contains
|
|
|
|
|
* enough information for the alternatives patching code to patch an
|
|
|
|
|
* instruction. See apply_alternatives().
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
.macro altinstr_entry orig alt ft_flags orig_len alt_len
|
2021-03-11 14:23:06 +00:00
|
|
|
.long \orig - .
|
|
|
|
|
.long \alt - .
|
2022-12-22 11:42:24 +00:00
|
|
|
.4byte \ft_flags
|
2021-03-11 14:23:06 +00:00
|
|
|
.byte \orig_len
|
|
|
|
|
.byte \alt_len
|
|
|
|
|
.endm
|
|
|
|
|
|
2023-12-10 06:21:36 +00:00
|
|
|
.macro ALT_CALL_INSTR
|
|
|
|
|
call BUG_func
|
|
|
|
|
.endm
|
|
|
|
|
|
2021-03-11 14:23:06 +00:00
|
|
|
/*
|
|
|
|
|
* Define an alternative between two instructions. If @feature is
|
|
|
|
|
* present, early code in apply_alternatives() replaces @oldinstr with
|
|
|
|
|
* @newinstr. ".skip" directive takes care of proper instruction padding
|
|
|
|
|
* in case @newinstr is longer than @oldinstr.
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
.macro ALTERNATIVE oldinstr, newinstr, ft_flags
|
2021-03-11 14:23:06 +00:00
|
|
|
140:
|
|
|
|
|
\oldinstr
|
|
|
|
|
141:
|
|
|
|
|
.skip -(((144f-143f)-(141b-140b)) > 0) * ((144f-143f)-(141b-140b)),0x90
|
|
|
|
|
142:
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstructions,"a"
|
2022-12-22 11:42:24 +00:00
|
|
|
altinstr_entry 140b,143f,\ft_flags,142b-140b,144f-143f
|
2021-03-11 14:23:06 +00:00
|
|
|
.popsection
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstr_replacement,"ax"
|
|
|
|
|
143:
|
|
|
|
|
\newinstr
|
|
|
|
|
144:
|
|
|
|
|
.popsection
|
|
|
|
|
.endm
|
|
|
|
|
|
|
|
|
|
#define old_len 141b-140b
|
|
|
|
|
#define new_len1 144f-143f
|
|
|
|
|
#define new_len2 145f-144f
|
2022-09-15 11:11:26 +00:00
|
|
|
#define new_len3 146f-145f
|
2021-03-11 14:23:06 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* gas compatible max based on the idea from:
|
|
|
|
|
* http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
|
|
|
|
|
*
|
|
|
|
|
* The additional "-" is needed because gas uses a "true" value of -1.
|
|
|
|
|
*/
|
2022-09-15 11:11:26 +00:00
|
|
|
#define alt_max_2(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
|
|
|
|
|
#define alt_max_3(a, b, c) (alt_max_2(alt_max_2(a, b), c))
|
2021-03-11 14:23:06 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Same as ALTERNATIVE macro above but for two alternatives. If CPU
|
|
|
|
|
* has @feature1, it replaces @oldinstr with @newinstr1. If CPU has
|
|
|
|
|
* @feature2, it replaces @oldinstr with @feature2.
|
|
|
|
|
*/
|
2022-12-22 11:42:24 +00:00
|
|
|
.macro ALTERNATIVE_2 oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2
|
2021-03-11 14:23:06 +00:00
|
|
|
140:
|
|
|
|
|
\oldinstr
|
|
|
|
|
141:
|
2022-09-15 11:11:26 +00:00
|
|
|
.skip -((alt_max_2(new_len1, new_len2) - (old_len)) > 0) * \
|
|
|
|
|
(alt_max_2(new_len1, new_len2) - (old_len)),0x90
|
2021-03-11 14:23:06 +00:00
|
|
|
142:
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstructions,"a"
|
2022-12-22 11:42:24 +00:00
|
|
|
altinstr_entry 140b,143f,\ft_flags1,142b-140b,144f-143f
|
|
|
|
|
altinstr_entry 140b,144f,\ft_flags2,142b-140b,145f-144f
|
2021-03-11 14:23:06 +00:00
|
|
|
.popsection
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstr_replacement,"ax"
|
|
|
|
|
143:
|
|
|
|
|
\newinstr1
|
|
|
|
|
144:
|
|
|
|
|
\newinstr2
|
|
|
|
|
145:
|
|
|
|
|
.popsection
|
|
|
|
|
.endm
|
|
|
|
|
|
2022-12-22 11:42:24 +00:00
|
|
|
.macro ALTERNATIVE_3 oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2, newinstr3, ft_flags3
|
2022-09-15 11:11:26 +00:00
|
|
|
140:
|
|
|
|
|
\oldinstr
|
|
|
|
|
141:
|
|
|
|
|
.skip -((alt_max_3(new_len1, new_len2, new_len3) - (old_len)) > 0) * \
|
|
|
|
|
(alt_max_3(new_len1, new_len2, new_len3) - (old_len)),0x90
|
|
|
|
|
142:
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstructions,"a"
|
2022-12-22 11:42:24 +00:00
|
|
|
altinstr_entry 140b,143f,\ft_flags1,142b-140b,144f-143f
|
|
|
|
|
altinstr_entry 140b,144f,\ft_flags2,142b-140b,145f-144f
|
|
|
|
|
altinstr_entry 140b,145f,\ft_flags3,142b-140b,146f-145f
|
2022-09-15 11:11:26 +00:00
|
|
|
.popsection
|
|
|
|
|
|
|
|
|
|
.pushsection .altinstr_replacement,"ax"
|
|
|
|
|
143:
|
|
|
|
|
\newinstr1
|
|
|
|
|
144:
|
|
|
|
|
\newinstr2
|
|
|
|
|
145:
|
|
|
|
|
\newinstr3
|
|
|
|
|
146:
|
|
|
|
|
.popsection
|
|
|
|
|
.endm
|
|
|
|
|
|
2021-03-11 14:23:11 +00:00
|
|
|
/* If @feature is set, patch in @newinstr_yes, otherwise @newinstr_no. */
|
2022-12-22 11:42:24 +00:00
|
|
|
#define ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
|
2021-03-11 14:23:11 +00:00
|
|
|
ALTERNATIVE_2 oldinstr, newinstr_no, X86_FEATURE_ALWAYS, \
|
2022-12-22 11:42:24 +00:00
|
|
|
newinstr_yes, ft_flags
|
2021-03-11 14:23:11 +00:00
|
|
|
|
2016-04-26 19:23:25 +00:00
|
|
|
#endif /* __ASSEMBLY__ */
|
|
|
|
|
|
2008-10-23 05:26:29 +00:00
|
|
|
#endif /* _ASM_X86_ALTERNATIVE_H */
|