2019-06-04 08:11:33 +00:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0-only */
|
2016-02-23 07:56:45 +00:00
|
|
|
/*
|
|
|
|
|
* Copyright (C) 2012 ARM Ltd.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifndef __ASM_BRK_IMM_H
|
|
|
|
|
#define __ASM_BRK_IMM_H
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* #imm16 values used for BRK instruction generation
|
2019-02-26 15:06:42 +00:00
|
|
|
* 0x004: for installing kprobes
|
|
|
|
|
* 0x005: for installing uprobes
|
2020-11-03 13:49:01 +00:00
|
|
|
* 0x006: for kprobe software single-step
|
arm64: kretprobes: acquire the regs via a BRK exception
On arm64, kprobes always take an exception and so create a struct
pt_regs through the usual exception entry logic. Similarly kretprobes
taskes and exception for function entry, but for function returns it
uses a trampoline which attempts to create a struct pt_regs without
taking an exception.
This is problematic for a few reasons, including:
1) The kretprobes trampoline neither saves nor restores all of the
portions of PSTATE. Before invoking the handler it saves a number of
portions of PSTATE, and after returning from the handler it restores
NZCV before returning to the original return address provided by the
handler.
2) The kretprobe trampoline constructs the PSTATE value piecemeal from
special purpose registers as it cannot read all of PSTATE atomically
without taking an exception. This is somewhat fragile, and it's not
possible to reliably recover PSTATE information which only exists on
some physical CPUs (e.g. when SSBS support is mismatched).
Today the kretprobes trampoline does not record:
- BTYPE
- SSBS
- ALLINT
- SS
- PAN
- UAO
- DIT
- TCO
... and this will only get worse with future architecture extensions
which add more PSTATE bits.
3) The kretprobes trampoline doesn't store portions of struct pt_regs
(e.g. the PMR value when using pseudo-NMIs). Due to this, helpers
which operate on a struct pt_regs, such as interrupts_enabled(), may
not work correctly.
4) The function entry and function exit handlers run in different
contexts. The entry handler will always be run in a debug exception
context (which is currently treated as an NMI), but the return will
be treated as whatever context the instrumented function was executed
in. The differences between these contexts are liable to cause
problems (e.g. as the two can be differently interruptible or
preemptible, adversely affecting synchronization between the
handlers).
5) As the kretprobes trampoline runs in the same context as the code
being probed, it is subject to the same single-stepping context,
which may not be desirable if this is being driven by the kprobes
handlers.
Overall, this is fragile, painful to maintain, and gets in the way of
supporting other things (e.g. RELIABLE_STACKTRACE, FEAT_NMI).
This patch addresses these issues by replacing the kretprobes trampoline
with a `BRK` instruction, and using an exception boundary to acquire and
restore the regs, in the same way as the regular kprobes trampoline.
Ive tested this atop v6.8-rc3:
| KTAP version 1
| 1..1
| KTAP version 1
| # Subtest: kprobes_test
| # module: test_kprobes
| 1..7
| ok 1 test_kprobe
| ok 2 test_kprobes
| ok 3 test_kprobe_missed
| ok 4 test_kretprobe
| ok 5 test_kretprobes
| ok 6 test_stacktrace_on_kretprobe
| ok 7 test_stacktrace_on_nested_kretprobe
| # kprobes_test: pass:7 fail:0 skip:0 total:7
| # Totals: pass:7 fail:0 skip:0 total:7
| ok 1 kprobes_test
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Florent Revest <revest@chromium.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Link: https://lore.kernel.org/r/20240208145916.2004154-1-mark.rutland@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2024-02-08 14:59:16 +00:00
|
|
|
* 0x007: for kretprobe return
|
2016-02-23 07:56:45 +00:00
|
|
|
* Allowed values for kgdb are 0x400 - 0x7ff
|
|
|
|
|
* 0x100: for triggering a fault on purpose (reserved)
|
|
|
|
|
* 0x400: for dynamic BRK instruction
|
|
|
|
|
* 0x401: for compile time BRK instruction
|
|
|
|
|
* 0x800: kernel-mode BUG() and WARN() traps
|
2018-12-28 08:30:54 +00:00
|
|
|
* 0x9xx: tag-based KASAN trap (allowed values 0x900 - 0x9ff)
|
2023-02-02 22:07:49 +00:00
|
|
|
* 0x55xx: Undefined Behavior Sanitizer traps ('U' << 8)
|
2022-09-08 21:54:52 +00:00
|
|
|
* 0x8xxx: Control-Flow Integrity traps
|
2016-02-23 07:56:45 +00:00
|
|
|
*/
|
2019-02-26 15:06:42 +00:00
|
|
|
#define KPROBES_BRK_IMM 0x004
|
|
|
|
|
#define UPROBES_BRK_IMM 0x005
|
2020-11-03 13:49:01 +00:00
|
|
|
#define KPROBES_BRK_SS_IMM 0x006
|
arm64: kretprobes: acquire the regs via a BRK exception
On arm64, kprobes always take an exception and so create a struct
pt_regs through the usual exception entry logic. Similarly kretprobes
taskes and exception for function entry, but for function returns it
uses a trampoline which attempts to create a struct pt_regs without
taking an exception.
This is problematic for a few reasons, including:
1) The kretprobes trampoline neither saves nor restores all of the
portions of PSTATE. Before invoking the handler it saves a number of
portions of PSTATE, and after returning from the handler it restores
NZCV before returning to the original return address provided by the
handler.
2) The kretprobe trampoline constructs the PSTATE value piecemeal from
special purpose registers as it cannot read all of PSTATE atomically
without taking an exception. This is somewhat fragile, and it's not
possible to reliably recover PSTATE information which only exists on
some physical CPUs (e.g. when SSBS support is mismatched).
Today the kretprobes trampoline does not record:
- BTYPE
- SSBS
- ALLINT
- SS
- PAN
- UAO
- DIT
- TCO
... and this will only get worse with future architecture extensions
which add more PSTATE bits.
3) The kretprobes trampoline doesn't store portions of struct pt_regs
(e.g. the PMR value when using pseudo-NMIs). Due to this, helpers
which operate on a struct pt_regs, such as interrupts_enabled(), may
not work correctly.
4) The function entry and function exit handlers run in different
contexts. The entry handler will always be run in a debug exception
context (which is currently treated as an NMI), but the return will
be treated as whatever context the instrumented function was executed
in. The differences between these contexts are liable to cause
problems (e.g. as the two can be differently interruptible or
preemptible, adversely affecting synchronization between the
handlers).
5) As the kretprobes trampoline runs in the same context as the code
being probed, it is subject to the same single-stepping context,
which may not be desirable if this is being driven by the kprobes
handlers.
Overall, this is fragile, painful to maintain, and gets in the way of
supporting other things (e.g. RELIABLE_STACKTRACE, FEAT_NMI).
This patch addresses these issues by replacing the kretprobes trampoline
with a `BRK` instruction, and using an exception boundary to acquire and
restore the regs, in the same way as the regular kprobes trampoline.
Ive tested this atop v6.8-rc3:
| KTAP version 1
| 1..1
| KTAP version 1
| # Subtest: kprobes_test
| # module: test_kprobes
| 1..7
| ok 1 test_kprobe
| ok 2 test_kprobes
| ok 3 test_kprobe_missed
| ok 4 test_kretprobe
| ok 5 test_kretprobes
| ok 6 test_stacktrace_on_kretprobe
| ok 7 test_stacktrace_on_nested_kretprobe
| # kprobes_test: pass:7 fail:0 skip:0 total:7
| # Totals: pass:7 fail:0 skip:0 total:7
| ok 1 kprobes_test
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Florent Revest <revest@chromium.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Link: https://lore.kernel.org/r/20240208145916.2004154-1-mark.rutland@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2024-02-08 14:59:16 +00:00
|
|
|
#define KRETPROBES_BRK_IMM 0x007
|
2016-02-23 07:56:45 +00:00
|
|
|
#define FAULT_BRK_IMM 0x100
|
|
|
|
|
#define KGDB_DYN_DBG_BRK_IMM 0x400
|
|
|
|
|
#define KGDB_COMPILED_DBG_BRK_IMM 0x401
|
|
|
|
|
#define BUG_BRK_IMM 0x800
|
2018-12-28 08:30:54 +00:00
|
|
|
#define KASAN_BRK_IMM 0x900
|
2019-02-26 12:52:47 +00:00
|
|
|
#define KASAN_BRK_MASK 0x0ff
|
2023-02-02 22:07:49 +00:00
|
|
|
#define UBSAN_BRK_IMM 0x5500
|
|
|
|
|
#define UBSAN_BRK_MASK 0x00ff
|
2016-02-23 07:56:45 +00:00
|
|
|
|
2022-09-08 21:54:52 +00:00
|
|
|
#define CFI_BRK_IMM_TARGET GENMASK(4, 0)
|
|
|
|
|
#define CFI_BRK_IMM_TYPE GENMASK(9, 5)
|
|
|
|
|
#define CFI_BRK_IMM_BASE 0x8000
|
|
|
|
|
#define CFI_BRK_IMM_MASK (CFI_BRK_IMM_TARGET | CFI_BRK_IMM_TYPE)
|
|
|
|
|
|
2016-02-23 07:56:45 +00:00
|
|
|
#endif
|