linux-stable/arch/arm64/kernel
Ard Biesheuvel 56796ed90e arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
commit 34e526cb7d upstream.

Even though the boot protocol stipulates otherwise, an exception has
been made for the EFI stub, and entering the core kernel with the MMU
enabled is permitted. This allows a substantial amount of cache
maintenance to be elided, wich is significant when fast boot times are
critical (e.g., for booting micro-VMs)

Once the initial ID map has been populated, the MMU is disabled as part
of the logic sequence that puts all system registers into a known state.
Any code that needs to execute within the window where the MMU is off is
cleaned to the PoC explicitly, which includes all of HYP text when
entering at EL2.

However, the current sequence of initializing the EL2 system registers
is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2
is reprogrammed, and this means that a VHE-to-nVHE switch may occur
while the MMU is enabled. This switch causes some system registers as
well as page table descriptors to be interpreted in a different way,
potentially resulting in spurious exceptions relating to MMU
translation.

So disable the MMU explicitly first when entering in EL2 with the MMU
and caches enabled.

Fixes: 6178617038 ("efi: arm64: enter with MMU and caches enabled")
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Cc: <stable@vger.kernel.org> # 6.3.x
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20240415075412.2347624-6-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-04-27 17:13:01 +02:00
..
pi arm64: kernel: Disable latent_entropy GCC plugin in early C runtime 2023-12-12 11:06:27 +00:00
probes arm64: kprobes: Drop ID map text from kprobes blacklist 2023-02-07 18:25:10 +00:00
vdso kbuild: unify vdso_install rules 2023-10-28 21:09:02 +09:00
vdso32 arm64: vdso32: Remove unused vdso32-offsets.h 2024-01-30 11:59:17 +00:00
.gitignore
Makefile arm64: scs: Disable LTO for SCS patching code 2024-01-30 11:52:46 +00:00
acpi.c arm64: acpi: Fix possible memory leak of ffh_ctxt 2023-02-24 14:21:49 +00:00
acpi_numa.c arm64: numa: Don't check node against MAX_NUMNODES 2022-07-19 19:10:28 +01:00
acpi_parking_protocol.c arm64: smp: Remove dedicated wakeup IPI 2023-09-25 17:15:28 +01:00
alternative.c Merge branches 'for-next/kpti', 'for-next/missing-proto-warn', 'for-next/iss2-decode', 'for-next/kselftest', 'for-next/misc', 'for-next/feat_mops', 'for-next/module-alloc', 'for-next/sysreg', 'for-next/cpucap', 'for-next/acpi', 'for-next/kdump', 'for-next/acpi-doc', 'for-next/doc' and 'for-next/tpidr2-fix', remote-tracking branch 'arm64/for-next/perf' into for-next/core 2023-06-23 18:32:20 +01:00
armv8_deprecated.c arm: Remove now superfluous sentinel elem from ctl_table arrays 2023-10-10 15:22:02 -07:00
asm-offsets.c arm64: fpsimd: Bring cond_yield asm macro in line with new rules 2024-01-12 12:48:27 +00:00
cacheinfo.c cacheinfo: Add arm64 early level initializer implementation 2023-04-13 09:32:33 +01:00
compat_alignment.c arm64: compat: Work around uninitialized variable warning 2023-04-05 17:51:47 +01:00
cpu-reset.S arm64: kernel: remove SHF_WRITE|SHF_EXECINSTR from .idmap.text 2023-05-02 12:42:22 +01:00
cpu_errata.c arm64: Subscribe Microsoft Azure Cobalt 100 to ARM Neoverse N2 errata 2024-02-15 11:47:22 +00:00
cpu_ops.c
cpufeature.c Generic: 2024-01-17 13:03:37 -08:00
cpuidle.c arm64: Explicitly include correct DT includes 2023-07-31 14:45:38 +01:00
cpuinfo.c arm64: Kill detection of VPIPT i-cache policy 2023-12-05 11:38:03 +00:00
crash_core.c arm64: move PAC masks to <asm/pointer_auth.h> 2023-04-13 12:27:11 +01:00
crash_dump.c
debug-monitors.c arm64: kgdb: Set PSTATE.SS to 1 to re-enable single-step 2023-04-14 13:39:47 +01:00
efi-header.S arm64: efi: Enable BTI codegen and add PE/COFF annotation 2023-04-20 15:43:45 +02:00
efi-rt-wrapper.S arm64: efi: Avoid workqueue to check whether EFI runtime is live 2023-01-16 15:27:31 +01:00
efi.c TTY/Serial changes for 6.7-rc1 2023-11-03 15:44:25 -10:00
elfcore.c arm64: mte: Avoid the racy walk of the vma list during core dump 2023-01-05 15:12:12 +00:00
entry-common.c arm64: syscall: unmask DAIF earlier for SVCs 2023-08-11 12:23:48 +01:00
entry-fpsimd.S arm64/sme: Implement context switching for ZT0 2023-01-20 12:23:06 +00:00
entry-ftrace.S arm64: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL 2023-06-20 18:38:37 -04:00
entry.S arm64: entry: simplify kernel_exit logic 2024-01-18 11:00:09 +00:00
fpsimd.c arm64/sme: Restore SMCR_EL1.EZT0 on exit from suspend 2024-02-20 12:19:16 +00:00
ftrace.c arm64: module: mandate MODULE_PLTS 2023-06-06 17:39:05 +01:00
head.S arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H 2024-04-27 17:13:01 +02:00
hibernate-asm.S
hibernate.c arm64: hibernate: remove WARN_ON in save_processor_state 2023-06-21 13:33:49 +01:00
hw_breakpoint.c hw_breakpoint: fix single-stepping when using bpf_overflow_handler 2023-08-18 17:04:09 +01:00
hyp-stub.S ARM64: 2023-07-03 15:32:22 -07:00
idle.c arm64: idle: Tag the arm64 idle functions as __cpuidle 2023-09-25 17:15:28 +01:00
idreg-override.c arm64/kernel: Move 'nokaslr' parsing out of early idreg code 2023-12-12 11:13:53 +00:00
image-vars.h efi: move screen_info into efi init code 2023-10-17 16:33:39 +02:00
image.h
io.c
irq.c arm64: irq: set the correct node for shadow call stack 2023-12-13 12:09:00 +00:00
jump_label.c jump_label: make initial NOP patching the special case 2022-06-24 09:48:55 +02:00
kaslr.c arm64/kernel: Move 'nokaslr' parsing out of early idreg code 2023-12-12 11:13:53 +00:00
kexec_image.c kexec_file, arm64: print out debugging message if required 2023-12-20 15:02:57 -08:00
kgdb.c arm64: kgdb: Set PSTATE.SS to 1 to re-enable single-step 2023-04-14 13:39:47 +01:00
kuser32.S arm64: Update Documentation/arm references 2023-06-12 06:33:48 -06:00
machine_kexec.c kexec_file, arm64: print out debugging message if required 2023-12-20 15:02:57 -08:00
machine_kexec_file.c kexec_file, arm64: print out debugging message if required 2023-12-20 15:02:57 -08:00
module-plts.c Merge branch 'for-next/cpus_have_const_cap' into for-next/core 2023-10-26 17:10:18 +01:00
module.c arm64: module: rework module VA range selection 2023-06-06 17:39:06 +01:00
mte.c Many singleton patches against the MM code. The patch series which are 2023-11-02 19:38:47 -10:00
paravirt.c arm64: paravirt: remove conduit check in has_pv_steal_clock 2022-11-09 18:11:56 +00:00
patch-scs.c arm64: Apply dynamic shadow call stack patching in two passes 2023-01-26 17:54:15 +00:00
patching.c arm64: patching: Add aarch64_insn_write_literal_u64() 2023-01-24 11:49:43 +00:00
pci.c arm64: Explicitly include correct DT includes 2023-07-31 14:45:38 +01:00
perf_callchain.c arm64: use XPACLRI to strip PAC 2023-04-13 12:27:11 +01:00
perf_regs.c perf: arm64: Add SVE vector granule register to user regs 2022-09-22 15:06:02 +01:00
pointer_auth.c
process.c sysctl-6.7-rc1 2023-11-01 20:51:41 -10:00
proton-pack.c arm64: Avoid cpus_have_const_cap() for ARM64_UNMAP_KERNEL_AT_EL0 2023-10-16 14:17:06 +01:00
psci.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
ptrace.c arm64/ptrace: Use saved floating point state type to determine SVE layout 2024-04-10 16:38:19 +02:00
reloc_test_core.c ARM64: reloc_test: add __init/__exit annotations to module init/exit funcs 2022-09-29 18:04:25 +01:00
reloc_test_syms.S
relocate_kernel.S
return_address.c
sdei.c arm64: sdei: abort running SDEI handlers during crash 2023-08-04 17:35:33 +01:00
setup.c arm64: convert to arch_cpu_is_hotpluggable() 2023-12-06 12:41:49 +09:00
signal.c arm64/signal: Don't assume that TIF_SVE means we saved SVE state 2024-02-09 16:34:23 +00:00
signal32.c x86/shstk: Add user control-protection fault handler 2023-08-02 15:01:50 -07:00
sigreturn32.S arm64: compat: Move sigreturn32.S to .rodata section 2022-07-01 12:05:45 +01:00
sleep.S arm64: kernel: remove SHF_WRITE|SHF_EXECINSTR from .idmap.text 2023-05-02 12:42:22 +01:00
smccc-call.S
smp.c arm64: Align boot cpucap handling with system cpucap handling 2023-12-13 16:02:01 +00:00
smp_spin_table.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
stacktrace.c arm64: prohibit probing on arch_kunwind_consume_entry() 2024-03-04 13:00:00 +00:00
suspend.c arm64/sme: Restore SME registers on exit from suspend 2024-02-20 12:19:15 +00:00
sys.c
sys32.c
sys_compat.c arm64: Avoid cpus_have_const_cap() for ARM64_WORKAROUND_1542419 2023-10-16 14:17:06 +01:00
syscall.c arm64: syscall: unmask DAIF earlier for SVCs 2023-08-11 12:23:48 +01:00
time.c
topology.c arm64/amu: Use capacity_ref_freq() to set AMU ratio 2023-12-23 15:52:36 +01:00
trace-events-emulation.h
traps.c ARM: 2023-11-02 15:45:15 -10:00
vdso-wrap.S
vdso.c arm64: Avoid cpus_have_const_cap() for ARM64_HAS_BTI 2023-10-16 14:17:04 +01:00
vdso32-wrap.S
vmlinux.lds.S arm64: lds: move .got section out of .text 2023-05-02 13:12:45 +01:00
watchdog_hld.c arm64: enable perf events based hard lockup detector 2023-06-09 17:44:22 -07:00