linux-stable/include/asm-generic
Alexandre Ghiti c4a05cf0ed mm: Introduce flush_cache_vmap_early()
[ Upstream commit 7a92fc8b4d ]

The pcpu setup when using the page allocator sets up a new vmalloc
mapping very early in the boot process, so early that it cannot use the
flush_cache_vmap() function which may depend on structures not yet
initialized (for example in riscv, we currently send an IPI to flush
other cpus TLB).

But on some architectures, we must call flush_cache_vmap(): for example,
in riscv, some uarchs can cache invalid TLB entries so we need to flush
the new established mapping to avoid taking an exception.

So fix this by introducing a new function flush_cache_vmap_early() which
is called right after setting the new page table entry and before
accessing this new mapping. This new function implements a local flush
tlb on riscv and is no-op for other architectures (same as today).

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Dennis Zhou <dennis@kernel.org>
Stable-dep-of: d9807d60c1 ("riscv: mm: execute local TLB flush after populating vmemmap")
Signed-off-by: Sasha Levin <sashal@kernel.org>
2024-02-16 19:10:52 +01:00
..
bitops locking/atomic: treewide: use raw_atomic*_<op>() 2023-06-05 09:57:20 +02:00
vdso
access_ok.h uaccess: remove CONFIG_SET_FS 2022-02-25 09:36:06 +01:00
agp.h char/agp: introduce asm-generic/agp.h 2023-02-13 22:13:29 +01:00
archrandom.h random: handle archrandom with multiple longs 2022-07-25 13:26:14 +02:00
asm-offsets.h
asm-prototypes.h
atomic.h locking/atomic: make atomic*_{cmp,}xchg optional 2023-06-05 09:57:14 +02:00
atomic64.h locking/atomic: delete !ARCH_ATOMIC remnants 2021-05-26 13:20:52 +02:00
audit_change_attr.h
audit_dir_write.h
audit_read.h
audit_signal.h
audit_write.h
barrier.h asm-generic: Add memory barrier dma_mb() 2022-06-23 18:34:58 +01:00
bitops.h include: move find.h from asm_generic to linux 2022-01-15 08:47:31 -08:00
bitsperlong.h lib: extend the scope of small_const_nbits() macro 2021-05-06 19:24:11 -07:00
bug.h panic: make function declarations visible 2023-06-09 17:44:15 -07:00
cache.h
cacheflush.h mm: Introduce flush_cache_vmap_early() 2024-02-16 19:10:52 +01:00
checksum.h unify generic instances of csum_partial_copy_nocheck() 2020-08-20 15:45:14 -04:00
cmpxchg-local.h asm-generic: Fix 32 bit __generic_cmpxchg_local 2024-01-25 15:35:19 -08:00
cmpxchg.h asm-generic: avoid __generic_cmpxchg_local warnings 2023-04-04 17:58:11 +02:00
compat.h asm-generic: compat: fix compat_arg_u64() and compat_arg_u64_dual() 2022-11-01 10:20:11 +11:00
current.h asm-generic: current: Don't include thread-info.h if building asm 2023-08-26 22:38:49 +02:00
delay.h
device.h
div64.h ARM: 9117/1: asm-generic: div64: Remove always-true __div64_const32_is_OK() 2021-08-20 11:39:28 +01:00
dma-mapping.h dma-mapping: no need to pass a bus_type into get_arch_dma_ops() 2023-02-15 12:35:20 +01:00
dma.h
early_ioremap.h mm/early_ioremap.c: remove redundant early_ioremap_shutdown() 2021-09-08 11:50:24 -07:00
emergency-restart.h
error-injection.h docs: fault-injection: add requirements of error injectable functions 2023-02-02 22:50:00 -08:00
exec.h
export.h ia64,export.h: replace EXPORT_DATA_SYMBOL* with EXPORT_SYMBOL* 2023-06-22 21:21:06 +09:00
extable.h
fb.h fbdev: Rename fb_mem*() helpers 2023-05-18 11:07:54 +02:00
fixmap.h
flat.h
ftrace.h
futex.h futex: Fix additional regressions 2021-12-11 23:31:51 +01:00
getorder.h asm-generic: force inlining of get_order() to work around gcc10 poor decision 2020-12-15 22:46:15 -08:00
hardirq.h irqstat: Move declaration into asm-generic/hardirq.h 2020-11-23 10:31:06 +01:00
hugetlb.h mm: hugetlb: add huge page size param to set_huge_pte_at() 2023-09-29 17:20:47 -07:00
hw_irq.h
hyperv-tlfs.h x86/hyperv: Add smp support for SEV-SNP guest 2023-08-22 00:38:20 +00:00
int-ll64.h
io.h mm: ioremap: remove unneeded ioremap_allowed and iounmap_allowed 2023-08-18 10:12:36 -07:00
ioctl.h
iomap.h asm-generic/iomap.h: remove ARCH_HAS_IOREMAP_xx macros 2023-08-18 10:12:32 -07:00
irq.h
irq_regs.h
irq_work.h
irqflags.h
Kbuild xen: branch for v6.0-rc1 2022-08-04 15:10:55 -07:00
kdebug.h
kmap_size.h mm/highmem: Provide and use CONFIG_DEBUG_KMAP_LOCAL 2020-11-24 14:42:08 +01:00
kprobes.h treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
kvm_para.h
kvm_types.h KVM: Move x86's version of struct kvm_mmu_memory_cache to common code 2020-07-09 13:29:42 -04:00
linkage.h
local.h locking/generic: Wire up local{,64}_try_cmpxchg() 2023-04-29 09:09:09 +02:00
local64.h locking/generic: Wire up local{,64}_try_cmpxchg() 2023-04-29 09:09:09 +02:00
logic_io.h logic_io instance of iounmap() needs volatile on argument 2021-12-21 21:31:08 +01:00
mcs_spinlock.h
memory_model.h mm, arch: add generic implementation of pfn_valid() for FLATMEM 2023-02-09 16:51:41 -08:00
mm_hooks.h
mmiowb.h asm-generic/mmiowb: Allow mmiowb_set_pending() when preemptible() 2020-07-17 10:02:03 +01:00
mmiowb_types.h
mmu.h
mmu_context.h asm-generic: add generic MMU versions of mmu context functions 2020-10-26 16:45:03 +01:00
module.h
module.lds.h kbuild: preprocess module linker script 2020-09-25 00:36:41 +09:00
mshyperv.h hyperv: reduce size of ms_hyperv_info 2023-09-22 20:28:16 +00:00
msi.h genirq: Get rid of GENERIC_MSI_IRQ_DOMAIN 2022-11-17 15:15:20 +01:00
nommu_context.h asm-generic: add generic MMU versions of mmu context functions 2020-10-26 16:45:03 +01:00
numa.h arm64: irq: set the correct node for VMAP stack 2024-02-05 20:14:14 +00:00
page.h asm-generic/page.h: Make pfn accessors static inlines 2023-05-29 11:27:08 +02:00
param.h
parport.h
pci.h asm-generic: Add new pci.h and use it 2022-07-22 17:34:57 -05:00
pci_iomap.h PCI: Stub __pci_ioport_map() for arches that don't support it at all 2022-07-29 12:01:00 -05:00
percpu.h percpu: Fix self-assignment of __old in raw_cpu_generic_try_cmpxchg() 2023-06-08 10:28:39 +02:00
pgalloc.h pgalloc: convert various functions to use ptdescs 2023-08-21 13:37:55 -07:00
pgtable-nop4d.h mm: rename p4d_page_vaddr to p4d_pgtable and make it return pud_t * 2021-07-08 11:48:22 -07:00
pgtable-nopmd.h riscv/mm: fix two page table check related issues 2022-05-19 14:08:48 -07:00
pgtable-nopud.h mm: rename p4d_page_vaddr to p4d_pgtable and make it return pud_t * 2021-07-08 11:48:22 -07:00
pgtable_uffd.h userfaultfd: wp: add pmd_swp_*uffd_wp() helpers 2020-04-07 10:43:39 -07:00
preempt.h riscv: support PREEMPT_DYNAMIC with static keys 2023-08-31 00:18:34 -07:00
qrwlock.h asm-generic changes for 5.19 2022-05-26 10:50:30 -07:00
qrwlock_types.h locking/qrwlock: Change "queue rwlock" to "queued rwlock" 2022-05-11 16:27:04 +02:00
qspinlock.h asm-generic: qspinlock: fix queued_spin_value_unlocked() implementation 2023-12-20 17:01:59 +01:00
qspinlock_types.h locking/qspinlock: Do not include atomic.h from qspinlock_types.h 2020-07-29 16:14:19 +02:00
resource.h
rwonce.h asm/rwonce: Don't pull <asm/barrier.h> into 'asm-generic/rwonce.h' 2020-07-21 10:50:36 +01:00
seccomp.h seccomp: Use -1 marker for end of mode 1 syscall list 2020-07-10 16:01:52 -07:00
sections.h asm-generic: sections: refactor memory_intersects 2022-08-28 14:02:45 -07:00
serial.h
set_memory.h
shmparam.h
signal.h asm-generic: Remove empty #ifdef SA_RESTORER 2022-09-10 09:56:53 +02:00
simd.h
softirq_stack.h asm-generic: Conditionally enable do_softirq_own_stack() via Kconfig. 2022-09-05 17:20:55 +02:00
spinlock.h asm-generic: ticket-lock: New generic ticket-based spinlock 2022-05-11 11:49:38 -07:00
spinlock_types.h asm-generic: ticket-lock: New generic ticket-based spinlock 2022-05-11 11:49:38 -07:00
statfs.h
string.h
switch_to.h
syscall.h ptrace: Create ptrace_report_syscall_{entry,exit} in ptrace.h 2022-03-10 13:35:08 -06:00
syscalls.h
timex.h
tlb.h mm: add utility functions for ptdesc 2023-08-21 13:37:52 -07:00
tlbflush.h
topology.h mm: replace CONFIG_NEED_MULTIPLE_NODES with CONFIG_NUMA 2021-06-29 10:53:55 -07:00
trace_clock.h
uaccess.h uaccess: remove CONFIG_SET_FS 2022-02-25 09:36:06 +01:00
unaligned.h asm-generic: make sparse happy with odd-sized put_unaligned_*() 2024-02-05 20:14:13 +00:00
user.h
vermagic.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
vga.h
vmlinux.lds.h vmlinux.lds.h: remove unused CPU_KEEP and CPU_DISCARD macros 2023-10-01 14:55:23 +09:00
vtime.h
word-at-a-time.h word-at-a-time: use the same return type for has_zero regardless of endianness 2023-08-02 10:23:36 -07:00
xor.h lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00