Joerg Roedel 763802b53a x86/mm: split vmalloc_sync_all() ... Commit 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in the vunmap() code-path. While this change was necessary to maintain correctness on x86-32-pae kernels, it also adds additional cycles for architectures that don't need it. Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported severe performance regressions in micro-benchmarks because it now also calls the x86-64 implementation of vmalloc_sync_all() on vunmap(). But the vmalloc_sync_all() implementation on x86-64 is only needed for newly created mappings. To avoid the unnecessary work on x86-64 and to gain the performance back, split up vmalloc_sync_all() into two functions: * vmalloc_sync_mappings(), and * vmalloc_sync_unmappings() Most call-sites to vmalloc_sync_all() only care about new mappings being synchronized. The only exception is the new call-site added in the above mentioned commit. Shile Zhang directed us to a report of an 80% regression in reaim throughput. Fixes: 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()") Reported-by: kernel test robot <oliver.sang@intel.com> Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Tested-by: Borislav Petkov <bp@suse.de> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> [GHES] Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: <stable@vger.kernel.org> Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/ Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2020-03-21 18:56:06 -07:00
..
boot	x86/boot/compressed: Don't declare __force_order in kaslr_64.c	2020-02-19 17:23:59 +01:00
configs
crypto	crypto: x86/curve25519 - support assemblers with no adx support	2020-03-05 18:28:09 +11:00
entry	kbuild: rename hostprogs-y/always to hostprogs/always-y	2020-02-04 01:53:07 +09:00
events	perf/amd/uncore: Replace manual sampling check with CAP_NO_INTERRUPT flag	2020-03-12 14:08:50 +01:00
hyperv	x86/hyperv: Suspend/resume the hypercall page for hibernation	2020-02-01 09:41:16 +01:00
ia32	x86: Remove force_iret()	2020-01-08 19:40:51 +01:00
include	Bugfixes, x86+s390.	2020-03-14 15:45:26 -07:00
kernel	Two fixes for x86:	2020-03-15 12:52:56 -07:00
kvm	Merge branch 'kvm-null-pointer-fix' into kvm-master	2020-03-14 12:49:37 +01:00
lib	Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2020-01-31 11:05:33 -08:00
math-emu
mm	x86/mm: split vmalloc_sync_all()	2020-03-21 18:56:06 -07:00
net	bpf: Introduce BPF_MAP_TYPE_STRUCT_OPS	2020-01-09 08:46:18 -08:00
oprofile
pci	pci-v5.6-changes	2020-01-31 14:48:54 -08:00
platform	efi/x86: Handle by-ref arguments covering multiple pages in mixed mode	2020-02-26 15:31:42 +01:00
power
purgatory
ras
realmode	kbuild: rename hostprogs-y/always to hostprogs/always-y	2020-02-04 01:53:07 +09:00
tools	kbuild: rename hostprogs-y/always to hostprogs/always-y	2020-02-04 01:53:07 +09:00
um	um: Implement copy_thread_tls	2020-01-07 13:31:29 +01:00
video
xen	x86/ioperm: Add new paravirt function update_io_bitmap()	2020-02-29 12:43:09 +01:00
.gitignore
Kbuild
Kconfig	asm-generic/tlb: rename HAVE_RCU_TABLE_FREE	2020-02-04 03:05:26 +00:00
Kconfig.cpu	x86/cpu: Detect VMX features on Intel, Centaur and Zhaoxin CPUs	2020-01-13 18:02:53 +01:00
Kconfig.debug	x86: mm: convert dump_pagetables to use walk_page_range	2020-02-04 03:05:25 +00:00
Makefile	crypto: x86/curve25519 - support assemblers with no adx support	2020-03-05 18:28:09 +11:00
Makefile.um
Makefile_32.cpu