linux-stable/arch
Dave Hansen d39268ad24 x86/mm/tlb: Revert retpoline avoidance approach
0day reported a regression on a microbenchmark which is intended to
stress the TLB flushing path:

	https://lore.kernel.org/all/20220317090415.GE735@xsang-OptiPlex-9020/

It pointed at a commit from Nadav which intended to remove retpoline
overhead in the TLB flushing path by taking the 'cond'-ition in
on_each_cpu_cond_mask(), pre-calculating it, and incorporating it into
'cpumask'.  That allowed the code to use a bunch of earlier direct
calls instead of later indirect calls that need a retpoline.

But, in practice, threads can go idle (and into lazy TLB mode where
they don't need to flush their TLB) between the early and late calls.
It works in this direction and not in the other because TLB-flushing
threads tend to hold mmap_lock for write.  Contention on that lock
causes threads to _go_ idle right in this early/late window.

There was not any performance data in the original commit specific
to the retpoline overhead.  I did a few tests on a system with
retpolines:

	https://lore.kernel.org/all/dd8be93c-ded6-b962-50d4-96b1c3afb2b7@intel.com/

which showed a possible small win.  But, that small win pales in
comparison with the bigger loss induced on non-retpoline systems.

Revert the patch that removed the retpolines.  This was not a
clean revert, but it was self-contained enough not to be too painful.

Fixes: 6035152d8e ("x86/mm/tlb: Open-code on_each_cpu_cond_mask() for tlb_is_not_lazy()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Nadav Amit <namit@vmware.com>
Cc: <stable@vger.kernel.org>
Link: https://lkml.kernel.org/r/164874672286.389.7021457716635788197.tip-bot2@tip-bot2
2022-04-04 19:41:36 +02:00
..
alpha Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
arc ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
arm ARM fixes for 5.18-rc1: 2022-04-03 10:17:48 -07:00
arm64 SoC: fixes for 5.18, part 1 2022-04-01 13:21:19 -07:00
csky ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
h8300 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
hexagon ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
ia64 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
m68k Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
microblaze Kbuild updates for v5.18 2022-03-31 11:59:03 -07:00
mips - build fix for gpio 2022-04-02 12:14:38 -07:00
nios2 ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
openrisc ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
parisc Kbuild updates for v5.18 2022-03-31 11:59:03 -07:00
powerpc Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
riscv RISC-V Patches for the 5.18 Merge Window, Part 2 2022-04-01 13:31:57 -07:00
s390 * Only do MSR filtering for MSRs accessed by rdmsr/wrmsr 2022-04-02 12:09:02 -07:00
sh Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2022-04-01 19:57:03 -07:00
sparc Kbuild updates for v5.18 2022-03-31 11:59:03 -07:00
um Kbuild fixes for v5.18 2022-04-02 12:33:31 -07:00
x86 x86/mm/tlb: Revert retpoline avoidance approach 2022-04-04 19:41:36 +02:00
xtensa Kbuild updates for v5.18 2022-03-31 11:59:03 -07:00
.gitignore
Kconfig Networking fixes for 5.18-rc1 and rethook patches. 2022-03-31 11:23:31 -07:00