linux-stable/arch/x86
David Matlack a3fe5dbda0 KVM: x86/mmu: Split huge pages mapped by the TDP MMU when dirty logging is enabled
When dirty logging is enabled without initially-all-set, try to split
all huge pages in the memslot down to 4KB pages so that vCPUs do not
have to take expensive write-protection faults to split huge pages.

Eager page splitting is best-effort only. This commit only adds the
support for the TDP MMU, and even there splitting may fail due to out
of memory conditions. Failures to split a huge page is fine from a
correctness standpoint because KVM will always follow up splitting by
write-protecting any remaining huge pages.

Eager page splitting moves the cost of splitting huge pages off of the
vCPU threads and onto the thread enabling dirty logging on the memslot.
This is useful because:

 1. Splitting on the vCPU thread interrupts vCPUs execution and is
    disruptive to customers whereas splitting on VM ioctl threads can
    run in parallel with vCPU execution.

 2. Splitting all huge pages at once is more efficient because it does
    not require performing VM-exit handling or walking the page table for
    every 4KiB page in the memslot, and greatly reduces the amount of
    contention on the mmu_lock.

For example, when running dirty_log_perf_test with 96 virtual CPUs, 1GiB
per vCPU, and 1GiB HugeTLB memory, the time it takes vCPUs to write to
all of their memory after dirty logging is enabled decreased by 95% from
2.94s to 0.14s.

Eager Page Splitting is over 100x more efficient than the current
implementation of splitting on fault under the read lock. For example,
taking the same workload as above, Eager Page Splitting reduced the CPU
required to split all huge pages from ~270 CPU-seconds ((2.94s - 0.14s)
* 96 vCPU threads) to only 1.55 CPU-seconds.

Eager page splitting does increase the amount of time it takes to enable
dirty logging since it has split all huge pages. For example, the time
it took to enable dirty logging in the 96GiB region of the
aforementioned test increased from 0.001s to 1.55s.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Matlack <dmatlack@google.com>
Message-Id: <20220119230739.2234394-16-dmatlack@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-02-10 13:50:42 -05:00
..
boot Kbuild updates for v5.17 2022-01-19 11:15:19 +02:00
configs x86/kbuild: Enable CONFIG_KALLSYMS_ALL=y in the defconfigs 2022-01-08 22:55:29 +01:00
crypto lib/crypto: blake2s: avoid indirect calls to compression function for Clang CFI 2022-02-04 19:22:32 +01:00
entry Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
events perf/x86/intel/pt: Fix crash with stop filters in single-range mode 2022-02-02 13:11:40 +01:00
hyperv hyperv-next for 5.17 2022-01-16 15:53:00 +02:00
ia32 audit/stable-5.16 PR 20211101 2021-11-01 21:17:39 -07:00
include KVM: x86/mmu: Split huge pages mapped by the TDP MMU when dirty logging is enabled 2022-02-10 13:50:42 -05:00
kernel x86/cpu: Add Xeon Icelake-D to list of CPUs that support PPIN 2022-01-25 18:40:30 +01:00
kvm KVM: x86/mmu: Split huge pages mapped by the TDP MMU when dirty logging is enabled 2022-02-10 13:50:42 -05:00
lib - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
math-emu x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
mm Merge branch 'akpm' (patches from Andrew) 2022-01-15 20:37:06 +02:00
net - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
pci PCI/sysfs: Find shadow ROM before static attribute initialization 2022-01-26 10:41:21 -06:00
platform - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
power x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
purgatory x86/purgatory: Remove -nostdlib compiler flag 2021-12-30 14:13:06 +01:00
ras
realmode - Flush *all* mappings from the TLB after switching to the trampoline 2022-01-10 09:51:38 -08:00
tools x86/build: Use the proper name CONFIG_FW_LOADER 2021-12-29 22:20:38 +01:00
um bitmap patches for 5.17-rc1 2022-01-23 06:20:44 +02:00
video
xen x86/Xen: streamline (and fix) PV CPU enumeration 2022-02-03 08:25:04 +01:00
.gitignore
Kbuild kbuild: use more subdir- for visiting subdirectories while cleaning 2021-10-24 13:49:46 +09:00
Kconfig ftrace: Have architectures opt-in for mcount build time sorting 2022-01-27 19:15:44 -05:00
Kconfig.assembler
Kconfig.cpu x86/mmx_32: Remove X86_USE_3DNOW 2021-12-11 09:09:45 +01:00
Kconfig.debug
Makefile x86: Add straight-line-speculation mitigation 2021-12-09 13:32:25 +01:00
Makefile.um
Makefile_32.cpu