linux-stable/arch/x86
Sean Christopherson d62007edf0 KVM: x86/mmu: Zap _all_ roots when unmapping gfn range in TDP MMU
Zap both valid and invalid roots when zapping/unmapping a gfn range, as
KVM must ensure it holds no references to the freed page after returning
from the unmap operation.  Most notably, the TDP MMU doesn't zap invalid
roots in mmu_notifier callbacks.  This leads to use-after-free and other
issues if the mmu_notifier runs to completion while an invalid root
zapper yields as KVM fails to honor the requirement that there must be
_no_ references to the page after the mmu_notifier returns.

The bug is most easily reproduced by hacking KVM to cause a collision
between set_nx_huge_pages() and kvm_mmu_notifier_release(), but the bug
exists between kvm_mmu_notifier_invalidate_range_start() and memslot
updates as well.  Invalidating a root ensures pages aren't accessible by
the guest, and KVM won't read or write page data itself, but KVM will
trigger e.g. kvm_set_pfn_dirty() when zapping SPTEs, and thus completing
a zap of an invalid root _after_ the mmu_notifier returns is fatal.

  WARNING: CPU: 24 PID: 1496 at arch/x86/kvm/../../../virt/kvm/kvm_main.c:173 [kvm]
  RIP: 0010:kvm_is_zone_device_pfn+0x96/0xa0 [kvm]
  Call Trace:
   <TASK>
   kvm_set_pfn_dirty+0xa8/0xe0 [kvm]
   __handle_changed_spte+0x2ab/0x5e0 [kvm]
   __handle_changed_spte+0x2ab/0x5e0 [kvm]
   __handle_changed_spte+0x2ab/0x5e0 [kvm]
   zap_gfn_range+0x1f3/0x310 [kvm]
   kvm_tdp_mmu_zap_invalidated_roots+0x50/0x90 [kvm]
   kvm_mmu_zap_all_fast+0x177/0x1a0 [kvm]
   set_nx_huge_pages+0xb4/0x190 [kvm]
   param_attr_store+0x70/0x100
   module_attr_store+0x19/0x30
   kernfs_fop_write_iter+0x119/0x1b0
   new_sync_write+0x11c/0x1b0
   vfs_write+0x1cc/0x270
   ksys_write+0x5f/0xe0
   do_syscall_64+0x38/0xc0
   entry_SYSCALL_64_after_hwframe+0x44/0xae
   </TASK>

Fixes: b7cccd397f ("KVM: x86/mmu: Fast invalidation for TDP MMU")
Cc: stable@vger.kernel.org
Cc: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20211215011557.399940-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-02-10 13:47:07 -05:00
..
boot Kbuild updates for v5.17 2022-01-19 11:15:19 +02:00
configs
crypto lib/crypto: blake2s: avoid indirect calls to compression function for Clang CFI 2022-02-04 19:22:32 +01:00
entry Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
events perf/x86/intel/pt: Fix crash with stop filters in single-range mode 2022-02-02 13:11:40 +01:00
hyperv hyperv-next for 5.17 2022-01-16 15:53:00 +02:00
ia32
include KVM: x86: SVM: move avic definitions from AMD's spec to svm.h 2022-02-08 13:30:50 -05:00
kernel x86/cpu: Add Xeon Icelake-D to list of CPUs that support PPIN 2022-01-25 18:40:30 +01:00
kvm KVM: x86/mmu: Zap _all_ roots when unmapping gfn range in TDP MMU 2022-02-10 13:47:07 -05:00
lib - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
math-emu
mm Merge branch 'akpm' (patches from Andrew) 2022-01-15 20:37:06 +02:00
net - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
pci PCI/sysfs: Find shadow ROM before static attribute initialization 2022-01-26 10:41:21 -06:00
platform - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
power
purgatory
ras
realmode - Flush *all* mappings from the TLB after switching to the trampoline 2022-01-10 09:51:38 -08:00
tools
um bitmap patches for 5.17-rc1 2022-01-23 06:20:44 +02:00
video
xen x86/Xen: streamline (and fix) PV CPU enumeration 2022-02-03 08:25:04 +01:00
.gitignore
Kbuild
Kconfig ftrace: Have architectures opt-in for mcount build time sorting 2022-01-27 19:15:44 -05:00
Kconfig.assembler
Kconfig.cpu
Kconfig.debug
Makefile
Makefile.um
Makefile_32.cpu