mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-08-23 17:30:02 +00:00
9e52fc2b50
There's a subtle bug in how some of the paravirt guest code handles page table freeing on x86: On x86 software page table walkers depend on the fact that remote TLB flush does an IPI: walk is performed lockless but with interrupts disabled and in case the page table is freed the freeing CPU will get blocked as remote TLB flush is required. On other architectures which don't require an IPI to do remote TLB flush we have an RCU-based mechanism (see include/asm-generic/tlb.h for more details). In virtualized environments we may want to override the ->flush_tlb_others callback in pv_mmu_ops and use a hypercall asking the hypervisor to do a remote TLB flush for us. This breaks the assumption about IPIs. Xen PV has been doing this for years and the upcoming remote TLB flush for Hyper-V will do it too. This is not safe, as software page table walkers may step on an already freed page. Fix the bug by enabling the RCU-based page table freeing mechanism, CONFIG_HAVE_RCU_TABLE_FREE=y. Testing with kernbench and mmap/munmap microbenchmarks, and neither showed any noticeable performance impact. Suggested-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com> Acked-by: Peter Zijlstra <peterz@infradead.org> Acked-by: Juergen Gross <jgross@suse.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Jork Loeser <Jork.Loeser@microsoft.com> Cc: KY Srinivasan <kys@microsoft.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Stephen Hemminger <sthemmin@microsoft.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: xen-devel@lists.xenproject.org Link: http://lkml.kernel.org/r/20170828082251.5562-1-vkuznets@redhat.com [ Rewrote/fixed/clarified the changelog. ] Signed-off-by: Ingo Molnar <mingo@kernel.org>
32 lines
1 KiB
C
32 lines
1 KiB
C
#ifndef _ASM_X86_TLB_H
|
|
#define _ASM_X86_TLB_H
|
|
|
|
#define tlb_start_vma(tlb, vma) do { } while (0)
|
|
#define tlb_end_vma(tlb, vma) do { } while (0)
|
|
#define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
|
|
|
|
#define tlb_flush(tlb) \
|
|
{ \
|
|
if (!tlb->fullmm && !tlb->need_flush_all) \
|
|
flush_tlb_mm_range(tlb->mm, tlb->start, tlb->end, 0UL); \
|
|
else \
|
|
flush_tlb_mm_range(tlb->mm, 0UL, TLB_FLUSH_ALL, 0UL); \
|
|
}
|
|
|
|
#include <asm-generic/tlb.h>
|
|
|
|
/*
|
|
* While x86 architecture in general requires an IPI to perform TLB
|
|
* shootdown, enablement code for several hypervisors overrides
|
|
* .flush_tlb_others hook in pv_mmu_ops and implements it by issuing
|
|
* a hypercall. To keep software pagetable walkers safe in this case we
|
|
* switch to RCU based table free (HAVE_RCU_TABLE_FREE). See the comment
|
|
* below 'ifdef CONFIG_HAVE_RCU_TABLE_FREE' in include/asm-generic/tlb.h
|
|
* for more details.
|
|
*/
|
|
static inline void __tlb_remove_table(void *table)
|
|
{
|
|
free_page_and_swap_cache(table);
|
|
}
|
|
|
|
#endif /* _ASM_X86_TLB_H */
|