mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-08-25 02:09:42 +00:00
x86/membarrier: Get rid of a dubious optimization
commita493d1ca1a
upstream. sync_core_before_usermode() had an incorrect optimization. If the kernel returns from an interrupt, it can get to usermode without IRET. It just has to schedule to a different task in the same mm and do SYSRET. Fortunately, there were no callers of sync_core_before_usermode() that could have had in_irq() or in_nmi() equal to true, because it's only ever called from the scheduler. While at it, clarify a related comment. Fixes:70216e18e5
("membarrier: Provide core serializing command, *_SYNC_CORE") Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/5afc7632be1422f91eaf7611aaaa1b5b8580a086.1607058304.git.luto@kernel.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This commit is contained in:
parent
6346ed69bc
commit
e3c1d51868
2 changed files with 13 additions and 6 deletions
|
@ -16,12 +16,13 @@ static inline void sync_core_before_usermode(void)
|
|||
/* With PTI, we unconditionally serialize before running user code. */
|
||||
if (static_cpu_has(X86_FEATURE_PTI))
|
||||
return;
|
||||
|
||||
/*
|
||||
* Return from interrupt and NMI is done through iret, which is core
|
||||
* serializing.
|
||||
* Even if we're in an interrupt, we might reschedule before returning,
|
||||
* in which case we could switch to a different thread in the same mm
|
||||
* and return using SYSRET or SYSEXIT. Instead of trying to keep
|
||||
* track of our need to sync the core, just sync right away.
|
||||
*/
|
||||
if (in_irq() || in_nmi())
|
||||
return;
|
||||
sync_core();
|
||||
}
|
||||
|
||||
|
|
|
@ -327,8 +327,14 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
|
|||
/*
|
||||
* The membarrier system call requires a full memory barrier and
|
||||
* core serialization before returning to user-space, after
|
||||
* storing to rq->curr. Writing to CR3 provides that full
|
||||
* memory barrier and core serializing instruction.
|
||||
* storing to rq->curr, when changing mm. This is because
|
||||
* membarrier() sends IPIs to all CPUs that are in the target mm
|
||||
* to make them issue memory barriers. However, if another CPU
|
||||
* switches to/from the target mm concurrently with
|
||||
* membarrier(), it can cause that CPU not to receive an IPI
|
||||
* when it really should issue a memory barrier. Writing to CR3
|
||||
* provides that full memory barrier and core serializing
|
||||
* instruction.
|
||||
*/
|
||||
if (real_prev == next) {
|
||||
VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[prev_asid].ctx_id) !=
|
||||
|
|
Loading…
Reference in a new issue