No description
Find a file
Yang Shi 70cbc3cc78 mm: gup: fix the fast GUP race against THP collapse
Since general RCU GUP fast was introduced in commit 2667f50e8b ("mm:
introduce a general RCU get_user_pages_fast()"), a TLB flush is no longer
sufficient to handle concurrent GUP-fast in all cases, it only handles
traditional IPI-based GUP-fast correctly.  On architectures that send an
IPI broadcast on TLB flush, it works as expected.  But on the
architectures that do not use IPI to broadcast TLB flush, it may have the
below race:

   CPU A                                          CPU B
THP collapse                                     fast GUP
                                              gup_pmd_range() <-- see valid pmd
                                                  gup_pte_range() <-- work on pte
pmdp_collapse_flush() <-- clear pmd and flush
__collapse_huge_page_isolate()
    check page pinned <-- before GUP bump refcount
                                                      pin the page
                                                      check PTE <-- no change
__collapse_huge_page_copy()
    copy data to huge page
    ptep_clear()
install huge pmd for the huge page
                                                      return the stale page
discard the stale page

The race can be fixed by checking whether PMD is changed or not after
taking the page pin in fast GUP, just like what it does for PTE.  If the
PMD is changed it means there may be parallel THP collapse, so GUP should
back off.

Also update the stale comment about serializing against fast GUP in
khugepaged.

Link: https://lkml.kernel.org/r/20220907180144.555485-1-shy828301@gmail.com
Fixes: 2667f50e8b ("mm: introduce a general RCU get_user_pages_fast()")
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Yang Shi <shy828301@gmail.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-09-26 12:14:33 -07:00
arch x86/mm: disable instrumentations of mm/pgprot.c 2022-09-11 16:22:30 -07:00
block block-6.0-2022-08-26 2022-08-26 11:05:54 -07:00
certs Kbuild updates for v5.20 2022-08-10 10:40:41 -07:00
crypto crypto: blake2b: effectively disable frame size warning 2022-08-10 17:59:11 -07:00
Documentation Misc fixes: 2022-08-28 10:10:23 -07:00
drivers Seventeen hotfixes. Mostly memory management things. Ten patches are 2022-08-28 14:49:59 -07:00
fs xfs: fix SB_BORN check in xfs_dax_notify_failure() 2022-09-11 16:22:29 -07:00
include mm/memory-failure: fix detection of memory_failure() handlers 2022-09-11 16:22:29 -07:00
init arm64 fixes for -rc3 2022-08-26 11:32:53 -07:00
io_uring io_uring/net: save address for sendzc async execution 2022-08-25 07:52:30 -06:00
ipc Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
kernel Seventeen hotfixes. Mostly memory management things. Ten patches are 2022-08-28 14:49:59 -07:00
lib bitmap fixes for v6.0-rc3 2022-08-28 14:36:27 -07:00
LICENSES LICENSES/LGPL-2.1: Add LGPL-2.1-or-later as valid identifiers 2021-12-16 14:33:10 +01:00
mm mm: gup: fix the fast GUP race against THP collapse 2022-09-26 12:14:33 -07:00
net Including fixes from ipsec and netfilter (with one broken Fixes tag). 2022-08-25 14:03:58 -07:00
samples Tracing updates for 5.20 / 6.0 2022-08-05 09:41:12 -07:00
scripts asm goto: eradicate CC_HAS_ASM_GOTO 2022-08-21 10:06:28 -07:00
security hardening fixes for v6.0-rc2 2022-08-19 13:56:14 -07:00
sound sound fixes for 6.0-rc2 2022-08-19 09:46:11 -07:00
tools tools: fix compilation after gfp_types.h split 2022-09-11 16:22:31 -07:00
usr Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
virt KVM: Drop unnecessary initialization of "ops" in kvm_ioctl_create_device() 2022-08-19 04:05:43 -04:00
.clang-format PCI/DOE: Add DOE mailbox support functions 2022-07-19 15:38:04 -07:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore kbuild: split the second line of *.mod into *.usyms 2022-05-08 03:16:59 +09:00
.mailmap .mailmap: update Luca Ceresoli's e-mail address 2022-08-28 14:02:46 -07:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS drm for 5.20/6.0 2022-08-03 19:52:08 -07:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS bitmap fixes for v6.0-rc3 2022-08-28 14:36:27 -07:00
Makefile Linux 6.0-rc3 2022-08-28 15:05:29 -07:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.