No description
Find a file
Peter Xu 57edfcfd34 mm/gup: accelerate thp gup even for "pages != NULL"
The acceleration of THP was done with ctx.page_mask, however it'll be
ignored if **pages is non-NULL.

The old optimization was introduced in 2013 in 240aadeedc ("mm:
accelerate mm_populate() treatment of THP pages").  It didn't explain why
we can't optimize the **pages non-NULL case.  It's possible that at that
time the major goal was for mm_populate() which should be enough back
then.

Optimize thp for all cases, by properly looping over each subpage, doing
cache flushes, and boost refcounts / pincounts where needed in one go.

This can be verified using gup_test below:

  # chrt -f 1 ./gup_test -m 512 -t -L -n 1024 -r 10

Before:    13992.50 ( +-8.75%)
After:       378.50 (+-69.62%)

Link: https://lkml.kernel.org/r/20230628215310.73782-6-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Lorenzo Stoakes <lstoakes@gmail.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: James Houghton <jthoughton@google.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kirill A . Shutemov <kirill@shutemov.name>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Mike Rapoport (IBM) <rppt@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Yang Shi <shy828301@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-08-18 10:12:03 -07:00
arch arm64: mte: simplify swap tag restoration logic 2023-08-18 10:12:02 -07:00
block block-6.5-2023-07-21 2023-07-22 11:05:15 -07:00
certs
crypto crypto: algif_hash - Fix race between MORE and non-MORE sends 2023-07-08 22:48:42 +10:00
Documentation TTY/Serial fixes for 6.5-rc4 2023-07-30 11:51:36 -07:00
drivers mm: remove arguments of show_mem() 2023-08-18 10:12:02 -07:00
fs mm: make MEMFD_CREATE into a selectable config option 2023-08-18 10:12:01 -07:00
include mm/hugetlb: add page_mask for hugetlb_follow_page_mask() 2023-08-18 10:12:03 -07:00
init mm: remove arguments of show_mem() 2023-08-18 10:12:02 -07:00
io_uring io_uring-6.5-2023-07-28 2023-07-28 10:19:44 -07:00
ipc
kernel mm: remove arguments of show_mem() 2023-08-18 10:12:02 -07:00
lib maple_tree: fix a few documentation issues 2023-08-18 10:12:00 -07:00
LICENSES
mm mm/gup: accelerate thp gup even for "pages != NULL" 2023-08-18 10:12:03 -07:00
net A patch to reduce the potential for erroneous RBD exclusive lock 2023-07-28 10:47:24 -07:00
rust rust: error: impl Debug for Error with errname() integration 2023-06-13 01:24:42 +02:00
samples arm64: ftrace: Add direct call trampoline samples support 2023-07-10 17:51:54 -04:00
scripts x86: 2023-07-30 11:19:08 -07:00
security security: keys: perform capable check only on privileged operations 2023-07-28 18:07:41 +00:00
sound ASoC: Fixes for v6.5 2023-07-27 14:54:23 +02:00
tools Probe fixes for 6.5-rc3: 2023-07-30 11:27:22 -07:00
usr initramfs: Encode dependency on KBUILD_BUILD_TIMESTAMP 2023-06-06 17:54:49 +09:00
virt KVM: Grab a reference to KVM for VM and vCPU stats file descriptors 2023-07-29 11:05:28 -04:00
.clang-format iommu: Add for_each_group_device() 2023-05-23 08:15:51 +02:00
.cocciconfig
.get_maintainer.ignore
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore Revert ".gitignore: ignore *.cover and *.mbx" 2023-07-04 15:05:12 -07:00
.mailmap mailmap: update remaining active codeaurora.org email addresses 2023-07-27 13:07:05 -07:00
.rustfmt.toml
COPYING
CREDITS - Address -Wmissing-prototype warnings 2023-06-26 16:43:54 -07:00
Kbuild
Kconfig
MAINTAINERS USB fixes for 6.5-rc4 2023-07-30 11:57:51 -07:00
Makefile Linux 6.5-rc4 2023-07-30 13:23:47 -07:00
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.