linux-stable/drivers/infiniband/core
Joao Martins 1d4b0166e3 RDMA/umem: batch page unpin in __ib_umem_release()
Use the newly added unpin_user_page_range_dirty_lock() for more quickly
unpinning a consecutive range of pages represented as compound pages.
This will also calculate number of pages to unpin (for the tail pages
which matching head page) and thus batch the refcount update.

Running a test program which calls memory range reg/unreg on a region 1G
in size and measures cost of both operations together (in a guest using
rxe) with THP and hugetlbfs:

Before:
  590 rounds in 5.003 sec: 8480.335 usec / round
  6898 rounds in 60.001 sec: 8698.367 usec / round

After:
  2688 rounds in 5.002 sec: 1860.786 usec / round
  32517 rounds in 60.001 sec: 1845.225 usec / round

Link: https://lkml.kernel.org/r/20210212130843.13865-5-joao.m.martins@oracle.com
Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
Acked-by: Jason Gunthorpe <jgg@nvidia.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Doug Ledford <dledford@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-04-30 11:20:37 -07:00
..
addr.c RDMA/addr: Be strict with gid size 2021-04-08 16:14:56 -03:00
agent.c
agent.h
cache.c RDMA/core/cache: Fix some misspellings, missing and superfluous param descriptions 2021-01-19 20:33:40 -04:00
cgroup.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 288 2019-06-05 17:36:37 +02:00
cm.c RDMA/cm: Fix IRQ restore in ib_send_cm_sidr_rep 2021-03-01 14:43:16 -04:00
cm_msgs.h RDMA/cm: Remove CM message structs 2020-01-25 15:11:37 -04:00
cm_trace.c RDMA/cm: Replace pr_debug() call sites with tracepoints 2020-08-24 19:41:41 -03:00
cm_trace.h RDMA/cm: Add tracepoints to track MAD send operations 2020-08-24 19:41:41 -03:00
cma.c RDMA/ucma: Fix use-after-free bug in ucma_create_uevent 2021-02-16 14:42:59 -04:00
cma_configfs.c Merge branch 'devx_set_get' into rdma.git for-next 2021-01-19 10:55:19 -04:00
cma_priv.h RDMA/ucma: Extend ucma_connect to receive ECE parameters 2020-05-27 16:05:05 -03:00
cma_trace.c RDMA/cma: Add trace points in RDMA Connection Manager 2020-01-07 16:10:53 -04:00
cma_trace.h RDMA/core: Move the rdma_show_ib_cm_event() macro 2020-08-24 16:01:47 -03:00
core_priv.h RDMA/core: Clean up cq pool mechanism 2020-12-10 15:05:17 -04:00
counters.c RDMA/core/counters: Demote non-conformant kernel-doc headers 2021-01-19 20:33:44 -04:00
cq.c RDMA/core: Clean up cq pool mechanism 2020-12-10 15:05:17 -04:00
device.c RDMA/core: Fix kernel doc warnings for ib_port_immutable_read() 2021-02-16 14:42:58 -04:00
ib_core_uverbs.c RDMA/core: Ensure that rdma_user_mmap_entry_remove() is a fence 2020-01-25 14:48:33 -04:00
iwcm.c RDMA/iwcm: Fix iwcm work deallocation 2020-03-04 14:28:25 -04:00
iwcm.h
iwpm_msg.c RDMA/core/iwpm_msg: Add proper descriptions for 'skb' param 2021-01-19 20:33:44 -04:00
iwpm_util.c RDMA/core/iwpm_util: Fix some param description misspellings 2021-01-19 20:33:44 -04:00
iwpm_util.h IB: Fix kernel-doc markups 2020-12-07 15:45:00 -04:00
lag.c RDMA/core: Consider flow label when building skb 2020-05-06 16:51:43 -03:00
mad.c RDMA 5.9 merge window pull request 2020-08-06 16:43:36 -07:00
mad_priv.h IB/mad: Change atomics to refcount API 2020-06-24 16:43:45 -03:00
mad_rmpp.c IB/mad: Delete RMPP_STATE_CANCELING state 2020-06-24 16:43:45 -03:00
mad_rmpp.h
Makefile RDMA/umem: Support importing dma-buf as user memory region 2021-01-20 16:07:52 -04:00
mr_pool.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
multicast.c RDMA/core/multicast: Provide description for 'ib_init_ah_from_mcmember()'s 'rec' param 2021-01-19 20:33:42 -04:00
netlink.c IB/core: Avoid deadlock during netlink message handling 2019-10-24 20:49:37 -03:00
nldev.c RDMA/nldev: Return an error message on failure to turn auto mode 2021-01-18 16:02:14 -04:00
opa_smi.h
packer.c
rdma_core.c RDMA/uverbs: Allow drivers to create a new HW object during rereg_mr 2020-12-07 14:06:23 -04:00
rdma_core.h IB/uverbs: Introduce create/destroy QP commands over ioctl 2020-05-21 20:39:36 -03:00
restrack.c RDMA/core/restrack: Fix kernel-doc formatting issue 2021-01-19 20:33:43 -04:00
restrack.h RDMA/restrack: Improve readability in task name management 2020-09-22 19:47:35 -03:00
roce_gid_mgmt.c RDMA/core/roce_gid_mgmt: Fix misnaming of 'rdma_roce_rescan_device()'s param 'ib_dev' 2021-01-19 20:33:41 -04:00
rw.c RDMA: Use kzalloc for allocating only one thing 2021-01-07 16:47:53 -04:00
sa.h
sa_query.c RDMA/ipoib: Remove racy Subnet Manager sendonly join checks 2021-02-16 14:42:58 -04:00
security.c RDMA/core: Ensure security pkey modify is not lost 2020-03-24 19:53:25 -03:00
smi.c
smi.h
sysfs.c IB/core: Add support for NDR link speed 2020-11-02 15:48:56 -04:00
trace.c RDMA/core: Clean up tracepoint headers 2020-07-06 14:54:46 -03:00
ucma.c RDMA/ucma: Do not miss ctx destruction steps in some cases 2021-01-06 17:15:15 -04:00
ud_header.c RDMA/core: Use sizeof_field() helper 2020-05-27 13:46:05 -03:00
umem.c RDMA/umem: batch page unpin in __ib_umem_release() 2021-04-30 11:20:37 -07:00
umem_dmabuf.c RDMA/umem: Support importing dma-buf as user memory region 2021-01-20 16:07:52 -04:00
umem_odp.c IB/core: Enable ODP sync without faulting 2020-10-01 16:44:05 -03:00
user_mad.c IB/umad: Return EPOLLERR in case of when device disassociated 2021-01-28 15:27:09 -04:00
uverbs.h IB/uverbs: Extend CQ to get its own asynchronous event FD 2020-05-21 20:34:53 -03:00
uverbs_cmd.c RDMA/uverbs: Don't set rcq for a QP if qp_type is IB_QPT_XRC_INI 2021-01-21 10:26:15 -04:00
uverbs_ioctl.c RDMA/uverbs: Fix kernel-doc warning of _uverbs_alloc 2021-03-03 13:22:10 -04:00
uverbs_main.c RDMA: Convert sysfs device * show functions to use sysfs_emit() 2020-10-26 19:53:21 -03:00
uverbs_marshall.c
uverbs_std_types.c RDMA/core: Make FD destroy callback void 2020-11-12 12:32:17 -04:00
uverbs_std_types_async_fd.c RDMA/core: Make FD destroy callback void 2020-11-12 12:32:17 -04:00
uverbs_std_types_counters.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_std_types_cq.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_std_types_device.c RDMA/uverbs: Fix incorrect variable type 2020-12-10 15:05:17 -04:00
uverbs_std_types_dm.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_std_types_flow_action.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_std_types_mr.c RDMA/uverbs: Add uverbs command for dma-buf based MR registration 2021-01-20 16:07:52 -04:00
uverbs_std_types_qp.c RDMA/restrack: Support all QP types 2020-11-27 11:38:46 -04:00
uverbs_std_types_srq.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_std_types_wq.c RDMA/core: Postpone uobject cleanup on failure till FD close 2020-11-12 12:32:17 -04:00
uverbs_uapi.c RDMA/core Remove uverbs_ex_cmd_mask 2020-10-26 19:27:59 -03:00
verbs.c IB/core: Use valid port number to check link layer 2021-02-02 19:25:51 -04:00