linux-stable/kernel/bpf
Yan Zhai 5ff8f56c39 bpf: report RCU QS in cpumap kthread
[ Upstream commit 00bf631224 ]

When there are heavy load, cpumap kernel threads can be busy polling
packets from redirect queues and block out RCU tasks from reaching
quiescent states. It is insufficient to just call cond_resched() in such
context. Periodically raise a consolidated RCU QS before cond_resched
fixes the problem.

Fixes: 6710e11269 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")
Reviewed-by: Jesper Dangaard Brouer <hawk@kernel.org>
Signed-off-by: Yan Zhai <yan@cloudflare.com>
Acked-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Jesper Dangaard Brouer <hawk@kernel.org>
Link: https://lore.kernel.org/r/c17b9f1517e19d813da3ede5ed33ee18496bb5d8.1710877680.git.yan@cloudflare.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2024-03-26 18:21:02 -04:00
..
preload bpf: iterators: Build and use lightweight bootstrap version of bpftool 2022-07-15 12:01:30 -07:00
Kconfig rcu: Make the TASKS_RCU Kconfig option be selected 2022-04-20 16:52:58 -07:00
Makefile bpf: Introduce any context BPF specific memory allocator. 2022-09-05 15:33:05 +02:00
arraymap.c bpf: Add map and need_defer parameters to .map_fd_put_ptr() 2024-01-25 15:27:26 -08:00
bloom_filter.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
bpf_inode_storage.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
bpf_iter.c bpf: Initialize the bpf_run_ctx in bpf_iter_run_prog() 2022-08-18 17:06:13 -07:00
bpf_local_storage.c bpf: Annotate data races in bpf_local_storage 2023-05-24 17:32:36 +01:00
bpf_lru_list.c bpf: Address KCSAN report on bpf_lru_list 2023-07-27 08:50:34 +02:00
bpf_lru_list.h bpf: Address KCSAN report on bpf_lru_list 2023-07-27 08:50:34 +02:00
bpf_lsm.c bpf: Fix the kernel crash caused by bpf_setsockopt(). 2023-02-09 11:28:02 +01:00
bpf_struct_ops.c bpf: Remove is_valid_bpf_tramp_flags() 2022-07-11 21:04:58 +02:00
bpf_struct_ops_types.h
bpf_task_storage.c bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy 2022-09-01 12:16:12 -07:00
btf.c bpf: Fix an error in verifying a field in a union 2023-09-13 09:42:30 +02:00
cgroup.c bpf: Don't EFAULT for {g,s}setsockopt with wrong optlen 2023-07-19 16:21:05 +02:00
cgroup_iter.c cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers 2023-06-21 16:00:51 +02:00
core.c bpf: hardcode BPF_PROG_PACK_SIZE to 2MB * num_possible_nodes() 2024-03-26 18:20:43 -04:00
cpumap.c bpf: report RCU QS in cpumap kthread 2024-03-26 18:21:02 -04:00
devmap.c bpf: Fix DEVMAP_HASH overflow check on 32-bit arches 2024-03-26 18:20:41 -04:00
disasm.c
disasm.h
dispatcher.c bpf: Add explicit cast to 'void *' for __BPF_DISPATCHER_UPDATE() 2022-11-07 16:07:35 -08:00
hashtab.c bpf: Fix hashtab overflow check on 32-bit arches 2024-03-26 18:20:41 -04:00
helpers.c bpf: Mark bpf_spin_{lock,unlock}() helpers with notrace correctly 2024-03-26 18:20:35 -04:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
link_iter.c bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
local_storage.c cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers 2023-06-21 16:00:51 +02:00
lpm_trie.c bpf, lpm: Fix check prefixlen before walking trie 2024-01-25 15:27:24 -08:00
map_in_map.c bpf: Defer the free of inner map when necessary 2024-01-25 15:27:26 -08:00
map_in_map.h bpf: Add map and need_defer parameters to .map_fd_put_ptr() 2024-01-25 15:27:26 -08:00
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
memalloc.c bpf: Zeroing allocated object from slab in bpf memory allocator 2023-03-10 09:33:06 +01:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf: restore the ebpf program ID for BPF_AUDIT_UNLOAD and PERF_BPF_EVENT_PROG_UNLOAD 2023-01-24 07:24:37 +01:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: Avoid deadlock when using queue and stack maps from NMI 2023-10-06 14:56:35 +02:00
reuseport_array.c net: Fix suspicious RCU usage in bpf_sk_reuseport_detach() 2022-08-17 16:42:59 -07:00
ringbuf.c bpf: Add bpf_user_ringbuf_drain() helper 2022-09-21 16:24:58 -07:00
stackmap.c bpf: Fix stackmap overflow check on 32-bit arches 2024-03-26 18:20:41 -04:00
syscall.c bpf: Set uattr->batch.count as zero before batched update or deletion 2024-02-05 20:12:51 +00:00
sysfs_btf.c
task_iter.c bpf: keep a reference to the mm, in case the task is dead. 2023-01-24 07:24:31 +01:00
tnum.c
trampoline.c bpf, x64: Fix tailcall infinite loop 2024-01-10 17:10:26 +01:00
verifier.c bpf: Add struct for bin_args arg in bpf_bprintf_prepare 2024-02-23 09:12:51 +01:00