Yan Zhai 5fcee137db bpf: report RCU QS in cpumap kthread ... [ Upstream commit 00bf631224 ] When there are heavy load, cpumap kernel threads can be busy polling packets from redirect queues and block out RCU tasks from reaching quiescent states. It is insufficient to just call cond_resched() in such context. Periodically raise a consolidated RCU QS before cond_resched fixes the problem. Fixes: 6710e11269 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP") Reviewed-by: Jesper Dangaard Brouer <hawk@kernel.org> Signed-off-by: Yan Zhai <yan@cloudflare.com> Acked-by: Paul E. McKenney <paulmck@kernel.org> Acked-by: Jesper Dangaard Brouer <hawk@kernel.org> Link: https://lore.kernel.org/r/c17b9f1517e19d813da3ede5ed33ee18496bb5d8.1710877680.git.yan@cloudflare.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>		2024-03-26 18:20:12 -04:00
..
preload	bpf: make preloaded map iterators to display map elements count	2023-07-06 12:42:25 -07:00
arraymap.c	bpf: Set need_defer as false when clearing fd array during map free	2024-02-05 20:14:20 +00:00
bloom_filter.c	bpf: Centralize permissions checks for all BPF map types	2023-06-19 14:04:04 +02:00
bpf_cgrp_storage.c	bpf: Teach verifier that certain helpers accept NULL pointer.	2023-04-04 16:57:16 -07:00
bpf_inode_storage.c	Networking changes for 6.4.	2023-04-26 16:07:23 -07:00
bpf_iter.c	bpf: implement numbers iterator	2023-03-08 16:19:51 -08:00
bpf_local_storage.c	bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc	2023-09-06 11:08:14 +02:00
bpf_lru_list.c	bpf: Address KCSAN report on bpf_lru_list	2023-05-12 12:01:03 -07:00
bpf_lru_list.h	bpf: lru: Remove unused declaration bpf_lru_promote()	2023-08-08 17:21:42 -07:00
bpf_lsm.c	bpf: Fix the kernel crash caused by bpf_setsockopt().	2023-01-26 23:26:40 -08:00
bpf_struct_ops.c	bpf: Support default .validate() and .update() behavior for struct_ops links	2023-08-14 22:23:39 -07:00
bpf_struct_ops_types.h
bpf_task_storage.c	bpf: Teach verifier that certain helpers accept NULL pointer.	2023-04-04 16:57:16 -07:00
btf.c	bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf	2024-01-31 16:19:04 -08:00
cgroup.c	bpf: Propagate modified uaddrlen from cgroup sockaddr programs	2024-01-31 16:19:04 -08:00
cgroup_iter.c	cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers	2023-03-17 12:07:13 -10:00
core.c	bpf: hardcode BPF_PROG_PACK_SIZE to 2MB * num_possible_nodes()	2024-03-26 18:19:41 -04:00
cpumap.c	bpf: report RCU QS in cpumap kthread	2024-03-26 18:20:12 -04:00
cpumask.c	bpf: Convert bpf_cpumask to bpf_mem_cache_free_rcu.	2023-07-12 23:45:23 +02:00
devmap.c	bpf: Fix DEVMAP_HASH overflow check on 32-bit arches	2024-03-26 18:19:39 -04:00
disasm.c	bpf: change bpf_alu_sign_string and bpf_movsx_string to static	2023-08-04 16:15:50 -07:00
disasm.h
dispatcher.c	bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func	2022-12-14 12:02:14 -08:00
hashtab.c	bpf: Fix hashtab overflow check on 32-bit arches	2024-03-26 18:19:39 -04:00
helpers.c	bpf: Mark bpf_spin_{lock,unlock}() helpers with notrace correctly	2024-03-26 18:19:29 -04:00
inode.c	bpf: convert to ctime accessor functions	2023-07-24 10:30:07 +02:00
Kconfig	bpf: Add fd-based tcx multi-prog infra with link support	2023-07-19 10:07:27 -07:00
link_iter.c	bpf: Add bpf_link iterator	2022-05-10 11:20:45 -07:00
local_storage.c	cgroup changes for v6.4-rc1	2023-04-29 10:05:22 -07:00
log.c	bpf: drop unnecessary user-triggerable WARN_ONCE in verifierl log	2023-05-16 22:34:50 -07:00
lpm_trie.c	bpf, lpm: Fix check prefixlen before walking trie	2024-01-25 15:35:19 -08:00
Makefile	bpf: Add fd-based tcx multi-prog infra with link support	2023-07-19 10:07:27 -07:00
map_in_map.c	bpf: Defer the free of inner map when necessary	2024-01-25 15:35:22 -08:00
map_in_map.h	bpf: Add map and need_defer parameters to .map_fd_put_ptr()	2024-01-25 15:35:22 -08:00
map_iter.c	bpf: allow any program to use the bpf_map_sum_elem_count kfunc	2023-07-19 09:48:53 -07:00
memalloc.c	bpf: Use c->unit_size to select target cache during free	2024-01-25 15:35:28 -08:00
mmap_unlock_work.h
mprog.c	bpf: Handle bpf_mprog_query with NULL entry	2023-10-06 17:11:20 -07:00
net_namespace.c	net: Add includes masked by netdevice.h including uapi/bpf.h	2021-12-29 20:03:05 -08:00
offload.c	bpf: Avoid dummy bpf_offload_netdev in __bpf_prog_dev_bound_init	2023-09-11 22:06:06 -07:00
percpu_freelist.c	bpf: Initialize same number of free nodes for each pcpu_freelist	2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c	bpf: Avoid deadlock when using queue and stack maps from NMI	2023-09-11 19:04:49 -07:00
reuseport_array.c	bpf: Centralize permissions checks for all BPF map types	2023-06-19 14:04:04 +02:00
ringbuf.c	bpf: Remove unnecessary ring buffer size check	2023-07-05 14:09:45 +02:00
stackmap.c	bpf: Fix stackmap overflow check on 32-bit arches	2024-03-26 18:19:39 -04:00
syscall.c	bpf: Set uattr->batch.count as zero before batched update or deletion	2024-02-05 20:14:21 +00:00
sysfs_btf.c
task_iter.c	bpf: keep a reference to the mm, in case the task is dead.	2022-12-28 14:11:48 -08:00
tcx.c	bpf: Handle bpf_mprog_query with NULL entry	2023-10-06 17:11:20 -07:00
tnum.c
trampoline.c	bpf, x64: Fix tailcall infinite loop	2023-11-20 11:58:55 +01:00
verifier.c	bpf: check bpf_func_state->callback_depth when pruning states	2024-03-15 10:48:17 -04:00