Andrii Nakryiko 4c8f30a2ad bpf: aggressively forget precise markings during state checkpointing ... [ Upstream commit 7a830b53c1 ] Exploit the property of about-to-be-checkpointed state to be able to forget all precise markings up to that point even more aggressively. We now clear all potentially inherited precise markings right before checkpointing and branching off into child state. If any of children states require precise knowledge of any SCALAR register, those will be propagated backwards later on before this state is finalized, preserving correctness. There is a single selftests BPF program change, but tremendous one: 25x reduction in number of verified instructions and states in trace_virtqueue_add_sgs. Cilium results are more modest, but happen across wider range of programs. SELFTESTS RESULTS ================= $ ./veristat -C -e file,prog,insns,states ~/imprecise-early-results.csv ~/imprecise-aggressive-results.csv | grep -v '+0' File Program Total insns (A) Total insns (B) Total insns (DIFF) Total states (A) Total states (B) Total states (DIFF) ------------------- ----------------------- --------------- --------------- ------------------ ---------------- ---------------- ------------------- loop6.bpf.linked1.o trace_virtqueue_add_sgs 398057 15114 -382943 (-96.20%) 8717 336 -8381 (-96.15%) ------------------- ----------------------- --------------- --------------- ------------------ ---------------- ---------------- ------------------- CILIUM RESULTS ============== $ ./veristat -C -e file,prog,insns,states ~/imprecise-early-results-cilium.csv ~/imprecise-aggressive-results-cilium.csv | grep -v '+0' File Program Total insns (A) Total insns (B) Total insns (DIFF) Total states (A) Total states (B) Total states (DIFF) ------------- -------------------------------- --------------- --------------- ------------------ ---------------- ---------------- ------------------- bpf_host.o tail_handle_nat_fwd_ipv4 23426 23221 -205 (-0.88%) 1537 1515 -22 (-1.43%) bpf_host.o tail_handle_nat_fwd_ipv6 13009 12904 -105 (-0.81%) 719 708 -11 (-1.53%) bpf_host.o tail_nodeport_nat_ingress_ipv6 5261 5196 -65 (-1.24%) 247 243 -4 (-1.62%) bpf_host.o tail_nodeport_nat_ipv6_egress 3446 3406 -40 (-1.16%) 203 198 -5 (-2.46%) bpf_lxc.o tail_handle_nat_fwd_ipv4 23426 23221 -205 (-0.88%) 1537 1515 -22 (-1.43%) bpf_lxc.o tail_handle_nat_fwd_ipv6 13009 12904 -105 (-0.81%) 719 708 -11 (-1.53%) bpf_lxc.o tail_ipv4_ct_egress 5074 4897 -177 (-3.49%) 255 248 -7 (-2.75%) bpf_lxc.o tail_ipv4_ct_ingress 5100 4923 -177 (-3.47%) 255 248 -7 (-2.75%) bpf_lxc.o tail_ipv4_ct_ingress_policy_only 5100 4923 -177 (-3.47%) 255 248 -7 (-2.75%) bpf_lxc.o tail_ipv6_ct_egress 4558 4536 -22 (-0.48%) 188 187 -1 (-0.53%) bpf_lxc.o tail_ipv6_ct_ingress 4578 4556 -22 (-0.48%) 188 187 -1 (-0.53%) bpf_lxc.o tail_ipv6_ct_ingress_policy_only 4578 4556 -22 (-0.48%) 188 187 -1 (-0.53%) bpf_lxc.o tail_nodeport_nat_ingress_ipv6 5261 5196 -65 (-1.24%) 247 243 -4 (-1.62%) bpf_overlay.o tail_nodeport_nat_ingress_ipv6 5261 5196 -65 (-1.24%) 247 243 -4 (-1.62%) bpf_overlay.o tail_nodeport_nat_ipv6_egress 3482 3442 -40 (-1.15%) 204 201 -3 (-1.47%) bpf_xdp.o tail_nodeport_nat_egress_ipv4 17200 15619 -1581 (-9.19%) 1111 1010 -101 (-9.09%) ------------- -------------------------------- --------------- --------------- ------------------ ---------------- ---------------- ------------------- Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20221104163649.121784-6-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>		2023-07-27 08:50:51 +02:00
..
preload	bpf: iterators: Build and use lightweight bootstrap version of bpftool	2022-07-15 12:01:30 -07:00
arraymap.c	bpf: Support kptrs in percpu arraymap	2022-09-07 11:46:08 -07:00
bloom_filter.c	treewide: use get_random_u32() when possible	2022-10-11 17:42:58 -06:00
bpf_inode_storage.c	bpf: Compute map_btf_id during build time	2022-04-26 11:35:21 -07:00
bpf_iter.c	bpf: Initialize the bpf_run_ctx in bpf_iter_run_prog()	2022-08-18 17:06:13 -07:00
bpf_local_storage.c	bpf: Annotate data races in bpf_local_storage	2023-05-24 17:32:36 +01:00
bpf_lru_list.c	bpf: Address KCSAN report on bpf_lru_list	2023-07-27 08:50:34 +02:00
bpf_lru_list.h	bpf: Address KCSAN report on bpf_lru_list	2023-07-27 08:50:34 +02:00
bpf_lsm.c	bpf: Fix the kernel crash caused by bpf_setsockopt().	2023-02-09 11:28:02 +01:00
bpf_struct_ops.c	bpf: Remove is_valid_bpf_tramp_flags()	2022-07-11 21:04:58 +02:00
bpf_struct_ops_types.h
bpf_task_storage.c	bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy	2022-09-01 12:16:12 -07:00
btf.c	bpf, btf: Warn but return no error for NULL btf from __register_btf_kfunc_id_set()	2023-07-19 16:22:05 +02:00
cgroup.c	bpf: Don't EFAULT for {g,s}setsockopt with wrong optlen	2023-07-19 16:21:05 +02:00
cgroup_iter.c	cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers	2023-06-21 16:00:51 +02:00
core.c	bpf: Adjust insufficient default bpf_jit_limit	2023-03-30 12:49:08 +02:00
cpumap.c	bpf: cpumap: Fix memory leak in cpu_map_update_elem	2023-07-23 13:49:26 +02:00
devmap.c	bpf: Use bpf_map_area_alloc consistently on bpf map creation	2022-08-10 11:50:43 -07:00
disasm.c
disasm.h
dispatcher.c	bpf: Add explicit cast to 'void *' for __BPF_DISPATCHER_UPDATE()	2022-11-07 16:07:35 -08:00
hashtab.c	bpf: fix a memory leak in the LRU and LRU_PERCPU hash maps	2023-05-30 14:03:21 +01:00
helpers.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2022-10-03 17:44:18 -07:00
inode.c	bpf: Convert bpf_preload.ko to use light skeleton.	2022-02-10 23:31:51 +01:00
Kconfig	rcu: Make the TASKS_RCU Kconfig option be selected	2022-04-20 16:52:58 -07:00
link_iter.c	bpf: Add bpf_link iterator	2022-05-10 11:20:45 -07:00
local_storage.c	cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers	2023-06-21 16:00:51 +02:00
lpm_trie.c	bpf: Use bpf_map_area_alloc consistently on bpf map creation	2022-08-10 11:50:43 -07:00
Makefile	bpf: Introduce any context BPF specific memory allocator.	2022-09-05 15:33:05 +02:00
map_in_map.c	bpf: Fix elem_size not being set for inner maps	2023-06-14 11:15:17 +02:00
map_in_map.h
map_iter.c	bpf: Introduce MEM_RDONLY flag	2021-12-18 13:27:41 -08:00
memalloc.c	bpf: Zeroing allocated object from slab in bpf memory allocator	2023-03-10 09:33:06 +01:00
mmap_unlock_work.h
net_namespace.c	net: Add includes masked by netdevice.h including uapi/bpf.h	2021-12-29 20:03:05 -08:00
offload.c	bpf: restore the ebpf program ID for BPF_AUDIT_UNLOAD and PERF_BPF_EVENT_PROG_UNLOAD	2023-01-24 07:24:37 +01:00
percpu_freelist.c	bpf: Initialize same number of free nodes for each pcpu_freelist	2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c	bpf: Remove unneeded memset in queue_stack_map creation	2022-08-10 11:48:22 -07:00
reuseport_array.c	net: Fix suspicious RCU usage in bpf_sk_reuseport_detach()	2022-08-17 16:42:59 -07:00
ringbuf.c	bpf: Add bpf_user_ringbuf_drain() helper	2022-09-21 16:24:58 -07:00
stackmap.c	perf/bpf: Always use perf callchains if exist	2022-09-13 15:03:22 +02:00
syscall.c	bpf: Print a warning only if writing to unprivileged_bpf_disabled.	2023-07-27 08:50:34 +02:00
sysfs_btf.c
task_iter.c	bpf: keep a reference to the mm, in case the task is dead.	2023-01-24 07:24:31 +01:00
tnum.c
trampoline.c	bpf: Fix memleak due to fentry attach failure	2023-07-19 16:21:05 +02:00
verifier.c	bpf: aggressively forget precise markings during state checkpointing	2023-07-27 08:50:51 +02:00