linux-stable/tools/testing/selftests/bpf/prog_tests
Stanislav Fomichev 9cacf81f81 bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE
Add custom implementation of getsockopt hook for TCP_ZEROCOPY_RECEIVE.
We skip generic hooks for TCP_ZEROCOPY_RECEIVE and have a custom
call in do_tcp_getsockopt using the on-stack data. This removes
3% overhead for locking/unlocking the socket.

Without this patch:
     3.38%     0.07%  tcp_mmap  [kernel.kallsyms]  [k] __cgroup_bpf_run_filter_getsockopt
            |
             --3.30%--__cgroup_bpf_run_filter_getsockopt
                       |
                        --0.81%--__kmalloc

With the patch applied:
     0.52%     0.12%  tcp_mmap  [kernel.kallsyms]  [k] __cgroup_bpf_run_filter_getsockopt_kern

Note, exporting uapi/tcp.h requires removing netinet/tcp.h
from test_progs.h because those headers have confliciting
definitions.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20210115163501.805133-2-sdf@google.com
2021-01-20 14:23:00 -08:00
..
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
align.c selftests/bpf: Fix "dubious pointer arithmetic" test 2020-12-10 13:11:30 -08:00
atomics.c bpf: Add tests for new BPF atomic operations 2021-01-14 18:34:29 -08:00
attach_probe.c selftests/bpf: Fix test_attach_probe 2019-12-19 16:14:08 +01:00
autoload.c selftests/bpf: Test auto-load disabling logic for BPF programs 2020-06-28 10:06:53 -07:00
bpf_iter.c selftests/bpf: Test bpf_sk_storage_get in tcp iterators 2020-12-04 22:32:40 +01:00
bpf_obj_id.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
bpf_tcp_ca.c bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 2021-01-20 14:23:00 -08:00
bpf_verif_scale.c selftests/bpf: Add subprogs to pyperf, strobemeta, and l4lb_noinline tests 2020-09-03 17:14:40 -07:00
btf.c bpf: Permit size-0 datasec 2021-01-20 14:14:09 -08:00
btf_dedup_split.c selftests/bpf: Add split BTF dedup selftests 2020-11-05 18:37:31 -08:00
btf_dump.c selftests/bpf: Test "incremental" btf_dump in C format 2020-09-30 12:30:46 -07:00
btf_endian.c selftests/bpf: Test BTF's handling of endianness 2020-09-29 12:21:23 -07:00
btf_map_in_map.c selftests/bpf: Sync RCU before unloading bpf_testmod 2021-01-12 17:23:47 -08:00
btf_skc_cls_ingress.c bpf: selftest: Use static globals in tcp_hdr_options and btf_skc_cls_ingress 2020-11-09 16:18:36 -08:00
btf_split.c selftests/bpf: Add split BTF basic test 2020-11-05 18:37:30 -08:00
btf_write.c selftests/bpf: Add checking of raw type dump in BTF writer APIs selftests 2020-11-05 18:37:30 -08:00
cg_storage_multi.c selftests/bpf: Don't destroy failed link 2020-07-31 00:43:49 +02:00
cgroup_attach_autodetach.c selftests/bpf: Declare bpf_log_buf variables as static 2020-03-02 17:00:41 -08:00
cgroup_attach_multi.c bpf: Rename BPF_XADD and prepare to encode other atomics in .imm 2021-01-14 18:34:29 -08:00
cgroup_attach_override.c selftests/bpf: Declare bpf_log_buf variables as static 2020-03-02 17:00:41 -08:00
cgroup_link.c selftests/bpf: Add link detach tests for cgroup, netns, and xdp bpf_links 2020-08-01 20:38:28 -07:00
cgroup_skb_sk_lookup.c bpf: selftests: A few improvements to network_helpers.c 2020-07-02 16:09:01 +02:00
cls_redirect.c bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 2021-01-20 14:23:00 -08:00
connect_force_port.c bpf: selftests: A few improvements to network_helpers.c 2020-07-02 16:09:01 +02:00
core_autosize.c selftests/bpf: Validate libbpf's auto-sizing of LD/ST/STX instructions 2020-10-07 18:50:27 -07:00
core_extern.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
core_read_macros.c selftests/bpf: Add tests for user- and non-CO-RE BPF_CORE_READ() variants 2021-01-08 13:39:24 -08:00
core_reloc.c selftests/bpf: Add tp_btf CO-RE reloc test for modules 2020-12-03 17:38:21 -08:00
core_retro.c selftests/bpf: Fix spurious test failures in core_retro selftest 2020-08-01 23:10:42 -07:00
cpu_mask.c selftests: Use consistent include paths for libbpf 2020-01-20 16:37:45 -08:00
d_path.c selftests/bpf: Fix stat probe in d_path test 2020-09-21 16:18:00 -07:00
enable_stats.c bpf: Add selftest for BPF_ENABLE_STATS 2020-05-01 10:36:32 -07:00
endian.c selftests/bpf: Add byte swapping selftest 2020-07-01 09:06:12 +02:00
fentry_fexit.c bpf: Add tests for PTR_TO_BTF_ID vs. null comparison 2020-06-30 22:21:29 +02:00
fentry_test.c bpf: Add test ops for BPF_PROG_TYPE_TRACING 2020-03-04 13:41:06 -08:00
fexit_bpf2bpf.c selftests: Add selftest for disallowing modify_return attachment to freplace 2020-09-29 13:09:24 -07:00
fexit_stress.c selftests/bpf: Add stress test for maximum number of progs 2019-11-15 23:43:53 +01:00
fexit_test.c bpf: Add test ops for BPF_PROG_TYPE_TRACING 2020-03-04 13:41:06 -08:00
flow_dissector.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
flow_dissector_load_bytes.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
flow_dissector_reattach.c selftests: bpf: Pass program and target_fd in flow_dissector_reattach 2020-06-30 10:46:39 -07:00
get_stack_raw_tp.c bpf: Test_progs, add test to catch retval refine error handling 2020-03-30 15:00:30 -07:00
get_stackid_cannot_attach.c selftests/bpf: Add get_stackid_cannot_attach 2020-07-25 20:16:35 -07:00
global_data.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
global_data_init.c selftests/bpf: Fix check in global_data_init. 2020-09-03 17:33:31 -07:00
hash_large_key.c bpf: Lift hashtab key_size limit 2020-11-05 20:04:46 -08:00
hashmap.c selftests/bpf: Convert test_hashmap into test_progs test 2020-04-28 19:48:05 -07:00
kfree_skb.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
ksyms.c selftests/bpf: Ksyms_btf to test typed ksyms 2020-10-02 14:59:25 -07:00
ksyms_btf.c bpf: selftest: Ensure the return value of the bpf_per_cpu_ptr() must be checked 2020-10-19 15:57:42 -07:00
ksyms_module.c selftests/bpf: Test kernel module ksym externs 2021-01-12 17:24:30 -08:00
l4lb_all.c selftests/bpf: Add subprogs to pyperf, strobemeta, and l4lb_noinline tests 2020-09-03 17:14:40 -07:00
link_pinning.c selftests/bpf: Add link pinning selftests 2020-03-02 22:06:27 -08:00
load_bytes_relative.c bpf: selftests: A few improvements to network_helpers.c 2020-07-02 16:09:01 +02:00
map_init.c bpf: Zero-fill re-used per-cpu map element 2020-11-05 19:55:57 -08:00
map_lock.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
map_ptr.c selftests/bpf: Test access to bpf map pointer 2020-06-22 22:22:59 +02:00
metadata.c selftests/bpf: Test load and dump metadata with btftool and skel 2020-09-15 18:28:27 -07:00
mmap.c selftests/bpf: Fix segmentation fault in test_progs 2020-08-11 15:36:45 +02:00
modify_return.c bpf: Add selftests for BPF_MODIFY_RETURN 2020-03-04 13:41:06 -08:00
module_attach.c selftests: bpf: Add a new test for bare tracepoints 2021-01-20 14:14:05 -08:00
ns_current_pid_tgid.c bpf, selftests: Fold test_current_pid_tgid_new_ns into test_progs. 2021-01-20 14:10:35 -08:00
obj_name.c selftests: bpf: break up test_progs - misc 2019-03-02 11:10:40 -08:00
pe_preserve_elems.c selftests/bpf: Add tests for BPF_F_PRESERVE_ELEMS 2020-09-30 23:21:06 -07:00
perf_branches.c selftests/bpf: Add bpf_read_branch_records() selftest 2020-02-19 15:01:07 -08:00
perf_buffer.c libbpf: Add perf_buffer APIs for better integration with outside epoll loop 2020-08-21 14:26:55 -07:00
perf_event_stackmap.c selftests/bpf: Add callchain_stackid 2020-07-25 20:16:35 -07:00
pinning.c selftest/bpf: Test pinning map with reused map fd 2020-10-06 11:10:20 -07:00
pkt_access.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
pkt_md_access.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
probe_read_user_str.c selftest/bpf: Test bpf_probe_read_user_str() strips trailing bytes after NUL 2020-11-19 11:58:15 -08:00
probe_user.c libbpf: Add generic bpf_program__attach() 2019-12-15 15:58:04 -08:00
prog_run_xattr.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
queue_stack_map.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
raw_tp_test_run.c selftests/bpf: Add raw_tp_test_run 2020-09-28 21:52:36 +02:00
raw_tp_writable_reject_nbd_invalid.c selftests: bpf: test writable buffers in raw tps 2019-04-26 19:04:19 -07:00
raw_tp_writable_test_run.c selftests: bpf: test writable buffers in raw tps 2019-04-26 19:04:19 -07:00
rdonly_maps.c libbpf: Expose BPF program's function name 2019-12-15 15:58:05 -08:00
reference_tracking.c selftests/bpf: Don't use deprecated libbpf APIs 2020-09-03 17:14:40 -07:00
resolve_btfids.c selftests/bpf: Fix alignment of .BTF_ids 2020-09-30 18:11:40 -07:00
ringbuf.c selftests/bpf: Drain ringbuf samples at the end of test 2020-12-01 20:21:45 -08:00
ringbuf_multi.c libbpf: Fix ring_buffer__poll() to return number of consumed samples 2020-12-01 20:21:45 -08:00
section_names.c libbpf: Add support for BPF XDP link 2020-07-25 20:37:02 -07:00
select_reuseport.c selftests: bpf: Enable UDP sockmap reuseport tests 2020-03-09 22:34:59 +01:00
send_signal.c bpf: Fix compilation warning of selftests 2020-08-06 16:58:42 -07:00
send_signal_sched_switch.c selftests/bpf: Add send_signal_sched_switch test 2020-03-05 14:02:41 -08:00
signal_pending.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
sk_assign.c selftests/bpf: Fix endianness issue in sk_assign 2020-09-18 22:54:52 +02:00
sk_lookup.c selftests/bpf: Fix v4_to_v6 in sk_lookup 2020-08-11 15:36:51 +02:00
sk_storage_tracing.c bpf: selftest: Use bpf_sk_storage in FENTRY/FEXIT/RAW_TP 2020-11-12 18:39:28 -08:00
skb_ctx.c selftest/bpf: Fix compilation warnings in 32-bit mode 2020-08-13 16:45:41 -07:00
skb_helpers.c bpf, selftests: Test probe_* helpers from SCHED_CLS 2020-06-01 14:38:21 -07:00
skeleton.c tools/bpftool: Strip away modifiers from global variables 2020-07-13 17:07:43 -07:00
snprintf_btf.c bpf, selftests: Fix warning in snprintf_btf where system() call unchecked 2020-09-29 13:21:23 -07:00
sock_fields.c selftests: Make sure all 'skel' variables are declared static 2020-09-29 11:43:43 -07:00
sockmap_basic.c bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 2021-01-20 14:23:00 -08:00
sockmap_ktls.c selftests/bpf: Fix build of sockmap_ktls.c 2020-02-20 01:17:24 +01:00
sockmap_listen.c selftests/bpf: Fix spurious failures in accept due to EAGAIN 2020-03-13 21:37:06 +01:00
sockopt.c selftests/bpf: test_progs: convert test_sockopt 2019-09-06 09:59:05 -07:00
sockopt_inherit.c selftests/bpf: Fix possible hang in sockopt_inherit 2020-07-16 20:57:09 +02:00
sockopt_multi.c selftests/bpf: Fix error return code in run_getsockopt_test() 2020-11-16 22:19:07 +01:00
sockopt_sk.c bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 2021-01-20 14:23:00 -08:00
spinlock.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
stacktrace_build_id.c bpftool, selftests/bpf: Embed object file inside skeleton 2019-12-17 22:16:35 -08:00
stacktrace_build_id_nmi.c bpf: Fix compilation warning of selftests 2020-08-06 16:58:42 -07:00
stacktrace_map.c selftests/bpf: test_progs: remove global fail/success counts 2019-08-28 00:35:40 +02:00
stacktrace_map_raw_tp.c selftests/bpf: test_progs: remove global fail/success counts 2019-08-28 00:35:40 +02:00
subprogs.c libbpf: Don't attempt to load unused subprog as an entry-point BPF program 2020-11-09 22:15:23 +01:00
tailcalls.c selftests/bpf: Add tailcall_bpf2bpf tests 2020-09-17 19:56:07 -07:00
task_fd_query_rawtp.c selftests/bpf: test_progs: remove global fail/success counts 2019-08-28 00:35:40 +02:00
task_fd_query_tp.c selftests/bpf: test_progs: remove global fail/success counts 2019-08-28 00:35:40 +02:00
tcp_estats.c selftests/bpf: test_progs: remove global fail/success counts 2019-08-28 00:35:40 +02:00
tcp_hdr_options.c bpf: selftest: Use static globals in tcp_hdr_options and btf_skc_cls_ingress 2020-11-09 16:18:36 -08:00
tcp_rtt.c bpf: selftests: A few improvements to network_helpers.c 2020-07-02 16:09:01 +02:00
tcpbpf_user.c selftests/bpf: Add Userspace tests for TCP_WINDOW_CLAMP 2020-12-03 17:23:24 -08:00
test_bpffs.c selftests/bpf: Add bpffs preload test. 2020-08-20 16:02:36 +02:00
test_bprm_opts.c bpf: Add tests for bpf_bprm_opts_set helper 2020-11-18 01:36:27 +01:00
test_global_funcs.c bpf: Relax return code check for subprograms 2020-11-14 08:17:27 -08:00
test_ima.c bpf: Add a selftest for bpf_ima_inode_hash 2020-11-26 00:25:47 +01:00
test_local_storage.c bpf: Update local storage test to check handling of null ptrs 2021-01-12 16:07:57 +01:00
test_lsm.c selftests/bpf: Remove duplicate include in test_lsm 2021-01-08 13:39:40 -08:00
test_overhead.c selftests: Remove fmod_ret from test_overhead 2020-09-28 17:20:28 -07:00
test_profiler.c selftests/bpf: Add profiler test 2020-10-09 22:03:06 +02:00
test_skb_pkt_end.c selftests/bpf: Add skb_pkt_end test 2020-11-13 01:42:11 +01:00
tp_attach_query.c selftests: bpf: initialize bpf_object pointers where needed 2019-05-09 15:53:56 -07:00
trace_ext.c selftests/bpf: Adding test for arg dereference in extension trace 2020-09-29 13:09:24 -07:00
trace_printk.c selftests/bpf: Add selftests verifying bpf_trace_printk() behaviour 2020-07-13 16:55:49 -07:00
trampoline_count.c selftests/bpf: Fix trampoline_count clean up logic 2020-02-20 18:03:10 -08:00
udp_limit.c selftests/bpf: Test BPF_CGROUP_INET_SOCK_RELEASE 2020-07-08 01:07:36 +02:00
varlen.c selftests/bpf: Make test_varlen work with 32-bit user-space arch 2020-08-13 16:45:41 -07:00
vmlinux.c selftests/bpf: Fix nanosleep for real this time 2020-03-17 19:29:12 +01:00
xdp.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
xdp_adjust_tail.c selftests/bpf: Xdp_adjust_tail add grow tail tests 2020-05-14 21:21:57 -07:00
xdp_attach.c selftests/bpf: Check for correct program attach/detach in xdp_attach test 2020-04-15 13:26:08 +02:00
xdp_bpf2bpf.c selftests/bpf: Move existing common networking parts into network_helpers 2020-05-09 00:48:20 +02:00
xdp_cpumap_attach.c selftest: Add tests for XDP programs in CPUMAP entries 2020-07-16 17:00:32 +02:00
xdp_devmap_attach.c bpf: Selftests and tools use struct bpf_devmap_val from uapi 2020-06-09 11:36:19 -07:00
xdp_info.c selftests/bpf: Add test for bpf_get_link_xdp_id 2020-04-08 01:35:24 +02:00
xdp_link.c selftests/bpf: Add link detach tests for cgroup, netns, and xdp bpf_links 2020-08-01 20:38:28 -07:00
xdp_noinline.c selftests/bpf: Initialize duration in xdp_noinline.c 2020-10-02 16:46:20 +02:00
xdp_perf.c selftests: bpf: Add xdp_perf test 2019-12-13 13:09:32 -08:00