mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-09-28 13:22:57 +00:00
4b2765ae41
-----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTFp0I1jqZrAX+hPRXbK58LschIgwUCZeEKVAAKCRDbK58LschI g7oYAQD5Jlv4fIVTvxvfZrTTZ2tU+OsPa75mc8SDKwpash3YygEA8kvESy8+t6pg D6QmSf1DIZdFoSp/bV+pfkNWMeR8gwg= =mTAj -----END PGP SIGNATURE----- Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next Daniel Borkmann says: ==================== pull-request: bpf-next 2024-02-29 We've added 119 non-merge commits during the last 32 day(s) which contain a total of 150 files changed, 3589 insertions(+), 995 deletions(-). The main changes are: 1) Extend the BPF verifier to enable static subprog calls in spin lock critical sections, from Kumar Kartikeya Dwivedi. 2) Fix confusing and incorrect inference of PTR_TO_CTX argument type in BPF global subprogs, from Andrii Nakryiko. 3) Larger batch of riscv BPF JIT improvements and enabling inlining of the bpf_kptr_xchg() for RV64, from Pu Lehui. 4) Allow skeleton users to change the values of the fields in struct_ops maps at runtime, from Kui-Feng Lee. 5) Extend the verifier's capabilities of tracking scalars when they are spilled to stack, especially when the spill or fill is narrowing, from Maxim Mikityanskiy & Eduard Zingerman. 6) Various BPF selftest improvements to fix errors under gcc BPF backend, from Jose E. Marchesi. 7) Avoid module loading failure when the module trying to register a struct_ops has its BTF section stripped, from Geliang Tang. 8) Annotate all kfuncs in .BTF_ids section which eventually allows for automatic kfunc prototype generation from bpftool, from Daniel Xu. 9) Several updates to the instruction-set.rst IETF standardization document, from Dave Thaler. 10) Shrink the size of struct bpf_map resp. bpf_array, from Alexei Starovoitov. 11) Initial small subset of BPF verifier prepwork for sleepable bpf_timer, from Benjamin Tissoires. 12) Fix bpftool to be more portable to musl libc by using POSIX's basename(), from Arnaldo Carvalho de Melo. 13) Add libbpf support to gcc in CORE macro definitions, from Cupertino Miranda. 14) Remove a duplicate type check in perf_event_bpf_event, from Florian Lehner. 15) Fix bpf_spin_{un,}lock BPF helpers to actually annotate them with notrace correctly, from Yonghong Song. 16) Replace the deprecated bpf_lpm_trie_key 0-length array with flexible array to fix build warnings, from Kees Cook. 17) Fix resolve_btfids cross-compilation to non host-native endianness, from Viktor Malik. * tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (119 commits) selftests/bpf: Test if shadow types work correctly. bpftool: Add an example for struct_ops map and shadow type. bpftool: Generated shadow variables for struct_ops maps. libbpf: Convert st_ops->data to shadow type. libbpf: Set btf_value_type_id of struct bpf_map for struct_ops. bpf: Replace bpf_lpm_trie_key 0-length array with flexible array bpf, arm64: use bpf_prog_pack for memory management arm64: patching: implement text_poke API bpf, arm64: support exceptions arm64: stacktrace: Implement arch_bpf_stack_walk() for the BPF JIT bpf: add is_async_callback_calling_insn() helper bpf: introduce in_sleepable() helper bpf: allow more maps in sleepable bpf programs selftests/bpf: Test case for lacking CFI stub functions. bpf: Check cfi_stubs before registering a struct_ops type. bpf: Clarify batch lookup/lookup_and_delete semantics bpf, docs: specify which BPF_ABS and BPF_IND fields were zero bpf, docs: Fix typos in instruction-set.rst selftests/bpf: update tcp_custom_syncookie to use scalar packet offset bpf: Shrink size of struct bpf_map/bpf_array. ... ==================== Link: https://lore.kernel.org/r/20240301001625.8800-1-daniel@iogearbox.net Signed-off-by: Jakub Kicinski <kuba@kernel.org> |
||
---|---|---|
.. | ||
benchs | ||
bpf_test_no_cfi | ||
bpf_testmod | ||
gnu | ||
map_tests | ||
prog_tests | ||
progs | ||
verifier | ||
.gitignore | ||
autoconf_helper.h | ||
bench.c | ||
bench.h | ||
bpf_experimental.h | ||
bpf_kfuncs.h | ||
bpf_legacy.h | ||
bpf_rand.h | ||
bpf_sockopt_helpers.h | ||
bpf_tcp_helpers.h | ||
bpf_util.h | ||
btf_helpers.c | ||
btf_helpers.h | ||
cap_helpers.c | ||
cap_helpers.h | ||
cgroup_getset_retval_hooks.h | ||
cgroup_helpers.c | ||
cgroup_helpers.h | ||
cgroup_tcp_skb.h | ||
config | ||
config.aarch64 | ||
config.s390x | ||
config.vm | ||
config.x86_64 | ||
DENYLIST | ||
DENYLIST.aarch64 | ||
DENYLIST.s390x | ||
disasm.c | ||
disasm.h | ||
flow_dissector_load.c | ||
flow_dissector_load.h | ||
generate_udp_fragments.py | ||
get_cgroup_id_user.c | ||
ima_setup.sh | ||
ip_check_defrag_frags.h | ||
json_writer.c | ||
json_writer.h | ||
liburandom_read.map | ||
Makefile | ||
Makefile.docs | ||
netcnt_common.h | ||
netlink_helpers.c | ||
netlink_helpers.h | ||
network_helpers.c | ||
network_helpers.h | ||
README.rst | ||
sdt-config.h | ||
sdt.h | ||
settings | ||
task_local_storage_helpers.h | ||
test_bpftool.py | ||
test_bpftool.sh | ||
test_bpftool_build.sh | ||
test_bpftool_metadata.sh | ||
test_bpftool_synctypes.py | ||
test_btf.h | ||
test_cgroup_storage.c | ||
test_cpp.cpp | ||
test_dev_cgroup.c | ||
test_doc_build.sh | ||
test_flow_dissector.c | ||
test_flow_dissector.sh | ||
test_ftrace.sh | ||
test_iptunnel_common.h | ||
test_kmod.sh | ||
test_lirc_mode2.sh | ||
test_lirc_mode2_user.c | ||
test_loader.c | ||
test_lpm_map.c | ||
test_lru_map.c | ||
test_lwt_ip_encap.sh | ||
test_lwt_seg6local.sh | ||
test_maps.c | ||
test_maps.h | ||
test_offload.py | ||
test_progs.c | ||
test_progs.h | ||
test_select_reuseport_common.h | ||
test_skb_cgroup_id.sh | ||
test_skb_cgroup_id_user.c | ||
test_sock.c | ||
test_sock_addr.c | ||
test_sock_addr.sh | ||
test_sockmap.c | ||
test_sysctl.c | ||
test_tag.c | ||
test_tc_edt.sh | ||
test_tc_tunnel.sh | ||
test_tcp_check_syncookie.sh | ||
test_tcp_check_syncookie_user.c | ||
test_tcp_hdr_options.h | ||
test_tcpbpf.h | ||
test_tcpnotify.h | ||
test_tcpnotify_user.c | ||
test_tunnel.sh | ||
test_verifier.c | ||
test_xdp_features.sh | ||
test_xdp_meta.sh | ||
test_xdp_redirect.sh | ||
test_xdp_redirect_multi.sh | ||
test_xdp_veth.sh | ||
test_xdp_vlan.sh | ||
test_xdp_vlan_mode_generic.sh | ||
test_xdp_vlan_mode_native.sh | ||
test_xdping.sh | ||
test_xsk.sh | ||
testing_helpers.c | ||
testing_helpers.h | ||
trace_helpers.c | ||
trace_helpers.h | ||
unpriv_helpers.c | ||
unpriv_helpers.h | ||
uprobe_multi.c | ||
urandom_read.c | ||
urandom_read_aux.c | ||
urandom_read_lib1.c | ||
urandom_read_lib2.c | ||
verify_sig_setup.sh | ||
veristat.c | ||
veristat.cfg | ||
vmtest.sh | ||
with_addr.sh | ||
with_tunnels.sh | ||
xdp_features.c | ||
xdp_features.h | ||
xdp_hw_metadata.c | ||
xdp_metadata.h | ||
xdp_redirect_multi.c | ||
xdp_synproxy.c | ||
xdping.c | ||
xdping.h | ||
xsk.c | ||
xsk.h | ||
xsk_prereqs.sh | ||
xsk_xdp_common.h | ||
xskxceiver.c | ||
xskxceiver.h |
================== BPF Selftest Notes ================== General instructions on running selftests can be found in `Documentation/bpf/bpf_devel_QA.rst`__. __ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests ============= BPF CI System ============= BPF employs a continuous integration (CI) system to check patch submission in an automated fashion. The system runs selftests for each patch in a series. Results are propagated to patchwork, where failures are highlighted similar to violations of other checks (such as additional warnings being emitted or a ``scripts/checkpatch.pl`` reported deficiency): https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173 The CI system executes tests on multiple architectures. It uses a kernel configuration derived from both the generic and architecture specific config file fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and ``config.x86_64``). Denylisting Tests ================= It is possible for some architectures to not have support for all BPF features. In such a case tests in CI may fail. An example of such a shortcoming is BPF trampoline support on IBM's s390x architecture. For cases like this, an in-tree deny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can be used to prevent the test from running on such an architecture. In addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is honored on every architecture running tests. These files are organized in three columns. The first column lists the test in question. This can be the name of a test suite or of an individual test. The remaining two columns provide additional meta data that helps identify and classify the entry: column two is a copy and paste of the error being reported when running the test in the setting in question. The third column, if available, summarizes the underlying problem. A value of ``trampoline``, for example, indicates that lack of trampoline support is causing the test to fail. This last entry helps identify tests that can be re-enabled once such support is added. ========================= Running Selftests in a VM ========================= It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. The script tries to ensure that the tests are run with the same environment as they would be run post-submit in the CI used by the Maintainers, with the exception that deny lists are not automatically honored. This script uses the in-tree kernel configuration and downloads a VM userspace image from the system used by the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by default in ``~/.bpf_selftests``). Script dependencies: - clang (preferably built from sources, https://github.com/llvm/llvm-project); - pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); - qemu; - docutils (for ``rst2man``); - libcap-devel. For more information about using the script, run: .. code-block:: console $ tools/testing/selftests/bpf/vmtest.sh -h In case of linker errors when running selftests, try using static linking: .. code-block:: console $ LDLIBS=-static PKG_CONFIG='pkg-config --static' vmtest.sh .. note:: Some distros may not support static linking. .. note:: The script uses pahole and clang based on host environment setting. If you want to change pahole and llvm, you can change `PATH` environment variable in the beginning of script. .. note:: The script currently only supports x86_64 and s390x architectures. Additional information about selftest failures are documented here. profiler[23] test failures with clang/llvm <12.0.0 ================================================== With clang/llvm <12.0.0, the profiler[23] test may fail. The symptom looks like .. code-block:: c // r9 is a pointer to map_value // r7 is a scalar 17: bf 96 00 00 00 00 00 00 r6 = r9 18: 0f 76 00 00 00 00 00 00 r6 += r7 math between map_value pointer and register with unbounded min value is not allowed // the instructions below will not be seen in the verifier log 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 20: bf 96 00 00 00 00 00 00 r6 = r9 // r6 is used here The verifier will reject such code with above error. At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and the insn 20 undoes map_value addition. It is currently impossible for the verifier to understand such speculative pointer arithmetic. Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. __ https://github.com/llvm/llvm-project/commit/ddf1864ace484035e3cde5e83b3a31ac81e059c6 The corresponding C code .. code-block:: c for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { filepart_length = bpf_probe_read_str(payload, ...); if (filepart_length <= MAX_PATH) { barrier_var(filepart_length); // workaround payload += filepart_length; } } bpf_iter test failures with clang/llvm 10.0.0 ============================================= With clang/llvm 10.0.0, the following two bpf_iter tests failed: * ``bpf_iter/ipv6_route`` * ``bpf_iter/netlink`` The symptom for ``bpf_iter/ipv6_route`` looks like .. code-block:: c 2: (79) r8 = *(u64 *)(r1 +8) ... 14: (bf) r2 = r8 15: (0f) r2 += r1 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 16: (7b) *(u64 *)(r8 +64) = r2 only read is supported The symptom for ``bpf_iter/netlink`` looks like .. code-block:: c ; struct netlink_sock *nlk = ctx->sk; 2: (79) r7 = *(u64 *)(r1 +8) ... 15: (bf) r2 = r7 16: (0f) r2 += r1 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 17: (7b) *(u64 *)(r7 +0) = r2 only read is supported This is due to a llvm BPF backend bug. `The fix`__ has been pushed to llvm 10.x release branch and will be available in 10.0.1. The patch is available in llvm 11.0.0 trunk. __ https://github.com/llvm/llvm-project/commit/3cb7e7bf959dcd3b8080986c62e10a75c7af43f0 bpf_verif_scale/loop6.bpf.o test failure with Clang 12 ====================================================== With Clang 12, the following bpf_verif_scale test failed: * ``bpf_verif_scale/loop6.bpf.o`` The verifier output looks like .. code-block:: c R1 type=ctx expected=fp The sequence of 8193 jumps is too complex. The reason is compiler generating the following code .. code-block:: c ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 15: bc 51 00 00 00 00 00 00 w1 = w5 16: 04 01 00 00 ff ff ff ff w1 += -1 17: 67 05 00 00 20 00 00 00 r5 <<= 32 18: 77 05 00 00 20 00 00 00 r5 >>= 32 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 20: b7 05 00 00 06 00 00 00 r5 = 6 00000000000000a8 <LBB0_4>: 21: b7 02 00 00 00 00 00 00 r2 = 0 22: b7 01 00 00 00 00 00 00 r1 = 0 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 Note that insn #15 has w1 = w5 and w1 is refined later but r5(w5) is eventually saved on stack at insn #24 for later use. This cause later verifier failure. The bug has been `fixed`__ in Clang 13. __ https://github.com/llvm/llvm-project/commit/1959ead525b8830cc8a345f45e1c3ef9902d3229 BPF CO-RE-based tests and Clang version ======================================= A set of selftests use BPF target-specific built-ins, which might require bleeding-edge Clang versions (Clang 12 nightly at this time). Few sub-tests of core_reloc test suit (part of test_progs test runner) require the following built-ins, listed with corresponding Clang diffs introducing them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too old to support them, they shouldn't cause build failures or runtime test failures: - __builtin_btf_type_id() [0_, 1_, 2_]; - __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. .. _0: https://github.com/llvm/llvm-project/commit/6b01b465388b204d543da3cf49efd6080db094a9 .. _1: https://github.com/llvm/llvm-project/commit/072cde03aaa13a2c57acf62d79876bf79aa1919f .. _2: https://github.com/llvm/llvm-project/commit/00602ee7ef0bf6c68d690a2bd729c12b95c95c99 .. _3: https://github.com/llvm/llvm-project/commit/6d218b4adb093ff2e9764febbbc89f429412006c .. _4: https://github.com/llvm/llvm-project/commit/6d6750696400e7ce988d66a1a00e1d0cb32815f8 Floating-point tests and Clang version ====================================== Certain selftests, e.g. core_reloc, require support for the floating-point types, which was introduced in `Clang 13`__. The older Clang versions will either crash when compiling these tests, or generate an incorrect BTF. __ https://github.com/llvm/llvm-project/commit/a7137b238a07d9399d3ae96c0b461571bd5aa8b2 Kernel function call test and Clang version =========================================== Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support to generate extern function in BTF. It was introduced in `Clang 13`__. Without it, the error from compiling bpf selftests looks like: .. code-block:: console libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 __ https://github.com/llvm/llvm-project/commit/886f9ff53155075bd5f1e994f17b85d1e1b7470c btf_tag test and Clang version ============================== The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_]. The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires pahole version ``1.23``. Without them, the btf_tag selftest will be skipped and you will observe: .. code-block:: console #<test_num> btf_tag:SKIP .. _0: https://github.com/llvm/llvm-project/commit/a162b67c98066218d0d00aa13b99afb95d9bb5e6 .. _1: https://github.com/llvm/llvm-project/commit/3466e00716e12e32fdb100e3fcfca5c2b3e8d784 Clang dependencies for static linking tests =========================================== linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to generate valid BTF information for weak variables. Please make sure you use Clang that contains the fix. __ https://github.com/llvm/llvm-project/commit/968292cb93198442138128d850fd54dc7edc0035 Clang relocation changes ======================== Clang 13 patch `clang reloc patch`_ made some changes on relocations such that existing relocation types are broken into more types and each new type corresponds to only one way to resolve relocation. See `kernel llvm reloc`_ for more explanation and some examples. Using clang 13 to compile old libbpf which has static linker support, there will be a compilation failure:: libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. To fix this issue, user newer libbpf. .. Links .. _clang reloc patch: https://github.com/llvm/llvm-project/commit/6a2ea84600ba4bd3b2733bd8f08f5115eb32164b .. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst Clang dependencies for the u32 spill test (xdpwall) =================================================== The xdpwall selftest requires a change in `Clang 14`__. Without it, the xdpwall selftest will fail and the error message from running test_progs will look like: .. code-block:: console test_xdpwall:FAIL:Does LLVM have https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5? unexpected error: -4007 __ https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5