mirrors/linux-stable
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git synced 2024-10-31 16:38:12 +00:00
Code Issues Releases Wiki Activity
linux-stable/tools/testing/selftests/bpf/verifier/atomic_cmpxchg.c

122 lines
3.5 KiB
C
Raw Normal View History

bpf: Add tests for new BPF atomic operations The prog_test that's added depends on Clang/LLVM features added by Yonghong in commit 286daafd6512 (was https://reviews.llvm.org/D72184). Note the use of a define called ENABLE_ATOMICS_TESTS: this is used to: - Avoid breaking the build for people on old versions of Clang - Avoid needing separate lists of test objects for no_alu32, where atomics are not supported even if Clang has the feature. The atomics_test.o BPF object is built unconditionally both for test_progs and test_progs-no_alu32. For test_progs, if Clang supports atomics, ENABLE_ATOMICS_TESTS is defined, so it includes the proper test code. Otherwise, progs and global vars are defined anyway, as stubs; this means that the skeleton user code still builds. The atomics_test.o userspace object is built once and used for both test_progs and test_progs-no_alu32. A variable called skip_tests is defined in the BPF object's data section, which tells the userspace object whether to skip the atomics test. Signed-off-by: Brendan Jackman <jackmanb@google.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20210114181751.768687-11-jackmanb@google.com
2021-01-14 18:17:50 +00:00
{
"atomic compare-and-exchange smoketest - 64bit",
.insns = {
/* val = 3; */
BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 3),
/* old = atomic_cmpxchg(&val, 2, 4); */
BPF_MOV64_IMM(BPF_REG_1, 4),
BPF_MOV64_IMM(BPF_REG_0, 2),
BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_10, BPF_REG_1, -8),
/* if (old != 3) exit(2); */
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV64_IMM(BPF_REG_0, 2),
BPF_EXIT_INSN(),
/* if (val != 3) exit(3); */
BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV64_IMM(BPF_REG_0, 3),
BPF_EXIT_INSN(),
/* old = atomic_cmpxchg(&val, 3, 4); */
BPF_MOV64_IMM(BPF_REG_1, 4),
BPF_MOV64_IMM(BPF_REG_0, 3),
BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_10, BPF_REG_1, -8),
/* if (old != 3) exit(4); */
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV64_IMM(BPF_REG_0, 4),
BPF_EXIT_INSN(),
/* if (val != 4) exit(5); */
BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 4, 2),
BPF_MOV64_IMM(BPF_REG_0, 5),
BPF_EXIT_INSN(),
/* exit(0); */
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
.result = ACCEPT,
},
{
"atomic compare-and-exchange smoketest - 32bit",
.insns = {
/* val = 3; */
BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 3),
/* old = atomic_cmpxchg(&val, 2, 4); */
BPF_MOV32_IMM(BPF_REG_1, 4),
BPF_MOV32_IMM(BPF_REG_0, 2),
BPF_ATOMIC_OP(BPF_W, BPF_CMPXCHG, BPF_REG_10, BPF_REG_1, -4),
/* if (old != 3) exit(2); */
BPF_JMP32_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV32_IMM(BPF_REG_0, 2),
BPF_EXIT_INSN(),
/* if (val != 3) exit(3); */
BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_10, -4),
BPF_JMP32_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV32_IMM(BPF_REG_0, 3),
BPF_EXIT_INSN(),
/* old = atomic_cmpxchg(&val, 3, 4); */
BPF_MOV32_IMM(BPF_REG_1, 4),
BPF_MOV32_IMM(BPF_REG_0, 3),
BPF_ATOMIC_OP(BPF_W, BPF_CMPXCHG, BPF_REG_10, BPF_REG_1, -4),
/* if (old != 3) exit(4); */
BPF_JMP32_IMM(BPF_JEQ, BPF_REG_0, 3, 2),
BPF_MOV32_IMM(BPF_REG_0, 4),
BPF_EXIT_INSN(),
/* if (val != 4) exit(5); */
BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_10, -4),
BPF_JMP32_IMM(BPF_JEQ, BPF_REG_0, 4, 2),
BPF_MOV32_IMM(BPF_REG_0, 5),
BPF_EXIT_INSN(),
/* exit(0); */
BPF_MOV32_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
.result = ACCEPT,
},
{
"Can't use cmpxchg on uninit src reg",
.insns = {
BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 3),
BPF_MOV64_IMM(BPF_REG_0, 3),
BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_10, BPF_REG_2, -8),
BPF_EXIT_INSN(),
},
.result = REJECT,
.errstr = "!read_ok",
},
{
"Can't use cmpxchg on uninit memory",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 3),
BPF_MOV64_IMM(BPF_REG_2, 4),
BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_10, BPF_REG_2, -8),
BPF_EXIT_INSN(),
},
.result = REJECT,
.errstr = "invalid read from stack",
},
bpf: Explicitly zero-extend R0 after 32-bit cmpxchg As pointed out by Ilya and explained in the new comment, there's a discrepancy between x86 and BPF CMPXCHG semantics: BPF always loads the value from memory into r0, while x86 only does so when r0 and the value in memory are different. The same issue affects s390. At first this might sound like pure semantics, but it makes a real difference when the comparison is 32-bit, since the load will zero-extend r0/rax. The fix is to explicitly zero-extend rax after doing such a CMPXCHG. Since this problem affects multiple archs, this is done in the verifier by patching in a BPF_ZEXT_REG instruction after every 32-bit cmpxchg. Any archs that don't need such manual zero-extension can do a look-ahead with insn_is_zext to skip the unnecessary mov. Note this still goes on top of Ilya's patch: https://lore.kernel.org/bpf/20210301154019.129110-1-iii@linux.ibm.com/T/#u Differences v5->v6[1]: - Moved is_cmpxchg_insn and ensured it can be safely re-used. Also renamed it and removed 'inline' to match the style of the is_*_function helpers. - Fixed up comments in verifier test (thanks for the careful review, Martin!) Differences v4->v5[1]: - Moved the logic entirely into opt_subreg_zext_lo32_rnd_hi32, thanks to Martin for suggesting this. Differences v3->v4[1]: - Moved the optimization against pointless zext into the correct place: opt_subreg_zext_lo32_rnd_hi32 is called _after_ fixup_bpf_calls. Differences v2->v3[1]: - Moved patching into fixup_bpf_calls (patch incoming to rename this function) - Added extra commentary on bpf_jit_needs_zext - Added check to avoid adding a pointless zext(r0) if there's already one there. Difference v1->v2[1]: Now solved centrally in the verifier instead of specifically for the x86 JIT. Thanks to Ilya and Daniel for the suggestions! [1] v5: https://lore.kernel.org/bpf/CA+i-1C3ytZz6FjcPmUg5s4L51pMQDxWcZNvM86w4RHZ_o2khwg@mail.gmail.com/T/#t v4: https://lore.kernel.org/bpf/CA+i-1C3ytZz6FjcPmUg5s4L51pMQDxWcZNvM86w4RHZ_o2khwg@mail.gmail.com/T/#t v3: https://lore.kernel.org/bpf/08669818-c99d-0d30-e1db-53160c063611@iogearbox.net/T/#t v2: https://lore.kernel.org/bpf/08669818-c99d-0d30-e1db-53160c063611@iogearbox.net/T/#t v1: https://lore.kernel.org/bpf/d7ebaefb-bfd6-a441-3ff2-2fdfe699b1d2@iogearbox.net/T/#t Reported-by: Ilya Leoshkevich <iii@linux.ibm.com> Fixes: 5ffa25502b5a ("bpf: Add instructions for atomic_[cmp]xchg") Signed-off-by: Brendan Jackman <jackmanb@google.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Acked-by: Ilya Leoshkevich <iii@linux.ibm.com> Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2021-03-05 02:56:46 +00:00
{
"BPF_W cmpxchg should zero top 32 bits",
.insns = {
/* r0 = U64_MAX; */
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_ALU64_IMM(BPF_SUB, BPF_REG_0, 1),
/* u64 val = r0; */
BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_0, -8),
/* r0 = (u32)atomic_cmpxchg((u32 *)&val, r0, 1); */
BPF_MOV32_IMM(BPF_REG_1, 1),
BPF_ATOMIC_OP(BPF_W, BPF_CMPXCHG, BPF_REG_10, BPF_REG_1, -8),
/* r1 = 0x00000000FFFFFFFFull; */
BPF_MOV64_IMM(BPF_REG_1, 1),
BPF_ALU64_IMM(BPF_LSH, BPF_REG_1, 32),
BPF_ALU64_IMM(BPF_SUB, BPF_REG_1, 1),
/* if (r0 != r1) exit(1); */
BPF_JMP_REG(BPF_JEQ, BPF_REG_0, BPF_REG_1, 2),
BPF_MOV32_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
/* exit(0); */
BPF_MOV32_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
.result = ACCEPT,
},
Reference in a new issue Copy permalink