tools/include/linux/kernel.h and kselftest_harness.h are missing
ifndef guard around ARRAY_SIZE define. Fix them to avoid duplicate
define errors during compile when another file defines it. This
problem was found when compiling selftests that include a header
with ARRAY_SIZE define.
ARRAY_SIZE is defined in several selftests. There are about 25+
duplicate defines in various selftests source and header files.
Add ARRAY_SIZE to kselftest.h in preparation for removing duplicate
ARRAY_SIZE defines from individual test files.
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Test can fail either immediately when ASSERT() failed or at the
end if one or more EXPECT() was not met. The exact return code
is decided based on the number of successful ASSERT()s.
If test has no ASSERT()s, however, the return code will be 0,
as if the test did not fail. Start counting ASSERT()s from 1.
Fixes: 369130b631 ("selftests: Enhance kselftest_harness.h to print which assert failed")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The kernel-doc markups on this file are weird: they don't
follow what's specified at:
Documentation/doc-guide/kernel-doc.rst
In particular, markups should use this format:
identifier - description
and not this:
identifier(args)
The way the definitions are inside this file cause the
parser to completely miss the identifier name of each
function.
This prevents improving the script to do some needed validation
tests.
Address this part. Yet, furter changes are needed in order
for it to fully follow the specs.
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/8383758160fdb4fcbb2ac56beeb874ca6dffc6b9.1610610937.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This Kselftest fixes update for Linux 5.10-rc3 consists of fixes to
ftrace test and several fixes from Tommi Rantala for several tests.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAl+lhLcACgkQCwJExA0N
QxwU6w/+O3riAjo+VscPD4dtC7BN3OwARBhGe0CgUuxOmrQrWdftZCpSA4PLw1xm
Iu0eDhZ+utfSZDyyBavkOftgVfV/VZC4hvFo2f5PG5QPc2tLkNDCFrCRXNhx1CU2
lyb4HEAfMfRknrS4xL4S/OmQLJmMpT4dqp5kE5fdtgOME/OhO7ZHy5P5cKXmAL+F
VkJGdbQ2q24jmYRvwlV9++gTfCJT14/364ET2ENTfsy4caHu4MU1te7Gz4T4kGGA
+JXWa7x3izPqVxgEnsPG8FFmD+7I/iJ9I2HWSZu/qoIYZ3q07I8kEIH7ap+Kb7Oj
jMO7ohXTmBQPNActaOOpcvUsUrIoQdFeZKdA3kk9jHQAelsiqlv/iastoAv3zThe
bc0SS5aF903qiE34EHsbnaL+TmEJfnBMTF2ugTSBkSuiDpUCEirCCX5O42ZR1hO+
utvVSL7joyDQjMf1LnBDSDzOehfv9PRAG6M859IE/p3kIHmGUXztLj+nGTgoSN+W
8DOcYsKv1RkGqGEfqjOwlFyv92bkOP7DzkLQGgwqAWGJQx7Nybjq7fJzXWGsPlCY
edF0TI+sFNv1Pk26H8BT9/dECbqi5SYHH2H+J5bhw+OtVyC319WnmO2r5JvfsIqd
+k11GB+qiosA/Tr3QDR5ROxYkTc0+ytMp5eibONtVv+QjZJ0rfE=
=A6KY
-----END PGP SIGNATURE-----
Merge tag 'linux-kselftest-fixes-5.10-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull Kselftest fixes from Shuah Khan:
"Fixes to the ftrace test and several fixes from Tommi Rantala for
various other tests"
* tag 'linux-kselftest-fixes-5.10-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
selftests: binderfs: use SKIP instead of XFAIL
selftests: clone3: use SKIP instead of XFAIL
selftests: core: use SKIP instead of XFAIL in close_range_test.c
selftests: proc: fix warning: _GNU_SOURCE redefined
selftests: pidfd: drop needless linux/kcmp.h inclusion in pidfd_setns_test.c
selftests: pidfd: add CONFIG_CHECKPOINT_RESTORE=y to config
selftests: pidfd: skip test on kcmp() ENOSYS
selftests: pidfd: use ksft_test_result_skip() when skipping test
selftests/harness: prettify SKIP message whitespace again
selftests: pidfd: fix compilation errors due to wait.h
selftests: filter kselftest headers from command in lib.mk
selftests/ftrace: check for do_sys_openat2 in user-memory test
selftests/ftrace: Use $FUNCTION_FORK to reference kernel fork function
The kernel-doc markups there is violating the expected
syntax, causing it to not parse the name of the
markup identifier properly, preventing it to check
if the kernel-doc matches the #define below each
markup.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/697640045663f1366beb15e76e78b420dac5f5a2.1603791716.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Commit 9847d24af9 ("selftests/harness: Refactor XFAIL into SKIP")
replaced XFAIL with SKIP in the output. Add one more space to make the
output aligned and pretty again.
Fixes: 9847d24af9 ("selftests/harness: Refactor XFAIL into SKIP")
Signed-off-by: Tommi Rantala <tommi.t.rantala@nokia.com>
Acked-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The test harness forks() a child to run each test. Both the parent and
the child print to stdout using libc functions. That can lead to
duplicated (or more) output if the libc buffers are not flushed before
forking.
It's generally not seen when running programs directly, because stdout
will usually be line buffered when it's pointing to a terminal.
This was noticed when running the seccomp_bpf test, eg:
$ ./seccomp_bpf | tee test.log
$ grep -c "TAP version 13" test.log
2
But we only expect the TAP header to appear once.
It can be exacerbated using stdbuf to increase the buffer size:
$ stdbuf -o 1MB ./seccomp_bpf > test.log
$ grep -c "TAP version 13" test.log
13
The fix is simple, we just flush stdout & stderr before fork. Usually
stderr is unbuffered, but that can be changed, so flush it as well
just to be safe.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Tested-by: Max Filippov <jcmvbkbc@gmail.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
When the selftest "step" counter grew beyond 255, non-fatal warnings
were being emitted, which is noisy and pointless. There are selftests
with more than 255 steps (especially those in loops, etc). Instead,
just cap "steps" to 254 and do not report the saturation.
Reported-by: Ralph Campbell <rcampbell@nvidia.com>
Tested-by: Ralph Campbell <rcampbell@nvidia.com>
Fixes: 9847d24af9 ("selftests/harness: Refactor XFAIL into SKIP")
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The FIXTURE*() macro kern-doc examples had the wrong names for the C code
examples associated with them. Fix those and clarify that FIXTURE_DATA()
usage should be avoided.
Cc: Shuah Khan <shuah@kernel.org>
Cc: Jakub Kicinski <kuba@kernel.org>
Fixes: 74bc7c97fa ("kselftest: add fixture variants")
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Use a share memory segment to pass string information between forked
test and the test runner for the skip reason.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Since forever the harness output for signed value tests have reported
unsigned values to avoid casting. Instead, actually test the variable
types and perform the correct casts and choose the correct format
specifiers.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Using the kselftest_harness.h would result in non-TAP test reporting,
which didn't make much sense given that all the requirements for using
the low-level API were met. Switch to using ksft_*() helpers while
retaining as much of a human-readability as possible.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Allow users to build parameterized variants of fixtures.
If fixtures want variants, they call FIXTURE_VARIANT() to declare
the structure to fill for each variant. Each fixture will be re-run
for each of the variants defined by calling FIXTURE_VARIANT_ADD()
with the differing parameters initializing the structure.
Since tests are being re-run, additional initialization (steps,
no_print) is also added.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Now that all tests have a fixture object move from a global
list of tests to a list of tests per fixture.
Order of tests may change as we will now group and run test
fixture by fixture, rather than in declaration order.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Grouping tests by fixture will allow us to parametrize
test runs. Create full objects for fixtures.
Add a "global" fixture for tests without a fixture.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Kees suggest to factor out the list append code to a macro,
since following commits need it, which leads to code duplication.
Suggested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
There a few identical spelling mistakes, fix these.
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
When a selftest would timeout before, the program would just fall over
and no accounting of failures would be reported (i.e. it would result in
an incomplete TAP report). Instead, add an explicit SIGALRM handler to
cleanly catch and report the timeout.
Before:
[==========] Running 2 tests from 2 test cases.
[ RUN ] timeout.finish
[ OK ] timeout.finish
[ RUN ] timeout.too_long
Alarm clock
After:
[==========] Running 2 tests from 2 test cases.
[ RUN ] timeout.finish
[ OK ] timeout.finish
[ RUN ] timeout.too_long
timeout.too_long: Test terminated by timeout
[ FAIL ] timeout.too_long
[==========] 1 / 2 tests passed.
[ FAILED ]
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
In order to better handle timeout failures, rearrange the child waiting
logic into a separate function. This is mostly a copy/paste with an
indentation change. To handle pid tracking, a new field is added for
the child pid. Also move the alarm() pairing into the function.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Based on 1 normalized pattern(s):
use of this source code is governed by the gplv2 license
extracted by the scancode license scanner the SPDX license identifier
GPL-2.0-only
has been chosen to replace the boilerplate/reference in 2 file(s).
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081204.507272547@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Commit a745f7af3c ("selftests/harness: Add 30 second timeout per test")
adds an hardcoded 30s timeout to all tests. Unfortunately, rtctest has two
tests taking up to 60s. Allow for individual tests to define their own
timeout.
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
In order to keep tests from hanging forever, this adds an alarm signal
to each test run. This assumes an individual test doesn't take longer
than 30 seconds.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The harness was still using old-style GNU named initializer syntax.
Fix this so Clang will stop warning:
seccomp_bpf.c:2924:1: warning: use of GNU old-style field designator extension
[-Wgnu-designator]
./../kselftest_harness.h:147:25: note: expanded from macro 'TEST'
^
./../kselftest_harness.h:172:5: note: expanded from macro '__TEST_IMPL'
fn: &test_name, termsig: _signal }; \
^
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Shuah Khan <shuah@kernel.org>
I've stumbled over the current macro-expand behaviour of the test
harness:
$ gcc -Wall -xc - <<'__EOF__'
TEST(macro) {
int status = 0;
ASSERT_TRUE(WIFSIGNALED(status));
}
TEST_HARNESS_MAIN
__EOF__
$ ./a.out
[==========] Running 1 tests from 1 test cases.
[ RUN ] global.macro
<stdin>:4:global.macro:Expected 0 (0) != (((signed char) (((status) & 0x7f) + 1) >> 1) > 0) (0)
global.macro: Test terminated by assertion
[ FAIL ] global.macro
[==========] 0 / 1 tests passed.
[ FAILED ]
With this change the output of the same test looks much more
comprehensible:
[==========] Running 1 tests from 1 test cases.
[ RUN ] global.macro
<stdin>:4:global.macro:Expected 0 (0) != WIFSIGNALED(status) (0)
global.macro: Test terminated by assertion
[ FAIL ] global.macro
[==========] 0 / 1 tests passed.
[ FAILED ]
The issue is very similar to the bug fixed in glibc assert(3)
three years ago:
https://sourceware.org/bugzilla/show_bug.cgi?id=18604
Cc: Shuah Khan <shuah@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Will Drewry <wad@chromium.org>
Cc: linux-kselftest@vger.kernel.org
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <shuah@kernel.org>
Since seccomp_get_metadata() depends on CHECKPOINT_RESTORE, XFAIL the
test if the ptrace reports it as missing.
Signed-off-by: Kees Cook <keescook@chromium.org>
Acked-by: Tycho Andersen <tycho@tycho.ws>
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
When a test process is not able to write to TH_LOG_STREAM, this step
mechanism enable to print the assert number which triggered the failure.
This can be enabled by setting _metadata->no_print to true at the
beginning of the test sequence.
Update the seccomp-bpf test to return 0 if a test succeeded.
This feature is needed for the Landlock tests.
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Kees Cook <keescook@chromium.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Will Drewry <wad@chromium.org>
Link: https://lkml.kernel.org/r/CAGXu5j+D-FP8Kt9unNOqKrQJP4DYTpmgkJxWykZyrYiVPz3Y3Q@mail.gmail.com
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Add ReST metadata to kselftest_harness.h to be able to include the
comments in the Sphinx documentation.
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kees Cook <keescook@chromium.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Will Drewry <wad@chromium.org>
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
The seccomp/test_harness.h file contains useful helpers to build tests.
Moving it to the selftest directory should benefit to other test
components.
Keep seccomp maintainers for this file.
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Acked-by: Kees Cook <keescook@chromium.org>
Acked-by: Will Drewry <wad@chromium.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Shuah Khan <shuah@kernel.org>
Link: https://lkml.kernel.org/r/CAGXu5j+8CVz8vL51DRYXqOY=xc3zuKFf=PTENe88XYHzFYidUQ@mail.gmail.com
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
2017-06-07 10:07:21 -06:00
Renamed from tools/testing/selftests/seccomp/test_harness.h (Browse further)
