kunit_tool's --alltests option was changed in commit
980ac3ad05 ("kunit: tool: rename all_test_uml.config, use it for --alltests")
to use a manually curated list of architecture-indpendent Kconfig
options, rather than attempting to use make allyesconfig on UML, which
was broken.
Update the kunit_tool documentation to reflect the new behaviour of
--alltests.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The numbered list contains full path to every files that need to be
modified or created in order to implement misc-example kunit test.
Except for .kunitconfig. Which might make a newcommer confused about
where the file exists. Since there are multiple .kunitconfig files.
Fix this by using the full path to .kunitconfig.
Signed-off-by: Khalid Masum <khalid.masum.92@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Replace out-of-date external links with references to the kernel
documentation, replacing TAP webpage for the more appropriate KTAP
documentation and the UML webpage by its documentation.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Describe the objective of the Getting Started page, which should be a
brief and beginner-friendly walkthrough for running and writing tests,
showing the reader where to find detailed instructions in other pages.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by:Sadiya Kazi<Sadiaykazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run"
covering the current alternatives for editing configs and glob-filtering
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by:Sadiya Kazi <Sadiyakazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The "Getting Started" guide should be beginner-friendly, therefore
add a note about the requirement of a clean source tree when running
kunit_tool for the first time, and its related error.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Delete "kunit-tool.rst" to remove repeated info from KUnit docs.
"What is kunit_tool?" was integrated into index.rst, the remaining
sections were moved into run_wrapper.rst and renamed as follows:
"What is a .kunitconfig?" -> "Creating a ``.kunitconfig`` file"
"Getting Started with kunit_tool" -> "Running tests with kunit_tool"
"Configuring, Building, and Running Tests" ->
"Configuring, building, and running tests"
"Running Tests on QEMU" -> "Running tests on QEMU"
"Parsing Test Results" -> "Parsing test results"
"Filtering Tests" -> "Filtering tests"
"Other Useful Options" -> "Running command-line arguments"
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Sadiya Kazi <sadiyakazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Add an organic link to the "other kinds of tests" in the index page
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The section was rewritten but its anchor got left behind.
Fix the anchor and add some references to running on QEMU.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Use the same wording when citing and describing Kunit parts.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Sadiya Kazi <sadiyakazi@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The Parameterized Testing example contains a compilation error, as the
signature for the description helper function is void(*)(const struct
sha1_test_case *, char *), and the struct is non-const. This is
warned by Clang:
error: initialization of ‘void (*)(struct sha1_test_case *, char *)’
from incompatible pointer type ‘void (*)(const struct sha1_test_case *,
char *)’ [-Werror=incompatible-pointer-types]
33 | KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc);
| ^~~~~~~~~~~~
../include/kunit/test.h:1339:70: note: in definition of macro
‘KUNIT_ARRAY_PARAM’
1339 | void
(*__get_desc)(typeof(__next), char *) = get_desc; \
Signed-off-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Without the quoting, the example will mess up invocations like
$ run_kunit "Something with spaces"
Note: this example isn't valid, but if ever a usecase arises where a
flag argument might have spaces in it, it'll break.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The "Run Tests on qemu" section of run_wrapper.rst had a few issues left
over from the last big documentation refactor[1]:
- It referenced a non_uml.rst page, which was integrated into the other
pages (including run_wrapper.rst).
- It skimmed over the use of --arch= and --cross_compile= in favour of
using a custom --qemu_config. Since most users will want to use the
former, let's give examples.
Remove the reference to the non-existant page, and add a couple of
examples to encourage the use of --arch= and --cross_compile=.
With this change, there should be no more broken references in the KUnit
documentation (i.e., the one mentioned in [2] is gone).
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=46201d47d6c4be594c1d57b7f3251c371626a9c4
[2]: https://lore.kernel.org/linux-doc/cover.1656234456.git.mchehab@kernel.org/
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Now that kunit.py's --kunitconfig is repeatable, let's create a file to
hold the various options needed to enable coverage under UML.
This can be used like so:
$ ./tools/testing/kunit/kunit.py run \
--kunitconfig=tools/testing/kunit/configs/all_tests_uml.config \
--kunitconfig=tools/testing/kunit/configs/coverage_uml.config \
--make_options=CC=/usr/bin/gcc-6
which on my system is enough to get coverage working [1].
This is still a clunky command, but far better than before.
[1] at the time of this commit, I get:
Overall coverage rate:
lines......: 11.6% (34112 of 295033 lines)
functions..: 15.3% (3721 of 24368 functions)
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
This KUnit update for Linux 5.19-rc1 consists of several fixes, cleanups,
and enhancements to tests and framework:
- introduces _NULL and _NOT_NULL macros to pointer error checks
- reworks kunit_resource allocation policy to fix memory leaks when
caller doesn't specify free() function to be used when allocating
memory using kunit_add_resource() and kunit_alloc_resource() funcs.
- adds ability to specify suite-level init and exit functions
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAmKLw4QACgkQCwJExA0N
Qxz9wRAA3PonJESDAFF2sXTDzQurEXdWoJHqNvO0JCObku8SDODEI7nozXOD0MBC
ASAXiX3HuNI0yESF27xECqu3xbe8KsYOtCN8vco/sYUroVGmzgAt/atsvrSUv2Oh
sEQbjrTMwkMUjL5ECvjR2dArd6bQew7PPBkl3HqOpyysL3b/EAMEAY0DmDXrrrwB
+oNvXGVAR1Tczg4ahcSSwDdZl1C41kREj5f8S/4+kohMdIjCUPWOAYnaWHpVdAOJ
C+LWkPSJ5IpgjU2urDX2kNfg32UxIJpFI009ovytBmwCbd+GEs24u7gtgtksPM2s
YypoPEqC40gxkbY99omojtADiDdZlKqlIipCTWYe/CpzgBD+WQ4PVqMGM4ZprP9w
Hrc6ulVmd8hZ4F9QQ3oN6W9L6pBCgdXtPPCsQtGoUTbw7r79BP67PjJ6Ko+usn3s
Jy0FR5LvzYBjykoJzKSIaJ8ONaX34DB6w5rB+q5mBGwPKPHWo3eAZVZDPEMVo3Z7
D9TW5UliGBt2y5YJZbPbSnhdJPMPHSK5ef9hIy0wYjVJFafirdgrQhgbWbVxalRT
eZz1edcs1sdU7GAzfMA/v+NqAAA3bFIUVr2b+GTc+4zzWhq+cwI2SNikgyhETv/f
xKq8Xek8EkOIdaa2lu9chTPT4sG7A6991EkRqfc7rL1IptkPiS8=
=DzVQ
-----END PGP SIGNATURE-----
Merge tag 'linux-kselftest-kunit-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull KUnit updates from Shuah Khan:
"Several fixes, cleanups, and enhancements to tests and framework:
- introduce _NULL and _NOT_NULL macros to pointer error checks
- rework kunit_resource allocation policy to fix memory leaks when
caller doesn't specify free() function to be used when allocating
memory using kunit_add_resource() and kunit_alloc_resource() funcs.
- add ability to specify suite-level init and exit functions"
* tag 'linux-kselftest-kunit-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (41 commits)
kunit: tool: Use qemu-system-i386 for i386 runs
kunit: fix executor OOM error handling logic on non-UML
kunit: tool: update riscv QEMU config with new serial dependency
kcsan: test: use new suite_{init,exit} support
kunit: tool: Add list of all valid test configs on UML
kunit: take `kunit_assert` as `const`
kunit: tool: misc cleanups
kunit: tool: minor cosmetic cleanups in kunit_parser.py
kunit: tool: make parser stop overwriting status of suites w/ no_tests
kunit: tool: remove dead parse_crash_in_log() logic
kunit: tool: print clearer error message when there's no TAP output
kunit: tool: stop using a shell to run kernel under QEMU
kunit: tool: update test counts summary line format
kunit: bail out of test filtering logic quicker if OOM
lib/Kconfig.debug: change KUnit tests to default to KUNIT_ALL_TESTS
kunit: Rework kunit_resource allocation policy
kunit: fix debugfs code to use enum kunit_status, not bool
kfence: test: use new suite_{init/exit} support, add .kunitconfig
kunit: add ability to specify suite-level init and exit functions
kunit: rename print_subtest_{start,end} for clarity (s/subtest/suite)
...
- After a long period of inactivity, the Japanese translations are seeing
some much-needed maintenance and updating.
- Reworked IOMMU documentation
- Some new documentation for static-analysis tools
- A new overall structure for the memory-management documentation. This
is an LSFMM outcome that, it is hoped, will help encourage developers to
fill in the many gaps. Optimism is eternal...but hopefully it will
work.
- More Chinese translations.
Plus the usual typo fixes, updates, etc.
-----BEGIN PGP SIGNATURE-----
iQFDBAABCAAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAmKLqZQPHGNvcmJldEBs
d24ubmV0AAoJEBdDWhNsDH5YdgQH/2/9+EgQDes93f/+iKtbO23EV67392dwrmXS
kYg8lR4948/Q3jzgMloUo6hNOoxXeV/sqmdHu0LjUhFN+BGsp9fFjd/jp0XhWcqA
nnc9foGbpmeFPxHeAg2aqV84eeasLoO5lUUm2rNoPBLd6HFV+IYC5R4VZ+w42StB
5bYEOYwHXMvQZXkivZDse82YmvQK3/2rRGTUoFhME/Aap6rFgWJJ+XQcSKA7WmwW
OpJqq+FOsjsxHe6IFVy6onzlqgGJM8zM2bLtqedid6yaE3uACcHMb/OyAjp0rdKF
BQvaG+d3f7DugABqM6Y1oU75iBtJWWYgGeAm36JtX+3mz2uR/f0=
=3UoR
-----END PGP SIGNATURE-----
Merge tag 'docs-5.19' of git://git.lwn.net/linux
Pull documentation updates from Jonathan Corbet:
"It was a moderately busy cycle for documentation; highlights include:
- After a long period of inactivity, the Japanese translations are
seeing some much-needed maintenance and updating.
- Reworked IOMMU documentation
- Some new documentation for static-analysis tools
- A new overall structure for the memory-management documentation.
This is an LSFMM outcome that, it is hoped, will help encourage
developers to fill in the many gaps. Optimism is eternal...but
hopefully it will work.
- More Chinese translations.
Plus the usual typo fixes, updates, etc"
* tag 'docs-5.19' of git://git.lwn.net/linux: (70 commits)
docs: pdfdocs: Add space for chapter counts >= 100 in TOC
docs/zh_CN: Add dev-tools/gdb-kernel-debugging.rst Chinese translation
input: Docs: correct ntrig.rst typo
input: Docs: correct atarikbd.rst typos
MAINTAINERS: Become the docs/zh_CN maintainer
docs/zh_CN: fix devicetree usage-model translation
mm,doc: Add new documentation structure
Documentation: drop more IDE boot options and ide-cd.rst
Documentation/process: use scripts/get_maintainer.pl on patches
MAINTAINERS: Add entry for DOCUMENTATION/JAPANESE
docs/trans/ja_JP/howto: Don't mention specific kernel versions
docs/ja_JP/SubmittingPatches: Request summaries for commit references
docs/ja_JP/SubmittingPatches: Add Suggested-by as a standard signature
docs/ja_JP/SubmittingPatches: Randy has moved
docs/ja_JP/SubmittingPatches: Suggest the use of scripts/get_maintainer.pl
docs/ja_JP/SubmittingPatches: Update GregKH links
Documentation/sysctl: document max_rcu_stall_to_panic
Documentation: add missing angle bracket in cgroup-v2 doc
Documentation: dev-tools: use literal block instead of code-block
docs/zh_CN: add vm numa translation
...
KUnit has support for setup/cleanup logic for each test case in a suite.
But it lacks the ability to specify setup/cleanup for the entire suite
itself.
This can be used to do setup that is too expensive or cumbersome to do
for each test.
Or it can be used to do simpler things like log debug information after
the suite completes.
It's a fairly common feature, so the lack of it is noticeable.
Some examples in other frameworks and languages:
* https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass
* https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite
Meta:
This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/
The changes from that patch:
* pass in `struct kunit *` so users can do stuff like
`kunit_info(suite, "debug message")`
* makes sure the init failure is bubbled up as a failure
* updates kunit-example-test.c to use a suite init
* Updates kunit/usage.rst to mention the new support
* some minor cosmetic things
* use `suite_{init,exit}` instead of `{init/exit}_suite`
* make suite init error message more consistent w/ test init
* etc.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Recent changes have made it so the current set is not sufficient.
Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
hopefully less fragile (esp. given we're tied to GCC 6 and lower).
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Background:
Currently, a reader looking at kunit/test.h will find the file is quite
long, and the first meaty comment is a doc comment about struct
kunit_resource.
Most users will not ever use the KUnit resource API directly.
They'll use kunit_kmalloc() and friends, or decide it's simpler to do
cleanups via labels (it often can be) instead of figuring out how to use
the API.
It's also logically separate from everything else in test.h.
Removing it from the file doesn't cause any compilation errors (since
struct kunit has `struct list_head resources` to store them).
This commit:
Let's move it into a kunit/resource.h file and give it a separate page
in the docs, kunit/api/resource.rst.
We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses `lock` inside `struct kunit` in a inline func
* so we can't just forward declare, and the alternatives require
uninlining the func, adding hepers to lock/unlock, or other more
invasive changes.
Now the first big comment in test.h is about kunit_case, which is a lot
more relevant to what a new user wants to know.
A side effect of this is git blame won't properly track history by
default, users need to run
$ git blame -L ,1 -C17 include/kunit/resource.h
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit ddbd60c779 ("kunit: use --build_dir=.kunit as default") changed
the default --build_dir, which had the side effect of making
`.kunitconfig` move to `.kunit/.kunitconfig`.
However, the first few lines of kunit/start.rst never got updated, oops.
Fix this by telling people to run kunit.py first, which will
automatically generate the .kunit directory and .kunitconfig file, and
then edit the file manually as desired.
Reported-by: Yifan Yuan <alpc_metic@live.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
fixes from Mauro and Akira; I also took the liberty of dropping in my
"messy diffstat" document.
-----BEGIN PGP SIGNATURE-----
iQFDBAABCAAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAmJFzSkPHGNvcmJldEBs
d24ubmV0AAoJEBdDWhNsDH5Yyp0H/jnK0edIgumAwGMBb2IeVImKsV8IU4Gdw1yo
RgetPAWP/i/KqR1JczLnwd+K/0KuOb1nifJO8MfCwN2GowlOdg5WuHkBfkUhoUy8
LK+Z1E7HN3riveBJKxRyOXTxvFkDVm8aD8s3C3NZjX1Tz8Xwe7v69YDJ3MCSBHZb
TxiffdSdBgDbL6A3nnH7fwUW94l0fv3RWhXU8ddW3i7rEmjafXlV+8MHniY1EGNz
r2Ej4J9/vdLIT37yl9vb28D45HdS937h4aQKhJJ7ZzLnwuOkgDUJ7LyL2GepATlN
v2nU0vge8125fDZQPGxJ00S5TITqN8PoQ9dkVRcOlzXR1n9Px70=
=Rbru
-----END PGP SIGNATURE-----
Merge tag 'docs-5.18-2' of git://git.lwn.net/linux
Pull more documentation updates from Jonathan Corbet:
"Some late-arriving documentation improvements.
This is mostly build-system fixes from Mauro and Akira; I also took
the liberty of dropping in my 'messy diffstat' document"
* tag 'docs-5.18-2' of git://git.lwn.net/linux:
docs: Add a document on how to fix a messy diffstat
docs: sphinx/requirements: Limit jinja2<3.1
Documentation: kunit: Fix cross-referencing warnings
scripts/kernel-doc: change the line number meta info
scripts/get_abi: change the file/line number meta info
docs: kernel_include.py: add sphinx build dependencies
docs: kernel_abi.py: add sphinx build dependencies
docs: kernel_feat.py: add build dependencies
scripts/get_feat.pl: allow output the parsed file names
docs: kfigure.py: Don't warn of missing PDF converter in 'make htmldocs'
Documentation: Fix duplicate statement about raw_spinlock_t type
The Architecture chapter of the KUnit documentation tried to include
copies of the kernel-doc for a couple of things, despite these already
existing in the API documentation. This lead to some warnings:
architecture:31: ./include/kunit/test.h:3: WARNING: Duplicate C declaration, also defined at dev-tools/kunit/api/test:66.
Declaration is '.. c:struct:: kunit_case'.
architecture:163: ./include/kunit/test.h:1217: WARNING: Duplicate C declaration, also defined at dev-tools/kunit/api/test:1217.
Declaration is '.. c:macro:: KUNIT_ARRAY_PARAM'.
architecture.rst:3: WARNING: Duplicate C declaration, also defined at dev-tools/kunit/api/test:66.
Declaration is '.. c:struct:: kunit_case'.
architecture.rst:1217: WARNING: Duplicate C declaration, also defined at dev-tools/kunit/api/test:1217.
Declaration is '.. c:macro:: KUNIT_ARRAY_PARAM'.
Get rid of these, and cleanup the mentions of the struct and macro in
question so that sphinx generates a link to the existing copy of the
documentation in the api/test document.
Fixes: bc145b370c ("Documentation: KUnit: Added KUnit Architecture")
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20220326054414.637293-1-davidgow@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
things still showed up:
- A documentation section for ARC processors
- Reworked and enhanced KUnit documentation
- The ability to pick your own theme for HTML builds; if the default
"Read the Docs" theme isn't ugly enough for you, you can now pick
an uglier one.
- More Chinese translation work
Plus the usual assortment of fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFDBAABCAAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAmHcgvUPHGNvcmJldEBs
d24ubmV0AAoJEBdDWhNsDH5YCkcH/0d/bDbjkwh/6fFleHJDic+jasuoAk4HTpKn
G3dMvEHDzYm2uXJiCZTaxagUtP8S8BdZ1cMrThZ0zcCni30+awvCmq/puK2zo3G3
urofj9FfjTjKBLOb8GFnLCITzKVeWlSojtSj6cLbd/bbMX4eCEliis0KexEBrKpF
rmlY7qVr0WeBiW0Wg7xXjTd0H2dCO9O+5IQ/BzSPDJOzxyBWGLg0B/mXcm+BeOJ7
MukRtR7e2QcAw6MIxUI4tc2yoeG8XssNbbIrmcya1xwsSiAtcxHv3V2qCrJqmAx1
8mK1q4Mot9zGcOcBQWJoRwotRuS8wqH0jr+XgRbg12AwEZc66zg=
=oPyM
-----END PGP SIGNATURE-----
Merge tag 'docs-5.17' of git://git.lwn.net/linux
Pull documentation updates from Jonathan Corbet:
"This isn't a hugely busy cycle for documentation, but a few
significant things still showed up:
- A documentation section for ARC processors
- Reworked and enhanced KUnit documentation
- The ability to pick your own theme for HTML builds; if the default
"Read the Docs" theme isn't ugly enough for you, you can now pick
an uglier one.
- More Chinese translation work
Plus the usual assortment of fixes and cleanups"
* tag 'docs-5.17' of git://git.lwn.net/linux: (53 commits)
scripts: sphinx-pre-install: Fix ctex support on Debian
docs: discourage use of list tables
docs: 5.Posting.rst: describe Fixes: and Link: tags
Documentation: kgdb: Replace deprecated remotebaud
docs: automarkup.py: Fix invalid HTML link output and broken URI fragments
Documentation: refer to config RANDOMIZE_BASE for kernel address-space randomization
Documentation: kgdb: properly capitalize the MAGIC_SYSRQ config
docs/zh_CN: Update and fix a couple of typos
scripts: sphinx-pre-install: add required ctex dependency
Documentation: KUnit: Restyled Frequently Asked Questions
Documentation: KUnit: Restyle Test Style and Nomenclature page
Documentation: KUnit: Rework writing page to focus on writing tests
Documentation: kunit: Reorganize documentation related to running tests
Documentation: KUnit: Added KUnit Architecture
Documentation: KUnit: Rewrite getting started
Documentation: KUnit: Rewrite main page
docs/zh_CN: Add zh_CN/accounting/delay-accounting.rst
Documentation/sphinx: fix typos of "its"
docs/zh_CN: Add sched-domains translation
doc: fs: remove bdev_try_to_free_page related doc
...
We now have dedicated pages on running tests. Therefore refocus the
usage page on writing tests and add content from tips page and
information on other architectures.
Signed-off-by: Harinder Singh <sharinder@google.com>
Reviewed-by: Tim Bird <tim.bird@sony.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-6-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Consolidate documentation running tests into two pages: "run tests with
kunit_tool" and "run tests without kunit_tool".
Signed-off-by: Harinder Singh <sharinder@google.com>
Reviewed-by: Tim Bird <tim.bird@sony.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-5-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Describe the components of KUnit and how the kernel mode parts
interact with kunit_tool.
Signed-off-by: Harinder Singh <sharinder@google.com>
Co-developed-by: Marco Elver <elver@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-4-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add a section on advantages of unit testing, how to write unit tests,
KUnit features and Prerequisites.
Signed-off-by: Harinder Singh <sharinder@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-2-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Problem: currently, if you remove something from your kunitconfig,
kunit.py will not regenerate the .config file.
The same thing happens if you did --kunitconfig_add=CONFIG_KASAN=y [1]
and then ran again without it. Your new run will still have KASAN.
The reason is that kunit.py won't regenerate the .config file if it's a
superset of the kunitconfig. This speeds it up a bit for iterating.
This patch adds an additional check that forces kunit.py to regenerate
the .config file if the current kunitconfig doesn't match the previous
one.
What this means:
* deleting entries from .kunitconfig works as one would expect
* dropping a --kunitconfig_add also triggers a rebuild
* you can still edit .config directly to turn on new options
We implement this by creating a `last_used_kunitconfig` file in the
build directory (so .kunit, by default) after we generate the .config.
When comparing the kconfigs, we compare python sets, so duplicates and
permutations don't trip us up.
The majority of this patch is adding unit tests for the existing logic
and for the new case where `last_used_kunitconfig` differs.
[1] https://lore.kernel.org/linux-kselftest/20211106013058.2621799-2-dlatypov@google.com/
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
KUnit does not have any first party support for "mocking".
The original RFC had some, but the code got dropped.
However, the documentation patches never got updated. This fixes that.
https://kunit.dev/mocking.html has a current writeup on the status quo
and will hopefully be eventually folded into the in-kernel
Documentation.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit 1d71307a6f ("kunit: add unit test for filtering suites by
names") introduced the ability to filter which suites we run via glob.
This change extends it so we can also filter individual test cases
inside of suites as well.
This is quite useful when, e.g.
* trying to run just the tests cases you've just added or are working on
* trying to debug issues with test hermeticity
Examples:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*exec*.parse*'
...
============================================================
======== [PASSED] kunit_executor_test ========
[PASSED] parse_filter_test
============================================================
Testing complete. 1 tests run. 0 failed. 0 crashed.
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*.no_matching_tests'
...
[ERROR] no tests run!
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
--raw_output is nice, but it would be nicer if could show only output
after KUnit tests have started.
So change the flag to allow specifying a string ('kunit').
Make it so `--raw_output` alone will default to `--raw_output=all` and
have the same original behavior.
Drop the small kunit_parser.raw_output() function since it feels wrong
to put it in "kunit_parser.py" when the point of it is to not parse
anything.
E.g.
$ ./tools/testing/kunit/kunit.py run --raw_output=kunit
...
[15:24:07] Starting KUnit Kernel ...
TAP version 14
1..1
# Subtest: example
1..3
# example_simple_test: initializing
ok 1 - example_simple_test
# example_skip_test: initializing
# example_skip_test: You should not see a line below.
ok 2 - example_skip_test # SKIP this test should be skipped
# example_mark_skipped_test: initializing
# example_mark_skipped_test: You should see a line below.
# example_mark_skipped_test: You should see this line.
ok 3 - example_mark_skipped_test # SKIP this test should be skipped
ok 1 - example
[15:24:10] Elapsed time: 6.487s total, 0.001s configuring, 3.510s building, 0.000s running
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit.py currently does not make it possible for users to specify module
parameters (/kernel arguments more generally) unless one directly tweaks
the kunit.py code itself.
This hasn't mattered much so far, but this would make it easier to port
existing tests that expose module parameters over to KUnit and/or let
current KUnit tests take advantage of them.
Tested using an kunit internal parameter:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit \
--kernel_args=kunit.filter_glob=kunit_status
...
Testing complete. 2 tests run. 0 failed. 0 crashed. 0 skipped.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit b6d5799b0b ("kunit: Add 'kunit_shutdown' option") changes KUnit
to call kernel_halt() by default when done testing.
This fixes the issue with not having .gcda files due to not calling
atexit() handlers, and therefore we can stop recommending people
manually tweak UML code.
The need to use older versions of GCC (<=6) remains however, due to
linktime issues, same as before. Note: There also might still be issues
with .gcda files as well in newer versions.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
This KUnit update for Linux 5.14-rc1 consists of fixes and features:
-- add support for skipped tests
-- introduce kunit_kmalloc_array/kunit_kcalloc() helpers
-- add gnu_printf specifiers
-- add kunit_shutdown
-- add unit test for filtering suites by names
-- convert lib/test_list_sort.c to use KUnit
-- code organization moving default config to tools/testing/kunit
-- refactor of internal parser input handling
-- cleanups and updates to documentation
-- code cleanup related to casts
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAmDfNHgACgkQCwJExA0N
QxzU5g/+ORbcbE5jhumo82xYUCGTScgIqPllNO000Vk9xydMqwx6tpY5puV0ig2z
5X7pxKEQTnype58yzY5mS1p336ryIx3q+AwEownxTW3YurXo2naQ59ZPjEvlAV+E
h+DMYLzEjrxzcyJATw02uC9YLUZ3w6FPJfiXViIv93YYrtcnM0u6JpwG0yfBlI4v
4KKB2Xu4K7T90C9/ADFYFKX3mjXQl5fQwvIdtA7wS90Cgq52LKp2mvg1XEiZE5d+
0dkTZ4Zo8TxxHt665o7vfnUjQQNmh45iGlW65wONxfAPb8BPoneGjVKWQTnN0Hor
W+93ZPbMuFMSSKJuoHY9U7sP5VySKvaiIYaGdi6prnZZu0zUabKnLZ6FOy7kEdfs
v09ulCBTVLslixVgNcp/kD9T+G/SXwCF5YBMAiMDQ0GNfUqlFtBkEA3gd44KwMI0
KwCcOgUSiaCkqyzOz/VeQsu/nhA5jdMO0KjiAs7Z3e7r7O/qKFs/ll7hZgDNCWSC
q8eIrcBkSL0EGgXR1iZ4AtGm8op6KKd4ACBM8NdtTyoGFl1npZOgZnHoIsy35G9K
9mhc7eXSoaDGqy9dONL1Tc8Neg7qLTXQNp2radqsnAAgNPUrJuC7+8YC+DdIsjBH
W7OyMjpfbwPws5rP4CS+JdwL+nQprKXZvFIhWGYhkDK44MbOngw=
=5QAv
-----END PGP SIGNATURE-----
Merge tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull KUnit update from Shuah Khan:
"Fixes and features:
- add support for skipped tests
- introduce kunit_kmalloc_array/kunit_kcalloc() helpers
- add gnu_printf specifiers
- add kunit_shutdown
- add unit test for filtering suites by names
- convert lib/test_list_sort.c to use KUnit
- code organization moving default config to tools/testing/kunit
- refactor of internal parser input handling
- cleanups and updates to documentation
- code cleanup related to casts"
* tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits)
kunit: add unit test for filtering suites by names
kasan: test: make use of kunit_skip()
kunit: test: Add example tests which are always skipped
kunit: tool: Support skipped tests in kunit_tool
kunit: Support skipped tests
thunderbolt: test: Reinstate a few casts of bitfields
kunit: tool: internal refactor of parser input handling
lib/test: convert lib/test_list_sort.c to use KUnit
kunit: introduce kunit_kmalloc_array/kunit_kcalloc() helpers
kunit: Remove the unused all_tests.config
kunit: Move default config from arch/um -> tools/testing/kunit
kunit: arch/um/configs: Enable KUNIT_ALL_TESTS by default
kunit: Add gnu_printf specifiers
lib/cmdline_kunit: Remove a cast which are no-longer required
kernel/sysctl-test: Remove some casts which are no-longer required
thunderbolt: test: Remove some casts which are no longer required
mmc: sdhci-of-aspeed: Remove some unnecessary casts from KUnit tests
iio: Remove a cast in iio-test-format which is no longer required
device property: Remove some casts in property-entry-test
Documentation: kunit: Clean up some string casts in examples
...
The default .kunitconfig file is currently kept in
arch/um/configs/kunit_defconfig, but -- with the impending QEMU patch
-- will no-longer be exclusively used for UML-based kernels.
Move it alongside the other KUnit configs in
tools/testing/kunit/configs, and give it a name which matches the
existing all_tests.config and broken_on_uml.config files.
Also update the Getting Started documentation to point to the new file.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
As the type checking is no longer excessively strict, get rid of the
unsightly (char*) casts -- and comment discussing them -- from the KUnit
usage page.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
We'll be replacing :doc:`foo` references to
Documentation/foo.rst. Yet, here it happens inside a table.
Doing a search-and-replace would break it.
Yet, as there's no good reason to use a table there,
let's just convert it into a list.
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The :doc:`foo` tag is auto-generated via automarkup.py.
So, use the filename at the sources, instead of :doc:`foo`.
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6fde409079959a95b62b9b2692503608d7ff0dbd.1623824363.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
We'll be replacing :doc:`foo` references to
Documentation/foo.rst. Yet, here it happens inside a table.
Doing a search-and-replace would break it.
Yet, as there's no good reason to use a table there,
let's just convert it into a list.
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/07d3a8ccafbb6345d6e78fb090290859e84361a1.1623824363.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Document QEMU support, what it does, and how to use it in kunit_tool.
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The kunit_tool documentation page was pretty minimal, and a bit
outdated. Update it and flesh it out a bit.
In particular,
- Mention that .kunitconfig is now in the build directory
- Describe the use of --kunitconfig to specify a different config
framgent
- Mention the split functionality (i.e., commands other than 'run')
- Describe --raw_output and kunit.py parse
- Mention the globbing support
- Provide a quick overview of other options, including --build_dir and
--alltests
Note that this does overlap a little with the new running_tips page. I
don't think it's a problem having both: this page is supposed to be a
bit more of a reference, rather than a list of useful tips, so the fact
that they both describe the same features isn't a problem.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
This is long overdue.
There are several things that aren't nailed down (in-tree
.kunitconfig's), or partially broken (GCOV on UML), but having them
documented, warts and all, is better than having nothing.
This covers a bunch of the more recent features
* kunit_filter_glob
* kunit.py run --kunitconfig
* slightly more detail on building tests as modules
* CONFIG_KUNIT_DEBUGFS
By my count, the only headline features now not mentioned are the KASAN
integration and KernelCI json output support (kunit.py run --json).
And then it also discusses how to get code coverage reports under UML
and non-UML since this is a question people have repeatedly asked.
Non-UML coverage collection is no different from normal, but we should
probably explicitly call this out.
As for UML, I was able to get it working again with two small hacks.*
E.g. with CONFIG_KUNIT=y && CONFIG_KUNIT_ALL_TESTS=y
Overall coverage rate:
lines......: 15.1% (18294 of 120776 lines)
functions..: 16.8% (1860 of 11050 functions)
Note: this doesn't document --alltests since this is not stable yet.
Hopefully being run more frequently as part of KernelCI will help...
*Using gcc/gcov-6 and not using uml_abort() in os_dump_core().
I've documented these hacks in "Notes" but left TODOs for
brendanhiggins@google.com who tracked down the runtime issue in GCC.
To be clear: these are not issues specific to KUnit, but rather to UML.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
