mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-09-30 14:19:16 +00:00
tools: tc-testing: Update README and TODO
Signed-off-by: Brenda J. Butler <bjb@mojatatu.com> Acked-by: Lucas Bates <lucasb@mojatatu.com> Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
Brenda J. Butler
David S. Miller
parent
c25e473686
commit
95ce14c3ff
2 changed files with 179 additions and 19 deletions
|
|
@ -14,11 +14,11 @@ REQUIREMENTS
|
|||
|
||||
* The kernel must have network namespace support
|
||||
|
||||
* The kernel must have veth support available, as a veth pair is created
|
||||
* The kernel must have veth support available, as a veth pair is created
|
||||
prior to running the tests.
|
||||
|
||||
* All tc-related features must be built in or available as modules.
|
||||
To check what is required in current setup run:
|
||||
* All tc-related features being tested must be built in or available as
|
||||
modules. To check what is required in current setup run:
|
||||
./tdc.py -c
|
||||
|
||||
Note:
|
||||
|
|
@ -44,10 +44,13 @@ using the -p option when running tdc:
|
|||
RUNNING TDC
|
||||
-----------
|
||||
|
||||
To use tdc, root privileges are required. tdc will not run otherwise.
|
||||
To use tdc, root privileges are required. This is because the
|
||||
commands being tested must be run as root. The code that enforces
|
||||
execution by root uid has been moved into a plugin (see PLUGIN
|
||||
ARCHITECTURE, below).
|
||||
|
||||
All tests are executed inside a network namespace to prevent conflicts
|
||||
within the host.
|
||||
If nsPlugin is linked, all tests are executed inside a network
|
||||
namespace to prevent conflicts within the host.
|
||||
|
||||
Running tdc without any arguments will run all tests. Refer to the section
|
||||
on command line arguments for more information, or run:
|
||||
|
|
@ -59,6 +62,33 @@ output captured from the failing test will be printed immediately following
|
|||
the failed test in the TAP output.
|
||||
|
||||
|
||||
OVERVIEW OF TDC EXECUTION
|
||||
-------------------------
|
||||
|
||||
One run of tests is considered a "test suite" (this will be refined in the
|
||||
future). A test suite has one or more test cases in it.
|
||||
|
||||
A test case has four stages:
|
||||
|
||||
- setup
|
||||
- execute
|
||||
- verify
|
||||
- teardown
|
||||
|
||||
The setup and teardown stages can run zero or more commands. The setup
|
||||
stage does some setup if the test needs it. The teardown stage undoes
|
||||
the setup and returns the system to a "neutral" state so any other test
|
||||
can be run next. These two stages require any commands run to return
|
||||
success, but do not otherwise verify the results.
|
||||
|
||||
The execute and verify stages each run one command. The execute stage
|
||||
tests the return code against one or more acceptable values. The
|
||||
verify stage checks the return code for success, and also compares
|
||||
the stdout with a regular expression.
|
||||
|
||||
Each of the commands in any stage will run in a shell instance.
|
||||
|
||||
|
||||
USER-DEFINED CONSTANTS
|
||||
----------------------
|
||||
|
||||
|
|
@ -70,23 +100,132 @@ executed as part of the test. More will be added as test cases require.
|
|||
Example:
|
||||
$TC qdisc add dev $DEV1 ingress
|
||||
|
||||
The NAMES values are used to substitute into the commands in the test cases.
|
||||
|
||||
|
||||
COMMAND LINE ARGUMENTS
|
||||
----------------------
|
||||
|
||||
Run tdc.py -h to see the full list of available arguments.
|
||||
|
||||
-p PATH Specify the tc executable located at PATH to be used on this
|
||||
test run
|
||||
-c Show the available test case categories in this test file
|
||||
-c CATEGORY Run only tests that belong to CATEGORY
|
||||
-f FILE Read test cases from the JSON file named FILE
|
||||
-l [CATEGORY] List all test cases in the JSON file. If CATEGORY is
|
||||
specified, list test cases matching that category.
|
||||
-s ID Show the test case matching ID
|
||||
-e ID Execute the test case identified by ID
|
||||
-i Generate unique ID numbers for test cases with no existing
|
||||
ID number
|
||||
usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
|
||||
[-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v]
|
||||
[-d DEVICE] [-n NS] [-V]
|
||||
|
||||
Linux TC unit tests
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-p PATH, --path PATH The full path to the tc executable to use
|
||||
-v, --verbose Show the commands that are being run
|
||||
-d DEVICE, --device DEVICE
|
||||
Execute the test case in flower category
|
||||
|
||||
selection:
|
||||
select which test cases: files plus directories; filtered by categories
|
||||
plus testids
|
||||
|
||||
-D DIR [DIR ...], --directory DIR [DIR ...]
|
||||
Collect tests from the specified directory(ies)
|
||||
(default [tc-tests])
|
||||
-f FILE [FILE ...], --file FILE [FILE ...]
|
||||
Run tests from the specified file(s)
|
||||
-c [CATG [CATG ...]], --category [CATG [CATG ...]]
|
||||
Run tests only from the specified category/ies, or if
|
||||
no category/ies is/are specified, list known
|
||||
categories.
|
||||
-e ID [ID ...], --execute ID [ID ...]
|
||||
Execute the specified test cases with specified IDs
|
||||
|
||||
action:
|
||||
select action to perform on selected test cases
|
||||
|
||||
-l, --list List all test cases, or those only within the
|
||||
specified category
|
||||
-s, --show Display the selected test cases
|
||||
-i, --id Generate ID numbers for new test cases
|
||||
|
||||
netns:
|
||||
options for nsPlugin(run commands in net namespace)
|
||||
|
||||
-n NS, --namespace NS
|
||||
Run commands in namespace NS
|
||||
|
||||
valgrind:
|
||||
options for valgrindPlugin (run command under test under Valgrind)
|
||||
|
||||
-V, --valgrind Run commands under valgrind
|
||||
|
||||
|
||||
PLUGIN ARCHITECTURE
|
||||
-------------------
|
||||
|
||||
There is now a plugin architecture, and some of the functionality that
|
||||
was in the tdc.py script has been moved into the plugins.
|
||||
|
||||
The plugins are in the directory plugin-lib. The are executed from
|
||||
directory plugins. Put symbolic links from plugins to plugin-lib,
|
||||
and name them according to the order you want them to run.
|
||||
|
||||
Example:
|
||||
|
||||
bjb@bee:~/work/tc-testing$ ls -l plugins
|
||||
total 4
|
||||
lrwxrwxrwx 1 bjb bjb 27 Oct 4 16:12 10-rootPlugin.py -> ../plugin-lib/rootPlugin.py
|
||||
lrwxrwxrwx 1 bjb bjb 25 Oct 12 17:55 20-nsPlugin.py -> ../plugin-lib/nsPlugin.py
|
||||
-rwxr-xr-x 1 bjb bjb 0 Sep 29 15:56 __init__.py
|
||||
|
||||
The plugins are a subclass of TdcPlugin, defined in TdcPlugin.py and
|
||||
must be called "SubPlugin" so tdc can find them. They are
|
||||
distinguished from each other in the python program by their module
|
||||
name.
|
||||
|
||||
This base class supplies "hooks" to run extra functions. These hooks are as follows:
|
||||
|
||||
pre- and post-suite
|
||||
pre- and post-case
|
||||
pre- and post-execute stage
|
||||
adjust-command (runs in all stages and receives the stage name)
|
||||
|
||||
The pre-suite hook receives the number of tests and an array of test ids.
|
||||
This allows you to dump out the list of skipped tests in the event of a
|
||||
failure during setup or teardown stage.
|
||||
|
||||
The pre-case hook receives the ordinal number and test id of the current test.
|
||||
|
||||
The adjust-command hook receives the stage id (see list below) and the
|
||||
full command to be executed. This allows for last-minute adjustment
|
||||
of the command.
|
||||
|
||||
The stages are identified by the following strings:
|
||||
|
||||
- pre (pre-suite)
|
||||
- setup
|
||||
- command
|
||||
- verify
|
||||
- teardown
|
||||
- post (post-suite)
|
||||
|
||||
|
||||
To write a plugin, you need to inherit from TdcPlugin in
|
||||
TdcPlugin.py. To use the plugin, you have to put the
|
||||
implementation file in plugin-lib, and add a symbolic link to it from
|
||||
plugins. It will be detected at run time and invoked at the
|
||||
appropriate times. There are a few examples in the plugin-lib
|
||||
directory:
|
||||
|
||||
- rootPlugin.py:
|
||||
implements the enforcement of running as root
|
||||
- nsPlugin.py:
|
||||
sets up a network namespace and runs all commands in that namespace
|
||||
- valgrindPlugin.py
|
||||
runs each command in the execute stage under valgrind,
|
||||
and checks for leaks.
|
||||
This plugin will output an extra test for each test in the test file,
|
||||
one is the existing output as to whether the test passed or failed,
|
||||
and the other is a test whether the command leaked memory or not.
|
||||
(This one is a preliminary version, it may not work quite right yet,
|
||||
but the overall template is there and it should only need tweaks.)
|
||||
|
||||
|
||||
ACKNOWLEDGEMENTS
|
||||
|
|
|
|||
|
|
@ -5,6 +5,27 @@ tc Testing Suite To-Do list:
|
|||
|
||||
- Add support for multiple versions of tc to run successively
|
||||
|
||||
- Improve error messages when tdc aborts its run
|
||||
- Improve error messages when tdc aborts its run. Partially done - still
|
||||
need to better handle problems in pre- and post-suite.
|
||||
|
||||
- Allow tdc to write its results to file
|
||||
- Use python logger module for debug/verbose output
|
||||
|
||||
- Allow tdc to write its results to file.
|
||||
Maybe use python logger module for this too.
|
||||
|
||||
- A better implementation of the "hooks". Currently, every plugin
|
||||
will attempt to run a function at every hook point. Could be
|
||||
changed so that plugin __init__ methods will register functions to
|
||||
be run in the various predefined times. Then if a plugin does not
|
||||
require action at a specific point, no penalty will be paid for
|
||||
trying to run a function that will do nothing.
|
||||
|
||||
- Proper exception handling - make an exception class and use it
|
||||
|
||||
- a TestCase class, for easier testcase handling, searching, comparison
|
||||
|
||||
- a TestSuite class
|
||||
and a way to configure a test suite,
|
||||
to automate running multiple "test suites" with different requirements
|
||||
|
||||
- super simple test case example using ls, touch, etc
|
||||
|
|
|
|||
Loading…
Reference in a new issue