selftest: Fix subunit stream to include the right prefixes rather than

[sfrench/samba-autobuild/.git] / selftest / README

diff --git a/selftest/README b/selftest/README
index f8be20a569c787a97ea7d049649093aea345d89e..a8c5101a61deca7ffbaa5d0ae259a4b214b067a1 100644 (file)
--- a/selftest/README
+++ b/selftest/README
@@ -11,7 +11,7 @@ the name of the test, the command to run and the environment that should be
 provided. Use the included "plantest" function to generate the required output.
 
 Testsuite behaviour
-================================
+===================
 
 Exit code
 ------------
@@ -31,41 +31,83 @@ for the last announced test.
 Accepted commands are:
 
 test
-~~~~~~~~~~~~
+~~~~
 test: <NAME>
 
 Announce that a new test with the specified name is starting
 
 success
-~~~~~~~~~~~~~~~
+~~~~~~~
 success: <NAME> 
 
 Announce that the test with the specified name is done and ran successfully.
 
 failure
-~~~~~~~~~~~~~~~
+~~~~~~~
 failure: <NAME> 
 failure: <NAME> [ REASON ]
 
 Announce that the test with the specified name failed. Optionally, it is 
 possible to specify a reason it failed.
 
+The alias "fail" will also work.
+
+xfail
+~~~~~
+xfail: <NAME>
+xfail: <NAME> [ REASON ]
+
+Announce that the test with the specified name failed but that the failure
+was expected, e.g. it's a test for a known bug that hasn't been fixed yet.
+Alternatively it is also possible to simply return "failure:" here but 
+specify in the samba4-knownfailures file that it is failing.
+
 skip
-~~~~~~~~~~~~
+~~~~
 skip: <NAME>
 skip: <NAME> [ REASON ]
 
 Announce that the test with the specified name was skipped. Optionally a 
 reason can be specified.
 
-knownfail
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-knownfail: <NAME>
-knownfail: <NAME> [ REASON ]
+time
+~~~~
+time: YYYY-MM-DD HH:mm:ssZ
 
-Announce that the test with the specified name was run and failed as expected.
-Alternatively it is also possible to simply return "failure:" here but 
-specify in the samba4-knownfailures file that it is failing.
+Announce the current time. This may be used to calculate the duration of 
+various tests.
+
+The following are Samba extensions to Subunit:
+
+testsuite-count
+~~~~~~~~~~~~~~~
+testsuite-count: number
+
+Announce the number of tests that is going to be run.
+
+start-testsuite
+~~~~~~~~~
+start-testsuite: name
+
+The testsuite name is used as prefix for all containing tests.
+
+skip-testsuite
+~~~~~~~~~~~~~~
+skip-testsuite: name
+
+Mark the testsuite with the specified name as skipped.
+
+testsuite-success
+~~~~~~~~~~~~~~~~~
+testsuite-success: name
+
+Indicate that the testsuite has succeeded successfully.
+
+testsuite-fail
+~~~~~~~~~~~~~~
+testsuite-fail: name
+
+Indicate that a testsuite has failed.
 
 Environments
 ============