1 === modified file 'AUTHORS'
2 --- AUTHORS 2009-02-03 12:38:12 +0000
3 +++ AUTHORS 2009-03-28 06:48:39 +0000
5 Daniel Gollub (pthreads support)
6 Friedrich Beckmann (mingw and msvc port)
7 Roy Merkel (specified test exit value)
8 + Robert Collins (subunit support)
10 Anybody who has contributed code to Check or Check's build system is
11 considered an author. Send patches to this file to
13 === modified file 'NEWS'
14 --- NEWS 2008-12-29 14:46:39 +0000
15 +++ NEWS 2009-03-28 06:48:39 +0000
19 +* Added CK_SUBUNIT support for outputting test information in the subunit wire
20 + protocol. See the check manual for more information. (Contributed by Robert
23 Mon, Dec 29, 2009: Released Check 0.9.6
24 based on r453 (2008-12-01 22:14:15).
27 === modified file 'configure.ac'
28 --- configure.ac 2008-12-31 01:23:32 +0000
29 +++ configure.ac 2009-03-28 07:31:57 +0000
32 AM_CONDITIONAL(NO_TIMEOUT_TESTS, test x"$enable_timeout_tests" = "xfalse")
34 +AC_ARG_ENABLE(subunit,
35 +AC_HELP_STRING([--enable-subunit],
36 + [enable support for the subunit test protocol @<:@default=autodetect@:>@]),
37 +[case "${enableval}" in
40 + echo "Enabled subunit support"
43 + enable_subunit=false
44 + echoo "Disabled subunit support"
47 + echo "Subunit support will enable automatically."
49 + *) AC_MSG_ERROR(bad value ${enableval} for --enable-subunit) ;;
51 +[echo "Subunit support will enable automatically."
52 + enable_subunit=autodetect])
54 # Checks for programs.
59 AC_CHECK_HEADERS([fcntl.h stddef.h stdint.h stdlib.h string.h sys/time.h unistd.h])
61 +if test xfalse != x"$enable_subunit"; then
62 +AC_CHECK_LIB(subunit, subunit_test_start, ,
63 +[case "$enable_subunit" in
65 + enable_subunit=false
68 + AC_MSG_ERROR([libsubunit is required for subunit protocol support. The homepage for subunit is https://launchpad.net/subunit/])
73 +if test xfalse != x"$enable_subunit"; then
74 +AC_CHECK_HEADER([subunit/child.h], ,
75 +[case "$enable_subunit" in
77 + enable_subunit=false
80 + AC_MSG_ERROR([The header subunit/child.h could not be succesfully included and is required for subunit protocol support. The homepage for subunit is https://launchpad.net/subunit/])
85 +if test xfalse = x"$enable_subunit"; then
90 +AC_SUBST(ENABLE_SUBUNIT)
91 +AC_DEFINE_UNQUOTED(ENABLE_SUBUNIT, $ENABLE_SUBUNIT, [Subunit protocol result output])
93 +AM_CONDITIONAL(SUBUNIT, test x"$enable_subunit" != "xfalse")
97 # Checks for typedefs, structures, and compiler characteristics.
110 === modified file 'doc/check.texi'
111 --- doc/check.texi 2009-03-12 21:05:05 +0000
112 +++ doc/check.texi 2009-03-28 06:48:39 +0000
114 @author Chris Pickett
115 @author Fredrik Hugosson
116 @author Robert Lemmen
117 +@author Robert Collins
119 @c The following two commands start the copyright page.
123 * Determining Test Coverage::
130 which can have the values "silent", "minimal", "normal", "verbose". If
131 the variable is not found or the value is not recognized, the print
132 mode is set to @code{CK_NORMAL}.
136 +Prints running progress through the @uref{https://launchpad.net/subunit/,
137 +subunit} test runner protocol. See 'subunit support' under the Advanced Features section for more information.
140 With the @code{CK_NORMAL} flag specified in our @code{main()}, let's
143 * Determining Test Coverage::
148 @node Running Multiple Cases, No Fork Mode, Advanced Features, Advanced Features
149 @@ -1179,7 +1187,7 @@
150 you. For more information or help with other compilers, please refer
151 to the relevant manuals.
153 -@node Test Logging, , Determining Test Coverage, Advanced Features
154 +@node Test Logging, Subunit Support, Determining Test Coverage, Advanced Features
155 @section Test Logging
157 @findex srunner_set_log()
158 @@ -1282,6 +1290,50 @@
162 +@node Subunit Support, , Test Logging, Advanced Features
163 +@section Subunit Support
165 +Check supports running test suites with subunit output. This can be useful to
166 +combine test results from multiple languages, or to perform programmatic
167 +analysis on the results of multiple check test suites or otherise handle test
168 +results in a programmatic manner. Using subunit with check is very straight
169 +forward. There are two steps:
170 +1) In your check test suite driver pass 'CK_SUBUNIT' as the output mode
175 +sr = srunner_create (make_s1_suite ());
176 +srunner_add_suite (sr, make_s2_suite ());
177 +srunner_run_all (sr, CK_SUBUNIT);
180 +2) Setup your main language test runner to run your check based test
181 +executable. For instance using python:
187 +class ShellTests(subunit.ExecTestCase):
188 + """Run some tests from the C codebase."""
190 + def test_group_one(self):
191 + """./foo/check_driver"""
193 + def test_group_two(self):
194 + """./foo/other_driver"""
198 +In this example, running the test suite ShellTests in python (using any test
199 +runner - unittest.py, tribunal, trial, nose or others) will run
200 +./foo/check_driver and ./foo/other_driver and report on their result.
202 +Subunit is hosted on launchpad - the @uref{https://launchpad.net/subunit/,
203 +subunit} project there contains bug tracker, future plans, and source code
206 @node Conclusion and References, AM_PATH_CHECK, Advanced Features, Top
207 @chapter Conclusion and References
208 The tutorial and description of advanced features has provided an
210 === modified file 'src/check.h.in'
211 --- src/check.h.in 2009-02-03 12:36:16 +0000
212 +++ src/check.h.in 2009-03-28 06:48:39 +0000
214 CK_NORMAL, /* All failed tests */
215 CK_VERBOSE, /* All tests */
216 CK_ENV, /* Look at environment var */
217 +#if @ENABLE_SUBUNIT@
218 + CK_SUBUNIT, /* Run as a subunit child process */
224 === modified file 'src/check_impl.h'
225 --- src/check_impl.h 2009-02-03 12:36:16 +0000
226 +++ src/check_impl.h 2009-03-28 06:48:39 +0000
231 + CLSTART_T, /* A test case is about to run */
236 === modified file 'src/check_log.c'
237 --- src/check_log.c 2008-12-31 21:41:27 +0000
238 +++ src/check_log.c 2009-03-28 06:48:39 +0000
243 +#if HAVE_SUBUNIT_CHILD_H
244 +#include <subunit/child.h>
247 #include "check_error.h"
248 #include "check_list.h"
249 #include "check_impl.h"
250 #include "check_log.h"
251 #include "check_print.h"
252 +#include "check_str.h"
255 static void srunner_send_evt (SRunner *sr, void *obj, enum cl_event evt);
257 srunner_send_evt (sr, s, CLEND_S);
260 +void log_test_start (SRunner *sr, TCase * tc, TF * tfun)
263 + snprintf(buffer, 99, "%s:%s", tc->name, tfun->name);
264 + srunner_send_evt (sr, buffer, CLSTART_T);
267 void log_test_end (SRunner *sr, TestResult *tr)
269 srunner_send_evt (sr, tr, CLEND_T);
271 void stdout_lfun (SRunner *sr, FILE *file, enum print_output printmode,
272 void *obj, enum cl_event evt)
277 if (printmode == CK_ENV) {
288 eprintf("Bad event type received in stdout_lfun", __FILE__, __LINE__);
289 @@ -195,12 +206,14 @@
297 tr_fprint(file, tr, CK_VERBOSE);
300 - eprintf("Bad event type received in stdout_lfun", __FILE__, __LINE__);
301 + eprintf("Bad event type received in lfile_lfun", __FILE__, __LINE__);
306 fprintf(file, " </suite>\n");
313 tr_xmlprint(file, tr, CK_VERBOSE);
319 +void subunit_lfun (SRunner *sr, FILE *file, enum print_output printmode,
320 + void *obj, enum cl_event evt)
326 + /* assert(printmode == CK_SUBUNIT); */
339 + if (printmode > CK_SILENT) {
340 + fprintf (file, "\n");
341 + srunner_fprint (file, sr, printmode);
349 + subunit_test_start(name);
354 + char *name = ck_strdup_printf ("%s:%s", tr->tcname, tr->tname);
355 + char *msg = tr_short_str (tr);
356 + switch (tr->rtype) {
358 + subunit_test_pass(name);
361 + subunit_test_fail(name, msg);
364 + subunit_test_error(name, msg);
367 + eprintf("Bad result type in subunit_lfun", __FILE__, __LINE__);
374 + eprintf("Bad event type received in subunit_lfun", __FILE__, __LINE__);
379 FILE *srunner_open_lfile (SRunner *sr)
384 sr->loglst = check_list_create();
385 - srunner_register_lfun (sr, stdout, 0, stdout_lfun, print_mode);
387 + if (print_mode != CK_SUBUNIT)
389 + srunner_register_lfun (sr, stdout, 0, stdout_lfun, print_mode);
392 + srunner_register_lfun (sr, stdout, 0, subunit_lfun, print_mode);
394 f = srunner_open_lfile (sr);
396 srunner_register_lfun (sr, f, 1, lfile_lfun, print_mode);
398 === modified file 'src/check_log.h'
399 --- src/check_log.h 2006-10-13 00:24:56 +0000
400 +++ src/check_log.h 2009-03-28 08:21:45 +0000
402 void log_suite_start (SRunner *sr, Suite *s);
403 void log_suite_end (SRunner *sr, Suite *s);
404 void log_test_end (SRunner *sr, TestResult *tr);
405 +void log_test_start (SRunner *sr, TCase *tc, TF *tfun);
407 void stdout_lfun (SRunner *sr, FILE *file, enum print_output,
408 void *obj, enum cl_event evt);
410 void xml_lfun (SRunner *sr, FILE *file, enum print_output,
411 void *obj, enum cl_event evt);
413 +void subunit_lfun (SRunner *sr, FILE *file, enum print_output,
414 + void *obj, enum cl_event evt);
416 void srunner_register_lfun (SRunner *sr, FILE *lfile, int close,
417 LFun lfun, enum print_output);
420 === modified file 'src/check_print.c'
421 --- src/check_print.c 2008-12-31 02:05:58 +0000
422 +++ src/check_print.c 2009-03-28 06:48:39 +0000
424 static void srunner_fprint_summary (FILE *file, SRunner *sr,
425 enum print_output print_mode)
428 + if (print_mode == CK_SUBUNIT)
432 if (print_mode >= CK_MINIMAL) {
436 enum print_output print_mode)
441 + if (print_mode == CK_SUBUNIT)
445 resultlst = sr->resultlst;
448 === modified file 'src/check_run.c'
449 --- src/check_run.c 2009-02-03 12:36:16 +0000
450 +++ src/check_run.c 2009-03-28 06:48:39 +0000
453 for (i = tfun->loop_start; i < tfun->loop_end; i++)
455 + log_test_start (sr, tc, tfun);
456 switch (srunner_fork_status(sr)) {
458 #ifdef _POSIX_VERSION
460 === modified file 'src/check_str.c'
461 --- src/check_str.c 2008-12-31 02:05:58 +0000
462 +++ src/check_str.c 2009-03-28 06:48:39 +0000
467 +char *tr_short_str (TestResult *tr)
469 + const char *exact_msg;
472 + exact_msg = (tr->rtype == CK_ERROR) ? "(after this point) ": "";
474 + rstr = ck_strdup_printf ("%s:%d: %s%s",
475 + tr->file, tr->line,
476 + exact_msg, tr->msg);
481 char *sr_stat_str (SRunner *sr)
485 === modified file 'src/check_str.h'
486 --- src/check_str.h 2006-10-13 00:24:56 +0000
487 +++ src/check_str.h 2009-03-28 06:48:39 +0000
489 value has been malloc'd, and must be freed by the caller */
490 char *tr_str (TestResult *tr);
492 +/* Return a string representation of the given TestResult message
493 + without the test id or result type. This is suitable for separate
494 + formatting of the test and the message. Return value has been
495 + malloc'd, and must be freed by the caller */
496 +char *tr_short_str (TestResult *tr);
498 /* Return a string representation of the given SRunner's run
499 statistics (% passed, num run, passed, errors, failures). Return
500 value has been malloc'd, and must be freed by the caller
502 === modified file 'tests/Makefile.am'
503 --- tests/Makefile.am 2009-02-03 12:36:16 +0000
504 +++ tests/Makefile.am 2009-03-28 08:26:21 +0000
509 -EXTRA_DIST = test_output.sh test_log_output.sh test_xml_output.sh
510 +EXTRA_DIST = test_output.sh test_log_output.sh test_vars.in test_xml_output.sh
513 check_check_CFLAGS = -DTIMEOUT_TESTS_ENABLED=0
515 check_check_export_LDADD = $(top_builddir)/src/libcheck.la $(top_builddir)/lib/libcompat.la
517 check_check_SOURCES = \
520 - check_check_sub.c \
521 - check_check_master.c \
522 - check_check_msg.c \
523 - check_check_log.c \
524 - check_check_limit.c \
525 - check_check_fork.c \
526 - check_check_fixture.c \
527 - check_check_pack.c \
528 - check_check_exit.c \
531 + check_check_sub.c \
532 + check_check_master.c \
533 + check_check_msg.c \
534 + check_check_log.c \
535 + check_check_log_internal.c \
536 + check_check_limit.c \
537 + check_check_fork.c \
538 + check_check_fixture.c \
539 + check_check_pack.c \
540 + check_check_exit.c \
542 check_check_LDADD = $(top_builddir)/src/libcheckinternal.la $(top_builddir)/lib/libcompat.la
545 === modified file 'tests/check_check.h'
546 --- tests/check_check.h 2009-02-03 12:36:16 +0000
547 +++ tests/check_check.h 2009-03-28 08:34:03 +0000
549 Suite *make_list_suite(void);
550 Suite *make_msg_suite(void);
551 Suite *make_log_suite(void);
552 +Suite *make_log_internal_suite(void);
553 Suite *make_limit_suite(void);
554 Suite *make_fork_suite(void);
555 Suite *make_fixture_suite(void);
557 === added file 'tests/check_check_log_internal.c'
558 --- tests/check_check_log_internal.c 1970-01-01 00:00:00 +0000
559 +++ tests/check_check_log_internal.c 2009-03-28 08:33:45 +0000
561 +#include "../lib/libcompat.h"
563 +/* Tests for log related stuff in check which need non-exported functions. */
569 +#include <check_list.h>
570 +#include <check_impl.h>
571 +#include <check_log.h>
572 +#include "check_check.h"
576 +START_TEST(test_init_logging_subunit)
578 + /* init_logging with CK_SUBUNIT sets stdout
579 + * to a subunit function, not any log.
581 + Log * first_log = NULL;
582 + Suite *s = suite_create("Suite");
583 + SRunner *sr = srunner_create(s);
584 + srunner_init_logging(sr, CK_SUBUNIT);
585 + list_front (sr->loglst);
586 + fail_if (list_at_end(sr->loglst), "No entries in log list");
587 + first_log = list_val(sr->loglst);
588 + fail_if (first_log == NULL, "log is NULL");
589 + list_advance(sr->loglst);
590 + fail_unless(list_at_end(sr->loglst), "More than one entry in log list");
591 + fail_unless(first_log->lfun == subunit_lfun,
592 + "Log function is not the subunit lfun.");
593 + srunner_end_logging(sr);
599 +Suite *make_log_internal_suite(void)
603 + TCase *tc_core_subunit;
605 + s = suite_create("Log");
606 + tc_core_subunit = tcase_create("Core SubUnit");
609 + suite_add_tcase(s, tc_core_subunit);
610 + tcase_add_test(tc_core_subunit, test_init_logging_subunit);
617 === modified file 'tests/check_check_main.c'
618 --- tests/check_check_main.c 2009-02-03 12:36:16 +0000
619 +++ tests/check_check_main.c 2009-03-28 08:34:51 +0000
621 srunner_add_suite(sr, make_list_suite());
622 srunner_add_suite(sr, make_msg_suite());
623 srunner_add_suite(sr, make_log_suite());
624 + srunner_add_suite(sr, make_log_internal_suite());
625 srunner_add_suite(sr, make_limit_suite());
626 srunner_add_suite(sr, make_fork_suite());
627 srunner_add_suite(sr, make_fixture_suite());
629 === modified file 'tests/ex_output.c'
630 --- tests/ex_output.c 2008-12-30 01:40:50 +0000
631 +++ tests/ex_output.c 2009-03-28 06:57:20 +0000
638 START_TEST(test_pass)
644 +static void print_usage(void)
646 + printf ("Usage: ex_output (CK_SILENT | CK_MINIMAL | CK_NORMAL | CK_VERBOSE");
648 + printf (" | CK_SUBUNIT");
653 int main (int argc, char **argv)
657 - printf ("Usage: ex_output (CK_SILENT | CK_MINIMAL | CK_NORMAL | CK_VERBOSE)\n");
663 run_tests(CK_NORMAL);
664 else if (strcmp (argv[1], "CK_VERBOSE") == 0)
665 run_tests(CK_VERBOSE);
667 + else if (strcmp (argv[1], "CK_SUBUNIT") == 0)
668 + run_tests(CK_SUBUNIT);
671 - printf ("Usage: ex_output (CK_SILENT | CK_MINIMAL | CK_NORMAL | CK_VERBOSE)\n");
677 === modified file 'tests/test_output.sh'
678 --- tests/test_output.sh 2008-12-30 05:47:35 +0000
679 +++ tests/test_output.sh 2009-03-28 07:59:52 +0000
683 +. "${srcdir}/"test_vars
685 if [ "${srcdir}" = "." ]; then
689 33%: Checks: 3, Failures: 1, Errors: 1"
690 t2="xRunning suite(s): Master
691 33%: Checks: 3, Failures: 1, Errors: 1
692 -${lsrc}ex_output.c:16:F:Core:test_fail:0: Failure
693 -${lsrc}ex_output.c:20:E:Core:test_exit:0: (after this point) Early exit with return value 1"
694 +${lsrc}ex_output.c:17:F:Core:test_fail:0: Failure
695 +${lsrc}ex_output.c:21:E:Core:test_exit:0: (after this point) Early exit with return value 1"
696 t3="xRunning suite(s): Master
697 33%: Checks: 3, Failures: 1, Errors: 1
698 -${lsrc}ex_output.c:10:P:Core:test_pass:0: Passed
699 -${lsrc}ex_output.c:16:F:Core:test_fail:0: Failure
700 -${lsrc}ex_output.c:20:E:Core:test_exit:0: (after this point) Early exit with return value 1"
701 +${lsrc}ex_output.c:11:P:Core:test_pass:0: Passed
702 +${lsrc}ex_output.c:17:F:Core:test_fail:0: Failure
703 +${lsrc}ex_output.c:21:E:Core:test_exit:0: (after this point) Early exit with return value 1"
704 +t4="xtest: Core:test_pass
705 +success: Core:test_pass
706 +test: Core:test_fail
707 +failure: Core:test_fail [
708 +${lsrc}ex_output.c:17: Failure
710 +test: Core:test_exit
711 +error: Core:test_exit [
712 +${lsrc}ex_output.c:21: (after this point) Early exit with return value 1
715 op0=`./ex_output CK_SILENT`
716 op1=`./ex_output CK_MINIMAL`
717 op2=`./ex_output CK_NORMAL`
718 op3=`./ex_output CK_VERBOSE`
719 +if test 1 -eq $ENABLE_SUBUNIT; then
720 +op4=`./ex_output CK_SUBUNIT`
726 test_output "$t1" x"$op1" "CK_MINIMAL";
727 test_output "$t2" x"$op2" "CK_NORMAL";
728 test_output "$t3" x"$op3" "CK_VERBOSE";
729 +if test 1 -eq $ENABLE_SUBUNIT; then
730 +test_output "$t4" x"$op4" "CK_SUBUNIT";
734 === added file 'tests/test_vars.in'
735 --- tests/test_vars.in 1970-01-01 00:00:00 +0000
736 +++ tests/test_vars.in 2009-03-28 06:48:39 +0000
738 +# defined to 1 if subunit is enabled
739 +export ENABLE_SUBUNIT=@ENABLE_SUBUNIT@