diff options
| author | xue <> | 2005-11-10 12:47:19 +0000 |
|---|---|---|
| committer | xue <> | 2005-11-10 12:47:19 +0000 |
| commit | 55c4ac1bfe565f1ca7f537fdd8b7a201be28e581 (patch) | |
| tree | a0599d5e36fdbb3f1e169ae56bab7d529597e3eb /tests/UnitTests/simpletest/docs/en/reporter_documentation.html | |
Initial import of prado framework
Diffstat (limited to 'tests/UnitTests/simpletest/docs/en/reporter_documentation.html')
| -rw-r--r-- | tests/UnitTests/simpletest/docs/en/reporter_documentation.html | 515 |
1 files changed, 515 insertions, 0 deletions
diff --git a/tests/UnitTests/simpletest/docs/en/reporter_documentation.html b/tests/UnitTests/simpletest/docs/en/reporter_documentation.html new file mode 100644 index 00000000..2794c5ab --- /dev/null +++ b/tests/UnitTests/simpletest/docs/en/reporter_documentation.html @@ -0,0 +1,515 @@ +<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>SimpleTest for PHP test runner and display documentation</title>
+<link rel="stylesheet" type="text/css" href="docs.css" title="Styles">
+</head>
+<body>
+<div class="menu_back">
+<div class="menu">
+<h2>
+<a href="index.html">SimpleTest</a>
+</h2>
+<ul>
+<li>
+<a href="overview.html">Overview</a>
+</li>
+<li>
+<a href="unit_test_documentation.html">Unit tester</a>
+</li>
+<li>
+<a href="group_test_documentation.html">Group tests</a>
+</li>
+<li>
+<a href="server_stubs_documentation.html">Server stubs</a>
+</li>
+<li>
+<a href="mock_objects_documentation.html">Mock objects</a>
+</li>
+<li>
+<a href="partial_mocks_documentation.html">Partial mocks</a>
+</li>
+<li>
+<span class="chosen">Reporting</span>
+</li>
+<li>
+<a href="expectation_documentation.html">Expectations</a>
+</li>
+<li>
+<a href="web_tester_documentation.html">Web tester</a>
+</li>
+<li>
+<a href="form_testing_documentation.html">Testing forms</a>
+</li>
+<li>
+<a href="authentication_documentation.html">Authentication</a>
+</li>
+<li>
+<a href="browser_documentation.html">Scriptable browser</a>
+</li>
+</ul>
+</div>
+</div>
+<h1>Test reporter documentation</h1>
+<div class="content">
+
+ <p>
+ SimpleTest pretty much follows the MVC pattern
+ (Model-View-Controller).
+ The reporter classes are the view and the model is your
+ test cases and their hiearchy.
+ The controller is mostly hidden from the user of
+ SimpleTest unless you want to change how the test cases
+ are actually run, in which case it is possible to
+ override the runner objects from within the test case.
+ As usual with MVC, the controller is mostly undefined
+ and there are other places to control the test run.
+ </p>
+
+ <p>
+<a class="target" name="html">
+<h2>Reporting results in HTML</h2>
+</a>
+</p>
+ <p>
+ The default test display is minimal in the extreme.
+ It reports success and failure with the conventional red and
+ green bars and shows a breadcrumb trail of test groups
+ for every failed assertion.
+ Here's a fail...
+ <div class="demo">
+ <h1>File test</h1>
+ <span class="fail">Fail</span>: createnewfile->True assertion failed.<br>
+ <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete.
+ <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div>
+ </div>
+ And here all tests passed...
+ <div class="demo">
+ <h1>File test</h1>
+ <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete.
+ <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div>
+ </div>
+ The good news is that there are several points in the display
+ hiearchy for subclassing.
+ </p>
+ <p>
+ For web page based displays there is the
+ <span class="new_code">HtmlReporter</span> class with the following
+ signature...
+<pre>
+class HtmlReporter extends SimpleReporter {
+ public HtmlReporter($encoding) { ... }
+ public makeDry(boolean $is_dry) { ... }
+ public void paintHeader(string $test_name) { ... }
+ public void sendNoCacheHeaders() { ... }
+ public void paintFooter(string $test_name) { ... }
+ public void paintGroupStart(string $test_name, integer $size) { ... }
+ public void paintGroupEnd(string $test_name) { ... }
+ public void paintCaseStart(string $test_name) { ... }
+ public void paintCaseEnd(string $test_name) { ... }
+ public void paintMethodStart(string $test_name) { ... }
+ public void paintMethodEnd(string $test_name) { ... }
+ public void paintFail(string $message) { ... }
+ public void paintPass(string $message) { ... }
+ public void paintError(string $message) { ... }
+ public void paintException(string $message) { ... }
+ public void paintMessage(string $message) { ... }
+ public void paintFormattedMessage(string $message) { ... }
+ protected string _getCss() { ... }
+ public array getTestList() { ... }
+ public integer getPassCount() { ... }
+ public integer getFailCount() { ... }
+ public integer getExceptionCount() { ... }
+ public integer getTestCaseCount() { ... }
+ public integer getTestCaseProgress() { ... }
+}
+</pre>
+ Here is what some of these methods mean. First the display methods
+ that you will probably want to override...
+ <ul class="api">
+ <li>
+ <span class="new_code">HtmlReporter(string $encoding)</span>
+<br>
+ is the constructor.
+ Note that the unit test sets up the link to the display
+ rather than the other way around.
+ The display is a mostly passive receiver of test events.
+ This allows easy adaption of the display for other test
+ systems beside unit tests, such as monitoring servers.
+ The encoding is the character encoding you wish to
+ display the test output in.
+ In order to correctly render debug output when
+ using the web tester, this should match the encoding
+ of the site you are trying to test.
+ The available character set strings are described in
+ the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a>
+ function.
+ </li>
+ <li>
+ <span class="new_code">void paintHeader(string $test_name)</span>
+<br>
+ is called once at the very start of the test when the first
+ start event arrives.
+ The first start event is usually delivered by the top level group
+ test and so this is where <span class="new_code">$test_name</span>
+ comes from.
+ It paints the page titles, CSS, body tag, etc.
+ It returns nothing (<span class="new_code">void</span>).
+ </li>
+ <li>
+ <span class="new_code">void paintFooter(string $test_name)</span>
+<br>
+ Called at the very end of the test to close any tags opened
+ by the page header.
+ By default it also displays the red/green bar and the final
+ count of results.
+ Actually the end of the test happens when a test end event
+ comes in with the same name as the one that started it all
+ at the same level.
+ The tests nest you see.
+ Closing the last test finishes the display.
+ </li>
+ <li>
+ <span class="new_code">void paintMethodStart(string $test_name)</span>
+<br>
+ is called at the start of each test method.
+ The name normally comes from method name.
+ The other test start events behave the same way except
+ that the group test one tells the reporter how large
+ it is in number of held test cases.
+ This is so that the reporter can display a progress bar
+ as the runner churns through the test cases.
+ </li>
+ <li>
+ <span class="new_code">void paintMethodEnd(string $test_name)</span>
+<br>
+ backs out of the test started with the same name.
+ </li>
+ <li>
+ <span class="new_code">void paintFail(string $message)</span>
+<br>
+ paints a failure.
+ By default it just displays the word fail, a breadcrumbs trail
+ showing the current test nesting and the message issued by
+ the assertion.
+ </li>
+ <li>
+ <span class="new_code">void paintPass(string $message)</span>
+<br>
+ by default does nothing.
+ </li>
+ <li>
+ <span class="new_code">string _getCss()</span>
+<br>
+ Returns the CSS styles as a string for the page header
+ method.
+ Additional styles have to be appended here if you are
+ not overriding the page header.
+ You will want to use this method in an overriden page header
+ if you want to include the original CSS.
+ </li>
+ </ul>
+ There are also some accessors to get information on the current
+ state of the test suite.
+ Use these to enrich the display...
+ <ul class="api">
+ <li>
+ <span class="new_code">array getTestList()</span>
+<br>
+ is the first convenience method for subclasses.
+ Lists the current nesting of the tests as a list
+ of test names.
+ The first, most deeply nested test, is first in the
+ list and the current test method will be last.
+ </li>
+ <li>
+ <span class="new_code">integer getPassCount()</span>
+<br>
+ returns the number of passes chalked up so far.
+ Needed for the display at the end.
+ </li>
+ <li>
+ <span class="new_code">integer getFailCount()</span>
+<br>
+ is likewise the number of fails so far.
+ </li>
+ <li>
+ <span class="new_code">integer getExceptionCount()</span>
+<br>
+ is likewise the number of errors so far.
+ </li>
+ <li>
+ <span class="new_code">integer getTestCaseCount()</span>
+<br>
+ is the total number of test cases in the test run.
+ This includes the grouping tests themselves.
+ </li>
+ <li>
+ <span class="new_code">integer getTestCaseProgress()</span>
+<br>
+ is the number of test cases completed so far.
+ </li>
+ </ul>
+ One simple modification is to get the HtmlReporter to display
+ the passes as well as the failures and errors...
+<pre>
+<strong>class ShowPasses extends HtmlReporter {
+
+ function paintPass($message) {
+ parent::paintPass($message);
+ print "&<span class=\"pass\">Pass</span>: ";
+ $breadcrumb = $this->getTestList();
+ array_shift($breadcrumb);
+ print implode("-&gt;", $breadcrumb);
+ print "-&gt;$message<br />\n";
+ }
+
+ function _getCss() {
+ return parent::_getCss() . ' .pass { color: green; }';
+ }
+}</strong>
+</pre>
+ </p>
+ <p>
+ One method that was glossed over was the <span class="new_code">makeDry()</span>
+ method.
+ If you run this method, with no parameters, on the reporter
+ before the test suite is run no actual test methods
+ will be called.
+ You will still get the events of entering and leaving the
+ test methods and test cases, but no passes or failures etc,
+ because the test code will not actually be executed.
+ </p>
+ <p>
+ The reason for this is to allow for more sophistcated
+ GUI displays that allow the selection of individual test
+ cases.
+ In order to build a list of possible tests they need a
+ report on the test structure for drawing, say a tree view
+ of the test suite.
+ With a reporter set to dry run that just sends drawing events
+ this is easily accomplished.
+ </p>
+
+ <p>
+<a class="target" name="other">
+<h2>Extending the reporter</h2>
+</a>
+</p>
+ <p>
+ Rather than simply modifying the existing display, you might want to
+ produce a whole new HTML look, or even generate text or XML.
+ Rather than override every method in
+ <span class="new_code">HtmlReporter</span> we can take one
+ step up the class hiearchy to <span class="new_code">SimpleReporter</span>
+ in the <em>simple_test.php</em> source file.
+ </p>
+ <p>
+ A do nothing display, a blank canvas for your own creation, would
+ be...
+<pre>
+<strong>require_once('simpletest/simple_test.php');</strong>
+
+class MyDisplay extends SimpleReporter {<strong>
+ </strong>
+ function paintHeader($test_name) {
+ }
+
+ function paintFooter($test_name) {
+ }
+
+ function paintStart($test_name, $size) {<strong>
+ parent::paintStart($test_name, $size);</strong>
+ }
+
+ function paintEnd($test_name, $size) {<strong>
+ parent::paintEnd($test_name, $size);</strong>
+ }
+
+ function paintPass($message) {<strong>
+ parent::paintPass($message);</strong>
+ }
+
+ function paintFail($message) {<strong>
+ parent::paintFail($message);</strong>
+ }
+}
+</pre>
+ No output would come from this class until you add it.
+ </p>
+
+ <p>
+<a class="target" name="cli">
+<h2>The command line reporter</h2>
+</a>
+</p>
+ <p>
+ SimpleTest also ships with a minimal command line reporter.
+ The interface mimics JUnit to some extent, but paints the
+ failure messages as they arrive.
+ To use the command line reporter simply substitute it
+ for the HTML version...
+<pre>
+<?php
+ require_once('simpletest/unit_tester.php');
+ require_once('simpletest/reporter.php');
+
+ $test = &new GroupTest('File test');
+ $test->addTestFile('tests/file_test.php');
+ $test->run(<strong>new TextReporter()</strong>);
+?>
+</pre>
+ Then invoke the test suite from the command line...
+<pre class="shell">
+php file_test.php
+</pre>
+ You will need the command line version of PHP installed
+ of course.
+ A passing test suite looks like this...
+<pre class="shell">
+File test
+OK
+Test cases run: 1/1, Failures: 0, Exceptions: 0
+</pre>
+ A failure triggers a display like this...
+<pre class="shell">
+File test
+1) True assertion failed.
+ in createnewfile
+FAILURES!!!
+Test cases run: 1/1, Failures: 1, Exceptions: 0
+</pre>
+ </p>
+ <p>
+ One of the main reasons for using a command line driven
+ test suite is of using the tester as part of some automated
+ process.
+ To function properly in shell scripts the test script should
+ return a non-zero exit code on failure.
+ If a test suite fails the value <span class="new_code">false</span>
+ is returned from the <span class="new_code">SimpleTest::run()</span>
+ method.
+ We can use that result to exit the script with the desired return
+ code...
+<pre>
+<?php
+ require_once('simpletest/unit_tester.php');
+ require_once('simpletest/reporter.php');
+
+ $test = &new GroupTest('File test');
+ $test->addTestFile('tests/file_test.php');
+ <strong>exit ($test->run(new TextReporter()) ? 0 : 1);</strong>
+?>
+</pre>
+ Of course we don't really want to create two test scripts,
+ a command line one and a web browser one, for each test suite.
+ The command line reporter includes a method to sniff out the
+ run time environment...
+<pre>
+<?php
+ require_once('simpletest/unit_tester.php');
+ require_once('simpletest/reporter.php');
+
+ $test = &new GroupTest('File test');
+ $test->addTestFile('tests/file_test.php');
+ <strong>if (TextReporter::inCli()) {</strong>
+ exit ($test->run(new TextReporter()) ? 0 : 1);
+ <strong>}</strong>
+ $test->run(new HtmlReporter());
+?>
+</pre>
+ This is the form used within SimpleTest itself.
+ </p>
+
+ <p>
+<a class="target" name="xml">
+<h2>Remote testing</h2>
+</a>
+</p>
+ <p>
+ SimpleTest ships with an <span class="new_code">XmlReporter</span> class
+ used for internal communication.
+ When run the output looks like...
+<pre class="shell">
+<?xml version="1.0"?>
+<run>
+ <group size="4">
+ <name>Remote tests</name>
+ <group size="4">
+ <name>Visual test with 48 passes, 48 fails and 4 exceptions</name>
+ <case>
+ <name>testofunittestcaseoutput</name>
+ <test>
+ <name>testofresults</name>
+ <pass>This assertion passed</pass>
+ <fail>This assertion failed</fail>
+ </test>
+ <test>
+ ...
+ </test>
+ </case>
+ </group>
+ </group>
+</run>
+</pre>
+ You can make use of this format with the parser
+ supplied as part of SimpleTest itself.
+ This is called <span class="new_code">SimpleTestXmlParser</span> and
+ resides in <em>xml.php</em> within the SimpleTest package...
+<pre>
+<?php
+ require_once('simpletest/xml.php');
+
+ ...
+ $parser = &new SimpleTestXmlParser(new HtmlReporter());
+ $parser->parse($test_output);
+?>
+</pre>
+ The <span class="new_code">$test_output</span> should be the XML format
+ from the XML reporter, and could come from say a command
+ line run of a test case.
+ The parser sends events to the reporter just like any
+ other test run.
+ There are some odd occasions where this is actually useful.
+ </p>
+ <p>
+ A problem with large test suites is thet they can exhaust
+ the default 8Mb memory limit on a PHP process.
+ By having the test groups output in XML and run in
+ separate processes, the output can be reparsed to
+ aggregate the results into a much smaller footprint top level
+ test.
+ </p>
+ <p>
+ Because the XML output can come from anywhere, this opens
+ up the possibility of aggregating test runs from remote
+ servers.
+ A test case already exists to do this within the SimpleTest
+ framework, but it is currently experimental...
+<pre>
+<?php
+ <strong>require_once('../remote.php');</strong>
+ require_once('../reporter.php');
+
+ $test_url = ...;
+ $dry_url = ...;
+
+ $test = &new GroupTest('Remote tests');
+ $test->addTestCase(<strong>new RemoteTestCase($test_url, $dry_url)</strong>);
+ $test->run(new HtmlReporter());
+?>
+</pre>
+ The <span class="new_code">RemoteTestCase</span> takes the actual location
+ of the test runner, basically a web page in XML format.
+ It also takes the URL of a reporter set to do a dry run.
+ This is so that progress can be reported upward correctly.
+ The <span class="new_code">RemoteTestCase</span> can be added to test suites
+ just like any other group test.
+ </p>
+
+ </div>
+<div class="copyright">
+ Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004
+ </div>
+</body>
+</html>
|