diff options
Diffstat (limited to 'tests/UnitTests/simpletest/docs/en/reporter_documentation.html')
| -rw-r--r-- | tests/UnitTests/simpletest/docs/en/reporter_documentation.html | 515 |
1 files changed, 0 insertions, 515 deletions
diff --git a/tests/UnitTests/simpletest/docs/en/reporter_documentation.html b/tests/UnitTests/simpletest/docs/en/reporter_documentation.html deleted file mode 100644 index 2794c5ab..00000000 --- a/tests/UnitTests/simpletest/docs/en/reporter_documentation.html +++ /dev/null @@ -1,515 +0,0 @@ -<html>
-<head>
-<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
-<title>SimpleTest for PHP test runner and display documentation</title>
-<link rel="stylesheet" type="text/css" href="docs.css" title="Styles">
-</head>
-<body>
-<div class="menu_back">
-<div class="menu">
-<h2>
-<a href="index.html">SimpleTest</a>
-</h2>
-<ul>
-<li>
-<a href="overview.html">Overview</a>
-</li>
-<li>
-<a href="unit_test_documentation.html">Unit tester</a>
-</li>
-<li>
-<a href="group_test_documentation.html">Group tests</a>
-</li>
-<li>
-<a href="server_stubs_documentation.html">Server stubs</a>
-</li>
-<li>
-<a href="mock_objects_documentation.html">Mock objects</a>
-</li>
-<li>
-<a href="partial_mocks_documentation.html">Partial mocks</a>
-</li>
-<li>
-<span class="chosen">Reporting</span>
-</li>
-<li>
-<a href="expectation_documentation.html">Expectations</a>
-</li>
-<li>
-<a href="web_tester_documentation.html">Web tester</a>
-</li>
-<li>
-<a href="form_testing_documentation.html">Testing forms</a>
-</li>
-<li>
-<a href="authentication_documentation.html">Authentication</a>
-</li>
-<li>
-<a href="browser_documentation.html">Scriptable browser</a>
-</li>
-</ul>
-</div>
-</div>
-<h1>Test reporter documentation</h1>
-<div class="content">
-
- <p>
- SimpleTest pretty much follows the MVC pattern
- (Model-View-Controller).
- The reporter classes are the view and the model is your
- test cases and their hiearchy.
- The controller is mostly hidden from the user of
- SimpleTest unless you want to change how the test cases
- are actually run, in which case it is possible to
- override the runner objects from within the test case.
- As usual with MVC, the controller is mostly undefined
- and there are other places to control the test run.
- </p>
-
- <p>
-<a class="target" name="html">
-<h2>Reporting results in HTML</h2>
-</a>
-</p>
- <p>
- The default test display is minimal in the extreme.
- It reports success and failure with the conventional red and
- green bars and shows a breadcrumb trail of test groups
- for every failed assertion.
- Here's a fail...
- <div class="demo">
- <h1>File test</h1>
- <span class="fail">Fail</span>: createnewfile->True assertion failed.<br>
- <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete.
- <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div>
- </div>
- And here all tests passed...
- <div class="demo">
- <h1>File test</h1>
- <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete.
- <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div>
- </div>
- The good news is that there are several points in the display
- hiearchy for subclassing.
- </p>
- <p>
- For web page based displays there is the
- <span class="new_code">HtmlReporter</span> class with the following
- signature...
-<pre>
-class HtmlReporter extends SimpleReporter {
- public HtmlReporter($encoding) { ... }
- public makeDry(boolean $is_dry) { ... }
- public void paintHeader(string $test_name) { ... }
- public void sendNoCacheHeaders() { ... }
- public void paintFooter(string $test_name) { ... }
- public void paintGroupStart(string $test_name, integer $size) { ... }
- public void paintGroupEnd(string $test_name) { ... }
- public void paintCaseStart(string $test_name) { ... }
- public void paintCaseEnd(string $test_name) { ... }
- public void paintMethodStart(string $test_name) { ... }
- public void paintMethodEnd(string $test_name) { ... }
- public void paintFail(string $message) { ... }
- public void paintPass(string $message) { ... }
- public void paintError(string $message) { ... }
- public void paintException(string $message) { ... }
- public void paintMessage(string $message) { ... }
- public void paintFormattedMessage(string $message) { ... }
- protected string _getCss() { ... }
- public array getTestList() { ... }
- public integer getPassCount() { ... }
- public integer getFailCount() { ... }
- public integer getExceptionCount() { ... }
- public integer getTestCaseCount() { ... }
- public integer getTestCaseProgress() { ... }
-}
-</pre>
- Here is what some of these methods mean. First the display methods
- that you will probably want to override...
- <ul class="api">
- <li>
- <span class="new_code">HtmlReporter(string $encoding)</span>
-<br>
- is the constructor.
- Note that the unit test sets up the link to the display
- rather than the other way around.
- The display is a mostly passive receiver of test events.
- This allows easy adaption of the display for other test
- systems beside unit tests, such as monitoring servers.
- The encoding is the character encoding you wish to
- display the test output in.
- In order to correctly render debug output when
- using the web tester, this should match the encoding
- of the site you are trying to test.
- The available character set strings are described in
- the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a>
- function.
- </li>
- <li>
- <span class="new_code">void paintHeader(string $test_name)</span>
-<br>
- is called once at the very start of the test when the first
- start event arrives.
- The first start event is usually delivered by the top level group
- test and so this is where <span class="new_code">$test_name</span>
- comes from.
- It paints the page titles, CSS, body tag, etc.
- It returns nothing (<span class="new_code">void</span>).
- </li>
- <li>
- <span class="new_code">void paintFooter(string $test_name)</span>
-<br>
- Called at the very end of the test to close any tags opened
- by the page header.
- By default it also displays the red/green bar and the final
- count of results.
- Actually the end of the test happens when a test end event
- comes in with the same name as the one that started it all
- at the same level.
- The tests nest you see.
- Closing the last test finishes the display.
- </li>
- <li>
- <span class="new_code">void paintMethodStart(string $test_name)</span>
-<br>
- is called at the start of each test method.
- The name normally comes from method name.
- The other test start events behave the same way except
- that the group test one tells the reporter how large
- it is in number of held test cases.
- This is so that the reporter can display a progress bar
- as the runner churns through the test cases.
- </li>
- <li>
- <span class="new_code">void paintMethodEnd(string $test_name)</span>
-<br>
- backs out of the test started with the same name.
- </li>
- <li>
- <span class="new_code">void paintFail(string $message)</span>
-<br>
- paints a failure.
- By default it just displays the word fail, a breadcrumbs trail
- showing the current test nesting and the message issued by
- the assertion.
- </li>
- <li>
- <span class="new_code">void paintPass(string $message)</span>
-<br>
- by default does nothing.
- </li>
- <li>
- <span class="new_code">string _getCss()</span>
-<br>
- Returns the CSS styles as a string for the page header
- method.
- Additional styles have to be appended here if you are
- not overriding the page header.
- You will want to use this method in an overriden page header
- if you want to include the original CSS.
- </li>
- </ul>
- There are also some accessors to get information on the current
- state of the test suite.
- Use these to enrich the display...
- <ul class="api">
- <li>
- <span class="new_code">array getTestList()</span>
-<br>
- is the first convenience method for subclasses.
- Lists the current nesting of the tests as a list
- of test names.
- The first, most deeply nested test, is first in the
- list and the current test method will be last.
- </li>
- <li>
- <span class="new_code">integer getPassCount()</span>
-<br>
- returns the number of passes chalked up so far.
- Needed for the display at the end.
- </li>
- <li>
- <span class="new_code">integer getFailCount()</span>
-<br>
- is likewise the number of fails so far.
- </li>
- <li>
- <span class="new_code">integer getExceptionCount()</span>
-<br>
- is likewise the number of errors so far.
- </li>
- <li>
- <span class="new_code">integer getTestCaseCount()</span>
-<br>
- is the total number of test cases in the test run.
- This includes the grouping tests themselves.
- </li>
- <li>
- <span class="new_code">integer getTestCaseProgress()</span>
-<br>
- is the number of test cases completed so far.
- </li>
- </ul>
- One simple modification is to get the HtmlReporter to display
- the passes as well as the failures and errors...
-<pre>
-<strong>class ShowPasses extends HtmlReporter {
-
- function paintPass($message) {
- parent::paintPass($message);
- print "&<span class=\"pass\">Pass</span>: ";
- $breadcrumb = $this->getTestList();
- array_shift($breadcrumb);
- print implode("-&gt;", $breadcrumb);
- print "-&gt;$message<br />\n";
- }
-
- function _getCss() {
- return parent::_getCss() . ' .pass { color: green; }';
- }
-}</strong>
-</pre>
- </p>
- <p>
- One method that was glossed over was the <span class="new_code">makeDry()</span>
- method.
- If you run this method, with no parameters, on the reporter
- before the test suite is run no actual test methods
- will be called.
- You will still get the events of entering and leaving the
- test methods and test cases, but no passes or failures etc,
- because the test code will not actually be executed.
- </p>
- <p>
- The reason for this is to allow for more sophistcated
- GUI displays that allow the selection of individual test
- cases.
- In order to build a list of possible tests they need a
- report on the test structure for drawing, say a tree view
- of the test suite.
- With a reporter set to dry run that just sends drawing events
- this is easily accomplished.
- </p>
-
- <p>
-<a class="target" name="other">
-<h2>Extending the reporter</h2>
-</a>
-</p>
- <p>
- Rather than simply modifying the existing display, you might want to
- produce a whole new HTML look, or even generate text or XML.
- Rather than override every method in
- <span class="new_code">HtmlReporter</span> we can take one
- step up the class hiearchy to <span class="new_code">SimpleReporter</span>
- in the <em>simple_test.php</em> source file.
- </p>
- <p>
- A do nothing display, a blank canvas for your own creation, would
- be...
-<pre>
-<strong>require_once('simpletest/simple_test.php');</strong>
-
-class MyDisplay extends SimpleReporter {<strong>
- </strong>
- function paintHeader($test_name) {
- }
-
- function paintFooter($test_name) {
- }
-
- function paintStart($test_name, $size) {<strong>
- parent::paintStart($test_name, $size);</strong>
- }
-
- function paintEnd($test_name, $size) {<strong>
- parent::paintEnd($test_name, $size);</strong>
- }
-
- function paintPass($message) {<strong>
- parent::paintPass($message);</strong>
- }
-
- function paintFail($message) {<strong>
- parent::paintFail($message);</strong>
- }
-}
-</pre>
- No output would come from this class until you add it.
- </p>
-
- <p>
-<a class="target" name="cli">
-<h2>The command line reporter</h2>
-</a>
-</p>
- <p>
- SimpleTest also ships with a minimal command line reporter.
- The interface mimics JUnit to some extent, but paints the
- failure messages as they arrive.
- To use the command line reporter simply substitute it
- for the HTML version...
-<pre>
-<?php
- require_once('simpletest/unit_tester.php');
- require_once('simpletest/reporter.php');
-
- $test = &new GroupTest('File test');
- $test->addTestFile('tests/file_test.php');
- $test->run(<strong>new TextReporter()</strong>);
-?>
-</pre>
- Then invoke the test suite from the command line...
-<pre class="shell">
-php file_test.php
-</pre>
- You will need the command line version of PHP installed
- of course.
- A passing test suite looks like this...
-<pre class="shell">
-File test
-OK
-Test cases run: 1/1, Failures: 0, Exceptions: 0
-</pre>
- A failure triggers a display like this...
-<pre class="shell">
-File test
-1) True assertion failed.
- in createnewfile
-FAILURES!!!
-Test cases run: 1/1, Failures: 1, Exceptions: 0
-</pre>
- </p>
- <p>
- One of the main reasons for using a command line driven
- test suite is of using the tester as part of some automated
- process.
- To function properly in shell scripts the test script should
- return a non-zero exit code on failure.
- If a test suite fails the value <span class="new_code">false</span>
- is returned from the <span class="new_code">SimpleTest::run()</span>
- method.
- We can use that result to exit the script with the desired return
- code...
-<pre>
-<?php
- require_once('simpletest/unit_tester.php');
- require_once('simpletest/reporter.php');
-
- $test = &new GroupTest('File test');
- $test->addTestFile('tests/file_test.php');
- <strong>exit ($test->run(new TextReporter()) ? 0 : 1);</strong>
-?>
-</pre>
- Of course we don't really want to create two test scripts,
- a command line one and a web browser one, for each test suite.
- The command line reporter includes a method to sniff out the
- run time environment...
-<pre>
-<?php
- require_once('simpletest/unit_tester.php');
- require_once('simpletest/reporter.php');
-
- $test = &new GroupTest('File test');
- $test->addTestFile('tests/file_test.php');
- <strong>if (TextReporter::inCli()) {</strong>
- exit ($test->run(new TextReporter()) ? 0 : 1);
- <strong>}</strong>
- $test->run(new HtmlReporter());
-?>
-</pre>
- This is the form used within SimpleTest itself.
- </p>
-
- <p>
-<a class="target" name="xml">
-<h2>Remote testing</h2>
-</a>
-</p>
- <p>
- SimpleTest ships with an <span class="new_code">XmlReporter</span> class
- used for internal communication.
- When run the output looks like...
-<pre class="shell">
-<?xml version="1.0"?>
-<run>
- <group size="4">
- <name>Remote tests</name>
- <group size="4">
- <name>Visual test with 48 passes, 48 fails and 4 exceptions</name>
- <case>
- <name>testofunittestcaseoutput</name>
- <test>
- <name>testofresults</name>
- <pass>This assertion passed</pass>
- <fail>This assertion failed</fail>
- </test>
- <test>
- ...
- </test>
- </case>
- </group>
- </group>
-</run>
-</pre>
- You can make use of this format with the parser
- supplied as part of SimpleTest itself.
- This is called <span class="new_code">SimpleTestXmlParser</span> and
- resides in <em>xml.php</em> within the SimpleTest package...
-<pre>
-<?php
- require_once('simpletest/xml.php');
-
- ...
- $parser = &new SimpleTestXmlParser(new HtmlReporter());
- $parser->parse($test_output);
-?>
-</pre>
- The <span class="new_code">$test_output</span> should be the XML format
- from the XML reporter, and could come from say a command
- line run of a test case.
- The parser sends events to the reporter just like any
- other test run.
- There are some odd occasions where this is actually useful.
- </p>
- <p>
- A problem with large test suites is thet they can exhaust
- the default 8Mb memory limit on a PHP process.
- By having the test groups output in XML and run in
- separate processes, the output can be reparsed to
- aggregate the results into a much smaller footprint top level
- test.
- </p>
- <p>
- Because the XML output can come from anywhere, this opens
- up the possibility of aggregating test runs from remote
- servers.
- A test case already exists to do this within the SimpleTest
- framework, but it is currently experimental...
-<pre>
-<?php
- <strong>require_once('../remote.php');</strong>
- require_once('../reporter.php');
-
- $test_url = ...;
- $dry_url = ...;
-
- $test = &new GroupTest('Remote tests');
- $test->addTestCase(<strong>new RemoteTestCase($test_url, $dry_url)</strong>);
- $test->run(new HtmlReporter());
-?>
-</pre>
- The <span class="new_code">RemoteTestCase</span> takes the actual location
- of the test runner, basically a web page in XML format.
- It also takes the URL of a reporter set to do a dry run.
- This is so that progress can be reported upward correctly.
- The <span class="new_code">RemoteTestCase</span> can be added to test suites
- just like any other group test.
- </p>
-
- </div>
-<div class="copyright">
- Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004
- </div>
-</body>
-</html>
|