diff options
Diffstat (limited to 'tests/test_tools/simpletest/docs/en')
14 files changed, 5606 insertions, 0 deletions
diff --git a/tests/test_tools/simpletest/docs/en/authentication_documentation.html b/tests/test_tools/simpletest/docs/en/authentication_documentation.html new file mode 100755 index 00000000..0623023c --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/authentication_documentation.html @@ -0,0 +1,320 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest documentation for testing log-in and authentication</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<span class="chosen">Authentication</span> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Authentication documentation</h1> +<div class="content"> + + <p> + One of the trickiest, and yet most important, areas + of testing web sites is the security. + Testing these schemes is one of the core goals of + the SimpleTest web tester. + </p> + + <p> +<a class="target" name="basic"> +<h2>Basic HTTP authentication</h2> +</a> +</p> + <p> + If you fetch a page protected by basic authentication then + rather than receiving content, you will instead get a 401 + header. + We can illustrate this with this test... +<pre> +class AuthenticationTest extends WebTestCase {<strong> + function test401Header() { + $this->get('http://www.lastcraft.com/protected/'); + $this->showHeaders(); + }</strong> +} +</pre> + This allows us to see the challenge header... + <div class="demo"> + <h1>File test</h1> +<pre style="background-color: lightgray; color: black"> +HTTP/1.1 401 Authorization Required +Date: Sat, 18 Sep 2004 19:25:18 GMT +Server: Apache/1.3.29 (Unix) PHP/4.3.4 +WWW-Authenticate: Basic realm="SimpleTest basic authentication" +Connection: close +Content-Type: text/html; charset=iso-8859-1 +</pre> + <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete. + <strong>0</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div> + </div> + We are trying to get away from visual inspection though, and so SimpleTest + allows to make automated assertions against the challenge. + Here is a thorough test of our header... +<pre> +class AuthenticationTest extends WebTestCase { + function test401Header() { + $this->get('http://www.lastcraft.com/protected/');<strong> + $this->assertAuthentication('Basic'); + $this->assertResponse(401); + $this->assertRealm('SimpleTest basic authentication');</strong> + } +} +</pre> + Any one of these tests would normally do on it's own depending + on the amount of detail you want to see. + </p> + <p> + Most of the time we are not interested in testing the + authentication itself, but want to get past it to test + the pages underneath. + As soon as the challenge has been issued we can reply with + an authentication response... +<pre> +class AuthenticationTest extends WebTestCase { + function testAuthentication() { + $this->get('http://www.lastcraft.com/protected/');<strong> + $this->authenticate('Me', 'Secret');</strong> + $this->assertTitle(...); + } +} +</pre> + The username and password will now be sent with every + subsequent request to that directory and subdirectories. + You will have to authenticate again if you step outside + the authenticated directory, but SimpleTest is smart enough + to merge subdirectories into a common realm. + </p> + <p> + You can shortcut this step further by encoding the log in + details straight into the URL... +<pre> +class AuthenticationTest extends WebTestCase { + function testCanReadAuthenticatedPages() { + $this->get('http://<strong>Me:Secret@</strong>www.lastcraft.com/protected/'); + $this->assertTitle(...); + } +} +</pre> + If your username or password has special characters, then you + will have to URL encode them or the request will not be parsed + correctly. + Also this header will not be sent on subsequent requests if + you request a page with a fully qualified URL. + If you navigate with relative URLs though, the authentication + information will be preserved. + </p> + <p> + Only basic authentication is currently supported and this is + only really secure in tandem with HTTPS connections. + This is usually enough to protect test server from prying eyes, + however. + Digest authentication and NTLM authentication may be added + in the future. + </p> + + <p> +<a class="target" name="cookies"> +<h2>Cookies</h2> +</a> +</p> + <p> + Basic authentication doesn't give enough control over the + user interface for web developers. + More likely this functionality will be coded directly into + the web architecture using cookies and complicated timeouts. + </p> + <p> + Starting with a simple log-in form... +<pre> +<form> + Username: + <input type="text" name="u" value="" /><br /> + Password: + <input type="password" name="p" value="" /><br /> + <input type="submit" value="Log in" /> +</form> +</pre> + Which looks like... + </p> + <p> + <form class="demo"> + Username: + <input type="text" name="u" value=""> +<br> + Password: + <input type="password" name="p" value=""> +<br> + <input type="submit" value="Log in"> + </form> + </p> + <p> + Let's suppose that in fetching this page a cookie has been + set with a session ID. + We are not going to fill the form in yet, just test that + we are tracking the user. + Here is the test... +<pre> +class LogInTest extends WebTestCase { + function testSessionCookieSetBeforeForm() { + $this->get('http://www.my-site.com/login.php');<strong> + $this->assertCookie('SID');</strong> + } +} +</pre> + All we are doing is confirming that the cookie is set. + As the value is likely to be rather cryptic it's not + really worth testing this. + </p> + <p> + The rest of the test would be the same as any other form, + but we might want to confirm that we still have the same + cookie after log-in as before we entered. + We wouldn't want to lose track of this after all. + Here is a possible test for this... +<pre> +class LogInTest extends WebTestCase { + ... + function testSessionCookieSameAfterLogIn() { + $this->get('http://www.my-site.com/login.php');<strong> + $session = $this->getCookie('SID'); + $this->setField('u', 'Me'); + $this->setField('p', 'Secret'); + $this->clickSubmit('Log in'); + $this->assertWantedPattern('/Welcome Me/'); + $this->assertCookie('SID', $session);</strong> + } +} +</pre> + This confirms that the session identifier is maintained + afer log-in. + </p> + <p> + We could even attempt to spoof our own system by setting + arbitrary cookies to gain access... +<pre> +class LogInTest extends WebTestCase { + ... + function testSessionCookieSameAfterLogIn() { + $this->get('http://www.my-site.com/login.php');<strong> + $this->setCookie('SID', 'Some other session'); + $this->get('http://www.my-site.com/restricted.php');</strong> + $this->assertWantedPattern('/Access denied/'); + } +} +</pre> + Is your site protected from this attack? + </p> + + <p> +<a class="target" name="session"> +<h2>Browser sessions</h2> +</a> +</p> + <p> + If you are testing an authentication system a critical piece + of behaviour is what happens when a user logs back in. + We would like to simulate closing and reopening a browser... +<pre> +class LogInTest extends WebTestCase { + ... + function testLoseAuthenticationAfterBrowserClose() { + $this->get('http://www.my-site.com/login.php'); + $this->setField('u', 'Me'); + $this->setField('p', 'Secret'); + $this->clickSubmit('Log in'); + $this->assertWantedPattern('/Welcome Me/');<strong> + + $this->restart(); + $this->get('http://www.my-site.com/restricted.php'); + $this->assertWantedPattern('/Access denied/');</strong> + } +} +</pre> + The <span class="new_code">WebTestCase::restart()</span> method will + preserve cookies that have unexpired timeouts, but throw away + those that are temporary or expired. + You can optionally specify the time and date that the restart + happened. + </p> + <p> + Expiring cookies can be a problem. + After all, if you have a cookie that expires after an hour, + you don't want to stall the test for an hour while the + cookie passes it's timeout. + </p> + <p> + To push the cookies over the hour limit you can age them + before you restart the session... +<pre> +class LogInTest extends WebTestCase { + ... + function testLoseAuthenticationAfterOneHour() { + $this->get('http://www.my-site.com/login.php'); + $this->setField('u', 'Me'); + $this->setField('p', 'Secret'); + $this->clickSubmit('Log in'); + $this->assertWantedPattern('/Welcome Me/'); + <strong> + $this->ageCookies(3600);</strong> + $this->restart(); + $this->get('http://www.my-site.com/restricted.php'); + $this->assertWantedPattern('/Access denied/'); + } +} +</pre> + After the restart it will appear that cookies are an + hour older and any that pass their expiry will have + disappeared. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/browser_documentation.html b/tests/test_tools/simpletest/docs/en/browser_documentation.html new file mode 100755 index 00000000..ef54aaea --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/browser_documentation.html @@ -0,0 +1,386 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest documentation for the scriptable web browser component</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<span class="chosen">Scriptable browser</span> +</li> +</ul> +</div> +</div> +<h1>PHP Scriptable Web Browser</h1> +<div class="content"> + + <p> + SimpleTest's web browser component can be used not just + outside of the <span class="new_code">WebTestCase</span> class, but also + independently of the SimpleTest framework itself. + </p> + + <p> +<a class="target" name="scripting"> +<h2>The Scriptable Browser</h2> +</a> +</p> + <p> + You can use the web browser in PHP scripts to confirm + services are up and running, or to extract information + from them at a regular basis. + For example, here is a small script to extract the current number of + open PHP 5 bugs from the <a href="http://www.php.net/">PHP web site</a>... +<pre> +<strong><?php + require_once('simpletest/browser.php'); + + $browser = &new SimpleBrowser(); + $browser->get('http://php.net/'); + $browser->clickLink('reporting bugs'); + $browser->clickLink('statistics'); + $page = $browser->clickLink('PHP 5 bugs only'); + preg_match('/status=Open.*?by=Any.*?(\d+)<\/a>/', $page, $matches); + print $matches[1]; +?></strong> +</pre> + There are simpler methods to do this particular example in PHP + of course. + For example you can just use the PHP <span class="new_code">file()</span> + command against what here is a pretty fixed page. + However, using the web browser for scripts allows authentication, + correct handling of cookies, automatic loading of frames, redirects, + form submission and the ability to examine the page headers. + Such methods are fragile against a site that is constantly + evolving and you would want a more direct way of accessing + data in a permanent set up, but for simple tasks this can provide + a very rapid solution. + </p> + <p> + All of the navigation methods used in the + <a href="web_tester_documentation.html">WebTestCase</a> + are present in the <span class="new_code">SimpleBrowser</span> class, but + the assertions are replaced with simpler accessors. + Here is a full list of the page navigation methods... + <table> +<tbody> + <tr> +<td><span class="new_code">addHeader($header)</span></td><td>Adds a header to every fetch</td> +</tr> + <tr> +<td><span class="new_code">useProxy($proxy, $username, $password)</span></td><td>Use this proxy from now on</td> +</tr> + <tr> +<td><span class="new_code">head($url, $parameters)</span></td><td>Perform a HEAD request</td> +</tr> + <tr> +<td><span class="new_code">get($url, $parameters)</span></td><td>Fetch a page with GET</td> +</tr> + <tr> +<td><span class="new_code">post($url, $parameters)</span></td><td>Fetch a page with POST</td> +</tr> + <tr> +<td><span class="new_code">clickLink($label)</span></td><td>Follows a link by label</td> +</tr> + <tr> +<td><span class="new_code">isLink($label)</span></td><td>See if a link is present by label</td> +</tr> + <tr> +<td><span class="new_code">clickLinkById($id)</span></td><td>Follows a link by attribute</td> +</tr> + <tr> +<td><span class="new_code">isLinkById($id)</span></td><td>See if a link is present by attribut</td> +</tr> + <tr> +<td><span class="new_code">getUrl()</span></td><td>Current URL of page or frame</td> +</tr> + <tr> +<td><span class="new_code">getTitle()</span></td><td>Page title</td> +</tr> + <tr> +<td><span class="new_code">getContent()</span></td><td>Raw page or frame</td> +</tr> + <tr> +<td><span class="new_code">getContentAsText()</span></td><td>HTML removed except for alt text</td> +</tr> + <tr> +<td><span class="new_code">retry()</span></td><td>Repeat the last request</td> +</tr> + <tr> +<td><span class="new_code">back()</span></td><td>Use the browser back button</td> +</tr> + <tr> +<td><span class="new_code">forward()</span></td><td>Use the browser forward button</td> +</tr> + <tr> +<td><span class="new_code">authenticate($username, $password)</span></td><td>Retry page or frame after a 401 response</td> +</tr> + <tr> +<td><span class="new_code">restart($date)</span></td><td>Restarts the browser for a new session</td> +</tr> + <tr> +<td><span class="new_code">ageCookies($interval)</span></td><td>Ages the cookies by the specified time</td> +</tr> + <tr> +<td><span class="new_code">setCookie($name, $value)</span></td><td>Sets an additional cookie</td> +</tr> + <tr> +<td><span class="new_code">getCookieValue($host, $path, $name)</span></td><td>Reads the most specific cookie</td> +</tr> + <tr> +<td><span class="new_code">getCurrentCookieValue($name)</span></td><td>Reads cookie for the current context</td> +</tr> + </tbody> +</table> + The methods <span class="new_code">SimpleBrowser::useProxy()</span> and + <span class="new_code">SimpleBrowser::addHeader()</span> are special. + Once called they continue to apply to all subsequent fetches. + </p> + <p> + Navigating forms is similar to the + <a href="form_testing_documentation.html">WebTestCase form navigation</a>... + <table> +<tbody> + <tr> +<td><span class="new_code">setField($name, $value)</span></td><td>Sets all form fields with that name</td> +</tr> + <tr> +<td><span class="new_code">setFieldById($id, $value)</span></td><td>Sets all form fields with that id</td> +</tr> + <tr> +<td><span class="new_code">getField($name)</span></td><td>Accessor for a form element value</td> +</tr> + <tr> +<td><span class="new_code">getFieldById($id)</span></td><td>Accessor for a form element value</td> +</tr> + <tr> +<td><span class="new_code">clickSubmit($label)</span></td><td>Submits form by button label</td> +</tr> + <tr> +<td><span class="new_code">clickSubmitByName($name)</span></td><td>Submits form by button attribute</td> +</tr> + <tr> +<td><span class="new_code">clickSubmitById($id)</span></td><td>Submits form by button attribute</td> +</tr> + <tr> +<td><span class="new_code">clickImage($label, $x, $y)</span></td><td>Clicks the image by alt text</td> +</tr> + <tr> +<td><span class="new_code">clickImageByName($name, $x, $y)</span></td><td>Clicks the image by attribute</td> +</tr> + <tr> +<td><span class="new_code">clickImageById($id, $x, $y)</span></td><td>Clicks the image by attribute</td> +</tr> + <tr> +<td><span class="new_code">submitFormById($id)</span></td><td>Submits by the form tag attribute</td> +</tr> + </tbody> +</table> + At the moment there aren't any methods to list available forms + and fields. + This will probably be added to later versions of SimpleTest. + </p> + <p> + Within a page, individual frames can be selected. + If no selection is made then all the frames are merged together + in one large conceptual page. + The content of the current page will be a concatenation of all of the + frames in the order that they were specified in the "frameset" + tags. + <table> +<tbody> + <tr> +<td><span class="new_code">getFrames()</span></td><td>A dump of the current frame structure</td> +</tr> + <tr> +<td><span class="new_code">getFrameFocus()</span></td><td>Current frame label or index</td> +</tr> + <tr> +<td><span class="new_code">setFrameFocusByIndex($choice)</span></td><td>Select a frame numbered from 1</td> +</tr> + <tr> +<td><span class="new_code">setFrameFocus($name)</span></td><td>Select frame by label</td> +</tr> + <tr> +<td><span class="new_code">clearFrameFocus()</span></td><td>Treat all the frames as a single page</td> +</tr> + </tbody> +</table> + When focused on a single frame, the content will come from + that frame only. + This includes links to click and forms to submit. + </p> + + <p> +<a class="target" name="debug"> +<h2>What went wrong?</h2> +</a> +</p> + <p> + All of this functionality is great when we actually manage to fetch pages, + but that doesn't always happen. + To help figure out what went wrong, the browser has some methods to + aid in debugging... + <table> +<tbody> + <tr> +<td><span class="new_code">setConnectionTimeout($timeout)</span></td><td>Close the socket on overrun</td> +</tr> + <tr> +<td><span class="new_code">getRequest()</span></td><td>Raw request header of page or frame</td> +</tr> + <tr> +<td><span class="new_code">getHeaders()</span></td><td>Raw response header of page or frame</td> +</tr> + <tr> +<td><span class="new_code">getTransportError()</span></td><td>Any socket level errors in the last fetch</td> +</tr> + <tr> +<td><span class="new_code">getResponseCode()</span></td><td>HTTP response of page or frame</td> +</tr> + <tr> +<td><span class="new_code">getMimeType()</span></td><td>Mime type of page or frame</td> +</tr> + <tr> +<td><span class="new_code">getAuthentication()</span></td><td>Authentication type in 401 challenge header</td> +</tr> + <tr> +<td><span class="new_code">getRealm()</span></td><td>Authentication realm in 401 challenge header</td> +</tr> + <tr> +<td><span class="new_code">setMaximumRedirects($max)</span></td><td>Number of redirects before page is loaded anyway</td> +</tr> + <tr> +<td><span class="new_code">setMaximumNestedFrames($max)</span></td><td>Protection against recursive framesets</td> +</tr> + <tr> +<td><span class="new_code">ignoreFrames()</span></td><td>Disables frames support</td> +</tr> + <tr> +<td><span class="new_code">useFrames()</span></td><td>Enables frames support</td> +</tr> + </tbody> +</table> + The methods <span class="new_code">SimpleBrowser::setConnectionTimeout()</span> + <span class="new_code">SimpleBrowser::setMaximumRedirects()</span>, + <span class="new_code">SimpleBrowser::setMaximumNestedFrames()</span>, + <span class="new_code">SimpleBrowser::ignoreFrames()</span> and + <span class="new_code">SimpleBrowser::useFrames()</span> continue to apply + to every subsequent request. + The other methods are frames aware. + This means that if you have an individual frame that is not + loading, navigate to it using <span class="new_code">SimpleBrowser::setFrameFocus()</span> + and you can then use <span class="new_code">SimpleBrowser::getRequest()</span>, etc to + see what happened. + </p> + + <p> +<a class="target" name="unit"> +<h2>Complex unit tests with multiple browsers</h2> +</a> +</p> + <p> + Anything that could be done in a + <a href="web_tester_documentation.html">WebTestCase</a> can + now be done in a <a href="unit_tester_documentation.html">UnitTestCase</a>. + This means that we can freely mix domain object testing with the + web interface... +<pre> +<strong> +class TestOfRegistration extends UnitTestCase { + function testNewUserAddedToAuthenticator() {</strong> + $browser = &new SimpleBrowser(); + $browser->get('http://my-site.com/register.php'); + $browser->setField('email', 'me@here'); + $browser->setField('password', 'Secret'); + $browser->clickSubmit('Register'); + <strong> + $authenticator = &new Authenticator(); + $member = &$authenticator->findByEmail('me@here'); + $this->assertEqual($member->getPassword(), 'Secret'); + } +}</strong> +</pre> + While this may be a useful temporary expediency, I am not a fan + of this type of testing. + The testing has cut across application layers, make it twice as + likely it will need refactoring when the code changes. + </p> + <p> + A more useful case of where using the browser directly can be helpful + is where the <span class="new_code">WebTestCase</span> cannot cope. + An example is where two browsers are needed at the same time. + </p> + <p> + For example, say we want to disallow multiple simultaneous + usage of a site with the same username. + This test case will do the job... +<pre> +class TestOfSecurity extends UnitTestCase { + function testNoMultipleLoginsFromSameUser() {<strong> + $first = &new SimpleBrowser(); + $first->get('http://my-site.com/login.php'); + $first->setField('name', 'Me'); + $first->setField('password', 'Secret'); + $first->clickSubmit('Enter'); + $this->assertEqual($first->getTitle(), 'Welcome'); + + $second = &new SimpleBrowser(); + $second->get('http://my-site.com/login.php'); + $second->setField('name', 'Me'); + $second->setField('password', 'Secret'); + $second->clickSubmit('Enter'); + $this->assertEqual($second->getTitle(), 'Access Denied');</strong> + } +} +</pre> + You can also use the <span class="new_code">SimpleBrowser</span> class + directly when you want to write test cases using a different + test tool than SimpleTest. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/docs.css b/tests/test_tools/simpletest/docs/en/docs.css new file mode 100755 index 00000000..93226cd7 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/docs.css @@ -0,0 +1,84 @@ +body { + padding-left: 3%; + padding-right: 3%; +} +pre { + font-family: courier; + font-size: 80%; + border: 1px solid; + background-color: #cccccc; + padding: 5px; + margin-left: 5%; + margin-right: 8%; +} +.code, .new_code, pre.new_code { + font-weight: bold; +} +div.copyright { + font-size: 80%; + color: gray; +} +div.copyright a { + color: gray; +} +ul.api { + padding-left: 0em; + padding-right: 25%; +} +ul.api li { + margin-top: 0.2em; + margin-bottom: 0.2em; + list-style: none; + text-indent: -3em; + padding-left: 3em; +} +div.demo { + border: 4px ridge; + border-color: gray; + padding: 10px; + margin: 5px; + margin-left: 20px; + margin-right: 40px; + background-color: white; +} +div.demo span.fail { + color: red; +} +div.demo span.pass { + color: green; +} +div.demo h1 { + font-size: 12pt; + text-align: left; + font-weight: bold; +} +table { + border: 2px outset; + border-color: gray; + background-color: white; + margin: 5px; + margin-left: 5%; + margin-right: 5%; +} +td { + font-size: 80%; +} +.shell { + color: white; +} +pre.shell { + border: 4px ridge; + border-color: gray; + padding: 10px; + margin: 5px; + margin-left: 20px; + margin-right: 40px; + background-color: black; +} +form.demo { + background-color: lightgray; + border: 4px outset; + border-color: lightgray; + padding: 10px; + margin-right: 40%; +} diff --git a/tests/test_tools/simpletest/docs/en/expectation_documentation.html b/tests/test_tools/simpletest/docs/en/expectation_documentation.html new file mode 100755 index 00000000..0165988c --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/expectation_documentation.html @@ -0,0 +1,356 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title> + Extending the SimpleTest unit tester with additional expectation classes + </title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<span class="chosen">Expectations</span> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Expectation documentation</h1> +<div class="content"> + <p> +<a class="target" name="mock"> +<h2>More control over mock objects</h2> +</a> +</p> + <p> + The default behaviour of the + <a href="mock_objects_documentation.html">mock objects</a> + in + <a href="http://sourceforge.net/projects/simpletest/">SimpleTest</a> + is either an identical match on the argument or to allow any argument at all. + For almost all tests this is sufficient. + Sometimes, though, you want to weaken a test case. + </p> + <p> + One place where a test can be too tightly coupled is with + text matching. + Suppose we have a component that outputs a helpful error + message when something goes wrong. + You want to test that the correct error was sent, but the actual + text may be rather long. + If you test for the text exactly, then every time the exact wording + of the message changes, you will have to go back and edit the test suite. + </p> + <p> + For example, suppose we have a news service that has failed + to connect to its remote source. +<pre> +<strong>class NewsService { + ... + function publish(&$writer) { + if (! $this->isConnected()) { + $writer->write('Cannot connect to news service "' . + $this->_name . '" at this time. ' . + 'Please try again later.'); + } + ... + } +}</strong> +</pre> + Here it is sending its content to a + <span class="new_code">Writer</span> class. + We could test this behaviour with a + <span class="new_code">MockWriter</span> like so... +<pre> +class TestOfNewsService extends UnitTestCase { + ... + function testConnectionFailure() {<strong> + $writer = &new MockWriter($this); + $writer->expectOnce('write', array( + 'Cannot connect to news service ' . + '"BBC News" at this time. ' . + 'Please try again later.')); + + $service = &new NewsService('BBC News'); + $service->publish($writer); + + $writer->tally();</strong> + } +} +</pre> + This is a good example of a brittle test. + If we decide to add additional instructions, such as + suggesting an alternative news source, we will break + our tests even though no underlying functionality + has been altered. + </p> + <p> + To get around this, we would like to do a regular expression + test rather than an exact match. + We can actually do this with... +<pre> +class TestOfNewsService extends UnitTestCase { + ... + function testConnectionFailure() { + $writer = &new MockWriter($this);<strong> + $writer->expectOnce( + 'write', + array(new WantedPatternExpectation('/cannot connect/i')));</strong> + + $service = &new NewsService('BBC News'); + $service->publish($writer); + + $writer->tally(); + } +} +</pre> + Instead of passing in the expected parameter to the + <span class="new_code">MockWriter</span> we pass an + expectation class called + <span class="new_code">WantedPatternExpectation</span>. + The mock object is smart enough to recognise this as special + and to treat it differently. + Rather than simply comparing the incoming argument to this + object, it uses the expectation object itself to + perform the test. + </p> + <p> + The <span class="new_code">WantedPatternExpectation</span> takes + the regular expression to match in its constructor. + Whenever a comparison is made by the <span class="new_code">MockWriter</span> + against this expectation class, it will do a + <span class="new_code">preg_match()</span> with this pattern. + With our test case above, as long as "cannot connect" + appears in the text of the string, the mock will issue a pass + to the unit tester. + The rest of the text does not matter. + </p> + <p> + The possible expectation classes are... + <table> +<tbody> + <tr> +<td><span class="new_code">EqualExpectation</span></td><td>An equality, rather than the stronger identity comparison</td> +</tr> + <tr> +<td><span class="new_code">NotEqualExpectation</span></td><td>An inequality comparison</td> +</tr> + <tr> +<td><span class="new_code">IndenticalExpectation</span></td><td>The default mock object check which must match exactly</td> +</tr> + <tr> +<td><span class="new_code">NotIndenticalExpectation</span></td><td>Inverts the mock object logic</td> +</tr> + <tr> +<td><span class="new_code">WantedPatternExpectation</span></td><td>Uses a Perl Regex to match a string</td> +</tr> + <tr> +<td><span class="new_code">NoUnwantedPatternExpectation</span></td><td>Passes only if failing a Perl Regex</td> +</tr> + <tr> +<td><span class="new_code">IsAExpectation</span></td><td>Checks the type or class name only</td> +</tr> + <tr> +<td><span class="new_code">NotAExpectation</span></td><td>Opposite of the <span class="new_code">IsAExpectation</span></td> +</tr> + <tr> +<td><span class="new_code">MethodExistsExpectation</span></td><td>Checks a method is available on an object</td> +</tr> + </tbody> +</table> + Most take the expected value in the constructor. + The exceptions are the pattern matchers, which take a regular expression, + and the <span class="new_code">IsAExpectation</span> and <span class="new_code">NotAExpectation</span> which takes a type + or class name as a string. + </p> + + <p> +<a class="target" name="behaviour"> +<h2>Using expectations to control stubs</h2> +</a> +</p> + <p> + The expectation classes can be used not just for sending assertions + from mock objects, but also for selecting behaviour for either + the + <a href="mock_objects_documentation.html">mock objects</a> + or the + <a href="server_stubs_documentation.html">server stubs</a>. + Anywhere a list of arguments is given, a list of expectation objects + can be inserted instead. + </p> + <p> + Suppose we want an authorisation server stub to simulate a successful login + only if it receives a valid session object. + We can do this as follows... +<pre> +Stub::generate('Authorisation'); +<strong> +$authorisation = new StubAuthorisation(); +$authorisation->setReturnValue( + 'isAllowed', + true, + array(new IsAExpectation('Session', 'Must be a session'))); +$authorisation->setReturnValue('isAllowed', false);</strong> +</pre> + We have set the default stub behaviour to return false when + <span class="new_code">isAllowed</span> is called. + When we call the method with a single parameter that + is a <span class="new_code">Session</span> object, it will return true. + We have also added a second parameter as a message. + This will be displayed as part of the mock object + failure message if this expectation is the cause of + a failure. + </p> + <p> + This kind of sophistication is rarely useful, but is included for + completeness. + </p> + + <p> +<a class="target" name="extending"> +<h2>Creating your own expectations</h2> +</a> +</p> + <p> + The expectation classes have a very simple structure. + So simple that it is easy to create your own versions for + commonly used test logic. + </p> + <p> + As an example here is the creation of a class to test for + valid IP addresses. + In order to work correctly with the stubs and mocks the new + expectation class should extend + <span class="new_code">SimpleExpectation</span>... +<pre> +<strong>class ValidIp extends SimpleExpectation { + + function test($ip) { + return (ip2long($ip) != -1); + } + + function testMessage($ip) { + return "Address [$ip] should be a valid IP address"; + } +}</strong> +</pre> + There are only two methods to implement. + The <span class="new_code">test()</span> method should + evaluate to true if the expectation is to pass, and + false otherwise. + The <span class="new_code">testMessage()</span> method + should simply return some helpful text explaining the test + that was carried out. + </p> + <p> + This class can now be used in place of the earlier expectation + classes. + </p> + + <p> +<a class="target" name="unit"> +<h2>Under the bonnet of the unit tester</h2> +</a> +</p> + <p> + The <a href="http://sourceforge.net/projects/simpletest/">SimpleTest unit testing framework</a> + also uses the expectation classes internally for the + <a href="unit_test_documentation.html">UnitTestCase class</a>. + We can also take advantage of these mechanisms to reuse our + homebrew expectation classes within the test suites directly. + </p> + <p> + The most crude way of doing this is to use the + <span class="new_code">SimpleTest::assertExpectation()</span> method to + test against it directly... +<pre> +<strong>class TestOfNetworking extends UnitTestCase { + ... + function testGetValidIp() { + $server = &new Server(); + $this->assertExpectation( + new ValidIp(), + $server->getIp(), + 'Server IP address->%s'); + } +}</strong> +</pre> + This is a little untidy compared with our usual + <span class="new_code">assert...()</span> syntax. + </p> + <p> + For such a simple case we would normally create a + separate assertion method on our test case rather + than bother using the expectation class. + If we pretend that our expectation is a little more + complicated for a moment, so that we want to reuse it, + we get... +<pre> +class TestOfNetworking extends UnitTestCase { + ...<strong> + function assertValidIp($ip, $message = '%s') { + $this->assertExpectation(new ValidIp(), $ip, $message); + }</strong> + + function testGetValidIp() { + $server = &new Server();<strong> + $this->assertValidIp( + $server->getIp(), + 'Server IP address->%s');</strong> + } +} +</pre> + It is unlikely we would ever need this degree of control + over the testing machinery. + It is rare to need the expectations for more than pattern + matching. + Also, complex expectation classes could make the tests + harder to read and debug. + These mechanisms are really of most use to authors of systems + that will extend the test framework to create their own tool set. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/form_testing_documentation.html b/tests/test_tools/simpletest/docs/en/form_testing_documentation.html new file mode 100755 index 00000000..b1e15b3d --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/form_testing_documentation.html @@ -0,0 +1,277 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>Simple Test documentation for testing HTML forms</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<span class="chosen">Testing forms</span> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Form testing documentation</h1> +<div class="content"> + <p> +<a class="target" name="submit"> +<h2>Submitting a simple form</h2> +</a> +</p> + <p> + When a page is fetched by the <span class="new_code">WebTestCase</span> + using <span class="new_code">get()</span> or + <span class="new_code">post()</span> the page content is + automatically parsed. + This results in any form controls that are inside <form> tags + being available from within the test case. + For example, if we have this snippet of HTML... +<pre> +<form> + <input type="text" name="a" value="A default" /> + <input type="submit" value="Go" /> +</form> +</pre> + Which looks like this... + </p> + <p> + <form class="demo"> + <input type="text" name="a" value="A default"> + <input type="submit" value="Go"> + </form> + </p> + <p> + We can navigate to this code, via the + <a href="http://www.lastcraft.com/form_testing_documentation.php">LastCraft</a> + site, with the following test... +<pre> +class SimpleFormTests extends WebTestCase { + <strong> + function testDefaultValue() { + $this->get('http://www.lastcraft.com/form_testing_documentation.php'); + $this->assertField('a', 'A default'); + }</strong> +} +</pre> + Immediately after loading the page all of the HTML controls are set at + their default values just as they would appear in the web browser. + The assertion tests that a HTML widget exists in the page with the + name "a" and that it is currently set to the value + "A default" + </p> + <p> + We could submit the form straight away, but first we'll change + the value of the text field and only then submit it... +<pre> +class SimpleFormTests extends WebTestCase { + + function testDefaultValue() { + $this->get('http://www.my-site.com/'); + $this->assertField('a', 'A default');<strong> + $this->setField('a', 'New value'); + $this->clickSubmit('Go');</strong> + } +} +</pre> + Because we didn't specify a method attribute on the form tag, and + didn't specify an action either, the test case will follow + the usual browser behaviour of submitting the form data as a <em>GET</em> + request back to the same location. + SimpleTest tries to emulate typical browser behaviour as much as possible, + rather than attempting to catch missing attributes on tags. + This is because the target of the testing framework is the PHP application + logic, not syntax or other errors in the HTML code. + For HTML errors, other tools such as + <a href="http://www.w3.org/People/Raggett/tidy/">HTMLTidy</a> should be used. + </p> + <p> + If a field is not present in any form, or if an option is unavailable, + then <span class="new_code">WebTestCase::setField()</span> will return + <span class="new_code">false</span>. + For example, suppose we wish to verify that a "Superuser" + option is not present in this form... +<pre> +<strong>Select type of user to add:</strong> +<select name="type"> + <option>Subscriber</option> + <option>Author</option> + <option>Administrator</option> +</select> +</pre> + Which looks like... + </p> + <p> + <form class="demo"> + <strong>Select type of user to add:</strong> + <select name="type"> + <option>Subscriber</option> + <option>Author</option> + <option>Administrator</option> + </select> + </form> + </p> + <p> + The following test will confirm it... +<pre> +class SimpleFormTests extends WebTestCase { + ... + function testNoSuperuserChoiceAvailable() {<strong> + $this->get('http://www.lastcraft.com/form_testing_documentation.php'); + $this->assertFalse($this->setField('type', 'Superuser'));</strong> + } +} +</pre> + The selection will not be changed on a failure to set + a widget value. + </p> + <p> + Here is the full list of widgets currently supported... + <ul> + <li>Text fields, including hidden and password fields.</li> + <li>Submit buttons including the button tag, although not yet reset buttons</li> + <li>Text area. This includes text wrapping behaviour.</li> + <li>Checkboxes, including multiple checkboxes in the same form.</li> + <li>Drop down selections, including multiple selects.</li> + <li>Radio buttons.</li> + <li>Images.</li> + </ul> + </p> + <p> + Although most standard HTML widgets are catered for by <em>SimpleTest</em>'s + built in parser, it is unlikely that JavaScript will be implemented + anytime soon. + </p> + + <p> +<a class="target" name="multiple"> +<h2>Fields with multiple values</h2> +</a> +</p> + <p> + SimpleTest can cope with two types of multivalue controls: Multiple + selection drop downs, and multiple checkboxes with the same name + within a form. + The multivalue nature of these means that setting and testing + are slightly different. + Using checkboxes as an example... +<pre> +<form class="demo"> + <strong>Create privileges allowed:</strong> + <input type="checkbox" name="crud" value="c" checked><br> + <strong>Retrieve privileges allowed:</strong> + <input type="checkbox" name="crud" value="r" checked><br> + <strong>Update privileges allowed:</strong> + <input type="checkbox" name="crud" value="u" checked><br> + <strong>Destroy privileges allowed:</strong> + <input type="checkbox" name="crud" value="d" checked><br> + <input type="submit" value="Enable Privileges"> +</form> +</pre> + Which renders as... + </p> + <p> + <form class="demo"> + <strong>Create privileges allowed:</strong> + <input type="checkbox" name="crud" value="c" checked> +<br> + <strong>Retrieve privileges allowed:</strong> + <input type="checkbox" name="crud" value="r" checked> +<br> + <strong>Update privileges allowed:</strong> + <input type="checkbox" name="crud" value="u" checked> +<br> + <strong>Destroy privileges allowed:</strong> + <input type="checkbox" name="crud" value="d" checked> +<br> + <input type="submit" value="Enable Privileges"> + </form> + </p> + <p> + If we wish to disable all but the retrieval privileges and + submit this information we can do it like this... +<pre> +class SimpleFormTests extends WebTestCase { + ...<strong> + function testDisableNastyPrivileges() { + $this->get('http://www.lastcraft.com/form_testing_documentation.php'); + $this->assertField('crud', array('c', 'r', 'u', 'd')); + $this->setField('crud', array('r')); + $this->clickSubmit('Enable Privileges'); + }</strong> +} +</pre> + Instead of setting the field to a single value, we give it a list + of values. + We do the same when testing expected values. + We can then write other test code to confirm the effect of this, perhaps + by logging in as that user and attempting an update. + </p> + <p> + <a class="target" name="raw"> +<h2>Raw posting</h2> +</a> + </p> + <p> + If you want to test a form handler, but have not yet written + or do not have access to the form itself, you can create a + form submission by hand. +<pre> +class SimpleFormTests extends WebTestCase { + ...<strong> + function testAttemptedHack() { + $this->post( + 'http://www.my-site.com/add_user.php', + array('type' => 'superuser')); + $this->assertNoUnwantedPattern('/user created/i'); + }</strong> +} +</pre> + By adding data to the <span class="new_code">WebTestCase::post()</span> + method, we are attempting to fetch the page as a form submission. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/group_test_documentation.html b/tests/test_tools/simpletest/docs/en/group_test_documentation.html new file mode 100755 index 00000000..adbc66ef --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/group_test_documentation.html @@ -0,0 +1,357 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP group test documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<span class="chosen">Group tests</span> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Group Test documentation</h1> +<div class="content"> + <p> +<a class="target" name="group"> +<h2>Grouping tests</h2> +</a> +</p> + <p> + To run test cases as part of a group the test cases should really + be placed in files without the runner code... +<pre> +<strong><?php + require_once('../classes/io.php'); + + class FileTester extends UnitTestCase { + ... + } + + class SocketTester extends UnitTestCase { + ... + } +?></strong> +</pre> + As many cases as needed can appear in a single file. + They should include any code they need, such as the library + being tested, but none of the simple test libraries. + </p> + <p> + If you have extended any test cases, you can include them + as well. +<pre> +<?php + require_once('../classes/io.php'); +<strong> + class MyFileTestCase extends UnitTestCase { + ... + } + SimpleTestOptions::ignore('MyFileTestCase');</strong> + + class FileTester extends MyFileTestCase { + ... + } + + class SocketTester extends UnitTestCase { + ... + } +?> +</pre> + The <span class="new_code">FileTester</span> class does + not contain any actual tests, but is a base class for other + test cases. + For this reason we use the + <span class="new_code">SimpleTestOptions::ignore()</span> directive + to tell the upcoming group test to ignore it. + This directive can appear anywhere in the file and works + when a whole file of test cases is loaded (see below). + We will call this sample <em>file_test.php</em>. + </p> + <p> + Next we create a group test file, called say <em>group_test.php</em>. + You will think of a better name I am sure. + We will add the test file using a safe method... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php');<strong> + require_once('file_test.php'); + + $test = &new GroupTest('All file tests'); + $test->addTestCase(new FileTestCase()); + $test->run(new HtmlReporter());</strong> +?> +</pre> + This instantiates the test case before the test suite is + run. + This could get a little expensive with a large number of test + cases, so another method is provided that will only + instantiate the class when it is needed... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + require_once('file_test.php'); + + $test = &new GroupTest('All file tests');<strong> + $test->addTestClass('FileTestCase');</strong> + $test->run(new HtmlReporter()); +?> +</pre> + The problem with this method is that for every test case + that we add we will have + to <span class="new_code">require_once()</span> the test code + file and manually instantiate each and every test case. + We can save a lot of typing with... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + + $test = &new GroupTest('All file tests');<strong> + $test->addTestFile('file_test.php');</strong> + $test->run(new HtmlReporter()); +?&gt; +</pre> + What happens here is that the <span class="new_code">GroupTest</span> + class has done the <span class="new_code">require_once()</span> + for us. + It then checks to see if any new test case classes + have been created by the new file and automatically adds + them to the group test. + Now all we have to do is add each new file. + </p> + <p> + There are two things that could go wrong and which require care... + <ol> + <li> + The file could already have been parsed by PHP and so no + new classes will have been added. You should make + sure that the test cases are only included in this file + and no others. + </li> + <li> + New test case extension classes that get included will be + placed in the group test and run also. + You will need to add a <span class="new_code">SimpleTestOptions::ignore()</span> + directive for these classes or make sure that they are included + before the <span class="new_code">GroupTest::addTestFile()</span> + line. + </li> + </ol> + </p> + + <p> +<a class="target" name="higher"> +<h2>Higher groupings</h2> +</a> +</p> + <p> + The above method places all of the test cases into one large group. + For larger projects though this may not be flexible enough; you + may want to group the tests in all sorts of ways. + </p> + <p> + To get a more flexible group test we can subclass + <span class="new_code">GroupTest</span> and then instantiate it as needed... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + <strong> + class FileGroupTest extends GroupTest { + function FileGroupTest() { + $this->GroupTest('All file tests'); + $this->addTestFile('file_test.php'); + } + }</strong> +?> +</pre> + This effectively names the test in the constructor and then + adds our test cases and a single group below. + Of course we can add more than one group at this point. + We can now invoke the tests from a separate runner file... +<pre> +<?php + require_once('file_group_test.php'); + <strong> + $test = &new FileGroupTest(); + $test->run(new HtmlReporter());</strong> +?> +</pre> + ...or we can group them into even larger group tests... +<pre> +<?php + require_once('file_group_test.php'); + <strong> + $test = &new BigGroupTest('Big group'); + $test->addTestCase(new FileGroupTest()); + $test->addTestCase(...); + $test->run(new HtmlReporter());</strong> +?> +</pre> + If we still wish to run the original group test and we + don't want all of these little runner files, we can + put the test runner code around guard bars when we create + each group. +<pre> +<?php + class FileGroupTest extends GroupTest { + function FileGroupTest() { + $this->GroupTest('All file tests'); + $test->addTestFile('file_test.php'); + } + } + <strong> + if (! defined('RUNNER')) { + define('RUNNER', true);</strong> + $test = &new FileGroupTest(); + $test->run(new HtmlReporter()); + } +?> +</pre> + This approach requires the guard to be set when including + the group test file, but this is still less hassle than + lots of separate runner files. + You include the same guard on the top level tests to make sure + that <span class="new_code">run()</span> will run once only + from the top level script that has been invoked. +<pre> +<?php<strong> + define('RUNNER', true);</strong> + require_once('file_group_test.php'); + + $test = &new BigGroupTest('Big group'); + $test->addTestCase(new FileGroupTest()); + $test->addTestCase(...); + $test->run(new HtmlReporter()); +?> +</pre> + As with the normal test cases, a <span class="new_code">GroupTest</span> can + be loaded with the <span class="new_code">GroupTest::addTestFile()</span> method. +<pre> +<?php + define('RUNNER', true); + + $test = &new BigGroupTest('Big group');<strong> + $test->addTestFile('file_group_test.php'); + $test->addTestFile(...);</strong> + $test->run(new HtmlReporter()); +?> +</pre> + </p> + + <p> +<a class="target" name="legacy"> +<h2>Integrating legacy test cases</h2> +</a> +</p> + <p> + If you already have unit tests for your code or are extending external + classes that have tests, it is unlikely that all of the test cases + are in SimpleTest format. + Fortunately it is possible to incorporate test cases from other + unit testers directly into SimpleTest group tests. + </p> + <p> + Say we have the following + <a href="http://sourceforge.net/projects/phpunit">PhpUnit</a> + test case in the file <em>config_test.php</em>... +<pre> +<strong>class ConfigFileTest extends TestCase { + function ConfigFileTest() { + $this->TestCase('Config file test'); + } + + function testContents() { + $config = new ConfigFile('test.conf'); + $this->assertRegexp('/me/', $config->getValue('username')); + } +}</strong> +</pre> + The group test can recognise this as long as we include + the appropriate adapter class before we add the test + file... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php');<strong> + require_once('simpletest/adapters/phpunit_test_case.php');</strong> + + $test = &new GroupTest('All file tests');<strong> + $test->addTestFile('config_test.php');</strong> + $test->run(new HtmlReporter()); +?> +</pre> + There are only two adapters, the other is for the + <a href="http://pear.php.net/manual/en/package.php.phpunit.php">PEAR</a> + 1.0 unit tester... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php');<strong> + require_once('simpletest/adapters/pear_test_case.php');</strong> + + $test = &new GroupTest('All file tests');<strong> + $test->addTestFile('some_pear_test_cases.php');</strong> + $test->run(new HtmlReporter()); +?> +</pre> + The PEAR test cases can be freely mixed with SimpleTest + ones even in the same test file, + but you cannot use SimpleTest assertions in the legacy + test case versions. + This is done as a check that you are not accidently making + your test cases completely dependent on SimpleTest. + You may want to do a PEAR release of your library for example + which would mean shipping it with valid PEAR::PhpUnit test + cases. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/index.html b/tests/test_tools/simpletest/docs/en/index.html new file mode 100755 index 00000000..04797272 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/index.html @@ -0,0 +1,467 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title> + Download the Simple Test testing framework - + Unit tests and mock objects for PHP + </title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<span class="chosen">SimpleTest</span> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Simple Test for PHP</h1> +<div class="content"> + + + <p> + The following assumes that you are familiar with the concept + of unit testing as well as the PHP web development language. + It is a guide for the impatient new user of + <a href="https://sourceforge.net/project/showfiles.php?group_id=76550">SimpleTest</a>. + For fuller documentation, especially if you are new + to unit testing see the ongoing + <a href="unit_test_documentation.html">documentation</a>, and for + example test cases see the + <a href="http://www.lastcraft.com/first_test_tutorial.php">unit testing tutorial</a>. + </p> + + <p> +<a class="target" name="unit"> +<h2>Using the tester quickly</h2> +</a> +</p> + <p> + Amongst software testing tools, a unit tester is the one + closest to the developer. + In the context of agile development the test code sits right + next to the source code as both are written simultaneously. + In this context SimpleTest aims to be a complete PHP developer + test solution and is called "Simple" because it + should be easy to use and extend. + It wasn't a good choice of name really. + It includes all of the typical functions you would expect from + <a href="http://www.junit.org/">JUnit</a> and the + <a href="http://sourceforge.net/projects/phpunit/">PHPUnit</a> + ports, but also adds + <a href="http://www.mockobjects.com">mock objects</a>. + It has some <a href="http://sourceforge.net/projects/jwebunit/">JWebUnit</a> + functionality as well. + This includes web page navigation, cookie testing and form submission. + </p> + <p> + The quickest way to demonstrate is with an example. + </p> + <p> + Let us suppose we are testing a simple file logging class called + <span class="new_code">Log</span> in <em>classes/log.php</em>. + We start by creating a test script which we will call + <em>tests/log_test.php</em> and populate it as follows... +<pre> +<strong><?php +require_once('simpletest/unit_tester.php'); +require_once('simpletest/reporter.php'); +require_once('../classes/log.php'); +?></strong> +</pre> + Here the <em>simpletest</em> folder is either local or in the path. + You would have to edit these locations depending on where you + placed the toolset. + Next we create a test case... +<pre> +<?php +require_once('simpletest/unit_tester.php'); +require_once('simpletest/reporter.php'); +require_once('../classes/log.php'); +<strong> +class TestOfLogging extends UnitTestCase { +}</strong> +?> +</pre> + Now we have five lines of scaffolding code and still no tests. + However from this part on we get return on our investment very quickly. + We'll assume that the <span class="new_code">Log</span> class + takes the file name to write to in the constructor and we have + a temporary folder in which to place this file... +<pre> +<?php +require_once('simpletest/unit_tester.php'); +require_once('simpletest/reporter.php'); +require_once('../classes/log.php'); + +class TestOfLogging extends UnitTestCase { + <strong> + function testCreatingNewFile() { + @unlink('/temp/test.log'); + $log = new Log('/temp/test.log'); + $this->assertFalse(file_exists('/temp/test.log')); + $log->message('Should write this to a file'); + $this->assertTrue(file_exists('/temp/test.log')); + }</strong> +} +?> +</pre> + When a test case runs it will search for any method that + starts with the string <span class="new_code">test</span> + and execute that method. + We would normally have more than one test method of course. + Assertions within the test methods trigger messages to the + test framework which displays the result immediately. + This immediate response is important, not just in the event + of the code causing a crash, but also so that + <span class="new_code">print</span> statements can display + their content right next to the test case concerned. + </p> + <p> + To see these results we have to actually run the tests. + If this is the only test case we wish to run we can achieve + it with... +<pre> +<?php +require_once('simpletest/unit_tester.php'); +require_once('simpletest/reporter.php'); +require_once('../classes/log.php'); + +class TestOfLogging extends UnitTestCase { + + function testCreatingNewFile() { + @unlink('/temp/test.log'); + $log = new Log('/temp/test.log'); + $this->assertFalse(file_exists('/temp/test.log')); + $log->message('Should write this to a file'); + $this->assertTrue(file_exists('/temp/test.log')); + } +} +<strong> +$test = &new TestOfLogging(); +$test->run(new HtmlReporter());</strong> +?> +</pre> + </p> + <p> + On failure the display looks like this... + <div class="demo"> + <h1>testoflogging</h1> + <span class="fail">Fail</span>: testcreatingnewfile->True assertion failed.<br> + <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete. + <strong>1</strong> passes and <strong>1</strong> fails.</div> + </div> + ...and if it passes like this... + <div class="demo"> + <h1>testoflogging</h1> + <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete. + <strong>2</strong> passes and <strong>0</strong> fails.</div> + </div> + And if you get this... + <div class="demo"> + <b>Fatal error</b>: Failed opening required '../classes/log.php' (include_path='') in <b>/home/marcus/projects/lastcraft/tutorial_tests/Log/tests/log_test.php</b> on line <b>7</b> + </div> + it means you're missing the <em>classes/Log.php</em> file that could look like... +<pre> +<?php +class Log { + + function Log($file_path) { + } + + function message() { + } +} +?>; +</pre> + </p> + + <p> +<a class="target" name="group"> +<h2>Building group tests</h2> +</a> +</p> + <p> + It is unlikely in a real application that we will only ever run + one test case. + This means that we need a way of grouping cases into a test + script that can, if need be, run every test in the application. + </p> + <p> + Our first step is to strip the includes and to undo our + previous hack... +<pre> +<?php<strong> +require_once('../classes/log.php');</strong> + +class TestOfLogging extends UnitTestCase { + + function testCreatingNewFile() { + @unlink('/temp/test.log'); + $log = new Log('/temp/test.log'); + $this->assertFalse(file_exists('/temp/test.log')); + $log->message('Should write this to a file'); + $this->assertTrue(file_exists('/temp/test.log'));<strong> + } +} +?></strong> +</pre> + Next we create a new file called <em>tests/all_tests.php</em> + and insert the following code... +<pre> +<strong><?php +require_once('simpletest/unit_tester.php'); +require_once('simpletest/reporter.php'); + +$test = &new GroupTest('All tests'); +$test->addTestFile('log_test.php'); +$test->run(new HtmlReporter()); +?></strong> +</pre> + The method <span class="new_code">GroupTest::addTestFile()</span> + will include the test case file and read any new classes created + that are descended from <span class="new_code">SimpleTestCase</span>, of which + <span class="new_code">UnitTestCase</span> is one example. + Just the class names are stored for now, so that the test runner + can instantiate the class when it works its way + through your test suite. + </p> + <p> + For this to work properly the test case file should not blindly include + any other test case extensions that do not actually run tests. + This could result in extra test cases being counted during the test + run. + Hardly a major problem, but to avoid this inconvenience simply add + a <span class="new_code">SimpleTestOptions::ignore()</span> directive + somewhere in the test case file. + Also the test case file should not have been included + elsewhere or no cases will be added to this group test. + This would be a more serious error as if the test case classes are + already loaded by PHP the <span class="new_code">GroupTest::addTestFile()</span> + method will not detect them. + </p> + <p> + To display the results it is necessary only to invoke + <em>tests/all_tests.php</em> from the web server. + </p> + + <p> +<a class="target" name="mock"> +<h2>Using mock objects</h2> +</a> +</p> + <p> + Let's move further into the future. + </p> + <p> + Assume that our logging class is tested and completed. + Assume also that we are testing another class that is + required to write log messages, say a + <span class="new_code">SessionPool</span>. + We want to test a method that will probably end up looking + like this... +<pre> +<strong> +class SessionPool { + ... + function logIn($username) { + ... + $this->_log->message("User $username logged in."); + ... + } + ... +} +</strong> +</pre> + In the spirit of reuse we are using our + <span class="new_code">Log</span> class. + A conventional test case might look like this... +<pre> +<strong> +<?php +require_once('../classes/log.php'); +require_once('../classes/session_pool.php'); + +class TestOfSessionLogging extends UnitTestCase { + + function setUp() { + @unlink('/temp/test.log'); + } + + function tearDown() { + @unlink('/temp/test.log'); + } + + function testLogInIsLogged() { + $log = new Log('/temp/test.log'); + $session_pool = &new SessionPool($log); + $session_pool->logIn('fred'); + $messages = file('/temp/test.log'); + $this->assertEqual($messages[0], "User fred logged in.\n"); + } +} +?></strong> +</pre> + This test case design is not all bad, but it could be improved. + We are spending time fiddling with log files which are + not part of our test. Worse, we have created close ties + with the <span class="new_code">Log</span> class and + this test. + What if we don't use files any more, but use ths + <em>syslog</em> library instead? + Did you notice the extra carriage return in the message? + Was that added by the logger? + What if it also added a time stamp or other data? + </p> + <p> + The only part that we really want to test is that a particular + message was sent to the logger. + We reduce coupling if we can pass in a fake logging class + that simply records the message calls for testing, but + takes no action. + It would have to look exactly like our original though. + </p> + <p> + If the fake object doesn't write to a file then we save on deleting + the file before and after each test. We could save even more + test code if the fake object would kindly run the assertion for us. + <p> + </p> + Too good to be true? + Luckily we can create such an object easily... +<pre> +<?php +require_once('../classes/log.php'); +require_once('../classes/session_pool.php');<strong> +Mock::generate('Log');</strong> + +class TestOfSessionLogging extends UnitTestCase { + + function testLogInIsLogged() {<strong> + $log = &new MockLog($this); + $log->expectOnce('message', array('User fred logged in.'));</strong> + $session_pool = &new SessionPool($log); + $session_pool->logIn('fred');<strong> + $log->tally();</strong> + } +} +?> +</pre> + The <span class="new_code">tally()</span> call is needed to + tell the mock object that time is up for the expected call + count. + Without it the mock would wait forever for the method + call to come in without ever actually notifying the test case. + The other test will be triggered when the call to + <span class="new_code">message()</span> is invoked on the + <span class="new_code">MockLog</span> object. + The mock call will trigger a parameter comparison and then send the + resulting pass or fail event to the test display. + Wildcards can be included here too so as to prevent tests + becoming too specific. + </p> + <p> + The mock objects in the SimpleTest suite can have arbitrary + return values set, sequences of returns, return values + selected according to the incoming arguments, sequences of + parameter expectations and limits on the number of times + a method is to be invoked. + </p> + <p> + For this test to run the mock objects library must have been + included in the test suite, say in <em>all_tests.php</em>. + </p> + + <p> +<a class="target" name="web"> +<h2>Web page testing</h2> +</a> +</p> + <p> + One of the requirements of web sites is that they produce web + pages. + If you are building a project top-down and you want to fully + integrate testing along the way then you will want a way of + automatically navigating a site and examining output for + correctness. + This is the job of a web tester. + </p> + <p> + The web testing in SimpleTest is fairly primitive, there is + no JavaScript for example. + To give an idea here is a trivial example where a home + page is fetched, from which we navigate to an "about" + page and then test some client determined content. +<pre> +<?php<strong> +require_once('simpletest/web_tester.php');</strong> +require_once('simpletest/reporter.php'); +<strong> +class TestOfAbout extends WebTestCase { + + function setUp() { + $this->get('http://test-server/index.php'); + $this->clickLink('About'); + } + + function testSearchEngineOptimisations() { + $this->assertTitle('A long title about us for search engines'); + $this->assertWantedPattern('/a popular keyphrase/i'); + } +}</strong> +$test = &new TestOfAbout(); +$test->run(new HtmlReporter()); +?> +</pre> + With this code as an acceptance test you can ensure that + the content always meets the specifications of both the + developers and the other project stakeholders. + </p> + <p> + <a href="http://sourceforge.net/projects/simpletest/"><img src="http://sourceforge.net/sflogo.php?group_id=76550&type=5" width="210" height="62" border="0" alt="SourceForge.net Logo"></a> + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/mock_objects_documentation.html b/tests/test_tools/simpletest/docs/en/mock_objects_documentation.html new file mode 100755 index 00000000..2f8a1f90 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/mock_objects_documentation.html @@ -0,0 +1,713 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP mock objects documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<span class="chosen">Mock objects</span> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Mock objects documentation</h1> +<div class="content"> + <p> +<a class="target" name="what"> +<h2>What are mock objects?</h2> +</a> +</p> + <p> + Mock objects have two roles during a test case: actor and critic. + </p> + <p> + The actor behaviour is to simulate objects that are difficult to + set up or time consuming to set up for a test. + The classic example is a database connection. + Setting up a test database at the start of each test would slow + testing to a crawl and would require the installation of the + database engine and test data on the test machine. + If we can simulate the connection and return data of our + choosing we not only win on the pragmatics of testing, but can + also feed our code spurious data to see how it responds. + We can simulate databases being down or other extremes + without having to create a broken database for real. + In other words, we get greater control of the test environment. + </p> + <p> + If mock objects only behaved as actors they would simply be + known as <a href="server_stubs_documentation.html">server stubs</a>. + </p> + <p> + However, the mock objects not only play a part (by supplying chosen + return values on demand) they are also sensitive to the + messages sent to them (via expectations). + By setting expected parameters for a method call they act + as a guard that the calls upon them are made correctly. + If expectations are not met they save us the effort of + writing a failed test assertion by performing that duty on our + behalf. + In the case of an imaginary database connection they can + test that the query, say SQL, was correctly formed by + the object that is using the connection. + Set them up with fairly tight expectations and you will + hardly need manual assertions at all. + </p> + + <p> +<a class="target" name="creation"> +<h2>Creating mock objects</h2> +</a> +</p> + <p> + In the same way that we create server stubs, all we need is an + existing class, say a database connection that looks like this... +<pre> +<strong>class DatabaseConnection { + function DatabaseConnection() { + } + + function query() { + } + + function selectQuery() { + } +}</strong> +</pre> + The class does not need to have been implemented yet. + To create a mock version of the class we need to include the + mock object library and run the generator... +<pre> +<strong>require_once('simpletest/unit_tester.php'); +require_once('simpletest/mock_objects.php'); +require_once('database_connection.php'); + +Mock::generate('DatabaseConnection');</strong> +</pre> + This generates a clone class called + <span class="new_code">MockDatabaseConnection</span>. + We can now create instances of the new class within + our test case... +<pre> +require_once('simpletest/unit_tester.php'); +require_once('simpletest/mock_objects.php'); +require_once('database_connection.php'); + +Mock::generate('DatabaseConnection'); +<strong> +class MyTestCase extends UnitTestCase { + + function testSomething() { + $connection = &new MockDatabaseConnection($this); + } +}</strong> +</pre> + Unlike the generated stubs the mock constructor needs a reference + to the test case so that it can dispatch passes and failures while + checking its expectations. + This means that mock objects can only be used within test cases. + Despite this their extra power means that stubs are hardly ever used + if mocks are available. + </p> + <p> + <a class="target" name="stub"> +<h2>Mocks as actors</h2> +</a> + </p> + <p> + The mock version of a class has all the methods of the original + so that operations like + <span class="new_code">$connection->query()</span> are still + legal. + As with stubs we can replace the default null return values... +<pre> +<strong>$connection->setReturnValue('query', 37);</strong> +</pre> + Now every time we call + <span class="new_code">$connection->query()</span> we get + the result of 37. + As with the stubs we can set wildcards and we can overload the + wildcard parameter. + We can also add extra methods to the mock when generating it + and choose our own class name... +<pre> +<strong>Mock::generate('DatabaseConnection', 'MyMockDatabaseConnection', array('setOptions'));</strong> +</pre> + Here the mock will behave as if the <span class="new_code">setOptions()</span> + existed in the original class. + This is handy if a class has used the PHP <span class="new_code">overload()</span> + mechanism to add dynamic methods. + You can create a special mock to simulate this situation. + </p> + <p> + All of the patterns available with server stubs are available + to mock objects... +<pre> +class Iterator { + function Iterator() { + } + + function next() { + } +} +</pre> + Again, assuming that this iterator only returns text until it + reaches the end, when it returns false, we can simulate it + with... +<pre> +Mock::generate('Iterator'); + +class IteratorTest extends UnitTestCase() { + + function testASequence() {<strong> + $iterator = &new MockIterator($this); + $iterator->setReturnValue('next', false); + $iterator->setReturnValueAt(0, 'next', 'First string'); + $iterator->setReturnValueAt(1, 'next', 'Second string');</strong> + ... + } +} +</pre> + When <span class="new_code">next()</span> is called on the + mock iterator it will first return "First string", + on the second call "Second string" will be returned + and on any other call <span class="new_code">false</span> will + be returned. + The sequenced return values take precedence over the constant + return value. + The constant one is a kind of default if you like. + </p> + <p> + A repeat of the stubbed information holder with name/value pairs... +<pre> +class Configuration { + function Configuration() { + } + + function getValue($key) { + } +} +</pre> + This is a classic situation for using mock objects as + actual configuration will vary from machine to machine, + hardly helping the reliability of our tests if we use it + directly. + The problem though is that all the data comes through the + <span class="new_code">getValue()</span> method and yet + we want different results for different keys. + Luckily the mocks have a filter system... +<pre> +<strong>$config = &new MockConfiguration($this); +$config->setReturnValue('getValue', 'primary', array('db_host')); +$config->setReturnValue('getValue', 'admin', array('db_user')); +$config->setReturnValue('getValue', 'secret', array('db_password'));</strong> +</pre> + The extra parameter is a list of arguments to attempt + to match. + In this case we are trying to match only one argument which + is the look up key. + Now when the mock object has the + <span class="new_code">getValue()</span> method invoked + like this... +<pre> +$config->getValue('db_user') +</pre> + ...it will return "admin". + It finds this by attempting to match the calling arguments + to its list of returns one after another until + a complete match is found. + </p> + <p> + There are times when you want a specific object to be + dished out by the mock rather than a copy. + Again this is identical to the server stubs mechanism... +<pre> +class Thing { +} + +class Vector { + function Vector() { + } + + function get($index) { + } +} +</pre> + In this case you can set a reference into the mock's + return list... +<pre> +$thing = new Thing();<strong> +$vector = &new MockVector($this); +$vector->setReturnReference('get', $thing, array(12));</strong> +</pre> + With this arrangement you know that every time + <span class="new_code">$vector->get(12)</span> is + called it will return the same + <span class="new_code">$thing</span> each time. + </p> + + <p> +<a class="target" name="expectations"> +<h2>Mocks as critics</h2> +</a> +</p> + <p> + Although the server stubs approach insulates your tests from + real world disruption, it is only half the benefit. + You can have the class under test receiving the required + messages, but is your new class sending correct ones? + Testing this can get messy without a mock objects library. + </p> + <p> + By way of example, suppose we have a + <span class="new_code">SessionPool</span> class that we + want to add logging to. + Rather than grow the original class into something more + complicated, we want to add this behaviour with a decorator (GOF). + The <span class="new_code">SessionPool</span> code currently looks + like this... +<pre> +<strong>class SessionPool { + function SessionPool() { + ... + } + + function &findSession($cookie) { + ... + } + ... +} + +class Session { + ... +}</strong> +</php> + While our logging code looks like this... +<php><strong> +class Log { + function Log() { + ... + } + + function message() { + ... + } +} + +class LoggingSessionPool { + function LoggingSessionPool(&$session_pool, &$log) { + ... + } + + function &findSession(\$cookie) { + ... + } + ... +}</strong> +</pre> + Out of all of this, the only class we want to test here + is the <span class="new_code">LoggingSessionPool</span>. + In particular we would like to check that the + <span class="new_code">findSession()</span> method is + called with the correct session ID in the cookie and that + it sent the message "Starting session $cookie" + to the logger. + </p> + <p> + Despite the fact that we are testing only a few lines of + production code, here is what we would have to do in a + conventional test case: + <ol> + <li>Create a log object.</li> + <li>Set a directory to place the log file.</li> + <li>Set the directory permissions so we can write the log.</li> + <li>Create a <span class="new_code">SessionPool</span> object.</li> + <li>Hand start a session, which probably does lot's of things.</li> + <li>Invoke <span class="new_code">findSession()</span>.</li> + <li>Read the new Session ID (hope there is an accessor!).</li> + <li>Raise a test assertion to confirm that the ID matches the cookie.</li> + <li>Read the last line of the log file.</li> + <li>Pattern match out the extra logging timestamps, etc.</li> + <li>Assert that the session message is contained in the text.</li> + </ol> + It is hardly surprising that developers hate writing tests + when they are this much drudgery. + To make things worse, every time the logging format changes or + the method of creating new sessions changes, we have to rewrite + parts of this test even though this test does not officially + test those parts of the system. + We are creating headaches for the writers of these other classes. + </p> + <p> + Instead, here is the complete test method using mock object magic... +<pre> +Mock::generate('Session'); +Mock::generate('SessionPool'); +Mock::generate('Log'); + +class LoggingSessionPoolTest extends UnitTestCase { + ... + function testFindSessionLogging() {<strong> + $session = &new MockSession($this); + $pool = &new MockSessionPool($this); + $pool->setReturnReference('findSession', $session); + $pool->expectOnce('findSession', array('abc')); + + $log = &new MockLog($this); + $log->expectOnce('message', array('Starting session abc')); + + $logging_pool = &new LoggingSessionPool($pool, $log); + $this->assertReference($logging_pool->findSession('abc'), $session); + $pool->tally(); + $log->tally();</strong> + } +} +</pre> + We start by creating a dummy session. + We don't have to be too fussy about this as the check + for which session we want is done elsewhere. + We only need to check that it was the same one that came + from the session pool. + </p> + <p> + <span class="new_code">findSession()</span> is a factory + method the simulation of which is described <a href="#stub">above</a>. + The point of departure comes with the first + <span class="new_code">expectOnce()</span> call. + This line states that whenever + <span class="new_code">findSession()</span> is invoked on the + mock, it will test the incoming arguments. + If it receives the single argument of a string "abc" + then a test pass is sent to the unit tester, otherwise a fail is + generated. + This was the part where we checked that the right session was asked for. + The argument list follows the same format as the one for setting + return values. + You can have wildcards and sequences and the order of + evaluation is the same. + </p> + <p> + If the call is never made then neither a pass nor a failure will + generated. + To get around this we must tell the mock when the test is over + so that the object can decide if the expectation has been met. + The unit tester assertion for this is triggered by the + <span class="new_code">tally()</span> call at the end of + the test. + </p> + <p> + We use the same pattern to set up the mock logger. + We tell it that it should have + <span class="new_code">message()</span> invoked + once only with the argument "Starting session abc". + By testing the calling arguments, rather than the logger output, + we insulate the test from any display changes in the logger. + </p> + <p> + We start to run our tests when we create the new + <span class="new_code">LoggingSessionPool</span> and feed + it our preset mock objects. + Everything is now under our control. + Finally we confirm that the + <span class="new_code">$session</span> we gave our decorator + is the one that we get back and tell the mocks to run their + internal call count tests with the + <span class="new_code">tally()</span> calls. + </p> + <p> + This is still quite a bit of test code, but the code is very + strict. + If it still seems rather daunting there is a lot less of it + than if we tried this without mocks and this particular test, + interactions rather than output, is always more work to set + up. + More often you will be testing more complex situations without + needing this level or precision. + Also some of this can be refactored into a test case + <span class="new_code">setUp()</span> method. + </p> + <p> + Here is the full list of expectations you can set on a mock object + in <a href="http://www.lastcraft.com/simple_test.php">SimpleTest</a>... + <table> +<thead> + <tr> +<th>Expectation</th><th>Needs <span class="new_code">tally()</span></th> +</tr> + </thead> +<tbody> +<tr> + <td><span class="new_code">expectArguments($method, $args)</span></td> + <td style="text-align: center">No</td> + </tr> + <tr> + <td><span class="new_code">expectArgumentsAt($timing, $method, $args)</span></td> + <td style="text-align: center">No</td> + </tr> + <tr> + <td><span class="new_code">expectCallCount($method, $count)</span></td> + <td style="text-align: center">Yes</td> + </tr> + <tr> + <td><span class="new_code">expectMaximumCallCount($method, $count)</span></td> + <td style="text-align: center">No</td> + </tr> + <tr> + <td><span class="new_code">expectMinimumCallCount($method, $count)</span></td> + <td style="text-align: center">Yes</td> + </tr> + <tr> + <td><span class="new_code">expectNever($method)</span></td> + <td style="text-align: center">No</td> + </tr> + <tr> + <td><span class="new_code">expectOnce($method, $args)</span></td> + <td style="text-align: center">Yes</td> + </tr> + <tr> + <td><span class="new_code">expectAtLeastOnce($method, $args)</span></td> + <td style="text-align: center">Yes</td> + </tr> + </tbody> +</table> + Where the parameters are... + <dl> + <dt class="new_code">$method</dt> + <dd>The method name, as a string, to apply the condition to.</dd> + <dt class="new_code">$args</dt> + <dd> + The arguments as a list. Wildcards can be included in the same + manner as for <span class="new_code">setReturn()</span>. + This argument is optional for <span class="new_code">expectOnce()</span> + and <span class="new_code">expectAtLeastOnce()</span>. + </dd> + <dt class="new_code">$timing</dt> + <dd> + The only point in time to test the condition. + The first call starts at zero. + </dd> + <dt class="new_code">$count</dt> + <dd>The number of calls expected.</dd> + </dl> + The method <span class="new_code">expectMaximumCallCount()</span> + is slightly different in that it will only ever generate a failure. + It is silent if the limit is never reached. + </p> + <p> + Like the assertions within test cases, all of the expectations + can take a message override as an extra parameter. + Also the original failure message can be embedded in the output + as "%s". + </p> + + <p> +<a class="target" name="approaches"> +<h2>Other approaches</h2> +</a> +</p> + <p> + There are three approaches to creating mocks including the one + that SimpleTest employs. + Coding them by hand using a base class, generating them to + a file and dynamically generating them on the fly. + </p> + <p> + Mock objects generated with <a href="simple_test.html">SimpleTest</a> + are dynamic. + They are created at run time in memory, using + <span class="new_code">eval()</span>, rather than written + out to a file. + This makes the mocks easy to create, a one liner, + especially compared with hand + crafting them in a parallel class hierarchy. + The problem is that the behaviour is usually set up in the tests + themselves. + If the original objects change the mock versions + that the tests rely on can get out of sync. + This can happen with the parallel hierarchy approach as well, + but is far more quickly detected. + </p> + <p> + The solution, of course, is to add some real integration + tests. + You don't need very many and the convenience gained + from the mocks more than outweighs the small amount of + extra testing. + You cannot trust code that was only tested with mocks. + </p> + <p> + If you are still determined to build static libraries of mocks + because you want to simulate very specific behaviour, you can + achieve the same effect using the SimpleTest class generator. + In your library file, say <em>mocks/connection.php</em> for a + database connection, create a mock and inherit to override + special methods or add presets... +<pre> +<?php + require_once('simpletest/mock_objects.php'); + require_once('../classes/connection.php'); +<strong> + Mock::generate('Connection', 'BasicMockConnection'); + class MockConnection extends BasicMockConnection { + function MockConnection(&$test, $wildcard = '*') { + $this->BasicMockConnection($test, $wildcard); + $this->setReturn('query', false); + } + }</strong> +?> +</pre> + The generate call tells the class generator to create + a class called <span class="new_code">BasicMockConnection</span> + rather than the usual <span class="new_code">MockConnection</span>. + We then inherit from this to get our version of + <span class="new_code">MockConnection</span>. + By intercepting in this way we can add behaviour, here setting + the default value of <span class="new_code">query()</span> to be false. + By using the default name we make sure that the mock class + generator will not recreate a different one when invoked elsewhere in the + tests. + It never creates a class if it already exists. + As long as the above file is included first then all tests + that generated <span class="new_code">MockConnection</span> should + now be using our one instead. + If we don't get the order right and the mock library + creates one first then the class creation will simply fail. + </p> + <p> + Use this trick if you find you have a lot of common mock behaviour + or you are getting frequent integration problems at later + stages of testing. + </p> + + <p> +<a class="target" name="other_testers"> +<h2>I think SimpleTest stinks!</h2> +</a> +</p> + <p> + But at the time of writing it is the only one with mock objects, + so are you stuck with it? + </p> + <p> + No, not at all. + <a href="simple_test.html">SimpleTest</a> is a toolkit and one of those + tools is the mock objects which can be employed independently. + Suppose you have your own favourite unit tester and all your current + test cases are written using it. + Pretend that you have called your unit tester PHPUnit (everyone else has) + and the core test class looks like this... +<pre> +class PHPUnit { + function PHPUnit() { + } + + function assertion($message, $assertion) { + } + ... +} +</pre> + All the <span class="new_code">assertion()</span> method does + is print some fancy output and the boolean assertion parameter determines + whether to print a pass or a failure. + Let's say that it is used like this... +<pre> +$unit_test = new PHPUnit(); +$unit_test>assertion('I hope this file exists', file_exists('my_file')); +</pre> + How do you use mocks with this? + </p> + <p> + There is a protected method on the base mock class + <span class="new_code">SimpleMock</span> called + <span class="new_code">_assertTrue()</span> and + by overriding this method we can use our own assertion format. + We start with a subclass, in say <em>my_mock.php</em>... +<pre> +<strong><?php + require_once('simpletest/mock_objects.php'); + + class MyMock extends SimpleMock() { + function MyMock(&$test, $wildcard) { + $this->SimpleMock($test, $wildcard); + } + + function _assertTrue($assertion, $message) { + $test = &$this->getTest(); + $test->assertion($message, $assertion); + } + } +?></strong> +</pre> + Now instantiating <span class="new_code">MyMock</span> will create + an object that speaks the same language as your tester. + The catch is of course that we never create such an object, the + code generator does. + We need just one more line of code to tell the generator to use + your mock instead... +<pre> +<?php + require_once('simpletst/mock_objects.php'); + + class MyMock extends SimpleMock() { + function MyMock($test, $wildcard) { + $this->SimpleMock(&$test, $wildcard); + } + + function _assertTrue($assertion, $message , &$test) { + $test->assertion($message, $assertion); + } + }<strong> + SimpleTestOptions::setMockBaseClass('MyMock');</strong> +?> +</pre> + From now on you just include <em>my_mock.php</em> instead of the + default <em>mock_objects.php</em> version and you can introduce + mock objects into your existing test suite. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/overview.html b/tests/test_tools/simpletest/docs/en/overview.html new file mode 100755 index 00000000..d4965de3 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/overview.html @@ -0,0 +1,422 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title> + Overview and feature list for the SimpleTest PHP unit tester and web tester + </title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<span class="chosen">Overview</span> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Overview of SimpleTest</h1> +<div class="content"> + <p> +<a class="target" name="summary"> +<h2>What is SimpleTest?</h2> +</a> +</p> + <p> + The heart of SimpleTest is a testing framework built around + test case classes. + These are written as extensions of base test case classes, + each extended with methods that actually contain test code. + Top level test scripts then invoke the <span class="new_code">run()</span> + methods on every one of these test cases in order. + Each test method is written to invoke various assertions that + the developer expects to be true such as + <span class="new_code">assertEqual()</span>. + If the expectation is correct, then a successful result is dispatched to the + observing test reporter, but any failure triggers an alert + and a description of the mismatch. + </p> + <p> + A <a href="unit_test_documentation.html">test case</a> looks like this... +<pre> +<?php +class <strong>MyTestCase</strong> extends UnitTestCase { + <strong> + function testLog() { + $log = &new Log('my.log'); + $log->message('Hello'); + $this->assertTrue(file_exists('my.log')); + }</strong> +} +?> +</pre> + </p> + <p> + These tools are designed for the developer. + Tests are written in the PHP language itself more or less + as the application itself is built. + The advantage of using PHP itself as the testing language is that + there are no new languages to learn, testing can start straight away, + and the developer can test any part of the code. + Basically, all parts that can be accessed by the application code can also be + accessed by the test code if they are in the same language. + </p> + <p> + The simplest type of test case is the + <a href="unit_tester_documentation.html">UnitTestCase</a>. + This class of test case includes standard tests for equality, + references and pattern matching. + All these test the typical expectations of what you would + expect the result of a function or method to be. + This is by far the most common type of test in the daily + routine of development, making up about 95% of test cases. + </p> + <p> + The top level task of a web application though is not to + produce correct output from its methods and objects, but + to generate web pages. + The <a href="web_tester_documentation.html">WebTestCase</a> class tests web + pages. + It simulates a web browser requesting a page, complete with + cookies, proxies, secure connections, authentication, forms, frames and most + navigation elements. + With this type of test case, the developer can assert that + information is present in the page and that forms and + sessions are handled correctly. + </p> + <p> + A <a href="web_tester_documentation.html">WebTestCase</a> looks like this... +<pre> +<?php +class <strong>MySiteTest</strong> extends WebTestCase { + <strong> + function testHomePage() { + $this->get('http://www.my-site.com/index.php'); + $this->assertTitle('My Home Page'); + $this->clickLink('Contact'); + $this->assertTitle('Contact me'); + $this->assertWantedPattern('/Email me at/'); + }</strong> +} +?> +</pre> + </p> + + <p> +<a class="target" name="features"> +<h2>Feature list</h2> +</a> +</p> + <p> + The following is a very rough outline of past and future features + and their expected point of release. + I am afraid it is liable to change without warning as meeting the + milestones rather depends on time available. + Green stuff has been coded, but not necessarily released yet. + If you have a pressing need for a green but unreleased feature + then you should check-out the code from sourceforge CVS directly. + A released feature is marked as "Done". + <table> +<thead> + <tr> +<th>Feature</th><th>Description</th><th>Release</th> +</tr> + </thead> +<tbody> +<tr> + <td>Unit test case</td> + <td>Core test case class and assertions</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Html display</td> + <td>Simplest possible display</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Autoloading of test cases</td> + <td> + Reading a file with test cases and loading them into a + group test automatically + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Mock objects code generator</td> + <td> + Objects capable of simulating other objects removing + test dependencies + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Server stubs</td> + <td> + Mocks without expectations to be used outside of test cases, + e.g. for prototyping + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Integration of other unit testers</td> + <td> + The ability to read and simulate test cases from PHPUnit + and PEAR::PhpUnit + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Web test case</td> + <td>Basic pattern matching of fetched pages</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>HTML parsing of pages</td> + <td>Allows link following and title tag matching</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Partial mocks</td> + <td> + Mocking parts of a class for testing less than a class + or for complex simulations + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Web cookie handling</td> + <td>Correct handling of cookies when fetching pages</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Following redirects</td> + <td>Page fetching automatically follows 300 redirects</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Form parsing</td> + <td>Ability to submit simple forms and read default form values</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Command line interface</td> + <td>Test display without the need of a web browser</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Exposure of expectation classes</td> + <td>Can create precise tests with mocks as well as test cases</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>XML output and parsing</td> + <td> + Allows multi host testing and the integration of acceptance + testing extensions + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Command line test case</td> + <td>Allows testing of utilities and file handling</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>PHP Documentor compatibility</td> + <td>Fully generated class level documentation</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Browser interface</td> + <td> + Exposure of lower level web browser interface for more + detailed test cases + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>HTTP authentication</td> + <td> + Fetching protected web pages with basic authentication + only + </td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Browser navigation buttons</td> + <td>Back, forward and retry</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>SSL support</td> + <td>Can connect to https: pages</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Proxy support</td> + <td>Can connect via. common proxies</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Frames support</td> + <td>Handling of frames in web test cases</td> + <td style="color: green;">Done</td> + </tr> + <tr> + <td>Improved display</td> + <td>Better web GUI with tree display of test cases</td> + <td style="color: red;">1.1</td> + </tr> + <tr> + <td>Localisation</td> + <td>Messages abstracted and code generated from XML</td> + <td style="color: red;">1.1</td> + </tr> + <tr> + <td>File upload testing</td> + <td>Can simulate the input type file tag</td> + <td style="color: red;">1.1</td> + </tr> + <tr> + <td>Mocking interfaces</td> + <td>Can generate mock objects to interfaces as well as classes</td> + <td style="color: red;">2.0</td> + </tr> + <tr> + <td>Testing exceptions</td> + <td>Similar to testing PHP errors</td> + <td style="color: red;">2.0</td> + </tr> + <tr> + <td>XPath searching of elements</td> + <td>Can make use of HTML tidy for faster and more flexible content matching</td> + <td style="color: red;">2.0</td> + </tr> + </tbody> +</table> + PHP5 migraton will start straight after the version 1.1 series, + whereupon PHP4 will no longer be supported. + SimpleTest is currently compatible with PHP5, but will not + make use of all of the new features until version 2. + </p> + + <p> +<a class="target" name="resources"> +<h2>Web resources for testing</h2> +</a> +</p> + <p> + Process is at least as important as tools. + The type of process that makes the heaviest use of a developer's + testing tool is of course + <a href="http://www.extremeprogramming.org/">Extreme Programming</a>. + This is one of the + <a href="http://www.agilealliance.com/articles/index">Agile Methodologies</a> + which combine various practices to "flatten the cost curve" of software development. + More extreme still is <a href="http://www.testdriven.com/modules/news/">Test Driven Development</a>, + where you very strictly adhere to the rule of no coding until you have a test. + If you're more of a planner or believe that experience trumps evolution, + you may prefer the + <a href="http://www.therationaledge.com/content/dec_01/f_spiritOfTheRUP_pk.html">RUP</a> approach. + I haven't tried it, but even I can see that you will need test tools (see figure 9). + </p> + <p> + Most unit testers clone <a href="http://www.junit.org/">JUnit</a> to some degree, + as far as the interface at least. There is a wealth of information on the + JUnit site including the + <a href="http://junit.sourceforge.net/doc/faq/faq.htm">FAQ</a> + which contains plenty of general advice on testing. + Once you get bitten by the bug you will certainly appreciate the phrase + <a href="http://junit.sourceforge.net/doc/testinfected/testing.htm">test infected</a> + coined by Eric Gamma. + If you are still reviewing which unit tester to use the main choices + are <a href="http://phpunit.sourceforge.net/">PHPUnit</a> + and <a href="http://pear.php.net/manual/en/package.php.phpunit.php">Pear PHP::PHPUnit</a>. + They currently lack a lot of features found in + <a href="http://www.lastcraft.com/simple_test.php">SimpleTest</a>, but the PEAR + version at least has been upgraded for PHP5 and is recommended if you are porting + existing <a href="http://www.junit.org/">JUnit</a> test cases. + </p> + <p> + Library writers don't seem to ship tests with their code very often + which is a shame. + Library code that includes tests can be more safely refactored and + the test code can act as additional documentation in a fairly standard + form. + This can save trawling the source code for clues when problems occour, + especially when upgrading such a library. + Libraries using SimpleTest for their unit testing include + <a href="http://wact.sourceforge.net/">WACT</a> and + <a href="http://sourceforge.net/projects/htmlsax">PEAR::XML_HTMLSax</a>. + </p> + <p> + There is currently a sad lack of material on mock objects, which is a shame + as unit testing without them is a lot more work. + The <a href="http://www.sidewize.com/company/mockobjects.pdf">original mock objects paper</a> + is very Java focused, but still worth a read. + As a new technology there are plenty of discussions and debate on how to use mocks, + often on Wikis such as + <a href="http://xpdeveloper.com/cgi-bin/oldwiki.cgi?MockObjects">Extreme Tuesday</a> + or <a href="http://www.mockobjects.com/wiki/MocksObjectsPaper">www.mockobjects.com</a> + or <a href="http://c2.com/cgi/wiki?MockObject">the original C2 Wiki</a>. + Injecting mocks into a class is the main area of debate for which this + <a href="http://www-106.ibm.com/developerworks/java/library/j-mocktest.html">paper on IBM</a> + makes a good starting point. + </p> + <p> + There are plenty of web testing tools, but most are written in Java and + tutorials and advice are rather thin on the ground. + The only hope is to look at the documentation for + <a href="http://httpunit.sourceforge.net/">HTTPUnit</a>, + <a href="http://htmlunit.sourceforge.net/">HTMLUnit</a> + or <a href="http://jwebunit.sourceforge.net/">JWebUnit</a> and hope for clues. + There are some XML driven test frameworks, but again most + require Java to run. + As SimpleTest does not support JavaScript you would probably + have to look at these tools anyway if you have highly dynamic + pages. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/partial_mocks_documentation.html b/tests/test_tools/simpletest/docs/en/partial_mocks_documentation.html new file mode 100755 index 00000000..20749415 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/partial_mocks_documentation.html @@ -0,0 +1,426 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP partial mocks documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<span class="chosen">Partial mocks</span> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Partial mock objects documentation</h1> +<div class="content"> + + <p> + A partial mock is simply a pattern to alleviate a specific problem + in testing with mock objects, + that of getting mock objects into tight corners. + It's quite a limited tool and possibly not even a good idea. + It is included with SimpleTest because I have found it useful + on more than one occasion and has saved a lot of work at that point. + </p> + + <p> +<a class="target" name="inject"> +<h2>The mock injection problem</h2> +</a> +</p> + <p> + When one object uses another it is very simple to just pass a mock + version in already set up with its expectations. + Things are rather tricker if one object creates another and the + creator is the one you want to test. + This means that the created object should be mocked, but we can + hardly tell our class under test to create a mock instead. + The tested class doesn't even know it is running inside a test + after all. + </p> + <p> + For example, suppose we are building a telnet client and it + needs to create a network socket to pass its messages. + The connection method might look something like... +<pre> +<strong><?php + require_once('socket.php'); + + class Telnet { + ... + function &connect($ip, $port, $username, $password) { + $socket = &new Socket($ip, $port); + $socket->read( ... ); + ... + } + } +?></strong> +</pre> + We would really like to have a mock object version of the socket + here, what can we do? + </p> + <p> + The first solution is to pass the socket in as a parameter, + forcing the creation up a level. + Having the client handle this is actually a very good approach + if you can manage it and should lead to factoring the creation from + the doing. + In fact, this is one way in which testing with mock objects actually + forces you to code more tightly focused solutions. + They improve your programming. + </p> + <p> + Here this would be... +<pre> +<?php + require_once('socket.php'); + + class Telnet { + ... + <strong>function &connect(&$socket, $username, $password) { + $socket->read( ... ); + ... + }</strong> + } +?> +</pre> + This means that the test code is typical for a test involving + mock objects. +<pre> +class TelnetTest extends UnitTestCase { + ... + function testConnection() {<strong> + $socket = &new MockSocket($this); + ... + $telnet = &new Telnet(); + $telnet->connect($socket, 'Me', 'Secret'); + ...</strong> + } +} +</pre> + It is pretty obvious though that one level is all you can go. + You would hardly want your top level application creating + every low level file, socket and database connection ever + needed. + It wouldn't know the constructor parameters anyway. + </p> + <p> + The next simplest compromise is to have the created object passed + in as an optional parameter... +<pre> +<?php + require_once('socket.php'); + + class Telnet { + ...<strong> + function &connect($ip, $port, $username, $password, $socket = false) { + if (!$socket) { + $socket = &new Socket($ip, $port); + } + $socket->read( ... );</strong> + ... + return $socket; + } + } +?> +</pre> + For a quick solution this is usually good enough. + The test now looks almost the same as if the parameter + was formally passed... +<pre> +class TelnetTest extends UnitTestCase { + ... + function testConnection() {<strong> + $socket = &new MockSocket($this); + ... + $telnet = &new Telnet(); + $telnet->connect('127.0.0.1', 21, 'Me', 'Secret', &$socket); + ...</strong> + } +} +</pre> + The problem with this approach is its untidiness. + There is test code in the main class and parameters passed + in the test case that are never used. + This is a quick and dirty approach, but nevertheless effective + in most situations. + </p> + <p> + The next method is to pass in a factory object to do the creation... +<pre> +<?php + require_once('socket.php'); + + class Telnet {<strong> + function Telnet(&$network) { + $this->_network = &$network; + }</strong> + ... + function &connect($ip, $port, $username, $password) {<strong> + $socket = &$this->_network->createSocket($ip, $port); + $socket->read( ... );</strong> + ... + return $socket; + } + } +?> +</pre> + This is probably the most highly factored answer as creation + is now moved into a small specialist class. + The networking factory can now be tested separately, but mocked + easily when we are testing the telnet class... +<pre> +class TelnetTest extends UnitTestCase { + ... + function testConnection() {<strong> + $socket = &new MockSocket($this); + ... + $network = &new MockNetwork($this); + $network->setReturnReference('createSocket', $socket); + $telnet = &new Telnet($network); + $telnet->connect('127.0.0.1', 21, 'Me', 'Secret'); + ...</strong> + } +} +</pre> + The downside is that we are adding a lot more classes to the + library. + Also we are passing a lot of factories around which will + make the code a little less intuitive. + The most flexible solution, but the most complex. + </p> + <p> + Is there a middle ground? + </p> + + <p> +<a class="target" name="creation"> +<h2>Protected factory method</h2> +</a> +</p> + <p> + There is a way we can circumvent the problem without creating + any new application classes, but it involves creating a subclass + when we do the actual testing. + Firstly we move the socket creation into its own method... +<pre> +<?php + require_once('socket.php'); + + class Telnet { + ... + function &connect($ip, $port, $username, $password) {<strong> + $socket = &$this->_createSocket($ip, $port);</strong> + $socket->read( ... ); + ... + }<strong> + + function &_createSocket($ip, $port) { + return new Socket($ip, $port); + }</strong> + } +?> +</pre> + This is the only change we make to the application code. + </p> + <p> + For the test case we have to create a subclass so that + we can intercept the socket creation... +<pre> +<strong>class TelnetTestVersion extends Telnet { + var $_mock; + + function TelnetTestVersion(&$mock) { + $this->_mock = &$mock; + $this->Telnet(); + } + + function &_createSocket() { + return $this->_mock; + } +}</strong> +</pre> + Here I have passed the mock in the constructor, but a + setter would have done just as well. + Note that the mock was set into the object variable + before the constructor was chained. + This is necessary in case the constructor calls + <span class="new_code">connect()</span>. + Otherwise it could get a null value from + <span class="new_code">_createSocket()</span>. + </p> + <p> + After the completion of all of this extra work the + actual test case is fairly easy. + We just test our new class instead... +<pre> +class TelnetTest extends UnitTestCase { + ... + function testConnection() {<strong> + $socket = &new MockSocket($this); + ... + $telnet = &new TelnetTestVersion($socket); + $telnet->connect('127.0.0.1', 21, 'Me', 'Secret'); + ...</strong> + } +} +</pre> + The new class is very simple of course. + It just sets up a return value, rather like a mock. + It would be nice if it also checked the incoming parameters + as well. + Just like a mock. + It seems we are likely to do this often, can + we automate the subclass creation? + </p> + + <p> +<a class="target" name="partial"> +<h2>A partial mock</h2> +</a> +</p> + <p> + Of course the answer is "yes" or I would have stopped writing + this by now! + The previous test case was a lot of work, but we can + generate the subclass using a similar approach to the mock objects. + </p> + <p> + Here is the partial mock version of the test... +<pre> +<strong>Mock::generatePartial( + 'Telnet', + 'TelnetTestVersion', + array('_createSocket'));</strong> + +class TelnetTest extends UnitTestCase { + ... + function testConnection() {<strong> + $socket = &new MockSocket($this); + ... + $telnet = &new TelnetTestVersion($this); + $telnet->setReturnReference('_createSocket', $socket); + $telnet->Telnet(); + $telnet->connect('127.0.0.1', 21, 'Me', 'Secret'); + ...</strong> + } +} +</pre> + The partial mock is a subclass of the original with + selected methods "knocked out" with test + versions. + The <span class="new_code">generatePartial()</span> call + takes three parameters: the class to be subclassed, + the new test class name and a list of methods to mock. + </p> + <p> + Instantiating the resulting objects is slightly tricky. + The only constructor parameter of a partial mock is + the unit tester reference. + As with the normal mock objects this is needed for sending + test results in response to checked expectations. + </p> + <p> + The original constructor is not run yet. + This is necessary in case the constructor is going to + make use of the as yet unset mocked methods. + We set any return values at this point and then run the + constructor with its normal parameters. + This three step construction of "new", followed + by setting up the methods, followed by running the constructor + proper is what distinguishes the partial mock code. + </p> + <p> + Apart from construction, all of the mocked methods have + the same features as mock objects and all of the unmocked + methods behave as before. + We can set expectations very easily... +<pre> +class TelnetTest extends UnitTestCase { + ... + function testConnection() { + $socket = &new MockSocket($this); + ... + $telnet = &new TelnetTestVersion($this); + $telnet->setReturnReference('_createSocket', $socket);<strong> + $telnet->expectOnce('_createSocket', array('127.0.0.1', 21));</strong> + $telnet->Telnet(); + $telnet->connect('127.0.0.1', 21, 'Me', 'Secret'); + ...<strong> + $telnet->tally();</strong> + } +} +</pre> + </p> + + <p> +<a class="target" name="less"> +<h2>Testing less than a class</h2> +</a> +</p> + <p> + The mocked out methods don't have to be factory methods, + they could be any sort of method. + In this way partial mocks allow us to take control of any part of + a class except the constructor. + We could even go as far as to mock every method + except one we actually want to test. + </p> + <p> + This last situation is all rather hypothetical, as I haven't + tried it. + I am open to the possibility, but a little worried that + forcing object granularity may be better for the code quality. + I personally use partial mocks as a way of overriding creation + or for occasional testing of the TemplateMethod pattern. + </p> + <p> + It's all going to come down to the coding standards of your + project to decide which mechanism you use. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/reporter_documentation.html b/tests/test_tools/simpletest/docs/en/reporter_documentation.html new file mode 100755 index 00000000..44be8b1e --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/reporter_documentation.html @@ -0,0 +1,515 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP test runner and display documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<span class="chosen">Reporting</span> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Test reporter documentation</h1> +<div class="content"> + + <p> + SimpleTest pretty much follows the MVC pattern + (Model-View-Controller). + The reporter classes are the view and the model is your + test cases and their hiearchy. + The controller is mostly hidden from the user of + SimpleTest unless you want to change how the test cases + are actually run, in which case it is possible to + override the runner objects from within the test case. + As usual with MVC, the controller is mostly undefined + and there are other places to control the test run. + </p> + + <p> +<a class="target" name="html"> +<h2>Reporting results in HTML</h2> +</a> +</p> + <p> + The default test display is minimal in the extreme. + It reports success and failure with the conventional red and + green bars and shows a breadcrumb trail of test groups + for every failed assertion. + Here's a fail... + <div class="demo"> + <h1>File test</h1> + <span class="fail">Fail</span>: createnewfile->True assertion failed.<br> + <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete. + <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div> + </div> + And here all tests passed... + <div class="demo"> + <h1>File test</h1> + <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete. + <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div> + </div> + The good news is that there are several points in the display + hiearchy for subclassing. + </p> + <p> + For web page based displays there is the + <span class="new_code">HtmlReporter</span> class with the following + signature... +<pre> +class HtmlReporter extends SimpleReporter { + public HtmlReporter($encoding) { ... } + public makeDry(boolean $is_dry) { ... } + public void paintHeader(string $test_name) { ... } + public void sendNoCacheHeaders() { ... } + public void paintFooter(string $test_name) { ... } + public void paintGroupStart(string $test_name, integer $size) { ... } + public void paintGroupEnd(string $test_name) { ... } + public void paintCaseStart(string $test_name) { ... } + public void paintCaseEnd(string $test_name) { ... } + public void paintMethodStart(string $test_name) { ... } + public void paintMethodEnd(string $test_name) { ... } + public void paintFail(string $message) { ... } + public void paintPass(string $message) { ... } + public void paintError(string $message) { ... } + public void paintException(string $message) { ... } + public void paintMessage(string $message) { ... } + public void paintFormattedMessage(string $message) { ... } + protected string _getCss() { ... } + public array getTestList() { ... } + public integer getPassCount() { ... } + public integer getFailCount() { ... } + public integer getExceptionCount() { ... } + public integer getTestCaseCount() { ... } + public integer getTestCaseProgress() { ... } +} +</pre> + Here is what some of these methods mean. First the display methods + that you will probably want to override... + <ul class="api"> + <li> + <span class="new_code">HtmlReporter(string $encoding)</span> +<br> + is the constructor. + Note that the unit test sets up the link to the display + rather than the other way around. + The display is a mostly passive receiver of test events. + This allows easy adaption of the display for other test + systems beside unit tests, such as monitoring servers. + The encoding is the character encoding you wish to + display the test output in. + In order to correctly render debug output when + using the web tester, this should match the encoding + of the site you are trying to test. + The available character set strings are described in + the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a> + function. + </li> + <li> + <span class="new_code">void paintHeader(string $test_name)</span> +<br> + is called once at the very start of the test when the first + start event arrives. + The first start event is usually delivered by the top level group + test and so this is where <span class="new_code">$test_name</span> + comes from. + It paints the page titles, CSS, body tag, etc. + It returns nothing (<span class="new_code">void</span>). + </li> + <li> + <span class="new_code">void paintFooter(string $test_name)</span> +<br> + Called at the very end of the test to close any tags opened + by the page header. + By default it also displays the red/green bar and the final + count of results. + Actually the end of the test happens when a test end event + comes in with the same name as the one that started it all + at the same level. + The tests nest you see. + Closing the last test finishes the display. + </li> + <li> + <span class="new_code">void paintMethodStart(string $test_name)</span> +<br> + is called at the start of each test method. + The name normally comes from method name. + The other test start events behave the same way except + that the group test one tells the reporter how large + it is in number of held test cases. + This is so that the reporter can display a progress bar + as the runner churns through the test cases. + </li> + <li> + <span class="new_code">void paintMethodEnd(string $test_name)</span> +<br> + backs out of the test started with the same name. + </li> + <li> + <span class="new_code">void paintFail(string $message)</span> +<br> + paints a failure. + By default it just displays the word fail, a breadcrumbs trail + showing the current test nesting and the message issued by + the assertion. + </li> + <li> + <span class="new_code">void paintPass(string $message)</span> +<br> + by default does nothing. + </li> + <li> + <span class="new_code">string _getCss()</span> +<br> + Returns the CSS styles as a string for the page header + method. + Additional styles have to be appended here if you are + not overriding the page header. + You will want to use this method in an overriden page header + if you want to include the original CSS. + </li> + </ul> + There are also some accessors to get information on the current + state of the test suite. + Use these to enrich the display... + <ul class="api"> + <li> + <span class="new_code">array getTestList()</span> +<br> + is the first convenience method for subclasses. + Lists the current nesting of the tests as a list + of test names. + The first, most deeply nested test, is first in the + list and the current test method will be last. + </li> + <li> + <span class="new_code">integer getPassCount()</span> +<br> + returns the number of passes chalked up so far. + Needed for the display at the end. + </li> + <li> + <span class="new_code">integer getFailCount()</span> +<br> + is likewise the number of fails so far. + </li> + <li> + <span class="new_code">integer getExceptionCount()</span> +<br> + is likewise the number of errors so far. + </li> + <li> + <span class="new_code">integer getTestCaseCount()</span> +<br> + is the total number of test cases in the test run. + This includes the grouping tests themselves. + </li> + <li> + <span class="new_code">integer getTestCaseProgress()</span> +<br> + is the number of test cases completed so far. + </li> + </ul> + One simple modification is to get the HtmlReporter to display + the passes as well as the failures and errors... +<pre> +<strong>class ShowPasses extends HtmlReporter { + + function paintPass($message) { + parent::paintPass($message); + print "&<span class=\"pass\">Pass</span>: "; + $breadcrumb = $this->getTestList(); + array_shift($breadcrumb); + print implode("-&gt;", $breadcrumb); + print "-&gt;$message<br />\n"; + } + + function _getCss() { + return parent::_getCss() . ' .pass { color: green; }'; + } +}</strong> +</pre> + </p> + <p> + One method that was glossed over was the <span class="new_code">makeDry()</span> + method. + If you run this method, with no parameters, on the reporter + before the test suite is run no actual test methods + will be called. + You will still get the events of entering and leaving the + test methods and test cases, but no passes or failures etc, + because the test code will not actually be executed. + </p> + <p> + The reason for this is to allow for more sophistcated + GUI displays that allow the selection of individual test + cases. + In order to build a list of possible tests they need a + report on the test structure for drawing, say a tree view + of the test suite. + With a reporter set to dry run that just sends drawing events + this is easily accomplished. + </p> + + <p> +<a class="target" name="other"> +<h2>Extending the reporter</h2> +</a> +</p> + <p> + Rather than simply modifying the existing display, you might want to + produce a whole new HTML look, or even generate text or XML. + Rather than override every method in + <span class="new_code">HtmlReporter</span> we can take one + step up the class hiearchy to <span class="new_code">SimpleReporter</span> + in the <em>simple_test.php</em> source file. + </p> + <p> + A do nothing display, a blank canvas for your own creation, would + be... +<pre> +<strong>require_once('simpletest/simple_test.php');</strong> + +class MyDisplay extends SimpleReporter {<strong> + </strong> + function paintHeader($test_name) { + } + + function paintFooter($test_name) { + } + + function paintStart($test_name, $size) {<strong> + parent::paintStart($test_name, $size);</strong> + } + + function paintEnd($test_name, $size) {<strong> + parent::paintEnd($test_name, $size);</strong> + } + + function paintPass($message) {<strong> + parent::paintPass($message);</strong> + } + + function paintFail($message) {<strong> + parent::paintFail($message);</strong> + } +} +</pre> + No output would come from this class until you add it. + </p> + + <p> +<a class="target" name="cli"> +<h2>The command line reporter</h2> +</a> +</p> + <p> + SimpleTest also ships with a minimal command line reporter. + The interface mimics JUnit to some extent, but paints the + failure messages as they arrive. + To use the command line reporter simply substitute it + for the HTML version... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + + $test = &new GroupTest('File test'); + $test->addTestFile('tests/file_test.php'); + $test->run(<strong>new TextReporter()</strong>); +?> +</pre> + Then invoke the test suite from the command line... +<pre class="shell"> +php file_test.php +</pre> + You will need the command line version of PHP installed + of course. + A passing test suite looks like this... +<pre class="shell"> +File test +OK +Test cases run: 1/1, Failures: 0, Exceptions: 0 +</pre> + A failure triggers a display like this... +<pre class="shell"> +File test +1) True assertion failed. + in createnewfile +FAILURES!!! +Test cases run: 1/1, Failures: 1, Exceptions: 0 +</pre> + </p> + <p> + One of the main reasons for using a command line driven + test suite is of using the tester as part of some automated + process. + To function properly in shell scripts the test script should + return a non-zero exit code on failure. + If a test suite fails the value <span class="new_code">false</span> + is returned from the <span class="new_code">SimpleTest::run()</span> + method. + We can use that result to exit the script with the desired return + code... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + + $test = &new GroupTest('File test'); + $test->addTestFile('tests/file_test.php'); + <strong>exit ($test->run(new TextReporter()) ? 0 : 1);</strong> +?> +</pre> + Of course we don't really want to create two test scripts, + a command line one and a web browser one, for each test suite. + The command line reporter includes a method to sniff out the + run time environment... +<pre> +<?php + require_once('simpletest/unit_tester.php'); + require_once('simpletest/reporter.php'); + + $test = &new GroupTest('File test'); + $test->addTestFile('tests/file_test.php'); + <strong>if (TextReporter::inCli()) {</strong> + exit ($test->run(new TextReporter()) ? 0 : 1); + <strong>}</strong> + $test->run(new HtmlReporter()); +?> +</pre> + This is the form used within SimpleTest itself. + </p> + + <p> +<a class="target" name="xml"> +<h2>Remote testing</h2> +</a> +</p> + <p> + SimpleTest ships with an <span class="new_code">XmlReporter</span> class + used for internal communication. + When run the output looks like... +<pre class="shell"> +<?xml version="1.0"?> +<run> + <group size="4"> + <name>Remote tests</name> + <group size="4"> + <name>Visual test with 48 passes, 48 fails and 4 exceptions</name> + <case> + <name>testofunittestcaseoutput</name> + <test> + <name>testofresults</name> + <pass>This assertion passed</pass> + <fail>This assertion failed</fail> + </test> + <test> + ... + </test> + </case> + </group> + </group> +</run> +</pre> + You can make use of this format with the parser + supplied as part of SimpleTest itself. + This is called <span class="new_code">SimpleTestXmlParser</span> and + resides in <em>xml.php</em> within the SimpleTest package... +<pre> +<?php + require_once('simpletest/xml.php'); + + ... + $parser = &new SimpleTestXmlParser(new HtmlReporter()); + $parser->parse($test_output); +?> +</pre> + The <span class="new_code">$test_output</span> should be the XML format + from the XML reporter, and could come from say a command + line run of a test case. + The parser sends events to the reporter just like any + other test run. + There are some odd occasions where this is actually useful. + </p> + <p> + A problem with large test suites is thet they can exhaust + the default 8Mb memory limit on a PHP process. + By having the test groups output in XML and run in + separate processes, the output can be reparsed to + aggregate the results into a much smaller footprint top level + test. + </p> + <p> + Because the XML output can come from anywhere, this opens + up the possibility of aggregating test runs from remote + servers. + A test case already exists to do this within the SimpleTest + framework, but it is currently experimental... +<pre> +<?php + <strong>require_once('../remote.php');</strong> + require_once('../reporter.php'); + + $test_url = ...; + $dry_url = ...; + + $test = &new GroupTest('Remote tests'); + $test->addTestCase(<strong>new RemoteTestCase($test_url, $dry_url)</strong>); + $test->run(new HtmlReporter()); +?> +</pre> + The <span class="new_code">RemoteTestCase</span> takes the actual location + of the test runner, basically a web page in XML format. + It also takes the URL of a reporter set to do a dry run. + This is so that progress can be reported upward correctly. + The <span class="new_code">RemoteTestCase</span> can be added to test suites + just like any other group test. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/server_stubs_documentation.html b/tests/test_tools/simpletest/docs/en/server_stubs_documentation.html new file mode 100755 index 00000000..4b18bb0b --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/server_stubs_documentation.html @@ -0,0 +1,388 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP server stubs documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<span class="chosen">Server stubs</span> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Server stubs documentation</h1> +<div class="content"> + <p> +<a class="target" name="what"> +<h2>What are server stubs?</h2> +</a> +</p> + <p> + This was originally a pattern named by Robert Binder (Testing + object-oriented systems: models, patterns, and tools, + Addison-Wesley) in 1999. + A server stub is a simulation of an object or component. + It should exactly replace a component in a system for test + or prototyping purposes, but remain lightweight. + This allows tests to run more quickly, or if the simulated + class has not been written, to run at all. + </p> + + <p> +<a class="target" name="creation"> +<h2>Creating server stubs</h2> +</a> +</p> + <p> + All we need is an existing class, say a database connection + that looks like this... +<pre> +<strong>class DatabaseConnection { + function DatabaseConnection() { + } + + function query() { + } + + function selectQuery() { + } +}</strong> +</pre> + The class does not need to have been implemented yet. + To create a stub version of the class we need to include the + server stub library and run the generator... +<pre> +<strong>require_once('simpletest/mock_objects.php'); +require_once('database_connection.php'); +Stub::generate('DatabaseConnection');</strong> +</pre> + This generates a clone class called + <span class="new_code">StubDatabaseConnection</span>. + We can now create instances of the new class within + our prototype script... +<pre> +require_once('simpletest/mock_objects.php'); +require_once('database_connection.php'); +Stub::generate('DatabaseConnection'); +<strong> +$connection = new StubDatabaseConnection(); +</strong> +</pre> + The stub version of a class has all the methods of the original + so that operations like + <span class="new_code">$connection->query()</span> are still + legal. + The return value will be <span class="new_code">null</span>, + but we can change that with... +<pre> +<strong>$connection->setReturnValue('query', 37)</strong> +</pre> + Now every time we call + <span class="new_code">$connection->query()</span> we get + the result of 37. + We can set the return value to anything, say a hash of + imaginary database results or a list of persistent objects. + Parameters are irrelevant here, we always get the same + values back each time once they have been set up this way. + That may not sound like a convincing replica of a + database connection, but for the half a dozen lines of + a test method it is usually all you need. + </p> + + <p> +<a class="target" name="patterns"> +<h2>Simulation patterns</h2> +</a> +</p> + <p> + Things aren't always that simple though. + One common problem is iterators, where constantly returning + the same value could cause an endless loop in the object + being tested. + For these we need to set up sequences of values. + Let's say we have a simple iterator that looks like this... +<pre> +class Iterator { + function Iterator() { + } + + function next() { + } +} +</pre> + This is about the simplest iterator you could have. + Assuming that this iterator only returns text until it + reaches the end, when it returns false, we can simulate it + with... +<pre> +<strong>Stub::generate('Iterator'); + +$iterator = new StubIterator(); +$iterator->setReturnValue('next', false); +$iterator->setReturnValueAt(0, 'next', 'First string'); +$iterator->setReturnValueAt(1, 'next', 'Second string');</strong> +</pre> + When <span class="new_code">next()</span> is called on the + stub iterator it will first return "First string", + on the second call "Second string" will be returned + and on any other call <span class="new_code">false</span> will + be returned. + The sequenced return values take precedence over the constant + return value. + The constant one is a kind of default if you like. + </p> + <p> + Another tricky situation is an overloaded + <span class="new_code">get()</span> operation. + An example of this is an information holder with name/value pairs. + Say we have a configuration class like... +<pre> +class Configuration { + function Configuration() { + } + + function getValue($key) { + } +} +</pre> + This is a classic situation for using stub objects as + actual configuration will vary from machine to machine, + hardly helping the reliability of our tests if we use it + directly. + The problem though is that all the data comes through the + <span class="new_code">getValue()</span> method and yet + we want different results for different keys. + Luckily the stubs have a filter system... +<pre> +<strong>Stub::generate('Configuration'); + +$config = &new StubConfiguration(); +$config->setReturnValue('getValue', 'primary', array('db_host')); +$config->setReturnValue('getValue', 'admin', array('db_user')); +$config->setReturnValue('getValue', 'secret', array('db_password'));</strong> +</pre> + The extra parameter is a list of arguments to attempt + to match. + In this case we are trying to match only one argument which + is the look up key. + Now when the server stub has the + <span class="new_code">getValue()</span> method invoked + like this... +<pre> +$config->getValue('db_user'); +</pre> + ...it will return "admin". + It finds this by attempting to match the calling arguments + to its list of returns one after another until + a complete match is found. + </p> + <p> + You can set a default argument argument like so... +<pre> +<strong> +$config->setReturnValue('getValue', false, array('*'));</strong> +</pre> + This is not the same as setting the return value without + any argument requirements like this... +<pre> +<strong> +$config->setReturnValue('getValue', false);</strong> +</pre> + In the first case it will accept any single argument, + but exactly one is required. + In the second case any number of arguments will do and + it acts as a catchall after all other matches. + Note that if we add further single parameter options after + the wildcard in the first case, they will be ignored as the wildcard + will match first. + With complex parameter lists the ordering could be important + or else desired matches could be masked by earlier wildcard + ones. + Declare the most specific matches first if you are not sure. + </p> + <p> + There are times when you want a specific object to be + dished out by the stub rather than just a copy. + The PHP copy semantics force us to use a different method + for this. + You might be simulating a container for example... +<pre> +class Thing { +} + +class Vector { + function Vector() { + } + + function get($index) { + } +} +</pre> + In this case you can set a reference into the stub's + return list... +<pre> +Stub::generate('Vector'); + +$thing = new Thing();<strong> +$vector = &new StubVector(); +$vector->setReturnReference('get', $thing, array(12));</strong> +</pre> + With this arrangement you know that every time + <span class="new_code">$vector->get(12)</span> is + called it will return the same + <span class="new_code">$thing</span> each time. + </p> + <p> + These three factors, timing, parameters and whether to copy, + can be combined orthogonally. + For example... +<pre> +$complex = &new StubComplexThing(); +$stuff = new Stuff();<strong> +$complex->setReturnReferenceAt(3, 'get', $stuff, array('*', 1));</strong> +</pre> + This will return the <span class="new_code">$stuff</span> only on the third + call and only if two parameters were set the second of + which must be the integer 1. + That should cover most simple prototyping situations. + </p> + <p> + A final tricky case is one object creating another, known + as a factory pattern. + Suppose that on a successful query to our imaginary + database, a result set is returned as an iterator with + each call to <span class="new_code">next()</span> giving + one row until false. + This sounds like a simulation nightmare, but in fact it can all + be stubbed using the mechanics above. + </p> + <p> + Here's how... +<pre> +Stub::generate('DatabaseConnection'); +Stub::generate('ResultIterator'); + +class DatabaseTest extends UnitTestCase { + + function testUserFinder() {<strong> + $result = &new StubResultIterator(); + $result->setReturnValue('next', false); + $result->setReturnValueAt(0, 'next', array(1, 'tom')); + $result->setReturnValueAt(1, 'next', array(3, 'dick')); + $result->setReturnValueAt(2, 'next', array(6, 'harry')); + + $connection = &new StubDatabaseConnection(); + $connection->setReturnValue('query', false); + $connection->setReturnReference( + 'query', + $result, + array('select id, name from users'));</strong> + + $finder = &new UserFinder($connection); + $this->assertIdentical( + $finder->findNames(), + array('tom', 'dick', 'harry')); + } +} +</pre> + Now only if our + <span class="new_code">$connection</span> is called with the correct + <span class="new_code">query()</span> will the + <span class="new_code">$result</span> be returned that is + itself exhausted after the third call to <span class="new_code">next()</span>. + This should be enough + information for our <span class="new_code">UserFinder</span> class, + the class actually + being tested here, to come up with goods. + A very precise test and not a real database in sight. + </p> + + <p> +<a class="target" name="options"> +<h2>Stub creation options</h2> +</a> +</p> + <p> + There are some additional options when creating stubs. + At the generation stage we can change the class name... +<pre> +<strong>Stub::generate('Iterator', 'MyStubIterator'); +$iterator = &new MyStubIterator(); +</strong> +</pre> + This is not very useful in itself as there would be no difference + in this class and the default except for the name. + However we can also add additional methods not found in the + original interface... +<pre> +class Iterator { +} +<strong>Stub::generate('Iterator', 'PrototypeIterator', array('next', 'isError')); +$iterator = &new PrototypeIterator(); +$iterator->setReturnValue('next', 0); +</strong> +</pre> + The <span class="new_code">next()</span> and + <span class="new_code">isError()</span> methods can now have + return values set just as if they existed in the original class. + </p> + <p> + One other esoteric way of customising the stubs is to change + the default wildcard used for parameter matching. +<pre> +<strong>Stub::generate('Connection'); +$iterator = &new StubConnection('wild'); +$iterator->setReturnValue('query', array('id' => 33), array('wild')); +</strong> +</pre> + The only reason to do this is if you genuinely wanted to test + against the literal string "*" and didn't want it + interpreted as "any". + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/unit_test_documentation.html b/tests/test_tools/simpletest/docs/en/unit_test_documentation.html new file mode 100755 index 00000000..6aa8d8a7 --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/unit_test_documentation.html @@ -0,0 +1,387 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>SimpleTest for PHP regression test documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<span class="chosen">Unit tester</span> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<a href="web_tester_documentation.html">Web tester</a> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>PHP Unit Test documentation</h1> +<div class="content"> + <p> +<a class="target" name="unit"> +<h2>Unit test cases</h2> +</a> +</p> + <p> + The core system is a regression testing framework built around + test cases. + A sample test case looks like this... +<pre> +<strong>class FileTestCase extends UnitTestCase { +}</strong> +</pre> + If no test name is supplied when chaining the constructor then + the class name will be taken instead. + This will be the name displayed in the test results. + </p> + <p> + Actual tests are added as methods in the test case whose names + by default start with the string "test" and + when the test case is invoked all such methods are run in + the order that PHP introspection finds them. + As many test methods can be added as needed. + For example... +<pre> +require_once('../classes/writer.php'); + +class FileTestCase extends UnitTestCase { + function FileTestCase() { + $this->UnitTestCase('File test'); + }<strong> + + function setUp() { + @unlink('../temp/test.txt'); + } + + function tearDown() { + @unlink('../temp/test.txt'); + } + + function testCreation() { + $writer = &new FileWriter('../temp/test.txt'); + $writer->write('Hello'); + $this->assertTrue(file_exists('../temp/test.txt'), 'File created'); + }</strong> +} +</pre> + The constructor is optional and usually omitted. + Without a name, the class name is taken as the name of the test case. + </p> + <p> + Our only test method at the moment is <span class="new_code">testCreation()</span> + where we check that a file has been created by our + <span class="new_code">Writer</span> object. + We could have put the <span class="new_code">unlink()</span> + code into this method as well, but by placing it in + <span class="new_code">setUp()</span> and + <span class="new_code">tearDown()</span> we can use it with + other test methods that we add. + </p> + <p> + The <span class="new_code">setUp()</span> method is run + just before each and every test method. + <span class="new_code">tearDown()</span> is run just after + each and every test method. + </p> + <p> + You can place some test case set up into the constructor to + be run once for all the methods in the test case, but + you risk test inteference that way. + This way is slightly slower, but it is safer. + Note that if you come from a JUnit background this will not + be the behaviour you are used to. + JUnit surprisingly reinstantiates the test case for each test + method to prevent such interference. + SimpleTest requires the end user to use <span class="new_code">setUp()</span>, but + supplies additional hooks for library writers. + </p> + <p> + The means of reporting test results (see below) are by a + visiting display class + that is notified by various <span class="new_code">assert...()</span> + methods. + Here is the full list for the <span class="new_code">UnitTestCase</span> + class, the default for SimpleTest... + <table> +<tbody> + <tr> +<td><span class="new_code">assertTrue($x)</span></td><td>Fail if $x is false</td> +</tr> + <tr> +<td><span class="new_code">assertFalse($x)</span></td><td>Fail if $x is true</td> +</tr> + <tr> +<td><span class="new_code">assertNull($x)</span></td><td>Fail if $x is set</td> +</tr> + <tr> +<td><span class="new_code">assertNotNull($x)</span></td><td>Fail if $x not set</td> +</tr> + <tr> +<td><span class="new_code">assertIsA($x, $t)</span></td><td>Fail if $x is not the class or type $t</td> +</tr> + <tr> +<td><span class="new_code">assertNotA($x, $t)</span></td><td>Fail if $x is of the class or type $t</td> +</tr> + <tr> +<td><span class="new_code">assertEqual($x, $y)</span></td><td>Fail if $x == $y is false</td> +</tr> + <tr> +<td><span class="new_code">assertNotEqual($x, $y)</span></td><td>Fail if $x == $y is true</td> +</tr> + <tr> +<td><span class="new_code">assertIdentical($x, $y)</span></td><td>Fail if $x == $y is false or a type mismatch</td> +</tr> + <tr> +<td><span class="new_code">assertNotIdentical($x, $y)</span></td><td>Fail if $x == $y is true and types match</td> +</tr> + <tr> +<td><span class="new_code">assertReference($x, $y)</span></td><td>Fail unless $x and $y are the same variable</td> +</tr> + <tr> +<td><span class="new_code">assertCopy($x, $y)</span></td><td>Fail if $x and $y are the same variable</td> +</tr> + <tr> +<td><span class="new_code">assertWantedPattern($p, $x)</span></td><td>Fail unless the regex $p matches $x</td> +</tr> + <tr> +<td><span class="new_code">assertNoUnwantedPattern($p, $x)</span></td><td>Fail if the regex $p matches $x</td> +</tr> + <tr> +<td><span class="new_code">assertNoErrors()</span></td><td>Fail if any PHP error occoured</td> +</tr> + <tr> +<td><span class="new_code">assertError($x)</span></td><td>Fail if no PHP error or incorrect message</td> +</tr> + <tr> +<td><span class="new_code">assertErrorPattern($p)</span></td><td>Fail unless the error matches the regex $p</td> +</tr> + </tbody> +</table> + All assertion methods can take an optional description to + label the displayed result with. + If omitted a default message is sent instead which is usually + sufficient. + This default message can still be embedded in your own message + if you include "%s" within the string. + All the assertions return true on a pass or false on failure. + </p> + <p> + Some examples... +<pre> +<strong>$variable = null; +$this->assertNull($variable, 'Should be cleared');</strong> +</pre> + ...will pass and normally show no message. + If you have + <a href="http://www.lastcraft.com/display_subclass_tutorial.php">set up the tester to display passes</a> + as well then the message will be displayed as is. +<pre> +<strong>$this->assertIdentical(0, false, 'Zero is not false [%s]');</strong> +</pre> + This will fail as it performs a type + check as well as a comparison between the two values. + The "%s" part is replaced by the default + error message that would have been shown if we had not + supplied our own. + This also allows us to nest test messages. +<pre> +<strong>$a = 1; +$b = $a; +$this->assertReference($a, $b);</strong> +</pre> + Will fail as the variable <span class="new_code">$a</span> is a copy of <span class="new_code">$b</span>. +<pre> +<strong>$this->assertWantedPattern('/hello/i', 'Hello world');</strong> +</pre> + This will pass as using a case insensitive match the string + <span class="new_code">hello</span> is contained in <span class="new_code">Hello world</span>. +<pre> +<strong>trigger_error('Disaster'); +trigger_error('Catastrophe'); +$this->assertError(); +$this->assertError('Catastrophe'); +$this->assertNoErrors();</strong> +</pre> + This one takes some explanation as in fact they all pass! + </p> + <p> + PHP errors in SimpleTest are trapped and placed in a queue. + Here the first error check catches the "Disaster" + message without checking the text and passes. + This removes the error from the queue. + The next error check tests not only the existence of the error, + but also the text which here matches so another pass. + With the queue now empty the last test will pass as well. + If any unchecked errors are left at the end of a test method then + an exception will be reported in the test. + Note that SimpleTest cannot catch compile time PHP errors. + </p> + <p> + The test cases also have some convenience methods for debugging + code or extending the suite... + <table> +<tbody> + <tr> +<td><span class="new_code">setUp()</span></td><td>Runs this before each test method</td> +</tr> + <tr> +<td><span class="new_code">tearDown()</span></td><td>Runs this after each test method</td> +</tr> + <tr> +<td><span class="new_code">pass()</span></td><td>Sends a test pass</td> +</tr> + <tr> +<td><span class="new_code">fail()</span></td><td>Sends a test failure</td> +</tr> + <tr> +<td><span class="new_code">error()</span></td><td>Sends an exception event</td> +</tr> + <tr> +<td><span class="new_code">sendMessage()</span></td><td>Sends a status message to those displays that support it</td> +</tr> + <tr> +<td><span class="new_code">signal($type, $payload)</span></td><td>Sends a user defined message to the test reporter</td> +</tr> + <tr> +<td><span class="new_code">dump($var)</span></td><td>Does a formatted <span class="new_code">print_r()</span> for quick and dirty debugging</td> +</tr> + <tr> +<td><span class="new_code">swallowErrors()</span></td><td>Clears the error queue</td> +</tr> + </tbody> +</table> + </p> + + <p> +<a class="target" name="extending_unit"> +<h2>Extending test cases</h2> +</a> +</p> + <p> + Of course additional test methods can be added to create + specific types of test case too so as to extend framework... +<pre> +require_once('simpletest/unit_tester.php'); +<strong> +class FileTester extends UnitTestCase { + function FileTester($name = false) { + $this->UnitTestCase($name); + } + + function assertFileExists($filename, $message = '%s') { + $this->assertTrue( + file_exists($filename), + sprintf($message, 'File [$filename] existence check')); + }</strong> +} +</pre> + Here the SimpleTest library is held in a folder called + <em>simpletest</em> that is local. + Substitute your own path for this. + </p> + <p> + This new case can be now be inherited just like + a normal test case... +<pre> +class FileTestCase extends <strong>FileTester</strong> { + + function setUp() { + @unlink('../temp/test.txt'); + } + + function tearDown() { + @unlink('../temp/test.txt'); + } + + function testCreation() { + $writer = &new FileWriter('../temp/test.txt'); + $writer->write('Hello');<strong> + $this->assertFileExists('../temp/test.txt');</strong> + } +} +</pre> + </p> + <p> + If you want a test case that does not have all of the + <span class="new_code">UnitTestCase</span> assertions, + only your own and <span class="new_code">assertTrue()</span>, + you need to extend the <span class="new_code">SimpleTestCase</span> + class instead. + It is found in <em>simple_test.php</em> rather than + <em>unit_tester.php</em>. + See <a href="group_test_documentation.html">later</a> if you + want to incorporate other unit tester's + test cases in your test suites. + </p> + + <p> +<a class="target" name="running_unit"> +<h2>Running a single test case</h2> +</a> +</p> + <p> + You won't often run single test cases except when bashing + away at a module that is having difficulty and you don't + want to upset the main test suite. + Here is the scaffolding needed to run the a lone test case... +<pre> +<?php + require_once('simpletest/unit_tester.php');<strong> + require_once('simpletest/reporter.php');</strong> + require_once('../classes/writer.php'); + + class FileTestCase extends UnitTestCase { + function FileTestCase() { + $this->UnitTestCase('File test'); + } + }<strong> + + $test = &new FileTestCase(); + $test->run(new HtmlReporter());</strong> +?> +</pre> + This script will run as is, but will output zero passes + and zero failures until test methods are added. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> diff --git a/tests/test_tools/simpletest/docs/en/web_tester_documentation.html b/tests/test_tools/simpletest/docs/en/web_tester_documentation.html new file mode 100755 index 00000000..51f604be --- /dev/null +++ b/tests/test_tools/simpletest/docs/en/web_tester_documentation.html @@ -0,0 +1,508 @@ +<html> +<head> +<META http-equiv="Content-Type" content="text/html; charset=UTF-8"> +<title>Simple Test for PHP web script testing documentation</title> +<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> +</head> +<body> +<div class="menu_back"> +<div class="menu"> +<h2> +<a href="index.html">SimpleTest</a> +</h2> +<ul> +<li> +<a href="overview.html">Overview</a> +</li> +<li> +<a href="unit_test_documentation.html">Unit tester</a> +</li> +<li> +<a href="group_test_documentation.html">Group tests</a> +</li> +<li> +<a href="server_stubs_documentation.html">Server stubs</a> +</li> +<li> +<a href="mock_objects_documentation.html">Mock objects</a> +</li> +<li> +<a href="partial_mocks_documentation.html">Partial mocks</a> +</li> +<li> +<a href="reporter_documentation.html">Reporting</a> +</li> +<li> +<a href="expectation_documentation.html">Expectations</a> +</li> +<li> +<span class="chosen">Web tester</span> +</li> +<li> +<a href="form_testing_documentation.html">Testing forms</a> +</li> +<li> +<a href="authentication_documentation.html">Authentication</a> +</li> +<li> +<a href="browser_documentation.html">Scriptable browser</a> +</li> +</ul> +</div> +</div> +<h1>Web tester documentation</h1> +<div class="content"> + <p> +<a class="target" name="fetch"> +<h2>Fetching a page</h2> +</a> +</p> + <p> + Testing classes is all very well, but PHP is predominately + a language for creating functionality within web pages. + How do we test the front end presentation role of our PHP + applications? + Well the web pages are just text, so we should be able to + examine them just like any other test data. + </p> + <p> + This leads to a tricky issue. + If we test at too low a level, testing for matching tags + in the page with pattern matching for example, our tests will + be brittle. + The slightest change in layout could break a large number of + tests. + If we test at too high a level, say using mock versions of a + template engine, then we lose the ability to automate some classes + of test. + For example, the interaction of forms and navigation will + have to be tested manually. + These types of test are extremely repetitive and error prone. + </p> + <p> + SimpleTest includes a special form of test case for the testing + of web page actions. + The <span class="new_code">WebTestCase</span> includes facilities + for navigation, content and cookie checks and form handling. + Usage of these test cases is similar to the + <a href="unit_tester_documentation.html">UnitTestCase</a>... +<pre> +<strong>class TestOfLastcraft extends WebTestCase { +}</strong> +</pre> + Here we are about to test the + <a href="http://www/lastcraft.com/">Last Craft</a> site itself. + If this test case is in a file called <em>lastcraft_test.php</em> + then it can be loaded in a runner script just like unit tests... +<pre> +<?php<strong> + require_once('simpletest/web_tester.php');</strong> + require_once('simpletest/reporter.php'); + + $test = &new GroupTest('Web site tests');<strong> + $test->addTestFile('lastcraft_test.php');</strong> + exit ($test->run(new TextReporter()) ? 0 : 1); +?> +</pre> + I am using the text reporter here to more clearly + distinguish the web content from the test output. + </p> + <p> + Nothing is being tested yet. + We can fetch the home page by using the + <span class="new_code">get()</span> method... +<pre> +class TestOfLastcraft extends WebTestCase { + <strong> + function testHomepage() { + $this->assertTrue($this->get('http://www.lastcraft.com/')); + }</strong> +} +</pre> + The <span class="new_code">get()</span> method will + return true only if page content was successfully + loaded. + It is a simple, but crude way to check that a web page + was actually delivered by the web server. + However that content may be a 404 response and yet + our <span class="new_code">get()</span> method will still return true. + </p> + <p> + Assuming that the web server for the Last Craft site is up + (sadly not always the case), we should see... +<pre class="shell"> +Web site tests +OK +Test cases run: 1/1, Failures: 0, Exceptions: 0 +</pre> + All we have really checked is that any kind of page was + returned. + We don't yet know if it was the right one. + </p> + + <p> +<a class="target" name="content"> +<h2>Testing page content</h2> +</a> +</p> + <p> + To confirm that the page we think we are on is actually the + page we are on, we need to verify the page content. +<pre> +class TestOfLastcraft extends WebTestCase { + + function testHomepage() {<strong> + $this->get('http://www.lastcraft.com/'); + $this->assertWantedPattern('/why the last craft/i');</strong> + } +} +</pre> + The page from the last fetch is held in a buffer in + the test case, so there is no need to refer to it directly. + The pattern match is always made against the buffer. + </p> + <p> + Here is the list of possible content assertions... + <table> +<tbody> + <tr> +<td><span class="new_code">assertTitle($title)</span></td><td>Pass if title is an exact match</td> +</tr> + <tr> +<td><span class="new_code">assertWantedPattern($pattern)</span></td><td>A Perl pattern match against the page content</td> +</tr> + <tr> +<td><span class="new_code">assertNoUnwantedPattern($pattern)</span></td><td>A Perl pattern match to not find content</td> +</tr> + <tr> +<td><span class="new_code">assertWantedText($text)</span></td><td>Pass if matches visible and "alt" text</td> +</tr> + <tr> +<td><span class="new_code">assertNoUnwantedText($text)</span></td><td>Pass if doesn't match visible and "alt" text</td> +</tr> + <tr> +<td><span class="new_code">assertLink($label)</span></td><td>Pass if a link with this text is present</td> +</tr> + <tr> +<td><span class="new_code">assertNoLink($label)</span></td><td>Pass if no link with this text is present</td> +</tr> + <tr> +<td><span class="new_code">assertLinkById($id)</span></td><td>Pass if a link with this id attribute is present</td> +</tr> + <tr> +<td><span class="new_code">assertNoLinkById($id)</span></td><td>Pass if no link with this id attribute is present</td> +</tr> + <tr> +<td><span class="new_code">assertField($name, $value)</span></td><td>Pass if an input tag with this name has this value</td> +</tr> + <tr> +<td><span class="new_code">assertFieldById($id, $value)</span></td><td>Pass if an input tag with this id has this value</td> +</tr> + <tr> +<td><span class="new_code">assertResponse($codes)</span></td><td>Pass if HTTP response matches this list</td> +</tr> + <tr> +<td><span class="new_code">assertMime($types)</span></td><td>Pass if MIME type is in this list</td> +</tr> + <tr> +<td><span class="new_code">assertAuthentication($protocol)</span></td><td>Pass if the current challenge is this protocol</td> +</tr> + <tr> +<td><span class="new_code">assertNoAuthentication()</span></td><td>Pass if there is no current challenge</td> +</tr> + <tr> +<td><span class="new_code">assertRealm($name)</span></td><td>Pass if the current challenge realm matches</td> +</tr> + <tr> +<td><span class="new_code">assertHeader($header, $content)</span></td><td>Pass if a header was fetched matching this value</td> +</tr> + <tr> +<td><span class="new_code">assertNoUnwantedHeader($header)</span></td><td>Pass if a header was not fetched</td> +</tr> + <tr> +<td><span class="new_code">assertHeaderPattern($header, $pattern)</span></td><td>Pass if a header was fetched matching this Perl regex</td> +</tr> + <tr> +<td><span class="new_code">assertCookie($name, $value)</span></td><td>Pass if there is currently a matching cookie</td> +</tr> + <tr> +<td><span class="new_code">assertNoCookie($name)</span></td><td>Pass if there is currently no cookie of this name</td> +</tr> + </tbody> +</table> + As usual with the SimpleTest assertions, they all return + false on failure and true on pass. + They also allow an optional test message and you can embed + the original test message inside using "%s" inside + your custom message. + </p> + <p> + So now we could instead test against the title tag with... +<pre> +<strong>$this->assertTitle('The Last Craft? Web developer tutorials on PHP, Extreme programming and Object Oriented development');</strong> +</pre> + As well as the simple HTML content checks we can check + that the MIME type is in a list of allowed types with... +<pre> +<strong>$this->assertMime(array('text/plain', 'text/html'));</strong> +</pre> + More interesting is checking the HTTP response code. + Like the MIME type, we can assert that the response code + is in a list of allowed values... +<pre> +class TestOfLastcraft extends WebTestCase { + + function testRedirects() { + $this->get('http://www.lastcraft.com/test/redirect.php'); + $this->assertResponse(200);</strong> + } +} +</pre> + Here we are checking that the fetch is successful by + allowing only a 200 HTTP response. + This test will pass, but it is not actually correct to do so. + There is no page, instead the server issues a redirect. + The <span class="new_code">WebTestCase</span> will + automatically follow up to three such redirects. + The tests are more robust this way and we are usually + interested in the interaction with the pages rather + than their delivery. + If the redirects are of interest then this ability must + be disabled... +<pre> +class TestOfLastcraft extends WebTestCase { + + function testHomepage() {<strong> + $this->setMaximumRedirects(0);</strong> + $this->get('http://www.lastcraft.com/test/redirect.php'); + $this->assertResponse(200); + } +} +</pre> + The assertion now fails as expected... +<pre class="shell"> +Web site tests +1) Expecting response in [200] got [302] + in testhomepage + in testoflastcraft + in lastcraft_test.php +FAILURES!!! +Test cases run: 1/1, Failures: 1, Exceptions: 0 +</pre> + We can modify the test to correctly assert redirects with... +<pre> +class TestOfLastcraft extends WebTestCase { + + function testHomepage() { + $this->setMaximumRedirects(0); + $this->get('http://www.lastcraft.com/test/redirect.php'); + $this->assertResponse(<strong>array(301, 302, 303, 307)</strong>); + } +} +</pre> + This now passes. + </p> + + <p> +<a class="target" name="navigation"> +<h2>Navigating a web site</h2> +</a> +</p> + <p> + Users don't often navigate sites by typing in URLs, but by + clicking links and buttons. + Here we confirm that the contact details can be reached + from the home page... +<pre> +class TestOfLastcraft extends WebTestCase { + ... + function testContact() { + $this->get('http://www.lastcraft.com/');<strong> + $this->clickLink('About'); + $this->assertTitle('About Last Craft');</strong> + } +} +</pre> + The parameter is the text of the link. + </p> + <p> + If the target is a button rather than an anchor tag, then + <span class="new_code">clickSubmit()</span> should be used + with the button title... +<pre> +<strong>$this->clickSubmit('Go!');</strong> +</pre> + </p> + <p> + The list of navigation methods is... + <table> +<tbody> + <tr> +<td><span class="new_code">getUrl()</span></td><td>The current location</td> +</tr> + <tr> +<td><span class="new_code">get($url, $parameters)</span></td><td>Send a GET request with these parameters</td> +</tr> + <tr> +<td><span class="new_code">post($url, $parameters)</span></td><td>Send a POST request with these parameters</td> +</tr> + <tr> +<td><span class="new_code">head($url, $parameters)</span></td><td>Send a HEAD request without replacing the page content</td> +</tr> + <tr> +<td><span class="new_code">retry()</span></td><td>Reload the last request</td> +</tr> + <tr> +<td><span class="new_code">back()</span></td><td>Like the browser back button</td> +</tr> + <tr> +<td><span class="new_code">forward()</span></td><td>Like the browser forward button</td> +</tr> + <tr> +<td><span class="new_code">authenticate($name, $password)</span></td><td>Retry after a challenge</td> +</tr> + <tr> +<td><span class="new_code">restart()</span></td><td>Restarts the browser as if a new session</td> +</tr> + <tr> +<td><span class="new_code">getCookie($name)</span></td><td>Gets the cookie value for the current context</td> +</tr> + <tr> +<td><span class="new_code">ageCookies($interval)</span></td><td>Ages current cookies prior to a restart</td> +</tr> + <tr> +<td><span class="new_code">clearFrameFocus()</span></td><td>Go back to treating all frames as one page</td> +</tr> + <tr> +<td><span class="new_code">clickSubmit($label)</span></td><td>Click the first button with this label</td> +</tr> + <tr> +<td><span class="new_code">clickSubmitByName($name)</span></td><td>Click the button with this name attribute</td> +</tr> + <tr> +<td><span class="new_code">clickSubmitById($id)</span></td><td>Click the button with this ID attribute</td> +</tr> + <tr> +<td><span class="new_code">clickImage($label, $x, $y)</span></td><td>Click an input tag of type image by title or alt text</td> +</tr> + <tr> +<td><span class="new_code">clickImageByName($name, $x, $y)</span></td><td>Click an input tag of type image by name</td> +</tr> + <tr> +<td><span class="new_code">clickImageById($id, $x, $y)</span></td><td>Click an input tag of type image by ID attribute</td> +</tr> + <tr> +<td><span class="new_code">submitFormById($id)</span></td><td>Submit a form without the submit value</td> +</tr> + <tr> +<td><span class="new_code">clickLink($label, $index)</span></td><td>Click an anchor by the visible label text</td> +</tr> + <tr> +<td><span class="new_code">clickLinkById($id)</span></td><td>Click an anchor by the ID attribute</td> +</tr> + <tr> +<td><span class="new_code">getFrameFocus()</span></td><td>The name of the currently selected frame</td> +</tr> + <tr> +<td><span class="new_code">setFrameFocusByIndex($choice)</span></td><td>Focus on a frame counting from 1</td> +</tr> + <tr> +<td><span class="new_code">setFrameFocus($name)</span></td><td>Focus on a frame by name</td> +</tr> + </tbody> +</table> + </p> + <p> + The parameters in the <span class="new_code">get()</span>, <span class="new_code">post()</span> or + <span class="new_code">head()</span> methods are optional. + The HTTP HEAD fetch does not change the browser context, only loads + cookies. + This can be useful for when an image or stylesheet sets a cookie + for crafty robot blocking. + </p> + <p> + The <span class="new_code">retry()</span>, <span class="new_code">back()</span> and + <span class="new_code">forward()</span> commands work as they would on + your web browser. + They use the history to retry pages. + This can be handy for checking the effect of hitting the + back button on your forms. + </p> + <p> + The frame methods need a little explanation. + By default a framed page is treated just like any other. + Content will be searced for throughout the entire frameset, + so clicking a link will work no matter which frame + the anchor tag is in. + You can override this behaviour by focusing on a single + frame. + If you do that, all searches and actions will apply to that + frame alone, such as authentication and retries. + If a link or button is not in a focused frame then it cannot + be clicked. + </p> + <p> + Testing navigation on fixed pages only tells you when you + have broken an entire script. + For highly dynamic pages, such as for bulletin boards, this can + be crucial for verifying the correctness of the application. + For most applications though, the really tricky logic is usually in + the handling of forms and sessions. + Fortunately SimpleTest includes + <a href="form_testing_documentation.html">tools for testing web forms</a> + as well. + </p> + + <p> +<a class="target" name="request"> +<h2>Modifying the request</h2> +</a> +</p> + <p> + Although SimpleTest does not have the goal of testing networking + problems, it does include some methods to modify and debug + the requests it makes. + Here is another method list... + <table> +<tbody> + <tr> +<td><span class="new_code">getTransportError()</span></td><td>The last socket error</td> +</tr> + <tr> +<td><span class="new_code">showRequest()</span></td><td>Dump the outgoing request</td> +</tr> + <tr> +<td><span class="new_code">showHeaders()</span></td><td>Dump the incoming headers</td> +</tr> + <tr> +<td><span class="new_code">showSource()</span></td><td>Dump the raw HTML page content</td> +</tr> + <tr> +<td><span class="new_code">ignoreFrames()</span></td><td>Do not load framesets</td> +</tr> + <tr> +<td><span class="new_code">setCookie($name, $value)</span></td><td>Set a cookie from now on</td> +</tr> + <tr> +<td><span class="new_code">addHeader($header)</span></td><td>Always add this header to the request</td> +</tr> + <tr> +<td><span class="new_code">setMaximumRedirects($max)</span></td><td>Stop after this many redirects</td> +</tr> + <tr> +<td><span class="new_code">setConnectionTimeout($timeout)</span></td><td>Kill the connection after this time between bytes</td> +</tr> + <tr> +<td><span class="new_code">useProxy($proxy, $name, $password)</span></td><td>Make requests via this proxy URL</td> +</tr> + </tbody> +</table> + These methods are principally for debugging. + </p> + + </div> +<div class="copyright"> + Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004 + </div> +</body> +</html> |