Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

PHPUnit integration: Difference between revisions

From MoodleDocs
(40 intermediate revisions by 8 users not shown)
Line 1: Line 1:
Moodle PHPUnit integration was created to simplify using of PHPUnit framework in Moodle. It consists of specialised bootstrap script, utility scripts that initialise testing environment and highly optimised custom test case classes that handle automatic global state resetting after test that includes global variables, database rollback and purging of dataroot. We also try to bridge the gap between the design and coding style of Moodle and PHPUnit.
Moodle PHPUnit integration was created to simplify using of PHPUnit framework in Moodle. It consists of specialised bootstrap script, utility scripts that initialise testing environment and highly optimised custom test case classes that handle automatic global state resetting after test that includes global variables, database rollback and purging of dataroot. We also try to bridge the gap between the design and coding style of Moodle and PHPUnit.


Most of the documentation at http://www.phpunit.de/manual/3.6/en/index.html is relevant to Moodle PHPUnit integration, exceptions and additions are described bellow.
Most of the documentation at http://www.phpunit.de/manual/3.6/en/index.html is relevant to Moodle PHPUnit integration, exceptions and additions are described below.


==Definitions==
=Definitions=
These definitions may differ in each testing framework or programming language. The definitions here should be valid for PHPUnit framework with our Moodle tweaks.
These definitions may differ in each testing framework or programming language. The definitions here should be valid for PHPUnit framework with our Moodle tweaks.


;Test suite: is a collection of test cases or other suites usually related to one area of the product. It is defined in PHPUnit configuration files (phpunit.xml by default).
;Test suite: is a collection of test cases or other suites usually related to one area of the product. It is defined in PHPUnit configuration files (phpunit.xml by default).


;Test file: is a file with with *_test.php name which contains test cases. (*_Test.php in PHPUnit)
;Test file: is a file with *_test.php name which contains a test case. (*_Test.php in PHPUnit)


;Test case: is a class with test methods. Moodle tests extend basic_testcase or advanced_testcase. (PHPUnit_Framework_TestCase in PHPUnit)
;Test case: is a class with test methods. Moodle tests extend basic_testcase or advanced_testcase. (PHPUnit_Framework_TestCase in PHPUnit)
Line 16: Line 16:
;Assertion: is the actual comparison of expected and actual behaviour of the tested code.
;Assertion: is the actual comparison of expected and actual behaviour of the tested code.


==Organisation of test files==
=Organisation of test files=
All testing related files are stored in <code>/tests/</code> subdirectories:
All testing related files are stored in <code>/tests/</code> subdirectories:


* <code>.../tests/*_test.php</code> are PHPUnit test files with classes that extend basic_testcase or advanced_testcase
* <code>.../tests/*_test.php</code> are PHPUnit test files
* <code>.../tests/fixtures/</code> contains auxiliary files used in tests
* <code>.../tests/fixtures/</code> contains auxiliary files used in tests
* <code>.../tests/performance/</code> is reserved for performance testing scripts
* <code>.../tests/performance/</code> is reserved for performance testing scripts
* <code>.../tests/other/</code> is used for everything else that does not fit
* <code>.../tests/other/</code> is used for everything else that does not fit


== basic_testcase==
==Class and file naming rules==
The class names of all testcases need to be unique, Frankenstyle prefix is used to guarantee this. Since 2.6 it is possible to use automatic class loader when executing individual unit tests.
 
* All test case classes start with Frankenstyle prefix, for example: '''mod_forum_''', '''block_html_''', '''core_'''.
* All test case classes end with '''_testcase''' suffix.
* File names are constructed from the class names - the Frankenstyle prefix is removed, suffix '''_testcase''' is replaced with '''_test'''
 
Finally, but not less important, test case classes can be (recommended) namespaces. Read about the rules to do it properly @ [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|coding style]] (grand summary = 100% the same rules that are applied to **/classes directories).
 
Example:
* class '''mod_forum_lib_testcase''' is expected in file '''/mod/forum/tests/lib_test.php''', executed as <code>vendor/bin/phpunit mod_forum_lib_testcase</code>
* class '''core_text_testcase''' is expected in file '''/lib/texts/text_test.php''', executed as <code>vendor/bin/phpunit core_text_testcase</code>
 
=basic_testcase=
''basic_testcase'' is useful for simple tests that do not modify database or global variables. If something accidentally changes global state the test fails. This test case class is nearly identical to PHPUnit_Framework_TestCase used in PHPUnit documentation.
''basic_testcase'' is useful for simple tests that do not modify database or global variables. If something accidentally changes global state the test fails. This test case class is nearly identical to PHPUnit_Framework_TestCase used in PHPUnit documentation.


Sample class for testing of HTML Purifier integration:
<code php>
<?php
 
namespace mod_myplugin; // Optional, but recommended.
 
class mod_myplugin_sample_basic_testcase extends basic_testcase {
    public function test_equals() {
        $a = 1 + 2;
        $this->assertEquals(3, $a);
    }
}
</code>
 
There are no extra methods that could be used in tests. The expected file name for this test case is '''/mod/myplugin/tests/sample_basic_test.php'''.
 
=advanced_testcase=
By default each test starts with fresh new moodle installation. Test may modify database content, files or global variables. It is possible to use data generators to create new course, categories, module instances and other objects, alternatively table data can be preloaded from XML or CSV files.


<code php>
<code php>
class core_htmlpurifier_testcase extends basic_testcase {
<?php
    /**
 
    * Verify our nolink tag accepted
namespace mod_myplugin; // Optional, but recommended.
    * @return void
 
    */
class mod_myplugin_some_permission_testcase extends advanced_testcase {
     public function test_nolink() {
     public function test_isadmin() {
         $text = '<nolink><div>no filters</div></nolink>';
        global $DB;
         $result = purify_html($text, array());
 
         $this->assertSame($text, $result);
         $this->resetAfterTest(true);          // reset all changes automatically after this test
 
         $this->assertFalse(is_siteadmin());   // by default no user is logged-in
         $this->setUser(2);                    // switch $USER
        $this->assertTrue(is_siteadmin());   // admin is logged-in now


         $text = '<nolink>xxx<em>xx</em><div>xxx</div></nolink>';
         $DB->delete_records('user', array()); // lets do something crazy
        $result = purify_html($text, array());
        $this->assertSame($text, $result);
    }


    /**
         $this->resetAllData();                // that was not a good idea, let's go back
    * Verify our tex tag accepted
         $this->assertTrue($admin = $DB->record_exists('user', array('id'=>2)));
    * @return void
         $this->assertFalse(is_siteadmin());
    */
    public function test_tex() {
         $text = '<tex>a+b=c</tex>';
         $result = purify_html($text, array());
         $this->assertSame($text, $result);
     }
     }
}
}
</code>
</code>


==advanced_testcase==
The expected file name for this test case is '''/mod/myplugin/tests/some_permission_test.php'''.


==Extra methods==
; resetAfterTest(bool) : true means reset automatically after test, false means keep changes to next test method, default null means detect changes
; resetAllData() : reset global state in the middle of a test
; setAdminUser() : set current $USER as admin
; setGuestUser() : set current $USER as guest
; setUser() : set current $USER to a specific user - use getDataGenerator() to create one
; getDataGenerator() : returns data generator instance - use if you need to add new courses, users, etc.
; preventResetByRollback() : terminates active transactions, useful only when test contains own database transaction handling
; createXXXDataSet() : creates in memory structure of database table contents, used in loadDataSet() (eg: createXMLDataSet(), createCsvDataSet(), createFlatXMLDataSet())
; loadDataSet() : bulk loading of table contents
; getDebuggingMessages() : Return debugging messages from the current test. (Moodle 2.4 and upwards)
; resetDebugging() : Clear all previous debugging messages in current test. (Moodle 2.4 and upwards)
; assertDebuggingCalled() : Assert that exactly debugging was just called once. (Moodle 2.4 and upwards)
; assertDebuggingNotCalled() : Assert no debugging happened. (Moodle 2.4 and upwards)
; assertDebuggingCalledCount() : Asserts how many times debugging has been called. (Moodle 3.1 and upwards)
; [[Writing PHPUnit tests#Testing sending of messages|redirectMessages()]]: Captures ongoing messages for later testing (Moodle 2.4 and upwards)
; [[Writing PHPUnit tests#Testing_sending_of_emails|redirectEmails()]]: Captures ongoing emails for later testing (Moodle 2.6 and upwards)


==Restrictions==
* it is not possible to modify database structure such as create new table or drop columns from advanced_testcase.


=Moodle specific features=
* detection of global state changes - helps with detection of unintended changes in database
* highly optimised global state reset
* dataset loading - this feature is copied from PHPUnit database testcases
* automatic generation of phpunit.xml - init script builds list of plugin testcases
* database driver testing class - used for functional DB tests
* SimpleTest emulation class - helps with migration of old tests
* debugging() interception - enables to control and test any expected debug output. (Moodle 2.4 and upwards)


==Extra features==
=Limitations=
* detection of global state changes
* no Selenium support
* global state reset
* no support for PHPUnit_Extensions_Database_TestCase - it is possible to use data set loader only
* transaction rollbacks
* dataset loading
* database driver testing class
* SimpleTest emulation class


=See also=
* [[Writing PHPUnit tests]]


[[Category:Unit testing]]
[[Category:Unit testing]]

Revision as of 21:41, 2 September 2020

Moodle PHPUnit integration was created to simplify using of PHPUnit framework in Moodle. It consists of specialised bootstrap script, utility scripts that initialise testing environment and highly optimised custom test case classes that handle automatic global state resetting after test that includes global variables, database rollback and purging of dataroot. We also try to bridge the gap between the design and coding style of Moodle and PHPUnit.

Most of the documentation at http://www.phpunit.de/manual/3.6/en/index.html is relevant to Moodle PHPUnit integration, exceptions and additions are described below.

Definitions

These definitions may differ in each testing framework or programming language. The definitions here should be valid for PHPUnit framework with our Moodle tweaks.

Test suite
is a collection of test cases or other suites usually related to one area of the product. It is defined in PHPUnit configuration files (phpunit.xml by default).
Test file
is a file with *_test.php name which contains a test case. (*_Test.php in PHPUnit)
Test case
is a class with test methods. Moodle tests extend basic_testcase or advanced_testcase. (PHPUnit_Framework_TestCase in PHPUnit)
Test
is a public method starting with "test_" prefix defined in test case class. It is the implementation of the test procedure.
Assertion
is the actual comparison of expected and actual behaviour of the tested code.

Organisation of test files

All testing related files are stored in /tests/ subdirectories:

  • .../tests/*_test.php are PHPUnit test files
  • .../tests/fixtures/ contains auxiliary files used in tests
  • .../tests/performance/ is reserved for performance testing scripts
  • .../tests/other/ is used for everything else that does not fit

Class and file naming rules

The class names of all testcases need to be unique, Frankenstyle prefix is used to guarantee this. Since 2.6 it is possible to use automatic class loader when executing individual unit tests.

  • All test case classes start with Frankenstyle prefix, for example: mod_forum_, block_html_, core_.
  • All test case classes end with _testcase suffix.
  • File names are constructed from the class names - the Frankenstyle prefix is removed, suffix _testcase is replaced with _test

Finally, but not less important, test case classes can be (recommended) namespaces. Read about the rules to do it properly @ coding style (grand summary = 100% the same rules that are applied to **/classes directories).

Example:

  • class mod_forum_lib_testcase is expected in file /mod/forum/tests/lib_test.php, executed as vendor/bin/phpunit mod_forum_lib_testcase
  • class core_text_testcase is expected in file /lib/texts/text_test.php, executed as vendor/bin/phpunit core_text_testcase

basic_testcase

basic_testcase is useful for simple tests that do not modify database or global variables. If something accidentally changes global state the test fails. This test case class is nearly identical to PHPUnit_Framework_TestCase used in PHPUnit documentation.

<?php

namespace mod_myplugin; // Optional, but recommended.

class mod_myplugin_sample_basic_testcase extends basic_testcase {

   public function test_equals() {
       $a = 1 + 2;
       $this->assertEquals(3, $a);
   }

}

There are no extra methods that could be used in tests. The expected file name for this test case is /mod/myplugin/tests/sample_basic_test.php.

advanced_testcase

By default each test starts with fresh new moodle installation. Test may modify database content, files or global variables. It is possible to use data generators to create new course, categories, module instances and other objects, alternatively table data can be preloaded from XML or CSV files.

<?php

namespace mod_myplugin; // Optional, but recommended.

class mod_myplugin_some_permission_testcase extends advanced_testcase {

   public function test_isadmin() {
       global $DB;
       $this->resetAfterTest(true);          // reset all changes automatically after this test
       $this->assertFalse(is_siteadmin());   // by default no user is logged-in
       $this->setUser(2);                    // switch $USER
       $this->assertTrue(is_siteadmin());    // admin is logged-in now
       $DB->delete_records('user', array()); // lets do something crazy
       $this->resetAllData();                // that was not a good idea, let's go back
       $this->assertTrue($admin = $DB->record_exists('user', array('id'=>2)));
       $this->assertFalse(is_siteadmin());
   }

}

The expected file name for this test case is /mod/myplugin/tests/some_permission_test.php.

Extra methods

resetAfterTest(bool)
true means reset automatically after test, false means keep changes to next test method, default null means detect changes
resetAllData()
reset global state in the middle of a test
setAdminUser()
set current $USER as admin
setGuestUser()
set current $USER as guest
setUser()
set current $USER to a specific user - use getDataGenerator() to create one
getDataGenerator()
returns data generator instance - use if you need to add new courses, users, etc.
preventResetByRollback()
terminates active transactions, useful only when test contains own database transaction handling
createXXXDataSet()
creates in memory structure of database table contents, used in loadDataSet() (eg: createXMLDataSet(), createCsvDataSet(), createFlatXMLDataSet())
loadDataSet()
bulk loading of table contents
getDebuggingMessages()
Return debugging messages from the current test. (Moodle 2.4 and upwards)
resetDebugging()
Clear all previous debugging messages in current test. (Moodle 2.4 and upwards)
assertDebuggingCalled()
Assert that exactly debugging was just called once. (Moodle 2.4 and upwards)
assertDebuggingNotCalled()
Assert no debugging happened. (Moodle 2.4 and upwards)
assertDebuggingCalledCount()
Asserts how many times debugging has been called. (Moodle 3.1 and upwards)
redirectMessages()
Captures ongoing messages for later testing (Moodle 2.4 and upwards)
redirectEmails()
Captures ongoing emails for later testing (Moodle 2.6 and upwards)

Restrictions

  • it is not possible to modify database structure such as create new table or drop columns from advanced_testcase.

Moodle specific features

  • detection of global state changes - helps with detection of unintended changes in database
  • highly optimised global state reset
  • dataset loading - this feature is copied from PHPUnit database testcases
  • automatic generation of phpunit.xml - init script builds list of plugin testcases
  • database driver testing class - used for functional DB tests
  • SimpleTest emulation class - helps with migration of old tests
  • debugging() interception - enables to control and test any expected debug output. (Moodle 2.4 and upwards)

Limitations

  • no Selenium support
  • no support for PHPUnit_Extensions_Database_TestCase - it is possible to use data set loader only

See also