Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Writing PHPUnit tests: Difference between revisions

From MoodleDocs
(→‎Generators: do not use them in @dataprovider function)
m (fixing link to changelog (6.0))
(31 intermediate revisions by 13 users not shown)
Line 2: Line 2:


Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).
Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).
=Namespaces=
All the stuff under **/tests directories is [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|subject to some simple rules]] when using namespaces. They apply to test cases, fixtures, generators and, in general, any class within those directories. Take a look to them! (grand summary = 100% the same rules that are applied to **/classes directories).


=Testcase classes=
=Testcase classes=


There are two basic test class that are supposed to used in all Moodle unit tests - basic_testcase and advanced_testcase. '''Please note it is strongly recommended to put only one testcase into each class file.'''
There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. '''Please note it is strongly recommended to put only one testcase into each class file.'''
;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code.
;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code.
;provider_testcase: Enhanced testcase class, enhanced for easy testing of [[Privacy API|Privacy Providers]].


There is a third testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.
There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.


== Assertions ==
== Assertions ==


The complete list of assertions can be found in the [http://www.phpunit.de/manual/3.7/en/appendixes.assertions.html phpunit manual].
The complete list of assertions can be found in the [https://phpunit.readthedocs.io/en/7.0/assertions.html].
 
==Sample plugin testcase==
==Sample plugin testcase==


PHPUnit tests are located in <code>tests/*_test.php</code> files in your plugin, for example mod/myplugin/tests/myplugin_test.php, the file should contain only one class that extends <code>advanced_testcase</code>:
PHPUnit tests are located in <code>tests/*_test.php</code> files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends <code>advanced_testcase</code>:


<code php>
<code php>
  class mod_myplugin_testcase extends advanced_testcase {
  class mod_myplugin_sample_testcase extends advanced_testcase {
     public function test_adding() {
     public function test_adding() {
         $this->assertEquals(2, 1+2);
         $this->assertEquals(2, 1+2);
Line 25: Line 31:
  }
  }
</code>
</code>
See [[PHPUnit integration#Class and file naming rules]] for more information.
==Inclusion of Moodle library files==
If you want to include some Moodle library files you should always declare '''global $CFG'''. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically.


==Automatic state reset==
==Automatic state reset==
By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.
By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.
If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest().


<code php>
<code php>
Line 48: Line 62:
Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.
Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.


Note if you are using PHPUnit @dataProvider functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function.
Note if you are using PHPUnit [https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers @dataProvider] functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function.


==Creating users==
==Creating users==
Line 58: Line 72:
You may also specify properties of the user account, for example:
You may also specify properties of the user account, for example:
<code php>
<code php>
  $user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1');
  $user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1'));
</code>
</code>


Line 103: Line 117:
  $generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
  $generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
  $generator->create_instance(array('course'=>$course->id));
  $generator->create_instance(array('course'=>$course->id));
</code>
The following is functionally the same, but a bit shorter:
<code php>
$course = $this->getDataGenerator()->create_course();
$page = $this->getDataGenerator()->create_module('page', array('course' => $course->id));
</code>
</code>


Line 122: Line 142:
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');
</code>
</code>
==Creating scales==
<code php>
$this->getDataGenerator()->create_scale();
$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat));
</code>
==Creating roles==
<code php>
$this->getDataGenerator()->create_role();
$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype));
</code>
==Creating tags==
<code php>
$this->getDataGenerator()->create_tag();
$this->getDataGenerator()->create_tag(array(
    'userid' => $userid,
    'rawname' => $rawname,
    'name' => $name,
    'description' => $description,
    'descriptionformat' => $descriptionformat,
    'flag' => $flag
));
</code>
==Groups==
===Creating groups===
<code php>
$this->getDataGenerator()->create_group(array('courseid' => $courseid));
$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>
===Adding users to groups===
<code php>
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid));
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid));
</code>
===Creating groupings===
<code php>
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid));
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>
===Adding groups to groupings===
<code php>
$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid));
</code>
==Repositories==
===Creating repository instances===
{{Moodle 2.5}}
Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php..
<code php>
$this->getDataGenerator()->create_repository($type, $record, $options);
</code>
===Creating repository types===
{{Moodle 2.5}}
Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php..
<code php>
$this->getDataGenerator()->create_repository_type($type, $record, $options);
</code>
==Creating grades==
===Grade categories===
<code php>
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid));
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname));
</code>
===Grade items===
<code php>
$this->getDataGenerator()->create_grade_item();
$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype));
</code>
===Outcomes===
<code php>
$this->getDataGenerator()->create_grade_outcome();
$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname));
</code>
==Other types of plugin==
{{Moodle 2.5}}
Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname);
For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith.


=Long tests=
=Long tests=
Line 132: Line 252:
=Testing sending of messages=
=Testing sending of messages=
{{Moodle 2.4}}
{{Moodle 2.4}}
You can temporarily redirect all messages sent via message_send() to a message sink object. This allow developer to verify that the tested code is sending expected messages.
You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages.


To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later.
To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later.
Line 144: Line 264:
</code>
</code>


=Testing sending of emails=
{{Moodle 2.6}}
You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails.
To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later.
<code php>
unset_config('noemailever');
$sink = $this->redirectEmails();
//... code that is sending email
$messages = $sink->get_messages();
$this->assertEquals(1, count($messages));
</code>
=Logstores=
You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute.
<code php>
$this->preventResetByRollback();
set_config('enabled_stores', 'logstore_standard', 'tool_log');
set_config('buffersize', 0, 'logstore_standard');
</code>
=Check your coverage=
PHPUnit has the ability to generate code coverage information for your unit tests.
Prior to Moodle 3.7, this coverage would load all files and generate coverage for everything regardless of whether that file could be covered at all, or whether it was intentionally covered.
Since Moodle 3.7 the '''phpunit.xml''' configuration includes  generated Whitelist and Blacklist information for each component.  In addition to this the '''strict-coverage''' option is also set in the '''phpunit.xml''' which ensures that code is only marked as covered if it was intentionally covered.
==Generating blacklist and whitelist configuration==
You can programatically describe which files will be checked for coverage by creating a coverage.php file alongside the tests that you are writing.
The coverage.php file allows you to list white-listed and black-listed files and folders within the component being
tested. All paths specified are relative to the component being tested. For example, when working with '''mod_forum''' your
code will be in '''mod/forum''', and its unit tests will be in '''mod/forum/tests/example_test.php'''. The coverage file
for this would be in '''mod/forum/tests/coverage.php''' and all paths specified would be relative to '''mod/forum'''.
It is possible to specify a combination of whilte-listed files, white-listed folders, black-listed files, and
black-listed folders. This would allow you, for example, to whitelist the entire '''classes''' directory, but blacklist
a specific file or folder within it.
The following is an example coverage file from '''mod_forum''':
<code php>
return new class extends phpunit_coverage_info {
    /** @var array The list of folders relative to the plugin root to whitelist in coverage generation. */
    protected $whitelistfolders = [
        'classes/local',
        'externallib.php',
    ];
    /** @var array The list of files relative to the plugin root to whitelist in coverage generation. */
    protected $whitelistfiles = [];
    /** @var array The list of folders relative to the plugin root to excludelist in coverage generation. */
    protected $excludelistfolders = [];
    /** @var array The list of files relative to the plugin root to excludelist in coverage generation. */
    protected $excludelistfiles = [];
};
</code>
==Strict coverage==
Since Moodle 3.7, PHPUnit code coverage is strict about what is covered. This is to ensure that tests only cover the functions that they intend to test which is a best practice. Without this, much of Moodle is accidentally covered by virtue of being in the code-path, and it becomes very difficult to accurately determine whether a class or function is intentionally covered fully and correctly.
This is a PHPUnit feature and is [https://phpunit.readthedocs.io/en/8.1/code-coverage-analysis.html#specifying-covered-code-parts described in their documention].
=Best practice=
There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below.
==Check your coverage==
PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since
Moodle 3.7. We recommend that you consider checking the coverage of your plugins when you write your code.
==Keep use of resetAfterTest to a minimum==
Although many of the examples described above use the <code>resetAfterTest</code> nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to.
Generally speaking you should aim to write code which is mockable, and does not require real fixtures.
Use of resetAfterTest will also slow your tests down.
==Be careful with shared setUp and instance variables==
You should be careful of how you create and use instance variables in PHPUnit tests for two main reasons:
Firstly, if you create any fixtures in the setUp, or call the resetAfterTest function, these fixtures and conditions will apply for _all_ tests in the testsuite.
You will not be able to add another test to the suite which does not require these conditions without those conditions being fulfilled anyway.
This can lead to slow tests.
Secondly, because of the way in which PHPUnit operates. it creates an instance of each testcase during its bootstrap phase. These are stored in memory until the _entire suite_ completes.
This means that any fixture which is setup and not actively discarded will not be garbage collected and lead to memory bloat.
In severe cases this can lead to memory exhaustion.
==Make use of the dataProvider functionality==
The dataProvider functionality of PHPUnit is an extremely powerful and useful feature which allows you to verify a function quickly and easily with a range of different conditions.
However, the following rules should be followed when using dataProviders:
* Keep addition of resettable data requring resetAfterTest to a minimum - this will lead to many slow tests
* You can only _describe_ data in a dataProvider, you cannot create the data. The dataProvider is called after the testSuite is instantiated, but before any tests are run. Each test will run a full setUp and tearDown, which will destroy any data which was created.
=Extra test settings=
=Extra test settings=


Line 153: Line 372:


These constants may be then used in your test or plugin code.
These constants may be then used in your test or plugin code.
=Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5)=
{{Moodle 3.7}}
With Moodle 3.7, '''PHPUnit was upgraded to 7.5''' (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see MDL-65204 and linked issues for more details). While internally [https://phpunit.de/announcements/phpunit-7.html a lot of things changed with PHPUnit 7] (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.
To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources:
* [https://phpunit.de/announcements/phpunit-7.html  PHPUnit 7 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/520723129e2b3fc1dc4c0953e43c9d40e1ecb352/ChangeLog-7.5.md Detailed changelog of the release], paying special attention to the added, deprecated, changed and deleted sections.
* [https://phpunit.readthedocs.io/en/7.5/ Official PHPUnit manual].
* Changes performed into core, mainly: new, stricter, signatures ([https://github.com/moodle/moodle/commit/26218b7 26218b7]) and assertEquals() changes, specially important when comparing strings, now performed using strict (===) equals ([https://github.com/moodle/moodle/commit/85f47ba 85f47ba]).
=Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6)=
{{Moodle 3.4}}
With Moodle 3.4, '''PHPUnit was upgraded to 6.4''' (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see MDL-60611 and linked issues for more details). While internally [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues a lot of things changed with PHPUnit 6] (namespaced classes being the more noticeable), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.
Still, in '''some cases, it will impossible to maintain compatibility''' of tests between old (pre 3.4) tests and new ones, especially '''when direct use of any phpunit class is performed'''. Luckily, both travis and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly.
To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources:
* [https://thephp.cc/news/2017/02/migrating-to-phpunit-6 A very good mini-guide] showing all the important changes.
* [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues PHPUnit 6 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/9d0c024d2099531442d862b66b0ad7cf35ed8e78/ChangeLog-6.0.md Detailed changelog of the release], paying special attention to the changed and deleted sections.
* Changes performed into core, mainly: namespace class renaming ([https://github.com/moodle/moodle/commit/801a372dadb6e11c8781547603e3f0a59ce5638f 801a372]) and deprecated stuff ([https://github.com/moodle/moodle/commit/796e48a58bf18533bdca423fff7949ab119101c4 796e48a])


=See also=
=See also=

Revision as of 15:36, 4 May 2020

Moodle 2.3


Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).

Namespaces

All the stuff under **/tests directories is subject to some simple rules when using namespaces. They apply to test cases, fixtures, generators and, in general, any class within those directories. Take a look to them! (grand summary = 100% the same rules that are applied to **/classes directories).

Testcase classes

There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. Please note it is strongly recommended to put only one testcase into each class file.

basic_testcase
Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
advanced_testcase
Enhanced testcase class enhanced for easy testing of Moodle code.
provider_testcase
Enhanced testcase class, enhanced for easy testing of Privacy Providers.

There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.

Assertions

The complete list of assertions can be found in the [1].

Sample plugin testcase

PHPUnit tests are located in tests/*_test.php files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends advanced_testcase:

class mod_myplugin_sample_testcase extends advanced_testcase {
    public function test_adding() {
        $this->assertEquals(2, 1+2);
    }
}

See PHPUnit integration#Class and file naming rules for more information.

Inclusion of Moodle library files

If you want to include some Moodle library files you should always declare global $CFG. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically.

Automatic state reset

By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.

If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest().

class mod_myplugin_testcase extends advanced_testcase {
    public function test_deleting() {
        global $DB;
        $this->resetAfterTest(true);
        $DB->delete_records('user');
        $this->assertEmpty($DB->get_records('user'));
    }
    public function test_user_table_was_reset() {
        global $DB;
        $this->assertEquals(2, $DB->count_records('user', array()));
    }
}

Generators

Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.

Note if you are using PHPUnit @dataProvider functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function.

Creating users

At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use:

$user = $this->getDataGenerator()->create_user();

You may also specify properties of the user account, for example:

$user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1'));

By default no user is logged-in, use setUser() method to change current $USER value:

$this->setUser($user1);

Guest and admin accounts have a shortcut methods:

$this->setGuestUser();
$this->setAdminUser();

Null can be used to set current user back to not-logged-in:

$this->setUser(null);

Creating course categories

$category1 = $this->getDataGenerator()->create_category();
$category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id));

Creating courses

$course1 = $this->getDataGenerator()->create_course();

$category = $this->getDataGenerator()->create_category();
$course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id));

Creating activities

Some activity plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.

Example of creation of new course with one page resource:

$course = $this->getDataGenerator()->create_course();
$generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
$generator->create_instance(array('course'=>$course->id));

The following is functionally the same, but a bit shorter:

$course = $this->getDataGenerator()->create_course();
$page = $this->getDataGenerator()->create_module('page', array('course' => $course->id));

Creating cohorts

Moodle 2.4

Since 2.4 there the data generator supports creation of new cohorts.

$cohort = $this->getDataGenerator()->create_cohort();

Simplified user enrolments

Moodle 2.4

Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins.

$this->getDataGenerator()->enrol_user($userid, $courseid); $this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid); $this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');

Creating scales

$this->getDataGenerator()->create_scale(); $this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat));

Creating roles

$this->getDataGenerator()->create_role(); $this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype));

Creating tags

$this->getDataGenerator()->create_tag(); $this->getDataGenerator()->create_tag(array(

   'userid' => $userid, 
   'rawname' => $rawname,
   'name' => $name, 
   'description' => $description, 
   'descriptionformat' => $descriptionformat,
   'flag' => $flag

));

Groups

Creating groups

$this->getDataGenerator()->create_group(array('courseid' => $courseid)); $this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));

Adding users to groups

$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid)); $this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid));

Creating groupings

$this->getDataGenerator()->create_grouping(array('courseid' => $courseid)); $this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));

Adding groups to groupings

$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid));

Repositories

Creating repository instances

Moodle 2.5

Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

$this->getDataGenerator()->create_repository($type, $record, $options);

Creating repository types

Moodle 2.5

Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

$this->getDataGenerator()->create_repository_type($type, $record, $options);

Creating grades

Grade categories

$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid)); $this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname));

Grade items

$this->getDataGenerator()->create_grade_item(); $this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype));

Outcomes

$this->getDataGenerator()->create_grade_outcome(); $this->getDataGenerator()->create_grade_item(array('fullname' => $fullname));

Other types of plugin

Moodle 2.5

Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname);

For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith.

Long tests

All standard test should execute as fast as possible. Tests that take a loner time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php.

Large test data

See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See PHPUnit_integration#Extra_methods

Testing sending of messages

Moodle 2.4

You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages.

To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later. $this->preventResetByRollback(); $sink = $this->redirectMessages(); //... code that is sending messages $messages = $sink->get_messages(); $this->assertEquals(3, count($messages)); //.. test messages were generated in correct order with appropriate content

Testing sending of emails

Moodle 2.6

You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails.

To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later. unset_config('noemailever'); $sink = $this->redirectEmails(); //... code that is sending email $messages = $sink->get_messages(); $this->assertEquals(1, count($messages));

Logstores

You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute. $this->preventResetByRollback(); set_config('enabled_stores', 'logstore_standard', 'tool_log'); set_config('buffersize', 0, 'logstore_standard');

Check your coverage

PHPUnit has the ability to generate code coverage information for your unit tests.

Prior to Moodle 3.7, this coverage would load all files and generate coverage for everything regardless of whether that file could be covered at all, or whether it was intentionally covered.

Since Moodle 3.7 the phpunit.xml configuration includes generated Whitelist and Blacklist information for each component. In addition to this the strict-coverage option is also set in the phpunit.xml which ensures that code is only marked as covered if it was intentionally covered.

Generating blacklist and whitelist configuration

You can programatically describe which files will be checked for coverage by creating a coverage.php file alongside the tests that you are writing.

The coverage.php file allows you to list white-listed and black-listed files and folders within the component being tested. All paths specified are relative to the component being tested. For example, when working with mod_forum your code will be in mod/forum, and its unit tests will be in mod/forum/tests/example_test.php. The coverage file for this would be in mod/forum/tests/coverage.php and all paths specified would be relative to mod/forum.

It is possible to specify a combination of whilte-listed files, white-listed folders, black-listed files, and black-listed folders. This would allow you, for example, to whitelist the entire classes directory, but blacklist a specific file or folder within it.


The following is an example coverage file from mod_forum: return new class extends phpunit_coverage_info {

   /** @var array The list of folders relative to the plugin root to whitelist in coverage generation. */
   protected $whitelistfolders = [
       'classes/local',
       'externallib.php',
   ];
   /** @var array The list of files relative to the plugin root to whitelist in coverage generation. */
   protected $whitelistfiles = [];
   /** @var array The list of folders relative to the plugin root to excludelist in coverage generation. */
   protected $excludelistfolders = [];
   /** @var array The list of files relative to the plugin root to excludelist in coverage generation. */
   protected $excludelistfiles = [];

};

Strict coverage

Since Moodle 3.7, PHPUnit code coverage is strict about what is covered. This is to ensure that tests only cover the functions that they intend to test which is a best practice. Without this, much of Moodle is accidentally covered by virtue of being in the code-path, and it becomes very difficult to accurately determine whether a class or function is intentionally covered fully and correctly.

This is a PHPUnit feature and is described in their documention.

Best practice

There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below.

Check your coverage

PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since Moodle 3.7. We recommend that you consider checking the coverage of your plugins when you write your code.

Keep use of resetAfterTest to a minimum

Although many of the examples described above use the resetAfterTest nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to. Generally speaking you should aim to write code which is mockable, and does not require real fixtures. Use of resetAfterTest will also slow your tests down.

Be careful with shared setUp and instance variables

You should be careful of how you create and use instance variables in PHPUnit tests for two main reasons:

Firstly, if you create any fixtures in the setUp, or call the resetAfterTest function, these fixtures and conditions will apply for _all_ tests in the testsuite. You will not be able to add another test to the suite which does not require these conditions without those conditions being fulfilled anyway. This can lead to slow tests.

Secondly, because of the way in which PHPUnit operates. it creates an instance of each testcase during its bootstrap phase. These are stored in memory until the _entire suite_ completes. This means that any fixture which is setup and not actively discarded will not be garbage collected and lead to memory bloat. In severe cases this can lead to memory exhaustion.


Make use of the dataProvider functionality

The dataProvider functionality of PHPUnit is an extremely powerful and useful feature which allows you to verify a function quickly and easily with a range of different conditions. However, the following rules should be followed when using dataProviders:

  • Keep addition of resettable data requring resetAfterTest to a minimum - this will lead to many slow tests
  • You can only _describe_ data in a dataProvider, you cannot create the data. The dataProvider is called after the testSuite is instantiated, but before any tests are run. Each test will run a full setUp and tearDown, which will destroy any data which was created.

Extra test settings

Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php.

There are several ways how to inject your custom settings:

  • define test setting constants in your phpunit.xml file
  • define test setting constants in your config.php

These constants may be then used in your test or plugin code.

Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5)

Moodle 3.7


With Moodle 3.7, PHPUnit was upgraded to 7.5 (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see MDL-65204 and linked issues for more details). While internally a lot of things changed with PHPUnit 7 (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our wrapping layer (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources:

Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6)

Moodle 3.4


With Moodle 3.4, PHPUnit was upgraded to 6.4 (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see MDL-60611 and linked issues for more details). While internally a lot of things changed with PHPUnit 6 (namespaced classes being the more noticeable), thanks to our wrapping layer (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

Still, in some cases, it will impossible to maintain compatibility of tests between old (pre 3.4) tests and new ones, especially when direct use of any phpunit class is performed. Luckily, both travis and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly.

To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources:

See also