Testing

This page is the top level page regarding all testing activities around the Moodle project. Testing is essential to make sure that developed code does what it is meant to do, without causing new problems.

Manual testing

Code testing

Code is tested as part of reviewing at some key parts of the Moodle development process.

• Development - the developer of some code should test their own work on a wide variety of environments for correctness and performance
• Peer review - developers often test each others work early in the development process
• Integration reviews - Our integration team tests code weekly while they are evaluating suitability for integration into Moodle.

Integration functional testing

On Wednesday (all timezones) our Moodle HQ developers spend the day to manually test the functionality of all the issues that have been integrated that week. Where possible, developers submitting patches should first try to cover the testing required with unit tests and then with Behat behavioural tests. If neither of these are possible only then will manual testing be performed by the Moodle HQ developers.

Expectation from iTeam

• Most (if not all) tests should be allocated by Wednesday Morning 10:00 a.m (WST). Tests that appear after this might not be tested. If tests are allocated after this point, the allocated tester should be contacted individually by private message and allowed to acknowledge.
• The iTeam member assigning tests should put a message in office chat that tests have been allocated.
• The iTeam may need to help/re-assign tests if developers are having problems.

Expectation from tester

• If HQ developer is not available for testing on Wednesday, this should be recorded in the calendar. If this has not been communicated in the leave calendar, then the developer or management should let iTeam know ASAP.
• Testers should try to finish testing as early as possible on Wednesdays, so when tests fail, the issue assignee has enough time to resolve it (if possible).
• When a test fails, or looks like it may fail, the tester should contact the assignee individually to raise the problem.
• Testers should let the iTeam know if they are facing any problem and need help.
• For any reason (Big test, not enough time, not started testing yet), if a test is dragged to next day (Thursday) then he/she should leave comment on tracker, updating the status of testing and adding the expected time needed to finish testing.

• Testing of integrated issues

QA regression testing

For versions up to 2.4 all the QA testing was conducted manually before each major release. This process will soon be replaced by a combination of automated systems (see below) and a more lightweight manual user acceptance testing (UAT) cycle involving the Moodle community prior to release.

• QA testing

Automated testing

Unit tests

Moodle 2.3 and later fully supports PHPUnit tests as part of the code. These are automated tests of very low-level code functionality that a developer should write as part of any new code.

• PHPUnit_integration

Continuous integration testing

As soon as code is added to the integration repository, our continuous integration server tests the new code for:

• Coding guidelines
• PHPUnit tests
• SimpleTest unit tests on older versions of Moodle
• Detect unresolved merge conflicts
• Compare databases upgraded from previous versions
• Check the version.php is correct

A failure here notifies the integrators that the build has failed.

QA regression testing

Every day an automated build in a test server runs a large number of tests on key functions in Moodle to make sure everything still works and that some new fix in Moodle hasn't caused problems elsewhere.

These tests must pass completely before a release can be made.

• Acceptance testing using the Behat framework
• Performance testing using JMeter.