Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Testing: Difference between revisions

From MoodleDocs
m (→‎Integration functional testing: Fix the docs, we always test patches even if they include unit tests and behat tests.)
(14 intermediate revisions by 7 users not shown)
Line 1: Line 1:
This page is the top level page regarding all testing activities around the Moodle project.  Testing is essential to make sure that developed code does what it is meant to do, without causing new problems.
This page is the top level page regarding all testing activities around the Moodle project.  Testing is essential to make sure that developed code does what it is meant to do, without causing new problems.


==Unit tests (AUTOMATED)==
==Manual testing==


Moodle 2.3 and later fully supports PHPUnit tests as part of the code.  These are automated tests of very low-level code functionality that a developer should write as part of any new code.
===Code testing===
 
* [[PHPUnit_integration]]
 
==Code testing (MANUAL)==


Code is tested as part of reviewing at some key parts of the [[Process|Moodle development process]].
Code is tested as part of reviewing at some key parts of the [[Process|Moodle development process]].
Line 15: Line 11:
* Integration reviews - Our integration team tests code weekly while they are evaluating suitability for integration into Moodle.
* Integration reviews - Our integration team tests code weekly while they are evaluating suitability for integration into Moodle.


===Integration functional testing===
On Wednesday (all timezones) our Moodle HQ developers spend the day to manually test the functionality of all the issues that have been integrated that week. Developers submitting patches should always cover the patch with unit tests and/or Behat behavioural tests. Still, all issues are tested by a human and it is usually worth it.
* [[Testing of integrated issues]]
===QA regression testing===
For versions up to 2.4 all the [[QA testing]] was conducted manually before each major release.  This process will soon be replaced by a combination of automated systems (see below) and a more lightweight manual user acceptance testing (UAT) cycle involving the Moodle community prior to release.
* [[QA testing]]


==Continuous integration testing (AUTOMATED)==
==Automated testing==
 
===Unit tests===
 
Moodle 2.3 and later fully supports PHPUnit tests as part of the code.  These are automated tests of very low-level code functionality that a developer should write as part of any new code.
 
* [[PHPUnit_integration]]
 
===Continuous integration testing===


As soon as code is added to the integration repository, our continuous integration server tests the new code for:
As soon as code is added to the integration repository, our continuous integration server tests the new code for:
Line 29: Line 44:
A failure here notifies the integrators that the build has failed.
A failure here notifies the integrators that the build has failed.


==Integration functional testing (MANUAL)==
===QA regression testing===


On Wednesday (all timezones) our Moodle HQ developers spend the day to manually test the functionality of all the issues that have been integrated that week.
Every day an automated build in a test server runs a large number of tests on key functions in Moodle to make sure everything still works and that some new fix in Moodle hasn't caused problems elsewhere.
 
* [[Testing of integrated issues]]
 
 
==QA regression testing (AUTOMATED)==
 
Every week an automated build and test server runs a large number of tests on key functions in Moodle to make sure everything still works and that some new fix in Moodle hasn't caused problems elsewhere.


These tests must pass completely before a release can be made.
These tests must pass completely before a release can be made.


* [https://docs.moodle.org/dev/Behat Selenium based functional testing using the Behat framework]
* [[Acceptance testing]] using the Behat framework
* Performance testing using JMeter.
* Performance testing using JMeter.


==QA regression testing (MANUAL)==
Moodle uses a sponsored version of [https://www.browserstack.com BrowserStack] for testing on multiple browsers.


For past versions all the [[QA testing]] was conducted manually before each major release.  This process will soon be replaced by the above automated systems.


* [[QA testing]]
[[Category:Quality Assurance]]

Revision as of 08:23, 18 January 2017

This page is the top level page regarding all testing activities around the Moodle project. Testing is essential to make sure that developed code does what it is meant to do, without causing new problems.

Manual testing

Code testing

Code is tested as part of reviewing at some key parts of the Moodle development process.

  • Development - the developer of some code should test their own work on a wide variety of environments for correctness and performance
  • Peer review - developers often test each others work early in the development process
  • Integration reviews - Our integration team tests code weekly while they are evaluating suitability for integration into Moodle.

Integration functional testing

On Wednesday (all timezones) our Moodle HQ developers spend the day to manually test the functionality of all the issues that have been integrated that week. Developers submitting patches should always cover the patch with unit tests and/or Behat behavioural tests. Still, all issues are tested by a human and it is usually worth it.

QA regression testing

For versions up to 2.4 all the QA testing was conducted manually before each major release. This process will soon be replaced by a combination of automated systems (see below) and a more lightweight manual user acceptance testing (UAT) cycle involving the Moodle community prior to release.

Automated testing

Unit tests

Moodle 2.3 and later fully supports PHPUnit tests as part of the code. These are automated tests of very low-level code functionality that a developer should write as part of any new code.

Continuous integration testing

As soon as code is added to the integration repository, our continuous integration server tests the new code for:

  • Coding guidelines
  • PHPUnit tests
  • SimpleTest unit tests on older versions of Moodle
  • Detect unresolved merge conflicts
  • Compare databases upgraded from previous versions
  • Check the version.php is correct

A failure here notifies the integrators that the build has failed.

QA regression testing

Every day an automated build in a test server runs a large number of tests on key functions in Moodle to make sure everything still works and that some new fix in Moodle hasn't caused problems elsewhere.

These tests must pass completely before a release can be made.

Moodle uses a sponsored version of BrowserStack for testing on multiple browsers.