QA testing

Quality Assurance tests look at the functionality of Moodle from a user's point of view.

Real users systematically try each feature in Moodle and test that it works in the current version of the Moodle code.

These tests are repeated in series of cycles, usually just before major releases.

QA testing latest: Heads-up: We'll be starting Moodle 2.4 QA testing very soon!

Getting involved

Testers can pass or fail QA tests and add comments

Would you like to help with QA testing? If so, please contact Helen and ensure you're subscribed to the Testing and QA forum in order to receive QA testing news updates.

All testers are members of the testers group in the Moodle Tracker, enabling them to pass or fail QA tests and add comments.

Running tests

1. Select a test from the list
2. Using either the Moodle QA Testing Site or your own test site running the latest Moodle 2.4dev (available from Git on the integration/master branch git://git.moodle.org/integration.git), perform each of the steps listed in the test
3. Choose an appropriate workflow action:
   • Pass - Test runs perfectly :)
   • Fail - See below for actions to take
   • Obsolete - Test is no longer relevant in the current Moodle version
4. Add a comment about the workflow action (e.g. "This test passes - yippee!") to confirm that the test wasn't accidentally passed without being run

Please note: If you are behind a firewall you do not control or content filtering system, it may interfere with the use of the Moodle QA site. It is advisable that testing is completed where such complications are not in place.

When you pass or fail a test, you are automatically named as the assignee.

If in doubt whether to pass a test, give it a fail and perhaps ask someone else to have a look by adding them as a watcher then adding a comment.

If you notice that the test description is out-of-date, add Helen as a watcher (username tsala) then add a comment suggesting how it can be updated.

Note: Always use the latest version of Moodle 2.4dev for testing or the Moodle QA Testing Site. Contact Helen if you would like admin access to the Moodle QA Testing Site.

Failed tests

So you ran a test and it failed? Congratulations on finding a bug! Please do the following.

1. Click the Fail button at the top of the page.
2. Add a comment to the QA test stating that there was a problem and that you will report it as a Moodle bug.
3. Note the MDLQA number; it will be something like MDLQA-448.
4. Report the problem as a bug so that developers can investigate and fix it. Do this by clicking the button labelled "+ Create Issue" on the top-right corner of the Tracker page. The project should be "Moodle" and the Issue Type will be "Bug".
   
5. Add as much detail as to the bug description. Include the steps that would be needed to reproduce the bug. For the environment, write qa.moodle.net.
6. Include screenshots that illustrate the problem, if possible.
7. Give the issue the label 'mdlqa'.
8. In the new Moodle (MDL) issue select 'Link' from the 'More actions' dropdown menu.
   
9. Link to the QA test by selecting 'blocks' as the link type, entering the MDLQA number that you noted earlier, and optionally adding a comment.
   
10. (Optional) Add yourself as a watcher to the MDL issue so that you receive email notification when the issue is fixed.
11. When the MDL issue is fixed, hopefully within a day or two, the QA test can be reset, the reset QA test can then be run again.

Note for testers:

The Moodle QA Testing Site is updated daily at 11:00 UTC with the latest bug fixes to enable you to re-run QA tests.

Note for integrators:

After integrating a fix, please firstly add a comment to the linked MDLQA test thanking the tester and informing them that the test can be run again, then click the reset link. The tester will then receive email notification that the bug is fixed and will hopefully decide to run the test again soon. (Otherwise when the test is reset, it's assigned to nobody, and if the tester isn't watching the MDL issue, they won't know about the bug being fixed.)

The role of Core developers in creating tests

When a developer adds functionality to Moodle, they should also do the following.

1. Write user documentation in the User docs about the new feature (this should happen as part of the normal development process). Ask someone to proof read this documentation.
2. Add a feature to the release notes. This feature should include a link to the appropriate Tracker issue, which will contain testing instructions. The feature should also include a link to the user documentation for the feature.

The QA tests will be written by the Community Manager or a Test Writer based on the documentation and release notes written by the developer.

Writing new tests

Would you like to help with writing new QA tests? If so, you'll need to be a member of the test writers group in the Tracker. Please contact Helen about being added.

To create a new QA test:

1. Create a sub-task of MDLQA-1
2. Select Master copy as affected version
3. Select appropriate components including at least one of the following: User, Student, Teacher, Administrator
4. Leave the assignee as automatic (nobody)

New tests will be included in the next QA cycle.

Feedback

Feedback on all aspects of our QA testing process is welcome. If you have any questions or comments, please post in the Testing and QA forum in Using Moodle.

See also

• Testing credits

Comments on tests from previous QA cycles:

• Moodle 2.0 QA Cycle 1
• Moodle 2.0 QA Cycle 2
• Moodle 2.0.2 QA
• Moodle 2.1 QA Cycle 1
• Moodle 2.2 QA
• Moodle 2.3 QA