Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

QA testing: Difference between revisions

From MoodleDocs
(QA testing latest update - 100% in 3.4 cycle)
(removing QA testing latest)
 
(87 intermediate revisions by 10 users not shown)
Line 1: Line 1:
'''Quality Assurance''' tests look at the functionality of Moodle from a user's point of view.
{{Template:Migrated|newDocId=/general/development/process/testing/qa}}
'''Quality Assurance''' tests look at the functionality of Moodle from a user's point of view.  


Real users systematically try each feature in Moodle and test that it works in the current version of the Moodle code. These tests are repeated in series of cycles, usually just before major releases.
Real users systematically try each feature in Moodle and test that it works in the current version of the Moodle code. These tests are repeated in series of cycles, around 4 - 6 weeks before a major release, once all major features have landed.
 
<p class="note">'''QA testing latest''': ''A 100% pass rate was obtained in Moodle 3.4 QA testing. Many thanks to all our [[Testing credits|QA testers]].''</p>


==Getting involved==
==Getting involved==
 
Would you like to help with QA testing? If so, please make sure you have created an account in the [[Tracker_introduction|Moodle tracker]] and you're subscribed to the [https://moodle.org/mod/forum/view.php?id=56 Testing and QA forum] in order to receive QA testing news updates.
Would you like to help with QA testing? If so, please make sure you have created an account in the [[Tracker_introduction|Moodle tracker]], then contact [https://moodle.org/user/profile.php?id=24152&course=1 Helen]. Also, make sure you're subscribed to the [https://moodle.org/mod/forum/view.php?id=56 Testing and QA forum] in order to receive QA testing news updates.
 
All testers are members of the testers group in the Moodle Tracker, enabling them to pass or fail QA tests and add comments.
 
==Running tests==
==Running tests==
 
# Go to the [https://tracker.moodle.org/secure/Dashboard.jspa?selectPageId=11454 Moodle QA testing dashboard] and choose a test from the list of current QA cycle open issues. When viewing a test, if you wish, you can click the 'Assign to me' link on the right, so that nobody else chooses the same test to run. (If you then find you are unable to run the test, you can click the Assign button and set the assignee as 'Unassigned'.) Please note:
# Select a test from the list. When viewing a test, if you wish, you can click the 'Assign to me' link on the right, so that nobody else chooses the same test to run. (If you then find you are unable to run the test, you can assign the issue back to 'Unassigned'.) Please note:
#* Only assign an issue to yourself which no one else is testing (Assignee = Unassigned).
#* Only assign an issue to yourself which no one else is testing (Assignee = Unassigned).
#* Only assign one issue at a time unless you plan to test a number of related issues within the next 24 hours. In other words, don't assign several issues to yourself then do nothing for several days. ;-)
#* Only assign one issue at a time unless you plan to test a number of related issues within the next 24 hours. In other words, don't assign several issues to yourself then do nothing for several days. ;-)
#* The label ''test_server_required'' indicates issues that can't be tested on the QA testing site. The label ''credentials_required'' indicates that credentials such as an OAuth 2 service client ID and secret are required.
#* The label ''test_server_required'' indicates issues that can't be tested on the QA testing site. The label ''credentials_required'' indicates that credentials such as an OAuth 2 service client ID and secret are required.
# Using either the [http://qa.moodle.net/ Moodle QA Testing Site] or your own test site running the latest Moodle 3.4 beta (available from Git on the integration/master branch ''<nowiki>git://git.moodle.org/integration.git</nowiki>'') with [[:en:Debugging|debugging]] set to developer, perform each of the steps listed in the test. ''Please test using both the Clean and Boost themes.''
# Using either the [https://qa.moodledemo.net/ Moodle QA Testing Site] or your own test site running the latest Moodle 4.0dev (available from Git on the integration/master branch ''<nowiki>git://git.moodle.org/integration.git</nowiki>'') with [[:en:Debugging|debugging]] set to developer, perform each of the steps listed in the test.
# ''Please attach screenshots of the steps where you verify or check something.''
# If it makes sense, please test using the currently supported themes, Boost and Classic.
# Choose an appropriate workflow action:
# Choose an appropriate workflow action:
#* ''Pass'' - Test runs perfectly. Add comment such as feedback about a new feature, browsers used for testing (if applicable), or simply "This test passes - yippee!"
#* ''Pass'' - Test runs perfectly. Add comment such as feedback about a new feature, browsers used for testing (if applicable; example: "This test passes on Q&A site with Teacher role using Boost theme"), or simply "This test passes - yippee!"
#* ''Fail'' - Something doesn't work, or you obtain debugging messages. Add comment describing the step that doesn't work. If in doubt whether to pass a test, give it a fail and add a comment describing your doubts.
#* ''Fail'' - Something doesn't work, or you obtain debugging messages. Add comment describing the step that doesn't work. If in doubt whether to pass a test, give it a fail and add a comment describing your doubts.
#* ''Obsolete'' - Test is no longer relevant in the current Moodle version. Add comment explaining why.
#* ''Obsolete'' - Test is no longer relevant in the current Moodle version. Add comment explaining why.
 
If you notice that the test description is out-of-date, add a comment mentioning that it needs updating. Alternatively, if you'd like to help with updating the test yourself, see below.
If you notice that the test description is out-of-date, add a comment suggesting how it can be updated and type @Helen Foster then select Helen from the menu below the input box. Alternatively, if you'd like to help with updating the test yourself, see below.
==Any questions?==
 
If there is anything you are unsure of, such as whether to mark a test as failed, or you have any other questions, please ask in one of the following places:
===Moodle QA Testing Site===
* [https://t.me/moodleqa Moodle QA Telegram chat room] - To join the room for the first time, please use the [https://t.me/+cXneE_ZeZ_A4ODRk Moodle QA Telegram invite link].
 
* [https://moodle.org/mod/forum/view.php?id=56 Testing and QA forum]
The [http://qa.moodle.net/ Moodle QA Testing Site] is updated daily at around 13:00 UTC with the latest bug fixes to enable you to re-run QA tests.
==Moodle QA Testing Site==
The [https://qa.moodledemo.net/ Moodle QA Testing Site] is updated daily at around 13:00 UTC with the latest bug fixes to enable you to re-run QA tests.


To prevent the site being used for sending spam, no emails are sent from it. Thus, tests involving email cannot be run using the Moodle QA Testing Site. (If such tests are attempted, an email debug message is displayed. This is not a bug but rather expected behaviour.)
To prevent the site being used for sending spam, no emails are sent from it. Thus, tests involving email cannot be run using the Moodle QA Testing Site. (If such tests are attempted, an email debug message is displayed. This is not a bug but rather expected behaviour.)


Manager, teacher and student accounts are provided. Please contact Helen if you would like admin access to the Moodle QA Testing Site for running certain tests.
Teacher and student accounts are provided. If you would like admin access to the Moodle QA Testing Site for running certain tests, please ask in the Moodle QA Telegram chat room or the Testing and QA forum for someone to send you the account credentials via private message.
 
==<div id="failedTests">Failed tests</div>==
==<div id="failedTests">Failed tests</div>==
So you ran a test and it failed? Congratulations on finding a bug! Please do the following.
So you ran a test and it failed? Congratulations on finding a bug! Please do the following.
# Click the Fail button at the top of the page.
# Click the Fail button at the top of the page.
# Add a comment to the QA test stating that there was a problem and that you will report it as a Moodle bug.
# Add a comment to the QA test stating that there was a problem and that you will report it as a Moodle bug.
Line 46: Line 40:
# (Optional) Add yourself as a watcher to the MDL issue so that you receive email notification when the issue is fixed.
# (Optional) Add yourself as a watcher to the MDL issue so that you receive email notification when the issue is fixed.
# When the MDL issue is fixed, hopefully within a day or two, the QA test can be reset and can then be run again.
# When the MDL issue is fixed, hopefully within a day or two, the QA test can be reset and can then be run again.
==Resetting tests==
==Resetting tests==
''Note for integrators:''
''Note for integrators:''


After integrating a fix,  
After integrating a fix,  
# Reset the MDLQA test, adding a comment.
# Reset the MDLQA test, adding a comment.
# Remove the 'mdlqa' label from the MDL issue.
# Remove the 'mdlqa' label from the MDL issue.
# If the issue doesn't have testing instructions, pass it with message "Will be tested by MDLQA-XXXX".
# If the issue doesn't have testing instructions, pass it with message "Will be tested by MDLQA-XXXX".
The tester will then receive email notification that the bug is fixed and will hopefully decide to run the test again soon.
The tester will then receive email notification that the bug is fixed and will hopefully decide to run the test again soon.
==Fixing existing bugs==
At the beginning of the QA cycle, all bugs identified (both new and existing) are investigated promptly and hopefully fixed.


Suggested comment:
When we are close to the scheduled release date (1-2 weeks prior), developers must focus on fixing new bugs (which affect the upcoming release version) only.
Thanks for running this QA test. The bug causing it to fail has now been fixed, so the test can be reset and run again.
Please note that if you are using http://qa.moodle.net for testing, you will need to wait for the site to be updated at 13:00 UTC in order to test the bug fix.


==Updating tests==
Thus, at this point in the QA cycle, any bugs which also affect existing versions of Moodle are labelled qa_identified (and the label mdlqa removed) for investigation after the release.
==Testing tips==
When entering text into a form, try things like:
* <syntaxhighlight lang="php">&</syntaxhighlight> (ampersand), <syntaxhighlight lang="php">></syntaxhighlight> (greater than) or <syntaxhighlight lang="php"><</syntaxhighlight> (less than) e.g. <syntaxhighlight lang="php">x < 1 && x > 0</syntaxhighlight>
* <syntaxhighlight lang="php">0</syntaxhighlight> (the single digit 0)
* <syntaxhighlight lang="php">'</syntaxhighlight> (single quote) e.g. <syntaxhighlight lang="php">Fergal.O'Brien@example.com</syntaxhighlight>
* special characters e.g. <syntaxhighlight lang="php">café</syntaxhighlight> or <syntaxhighlight lang="php">囲碁</syntaxhighlight>
* very long strings
* different languages, such as a RTL language
==New QA tests required==
''Note for developers:''


QA tests often become out-of-date due to new developments. If you would like to help with updating tests, you'll need to be a member of the test writers group in the Tracker. Please contact Helen about being added.
If an issue fix cannot be covered by automated tests,  
# Add the label 'qa_test_required' to the issue.
# Add a comment explaining why it can't be covered by automated tests and suggesting which steps of the testing instructions should be included in a QA test e.g. steps 6-10 or all steps.
QA tests will then be written and included in the next QA cycle. For issues with long testing instructions, several QA tests will be written to cover the issue. If appropriate, activities etc. will be set up on the [https://qa.moodledemo.net/ Moodle QA Testing Site] to enable the issue to be easily tested in future.  


To update a QA test:
Similarly, for new features and improvements which would benefit from exploratory testing,
# Add the label 'qa_test_required' to the issue.
# Add a comment mentioning that exploratory testing is required.
Exploratory QA tests will then be written and included in the next QA cycle and then removed.
==Updating tests==
QA tests often become out-of-date due to new developments. If you would like to help with updating tests, you'll need to be a member of the test writers group in the Tracker.


# Search for the master copy of the test - with affects version as 'Master copy' and MDLQA-1 as parent.
To update a QA test original:
# Search for the test in MDLQA-1.
# Edit the test description.
# Edit the test description.
# Optional: Add a comment describing your edit.
If a test in the current QA cycle is marked as failed because it is out-of-date, the description should be updated (as well as the test original) before resetting the test. If the test in the current QA cycle is marked as passed, then only the test original needs to be updated.
==Writing new tests==
Would you like to help with writing new QA tests? If so, as for updating tests, you'll need to be a member of the test writers group in the Tracker.


==Writing new tests==
QA tests are needed for any features which can't be tested with automated testing, such as connecting to an external system, drag and drop functionality or a CLI script. Also if it requires a person to detect if something is 'correct' vs. present/absent on the page.


Would you like to help with writing new QA tests? If so, as for updating tests, you'll need to be a member of the test writers group in the Tracker. Please contact Helen about being added.
In addition, new features can benefit from exploratory testing by community volunteers.


To create a new QA test:
To create a new QA test:
# If appropriate, do a quick search of MDLQA-1 to check if there is an existing test which can be updated.
# If not, in MDLQA-1 from the More menu select 'Create sub-task'.
# Enter a summary such as 'A teacher can ...'.
# Select 'Original' as affected version and select appropriate components.
# In the description field add the test steps (usually between 3 and 10), similar to the issue's [[Testing instructions guide|testing instructions]], starting with 'Log in as a teacher...' or similar. It's a good idea to try doing the steps yourself as you write the test.
# Start some steps with 'Verify that ...' or similar.
# Click the Create button.
# Go to the MDL issue and create a ‘has a QA test’ link to the new QA test, adding a comment “This feature is now covered by the QA test MDLQA....”.


# Choose an issue from the  [https://tracker.moodle.org/issues/?jql=labels%20%3D%20qa_test_required%20AND%20status%20%3D%20Closed closed qa_test_required-labelled issues]
# Create a sub-task of MDLQA-1
# Select Master copy as affected version
# Select appropriate components including at least one of the following: User, Student, Teacher, Administrator
# Set assignee as unassigned
# Write the test (usually between 3 and 10 steps). It's a good idea to try doing the steps yourself as you write the test.
# Go to the MDL issue and create a ‘has a QA test’ link to the new QA test, adding a comment “Thanks for this improvement which is now covered by the QA test MDLQA....”, and if necessary remove the qa_test_required label. ''Make sure that all QA tests are linked back to a corresponding MDL issue.''
New tests will be included in the next QA cycle.


Notes:
* Add the label ''new''
* For tests which can’t be run on the QA testing site, such as ones where you need to check an email, label ''test_server_required''.
* For OAuth 2 tests and any other tests which require a client ID or secret to be entered, label ''credentials_required''.
* For issues which specifically mention in the testing instructions to test in different browsers, use the phrase "Test in as many browsers as possible and mention in a comment which ones you’ve used."
* For an exploratory test, begin the test description with "This is an exploratory test of a new feature or improvement, so please feel free to try anything you like and not just the test steps!"
* For a test requiring admin access which can be run on the QA site, add:
This test requires admin access. If you would like to use the [QA testing site|https://qa.moodledemo.net/] for running it, please see the [QA testing guide|https://docs.moodle.org/dev/QA_testing] for details of how to request admin access. Begin just after the hourly reset to give yourself plenty of time to complete the test!
==Feedback==
==Feedback==
 
Feedback on all aspects of our QA testing process is welcome. If you have any questions or comments, please post in the [https://moodle.org/mod/forum/view.php?id=56 Testing and QA forum].
Feedback on all aspects of our QA testing process is welcome. If you have any questions or comments, please post in the [http://moodle.org/mod/forum/view.php?id=56 Testing and QA forum].
 
==See also==
==See also==
* [https://tracker.moodle.org/secure/Dashboard.jspa?selectPageId=11454 QA testing dashboard]
* [https://tracker.moodle.org/secure/Dashboard.jspa?selectPageId=11454 QA testing dashboard]
* [[Testing credits]]
* [[Testing credits]]
* [[MDLQA-features]]
* [[MDLQA-features]]
 
* [https://moodle.org/mod/forum/discuss.php?d=351302 Useful tips for QA testing]
Comments on tests from previous QA cycles:
Comments on tests from previous QA cycles:
* [http://tracker.moodle.org/browse/MDLQA-150 Moodle 2.0 QA Cycle 1]
* [https://tracker.moodle.org/browse/MDLQA-150 Moodle 2.0 QA Cycle 1]
* [http://tracker.moodle.org/browse/MDLQA-328 Moodle 2.0 QA Cycle 2]
* [https://tracker.moodle.org/browse/MDLQA-328 Moodle 2.0 QA Cycle 2]
* [http://tracker.moodle.org/browse/MDLQA-540 Moodle 2.0.2 QA]
* [https://tracker.moodle.org/browse/MDLQA-540 Moodle 2.0.2 QA]
* [http://tracker.moodle.org/browse/MDLQA-944 Moodle 2.1 QA Cycle 1]
* [https://tracker.moodle.org/browse/MDLQA-944 Moodle 2.1 QA Cycle 1]
* [http://tracker.moodle.org/browse/MDLQA-1190 Moodle 2.2 QA]
* [https://tracker.moodle.org/browse/MDLQA-1190 Moodle 2.2 QA]
* [http://tracker.moodle.org/browse/MDLQA-1814 Moodle 2.3 QA]
* [https://tracker.moodle.org/browse/MDLQA-1814 Moodle 2.3 QA]
* [http://tracker.moodle.org/browse/MDLQA-4602 Moodle 2.4 QA]
* [https://tracker.moodle.org/browse/MDLQA-4602 Moodle 2.4 QA]
* [http://tracker.moodle.org/browse/MDLQA-5267 Moodle 2.5 QA]
* [https://tracker.moodle.org/browse/MDLQA-5267 Moodle 2.5 QA]
* [https://tracker.moodle.org/browse/MDLQA-5740 Moodle 2.6 QA]
* [https://tracker.moodle.org/browse/MDLQA-5740 Moodle 2.6 QA]
* [https://tracker.moodle.org/browse/MDLQA-6693 Moodle 2.7 QA]
* [https://tracker.moodle.org/browse/MDLQA-6693 Moodle 2.7 QA]
Line 117: Line 132:
* [https://tracker.moodle.org/browse/MDLQA-10403 Moodle 3.3 QA]
* [https://tracker.moodle.org/browse/MDLQA-10403 Moodle 3.3 QA]
* [https://tracker.moodle.org/browse/MDLQA-10999 Moodle 3.4 QA]
* [https://tracker.moodle.org/browse/MDLQA-10999 Moodle 3.4 QA]
 
* [https://tracker.moodle.org/browse/MDLQA-11698 Moodle 3.5 QA]
* [https://tracker.moodle.org/browse/MDLQA-12282 Moodle 3.6 QA]
* [https://tracker.moodle.org/browse/MDLQA-12911 Moodle 3.7 QA]
* [https://tracker.moodle.org/browse/MDLQA-13517 Moodle 3.8 QA]
* [https://tracker.moodle.org/browse/MDLQA-14131 Moodle 3.9 QA]
* [https://tracker.moodle.org/browse/MDLQA-14813 Moodle 3.10 QA]
* [https://tracker.moodle.org/browse/MDLQA-15457 Moodle 3.11 QA]
* [https://tracker.moodle.org/secure/Dashboard.jspa?selectPageId=11454 Moodle 4.0 QA]
[[Category:Quality Assurance]]
[[Category:Quality Assurance]]

Latest revision as of 04:44, 15 July 2022

Important:

This content of this page has been updated and migrated to the new Moodle Developer Resources. The information contained on the page should no longer be seen up-to-date.

Why not view this page on the new site and help us to migrate more content to the new site!

Quality Assurance tests look at the functionality of Moodle from a user's point of view.

Real users systematically try each feature in Moodle and test that it works in the current version of the Moodle code. These tests are repeated in series of cycles, around 4 - 6 weeks before a major release, once all major features have landed.

Getting involved

Would you like to help with QA testing? If so, please make sure you have created an account in the Moodle tracker and you're subscribed to the Testing and QA forum in order to receive QA testing news updates.

Running tests

  1. Go to the Moodle QA testing dashboard and choose a test from the list of current QA cycle open issues. When viewing a test, if you wish, you can click the 'Assign to me' link on the right, so that nobody else chooses the same test to run. (If you then find you are unable to run the test, you can click the Assign button and set the assignee as 'Unassigned'.) Please note:
    • Only assign an issue to yourself which no one else is testing (Assignee = Unassigned).
    • Only assign one issue at a time unless you plan to test a number of related issues within the next 24 hours. In other words, don't assign several issues to yourself then do nothing for several days. ;-)
    • The label test_server_required indicates issues that can't be tested on the QA testing site. The label credentials_required indicates that credentials such as an OAuth 2 service client ID and secret are required.
  2. Using either the Moodle QA Testing Site or your own test site running the latest Moodle 4.0dev (available from Git on the integration/master branch git://git.moodle.org/integration.git) with debugging set to developer, perform each of the steps listed in the test.
  3. Please attach screenshots of the steps where you verify or check something.
  4. If it makes sense, please test using the currently supported themes, Boost and Classic.
  5. Choose an appropriate workflow action:
    • Pass - Test runs perfectly. Add comment such as feedback about a new feature, browsers used for testing (if applicable; example: "This test passes on Q&A site with Teacher role using Boost theme"), or simply "This test passes - yippee!"
    • Fail - Something doesn't work, or you obtain debugging messages. Add comment describing the step that doesn't work. If in doubt whether to pass a test, give it a fail and add a comment describing your doubts.
    • Obsolete - Test is no longer relevant in the current Moodle version. Add comment explaining why.

If you notice that the test description is out-of-date, add a comment mentioning that it needs updating. Alternatively, if you'd like to help with updating the test yourself, see below.

Any questions?

If there is anything you are unsure of, such as whether to mark a test as failed, or you have any other questions, please ask in one of the following places:

Moodle QA Testing Site

The Moodle QA Testing Site is updated daily at around 13:00 UTC with the latest bug fixes to enable you to re-run QA tests.

To prevent the site being used for sending spam, no emails are sent from it. Thus, tests involving email cannot be run using the Moodle QA Testing Site. (If such tests are attempted, an email debug message is displayed. This is not a bug but rather expected behaviour.)

Teacher and student accounts are provided. If you would like admin access to the Moodle QA Testing Site for running certain tests, please ask in the Moodle QA Telegram chat room or the Testing and QA forum for someone to send you the account credentials via private message.

Failed tests

So you ran a test and it failed? Congratulations on finding a bug! Please do the following.

  1. Click the Fail button at the top of the page.
  2. Add a comment to the QA test stating that there was a problem and that you will report it as a Moodle bug.
  3. Note the MDLQA number; it will be something like MDLQA-448.
  4. Try searching for whether the bug has been reported previously, and if not create a new issue for it (as described in Tracker introduction).
  5. In the new Moodle (MDL) issue select 'Link' from the 'More actions' dropdown menu.
    Linking to the QA issue in the tracker
  6. Link to the QA test by selecting 'blocks' as the link type, entering the MDLQA number that you noted earlier, and optionally adding a comment.
    Adding details for a link to the QA issue
  7. Give the issue the label 'mdlqa'.
  8. (Optional) Add yourself as a watcher to the MDL issue so that you receive email notification when the issue is fixed.
  9. When the MDL issue is fixed, hopefully within a day or two, the QA test can be reset and can then be run again.

Resetting tests

Note for integrators:

After integrating a fix,

  1. Reset the MDLQA test, adding a comment.
  2. Remove the 'mdlqa' label from the MDL issue.
  3. If the issue doesn't have testing instructions, pass it with message "Will be tested by MDLQA-XXXX".

The tester will then receive email notification that the bug is fixed and will hopefully decide to run the test again soon.

Fixing existing bugs

At the beginning of the QA cycle, all bugs identified (both new and existing) are investigated promptly and hopefully fixed.

When we are close to the scheduled release date (1-2 weeks prior), developers must focus on fixing new bugs (which affect the upcoming release version) only.

Thus, at this point in the QA cycle, any bugs which also affect existing versions of Moodle are labelled qa_identified (and the label mdlqa removed) for investigation after the release.

Testing tips

When entering text into a form, try things like:

  • &
    
    (ampersand),
    >
    
    (greater than) or
    <
    
    (less than) e.g.
    x < 1 && x > 0
    
  • 0
    
    (the single digit 0)
  • '
    
    (single quote) e.g.
    Fergal.O'Brien@example.com
    
  • special characters e.g.
    café
    
    or
    囲碁
    
  • very long strings
  • different languages, such as a RTL language

New QA tests required

Note for developers:

If an issue fix cannot be covered by automated tests,

  1. Add the label 'qa_test_required' to the issue.
  2. Add a comment explaining why it can't be covered by automated tests and suggesting which steps of the testing instructions should be included in a QA test e.g. steps 6-10 or all steps.

QA tests will then be written and included in the next QA cycle. For issues with long testing instructions, several QA tests will be written to cover the issue. If appropriate, activities etc. will be set up on the Moodle QA Testing Site to enable the issue to be easily tested in future.

Similarly, for new features and improvements which would benefit from exploratory testing,

  1. Add the label 'qa_test_required' to the issue.
  2. Add a comment mentioning that exploratory testing is required.

Exploratory QA tests will then be written and included in the next QA cycle and then removed.

Updating tests

QA tests often become out-of-date due to new developments. If you would like to help with updating tests, you'll need to be a member of the test writers group in the Tracker.

To update a QA test original:

  1. Search for the test in MDLQA-1.
  2. Edit the test description.

If a test in the current QA cycle is marked as failed because it is out-of-date, the description should be updated (as well as the test original) before resetting the test. If the test in the current QA cycle is marked as passed, then only the test original needs to be updated.

Writing new tests

Would you like to help with writing new QA tests? If so, as for updating tests, you'll need to be a member of the test writers group in the Tracker.

QA tests are needed for any features which can't be tested with automated testing, such as connecting to an external system, drag and drop functionality or a CLI script. Also if it requires a person to detect if something is 'correct' vs. present/absent on the page.

In addition, new features can benefit from exploratory testing by community volunteers.

To create a new QA test:

  1. If appropriate, do a quick search of MDLQA-1 to check if there is an existing test which can be updated.
  2. If not, in MDLQA-1 from the More menu select 'Create sub-task'.
  3. Enter a summary such as 'A teacher can ...'.
  4. Select 'Original' as affected version and select appropriate components.
  5. In the description field add the test steps (usually between 3 and 10), similar to the issue's testing instructions, starting with 'Log in as a teacher...' or similar. It's a good idea to try doing the steps yourself as you write the test.
  6. Start some steps with 'Verify that ...' or similar.
  7. Click the Create button.
  8. Go to the MDL issue and create a ‘has a QA test’ link to the new QA test, adding a comment “This feature is now covered by the QA test MDLQA....”.


Notes:

  • Add the label new
  • For tests which can’t be run on the QA testing site, such as ones where you need to check an email, label test_server_required.
  • For OAuth 2 tests and any other tests which require a client ID or secret to be entered, label credentials_required.
  • For issues which specifically mention in the testing instructions to test in different browsers, use the phrase "Test in as many browsers as possible and mention in a comment which ones you’ve used."
  • For an exploratory test, begin the test description with "This is an exploratory test of a new feature or improvement, so please feel free to try anything you like and not just the test steps!"
  • For a test requiring admin access which can be run on the QA site, add:
This test requires admin access. If you would like to use the [QA testing site|https://qa.moodledemo.net/] for running it, please see the [QA testing guide|https://docs.moodle.org/dev/QA_testing] for details of how to request admin access. Begin just after the hourly reset to give yourself plenty of time to complete the test!

Feedback

Feedback on all aspects of our QA testing process is welcome. If you have any questions or comments, please post in the Testing and QA forum.

See also

Comments on tests from previous QA cycles: