Acceptance testing/Contributing automated tests: Difference between revisions
(Update link for Top priority tests for automation) |
Helen Foster (talk | contribs) (Choosing QA tests for automating, Automating a test update) |
||
Line 2: | Line 2: | ||
We are aiming to automate as many of these tests as possible using Moodle's new behat-based testing framework, so that they can be run weekly, and so that real users can focus on more exploratory testing instead. Currently there are over 400 MDLQA tests, so help in automating them is much appreciated! | We are aiming to automate as many of these tests as possible using Moodle's new behat-based testing framework, so that they can be run weekly, and so that real users can focus on more exploratory testing instead. Currently there are over 400 MDLQA tests, so help in automating them is much appreciated! | ||
==Automating a test== | ==Automating a test== | ||
# Choose a test from | |||
# Choose a test from the list [https://tracker.moodle.org/issues/?jql=project%20%3D%20MDLQA%20AND%20labels%20%3D%20mdlqa_conversion MDLQA tests with label mdlqa_conversion]. | |||
# Create a new MDL issue with summary 'Automate MDLQA-wxyz' and component 'Automated functional tests' and any MDLQA component such as 'Forum' which is also an MDL component. | # Create a new MDL issue with summary 'Automate MDLQA-wxyz' and component 'Automated functional tests' and any MDLQA component such as 'Forum' which is also an MDL component. | ||
# Add the '''mdlqa_conversion''' label to the new MDL issue | # Add the '''mdlqa_conversion''' label to the new MDL issue | ||
# Link to the MDLQA test by selecting 'Will help resolve' as the link type. | # Link to the MDLQA test by selecting 'Will help resolve' as the link type. | ||
# Follow the instructions provided in the | # Follow the instructions provided in the guide [[Writing acceptance tests]]. | ||
# Include 'MDLQA-wxyz' and a short description of what is being automated to the commit message/s where the .feature files are added/edited. | # Include 'MDLQA-wxyz' and a short description of what is being automated to the commit message/s where the .feature files are added/edited. | ||
''Many thanks for helping with test automating!'' | ''Many thanks for helping with test automating!'' | ||
==New features with acceptance tests== | ==New features with acceptance tests== | ||
New features with acceptance test will be tested in '''next''' QA cycle '''only'''. After QA cycle is over, it will be moved from MDLQA-1 to MDLQA-5249. | New features with acceptance test will be tested in '''next''' QA cycle '''only'''. After QA cycle is over, it will be moved from MDLQA-1 to MDLQA-5249. | ||
* Issues should be linked to new QA issue as "is a QA test written for" | * Issues should be linked to new QA issue as "is a QA test written for" | ||
* QA issue should be marked with priority '''blocker''' | * QA issue should be marked with priority '''blocker''' | ||
==Choosing QA tests for automating== | |||
* | |||
To decide if a QA test can be automated we ask: | |||
* Does it require any third-party servers or infrastructure e.g. OAuth 2 or CLI scripts? | |||
* Does it require a person to detect if something is 'correct' vs. present/absent on the page e.g. drag and drop functionality? | |||
If the answer to both questions is no, then it is likely that the QA test can be automated. | |||
New features and improvements which cannot have an automated test written for them should be labelled [https://tracker.moodle.org/issues/?jql=labels%20%3D%20qa_test_required qa_test_required] and detailed testing instructions provided which can then be used when writing a new MDLQA test. | New features and improvements which cannot have an automated test written for them should be labelled [https://tracker.moodle.org/issues/?jql=labels%20%3D%20qa_test_required qa_test_required] and detailed testing instructions provided which can then be used when writing a new MDLQA test. | ||
==Removing automated tests from the list of manual QA tests== | ==Removing automated tests from the list of manual QA tests== | ||
When a test is automated, it needs to be moved from MDLQA-1 to MDLQA-5249. | When a test is automated, it needs to be moved from MDLQA-1 to MDLQA-5249. | ||
[[Category:Quality Assurance]][[Category:Behat]] | [[Category:Quality Assurance]][[Category:Behat]] |
Revision as of 08:54, 13 September 2021
The issue MDLQA-1 contains testing tasks required to manually test many of the features and functions of Moodle.
We are aiming to automate as many of these tests as possible using Moodle's new behat-based testing framework, so that they can be run weekly, and so that real users can focus on more exploratory testing instead. Currently there are over 400 MDLQA tests, so help in automating them is much appreciated!
Automating a test
- Choose a test from the list MDLQA tests with label mdlqa_conversion.
- Create a new MDL issue with summary 'Automate MDLQA-wxyz' and component 'Automated functional tests' and any MDLQA component such as 'Forum' which is also an MDL component.
- Add the mdlqa_conversion label to the new MDL issue
- Link to the MDLQA test by selecting 'Will help resolve' as the link type.
- Follow the instructions provided in the guide Writing acceptance tests.
- Include 'MDLQA-wxyz' and a short description of what is being automated to the commit message/s where the .feature files are added/edited.
Many thanks for helping with test automating!
New features with acceptance tests
New features with acceptance test will be tested in next QA cycle only. After QA cycle is over, it will be moved from MDLQA-1 to MDLQA-5249.
- Issues should be linked to new QA issue as "is a QA test written for"
- QA issue should be marked with priority blocker
Choosing QA tests for automating
To decide if a QA test can be automated we ask:
- Does it require any third-party servers or infrastructure e.g. OAuth 2 or CLI scripts?
- Does it require a person to detect if something is 'correct' vs. present/absent on the page e.g. drag and drop functionality?
If the answer to both questions is no, then it is likely that the QA test can be automated.
New features and improvements which cannot have an automated test written for them should be labelled qa_test_required and detailed testing instructions provided which can then be used when writing a new MDLQA test.
Removing automated tests from the list of manual QA tests
When a test is automated, it needs to be moved from MDLQA-1 to MDLQA-5249.