2.4 Test Plan
The general aim of testing applied to Moodle 2.4 is to provide early feedback and critique to developers about the system under test. This in turn will hopefully mitigate the bulk of issues earlier in the release cycle than Moodle QA. It may also increase flexibility and mitigate the risk posed by changing requirements; thus allowing requirements to change if necessary with the minimum of impact on the delivery of the project.
Testing Applied to the Roadmap
Suitable testing methods will be applied to the tracker issues in the roadmap for 2.4. Testing will be performed as features are implemented throughout the integration process. This includes pulling testable versions of Moodle from developers local branches and prototypes.
PHP Unit tests will be provided where applicable(tba) and scheduled to run using the CI Server.
Feature Comparison Testing
Feature Comparison testing of MDL-21538 and MDL-31830 will take place at regular intervals during implementation. These tracker issues involve completely replacing existing Moodle plugins/features. The purpose of this testing is to ensure that no features are lost during implementation i.e. when implemented the features/plugins being implemented retain all of the functionality of what they are replacing without prior agreement otherwise.
The testing will provide a critique of the changes and the method used to test will be in the form of session based testing.
MOBILE-153, MDL-34080 and MDL-31830 require usability testing. The method to be used for this is persona based testing. This involves creating a number of personas of potential users of these features and then running exploratory tests role playing as these users. e.g. a study at home single mum or a super geek who knows everything about everything and highly critical of Moodle. Persona testing will take place regularly during the implementation of 2.4.
Session Based Exploratory Testing
In Moodle QA Cycle 2 for Moodle 2.3 the Moodle HQ team took part in session based testing. Session based exploratory testing will take place at regular intervals for MDL-31830, MDL-21538, MDL-16660, MDL-34086 and MOBILE-153. Test sessions will be 1 hour long and of limited scope.
MDL-34086 tackles upgrading from version 1.9 to 2.2 of Moodle. Selenium tests can be created to automate the upgrade part of this testing and some sanity testing of the application after the upgrade. This will be done in conjunction with session based testing.
MDL-25290 aims to improve the performance of Moodle. A performance comparison of Moodle 2.3 to Moodle 2.4 will critique theses changes. This testing will be automated so the early involvement of the Moodle HQ sysadmin is required to ensure the correct monitors and testing tools are implemented.
Moodle have access to automated functional regression and browser interoperability testing via Selenium WebDriver. To execute these tests the following is requred:
- A GIT repository for the test and test harness application.
- A CI server to schedule tests and report results.
- Server VMs to host webservers and databases of the supported types.
- Client VM's to support supported client operating systems and browsers.
There is currently no suitable method of executing these tests as point 4 remains unimplemented.
Sam Hemelryk has developed a performance test monitor web application based upon JMeter, this can be used to monitor performance.
Moodle unit tests are written using the PHPUnit framework.
Additional monitors may be required to measure the performance of the system.
Dedicated Test Management Plugin for Jira
In the past Moodle tracker has been used to manage test cases and test runs. Jira is not a test management tool and using it as such increases the documentation overhead for any scripted testing. Replacements are currently being evaluated.
The Bonfire Jira plugin was successfully used to run session based testing for Moodle 2.3. This will be used to manage and report upon exploratory test sessions during 2.4 testing.
Moodle 2.4 QA
Moodle 2.4 QA will go ahead as usual and the current test cases will need to be reviewed and updated as required. It will be possible, with a dedicated test management tool that supports test runs to remove some test cases for things such as browser interoperability and testing specific test environments.
The aim of Moodle 2.4 QA will be to put a gold stamp from the Moodle community on the release, to receive critique from our users and involve the Moodle community in product developement. The aim of the other methods of testing documented here is to take the burden of issue finding off the QA process late in the developement cycle.
Early and consistent promotion of Moodle QA and personal contact with existing Moodle QA testers will facilitate wider involvement in Moodle QA.
Change of requirements
Having a lightweight, flexible, early testing process will mitigate the risk posed by changes to requirements.
No Replacement Test Management Tool is Found
If this happens we can continue to use the existing tools.