2.4 Test Plan

Jump to: navigation, search

General Aim

The general aim of testing applied to Moodle 2.4 is to provide early feedback and critique to developers about the system under test. This in turn will hopefully mitigate the bulk of issues earlier in the release cycle than Moodle QA. It may also increase flexibility and mitigate the risk posed by changing requirements; thus allowing requirements to change if necessary with the minimum of impact on the delivery of the project.

Testing Applied to the Roadmap

Suitable testing methods will be applied to the tracker issues in the roadmap for 2.4. Testing will be performed as features are implemented throughout the integration process. This includes pulling testable versions of Moodle from developers local branches and prototypes.

Unit Testing

PHP Unit tests will be provided where applicable(tba) and scheduled to run using the CI Server.

Feature Comparison Testing

Feature Comparison testing of MDL-21538 and MDL-31830 will take place at regular intervals during implementation. These tracker issues involve completely replacing existing Moodle plugins/features. The purpose of this testing is to ensure that no features are lost during implementation i.e. when implemented the features/plugins being implemented retain all of the functionality of what they are replacing without prior agreement otherwise.

The testing will provide a critique of the changes and the method used to test will be in the form of session based testing.

Usability Testing

MOBILE-153, MDL-34080 and MDL-31830 require usability testing. The method to be used for this is persona based testing. This involves creating a number of personas of potential users of these features and then running exploratory tests role playing as these users. e.g. a study at home single mum or a super geek who knows everything about everything and highly critical of Moodle. Persona testing will take place regularly during the implementation of 2.4.

Session Based Exploratory Testing

In Moodle QA Cycle 2 for Moodle 2.3 the Moodle HQ team took part in session based testing. Session based exploratory testing will take place at regular intervals for MDL-31830, MDL-21538, MDL-16660, MDL-34086 and MOBILE-153. Test sessions will be 1 hour long and of limited scope.

Installability Testing

MDL-34086 tackles upgrading from version 1.9 to 2.2 of Moodle. Selenium tests can be created to automate the upgrade part of this testing and some sanity testing of the application after the upgrade. This will be done in conjunction with session based testing.

Performance Testing

MDL-25290 aims to improve the performance of Moodle. A performance comparison of Moodle 2.3 to Moodle 2.4 will critique theses changes. This testing will be automated so the early involvement of the Moodle HQ sysadmin is required to ensure the correct monitors and testing tools are implemented.

Test Tools

Test Automation

Selenium Grid

Moodle have access to automated functional regression and browser interoperability testing via Selenium WebDriver. To execute these tests the following is requred:

  1. A GIT repository for the test and test harness application.
  2. A CI server to schedule tests and report results.
  3. Server VMs to host webservers and databases of the supported types.
  4. Client VM's to support supported client operating systems and browsers.

There is currently no suitable method of executing these tests as point 4 remains unimplemented.

JMeter

Sam Hemelryk has developed a performance test monitor web application based upon JMeter, this can be used to monitor performance.

PHPUnit

Moodle unit tests are written using the PHPUnit framework.

Miscellaneous

Additional monitors may be required to measure the performance of the system.

Test Management

Dedicated Test Management Plugin for Jira

In the past Moodle tracker has been used to manage test cases and test runs. Jira is not a test management tool and using it as such increases the documentation overhead for any scripted testing. Replacements are currently being evaluated.

Bonfire

The Bonfire Jira plugin was successfully used to run session based testing for Moodle 2.3. This will be used to manage and report upon exploratory test sessions during 2.4 testing.

Moodle 2.4 QA

Moodle 2.4 QA will go ahead as usual and the current test cases will need to be reviewed and updated as required. It will be possible, with a dedicated test management tool that supports test runs to remove some test cases for things such as browser interoperability and testing specific test environments.

The aim of Moodle 2.4 QA will be to put a gold stamp from the Moodle community on the release, to receive critique from our users and involve the Moodle community in product developement. The aim of the other methods of testing documented here is to take the burden of issue finding off the QA process late in the developement cycle.

Early and consistent promotion of Moodle QA and personal contact with existing Moodle QA testers will facilitate wider involvement in Moodle QA.

Risks

Change of requirements

Having a lightweight, flexible, early testing process will mitigate the risk posed by changes to requirements.

No Replacement Test Management Tool is Found

If this happens we can continue to use the existing tools.