Testing strategy/Testing Methodology At Moodle
Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.
Testing Methodology at Moodle
Testing Methodology
This section provides a definition of distinct testing methodologies at Moodle. It gives context to what types of tests should run and when and also categorizes distinct types of test and their purpose. This section is intended as reference material; defining a language used to describe testing methods.
It is desirable on an agile project that a process of test driven development and continuous integration (CI) is performed. The aim of this is to increase business value and return on investment (ROI) by allowing software defects and issues - such as conflicting or problematic requirements - to be detected early in the software development life cycle (SDLC) when they can be resolved more easily. Although some manual testing is still required; to provide frequent and swift feedback, it is desirable that test automation is used where possible.
Leading agile testing exponent, Brian Marick, introduced the concept of testing quadrants to define different categories of tests that accomplish different purposes:
Fig.1 Agile testing matrix
IMAGE TO BE INCLUDED
Lisa Crispin and Janet Gregory explain these quadrants in detail in their book: “Agile Testing: A Practical Guide or Testers and Agile Teams”. To summarise, this is a list of traditional testing disciplines categorised relative to their context in an Agile process such as scrum.
The agile testing quadrant defines testing categories relevant to the organisation and facilitates communication of required testing on Moodle projects.
Quadrant Terms Explained
Tests that support the team – tests that assist with development and requirements gathering.
Tests that critique the product – constructive appraisal of the software under test to drive future development.
Technology facing – tests that verify scientific and technological aspects of the software under test.
Business facing – tests that verify the business logic, processes and communicate user requirements.
Testing Disciplines Explained
Functional Testing – Interacting positively and negatively with the software under test the way a user or process would, with specific expectations of inputs and outputs from the system under test.
Examples – Examples of user scenarios.
Story Tests – Defining a development story using a test.
Prototypes – e.g. paper prototyping of the system under development to gather and communicate customer requirements for the user interface.
Simulations – e.g. Wizard of Oz testing; similar to paper prototyping but involving inputs to and outputs from a human representing the system. Used for gathering functional requirements from users.
Unit testing – testing the behaviour of a particular piece of code.
Component testing – testing the interaction of interrelated pieces of code.
Exploratory testing – freestyle testing of the system under test. The scope of exploratory testing can be limited by defined constraints.
Scenarios – Specific business scenarios.
Usability testing – Using roles to exercise the system in a way the user might (something Moodle already does).
UAT (User Acceptance Testing) – At Moodle QA testing is User Acceptance Testing i.e. Real users executing tests to evaluate it's suitability relevant to business requirements.
Alpha/Beta – Suitable for software distributed to a large customer base where formal UAT may not be practical. Both have a similar goal to UAT. Alpha is an early distribution of the software, that may not be fully complete, to selected users for early feedback. Beta is distribution of almost finished software to users.
Performance and Load Testing – Testing the responsiveness of the software in normal circumstances and under high usage conditions.
Security Testing – Testing application security features.
'ility' Testing – Security, Maintainability, Interoperability, Compatibility, Scalability etc. etc.
