Quality assurance

Introduction

What is QA

Here few documents related to QA/Testing on Wikipedia: [1], [2]

QA Objective

Main objective: is a major version right for release?

A major version is right when (TBD):

• blocker bug number < 1%
• critical bug number < 5%
• major bug number < 20%
• test cases have been written/updated for new features
• all test cases are marked as Passed

A successful QA cycle assures that we have tested all major Moodle functionalities.

Participate to QA for your benefit

The document described the Moodle QA. Any participation is welcome. Please use the tab "page comments" in order to participate. A good Moodle QA will be a great benefit for Moodle project but also for any entity using Moodle.

QA cycle

We use Jira as Test Case Management System, see the specific chapter at the bottom of this page.

Setting a QA cycle

We will need to write into Jira the test cases the first time, then update them for any new QA cycle. In fact it would be good that people writing/updating the functional specs, update the test cases in the same time. It would not take so long for them. Another possibility: create a test case status "Cannot be run". So when a tester cannot run a test case because the feature changed, the QA manager is aware that this specific test case needs to be updated.

Running a QA cycle

QA participants will choose into Jira the feature they want to test (status set to "Not run" or "to retest").

• If the tester finds a blocker/critical/major bug, he sets the test case to "Failed". Then he writes a bug issue and links the test case to the bug issue. "Failed" test cases will be set to "to retest" by the bug fixer.
• If the bug is minor and the feature is working, the tester still writes a bug issue and still links the test case to the bug issue. However he sets the test case to "Passed".

Validate a QA cycle

The Test Case management system (Jira) needs to display:

• how many test cases are "failed","passed","not run","to retest" for a specific version/component. (test case issues)
• how many critical/major bugs for a specific version/component (bug issues)

From these results the QA manager valid the version for releasing.

Draft/Ideas/Opinion/Requirements/Notes

Need to be identified

QA Cost (in persons-hours) for:

• writting test cases (the first time)
• preparing a QA cycle (update test cases, get a new QA system ready for testing)
• running all test cases

When should a QA cycle be run?
What function specs/Use Cases have we got?
Who can write test cases?
Who can run test cases?
Do we need test data?

Ideas/Notes

• We need to make a clear statement of the Moodle docs in order to make a difference between: User Manuals, Function Specifications, Technical Specifications, Requirements, ... and many existing mixed document. (The moodle docs are a powerful resource of information, however at the current time Google is your best friend in order to find information.)
• There is a lack of detailed functional specifications and Use Cases => at this moment only developers and experimented users would be able to write test cases with missing use cases.
• I need to study Quality Assurance into open source world: how does the other projects manage their QA department?
• Moodle doesn't have deadline. It is released when it will be ready. However take care about no-end QA phase.
• there are many experimented users that can test Moodle.
• some people/company using Moodle probably have already written test plans, test cases, maybe even use cases.
• need documentation on how to write a test cases and how to run test case. Need also document for QA administrator (setting a QA cycle, read the result, managing the QA cycle)
• need to link Moodledocs QA documents with Moodle docs tracker documents, organize an easy to read Quality Engineering section into Moodledocs (include Code Testing as well).
• we could create a testing program as Netbeans [3]
• create a Moodle package including test data for people testing on local

Jira as Test Case Management System

The main reason to use Jira as Test Case Management System is that the community is familiar with. It will also be easy to link test case issue to bug issue.

• The search tool of Jira will provide the QA results.
• we will create a main issue containing thousand of sub-tasks. These subtasks will be the test cases. Then for every new QA cycle we will clone this main issue. Need to identify how change version of this subtask quicly/easily.

QA from other projects

Netbeans [4]