Quality assurance

Introduction

What is QA

Here few documents related to QA/Testing on Wikipedia: [1], [2]

QA Objective

Main objective: is a major version right for release?

A major version is right when (TBD):

• blocker bug number < 1%
• critical bug number < 5%
• major bug number < 20%
• test cases have been written/updated for new features
• all test cases are marked as Passed

A successful QA cycle assures that we have tested all major Moodle functionalities.

Participate to QA for your benefit

The document described the Moodle QA. Any participation is welcome. Please use the tab "page comments" in order to participate. A good Moodle QA will be a great benefit for Moodle project but also for any entity using Moodle.

QA cycle

We use Jira as Test Case Management System, see the specific chapter at the bottom of this page.

Setting a QA cycle

We will need to write into Jira the test cases the first time, then update them for any new QA cycle. In fact it would be good that people writing/updating the functional specs, update the test cases in the same time. It would not take so long for them. Another possibility: create a test case status "Cannot be run". So when a tester cannot run a test case because the feature changed, the QA manager is aware that this specific test case needs to be updated.

Running a QA cycle

QA participants will choose into Jira the feature they want to test (status set to "Not run" or "to retest").

• If the tester find a blocker/critical/major bug, he sets the test case to "Failed". Then he writes a bug issue and links the test case to the bug issue. "Failed" test cases will be set to "to retest" by the bug fixer.
• If the bug is minor and the feature is working, the tester still write a bug issue and link the test case to the bug issue. However he set the test case to "Passed".

Validate a QA cycle

The Test Case management system (Jira) needs to display:

• how many test cases are "failed","passed","not run","to retest" for a specific version/component. (test case issues)
• how many critical/major bugs for a specific version/component (bug issues)

From these results the QA manager valid the version for releasing.

Draft/Ideas/Opinion/Requirements/Notes

Need to be identified

QA Cost (in persons-hours) for:

• writting test cases (the first time)
• preparing the QA cycle (update test case, get a new QA system ready for testing, update test data)
• create a test data package for people testing on local?
• running all test cases

When should a QA cycle be run?
What function specs/Use Cases have we got?
Who can write test cases?
Who can run test cases?

Ideas/Notes

• We need to make a clear statement of the Moodle docs in order to make a difference between: User Manuals, Function Specifications, Technical Specifications, Requirements, ... and many mixed document. (The moodle docs are a powerful resource of information, however at the current time Google is your best friend in order to find information.)
• There is a lack of detailed functional specifications and Use Cases => at this moment only developers and experimented user would be able to write functionality with missing use cases. (not so good)
• use JIRA tracking system: we will use Jira as Test Case Managment system. Jira as not been developed for Test Case Management system but it can be used in order to manage a simple QA environment. The main reason to use Jira is to use a software that the community is familiar with.
• I need to study Quality Assurance into open source world: how does the other projects manage their QA department?
• Moodle doesn't have deadline. It is released when it will be ready. However take care about no-end QA phase.
• there are many experimented users that can test Moodle.
• some people/company using Moodle probably have already written test plans, test cases, maybe even use cases.
• need documentation on how to write a test cases and how to run test case. Need also document for QA administrator (setting a QA cycle, read the result, managing the QA cycle)
• need to link Moodledocs QA documents with Moodle docs tracker documents, organize an easy to read Quality Engineering section into Moodledocs (include Code Testing as well).
• we could create a testing program as Netbeans [3]

Jira as Test Case Management System

• The search tool of Jira will provide the QA results. Need to identify exatcly how.
• JIRA as a Test Case Management System: We'll create a main issue containing thousand of sub-tasks. These subtasks will be Test Case.

QA from other projects

Netbeans [4]