Quality assurance
What is QA
Here few documents related to QA/Testing on Wikipedia: [1], [2]
Participate to QA for your benefit
The document described the Moodle QA. Any participation is welcome. Please use the tab "page comments" in order to participate. A good Moodle QA will be a great benefit for Moodle project but also for any company using Moodle.
QA Objective
Main objective: is a major version ok for release?
A major version is ok when (TBD):
- blocker bug number < 1%
- critical bug number < 5%
- major bug number < 20%
- test cases has been written/updated for new features
- all test cases are marked as Passed
A successful QA cycle assures that we have tested all major Moodle functionalities.
QA cycle
Setting a QA cycle
We will need to write the test cases the first time, then update them for any new QA cycle. In fact it would be good that people writting/updating the functional specs update the test cases in the same time. It would not take so long for them. Other possibility, add a status "cannot be run". So when a tester cannot run a test case because the feature changed, the QA administrator is aware that this specific test case needs to be updated.
Running a QA cycle
All participants choose the feature they want to test (status set to "Not run" or "to retest").
- If they found a blocker/critical/major bug, they mark the test case as "failed". Then he writes a bug and link the test case to the bug issue. All "failed" test cases will be marked as "to retest" by the bug fixer.
- If the bug is minor and the feature is working, the tester still write a bug issue and link the test case to the bug issue. However he marks the test case has passed.
Validate a QA cycle
The Test Case management system needs to display:
- how many test cases are "failed","passed","not run","to retest" for a specific version/component. (test case issues)
- how many critical/major bugs for a specific version/component (bug issues)
From these results the QA manager valid the version for releasing.
Draft/Ideas/Opinion/Requirements/Notes
Need to be identified
QA Cost (in persons-hours) for:
- writting test cases (the first time)
- preparing the QA cycle (update test case, get a new QA system ready for testing, update test data)
- create a test data package for people testing on local?
- running all test cases
When should a QA cycle be run?
What function specs/Use Cases have we got?
Who can write test cases?
Who can run test cases?
Ideas/Notes
- We need to make a clear statement of the Moodle docs in order to make a difference between: User Manuals, Function Specifications, Technical Specifications, Requirements, ... and many mixed document. (The moodle docs are a powerful resource of information, however at the current time Google is your best friend in order to find information.)
- There is a lack of detailed functional specifications and Use Cases => at this moment only developers and experimented user would be able to write functionality with missing use cases. (not so good)
- use JIRA tracking system: we will use Jira as Test Case Managment system. Jira as not been developed for Test Case Management system but it can be used in order to manage a simple QA environment. The main reason to use Jira is to use a software that the community is familiar with.
- I need to study Quality Assurance into open source world: how does the other projects manage their QA department?
- Moodle doesn't have deadline. It is released when it will be ready. However take care about no-end QA phase.
- there are many experimented users that can test Moodle.
- some people/company using Moodle probably have already written test plans, test cases, maybe even use cases.
- need documentation on how to write a test cases and how to run test case. Need also document for QA administrator (setting a QA cycle, read the result, managing the QA cycle)
- need to link Moodledocs QA documents with Moodle docs tracker documents, organize an easy to read Quality Engineering section into Moodledocs (include Code Testing as well).
- we could create a testing program as Netbeans [3]
Jira
- The search tool of Jira will provide the QA results. Need to identify exatcly how.
- JIRA as a Test Case Management System: We'll create a main issue containing thousand of sub-tasks. These subtasks will be Test Case.
QA from other projects
Netbeans [4]