|
|
| (4 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| {{Template:Work in progress}}[[Category:Quality Assurance]] | | {{Moved_to_dev_docs}} |
| =Introduction=
| |
| ==What is QA==
| |
| QA is known as one of the lastest stage of the software development. However sometimes QA starts even before the developers write any code. From the functional specification and use case, the QA test plans writer write test case. Usually it is one test case per use case. Once the test cases are written, QA testers will test the software following these test cases. When they find a bug, they will notify an issue (Jira). Once all the test cases have been executed, the QA manager will read the results and know how far from completion is the software. In order to make the QA people life easier, the QA team use a Test case management system. On this software they can write and execute test cases. They also read/analyse the results.
| |
| | |
| ==QA Objective==
| |
| The main objective is to know if a major version is right for release.
| |
| | |
| A major version is right for release when:
| |
| * blocker bug number < 1%
| |
| * critical bug number < 5%
| |
| * major bug number < 20%
| |
| * all test cases have been written/updated for new features
| |
| * all test cases are marked as ''Passed''
| |
| | |
| A successful QA cycle assures that we have tested all major Moodle functionalities.
| |
| | |
| ==Participate to QA for your benefit==
| |
| The document described the Moodle QA. Any participation is welcome. Please use the tab "page comments" in order to participate. A good Moodle QA will be a great benefit for Moodle project but also for any entity using Moodle.
| |
| | |
| | |
| | |
| = QA cycle =
| |
| We use Jira as Test Case Management System, see the specific chapter at the bottom of this page.
| |
| | |
| == Setting a QA cycle ==
| |
| We will need to write into Jira the test cases the first time, then update them for any new QA cycle. In fact it would be good that people writing/updating the functional specs, update the test cases in the same time. It would not take so long for them. Another possibility: create a test case status "''Cannot be run''". So when a tester cannot run a test case because the feature changed, the QA manager is aware that this specific test case needs to be updated.
| |
| | |
| == Running a QA cycle ==
| |
| QA participants will choose into Jira the feature they want to test (status set to "''Not run''" or "''to retest''").
| |
| * If the tester finds a blocker/critical/major bug, he sets the test case to "''Failed''". Then he writes a bug issue and links the test case to the bug issue. "''Failed''" test cases will be set to "''to retest''" by the bug fixer.
| |
| * If the bug is minor and the feature is working, the tester still writes a bug issue and still links the test case to the bug issue. However he sets the test case to "''Passed''".
| |
| | |
| == Validate a QA cycle ==
| |
| The Test Case management system (Jira) needs to display:
| |
| * how many test cases are "''failed''","''passed''","''not run''","''to retest''" for a specific version/component. (test case issues)
| |
| * how many critical/major bugs for a specific version/component (bug issues)
| |
| <br>From these results the QA manager valid the version for releasing.
| |
| | |
| =Draft/Ideas/Opinion/Requirements/Notes=
| |
| ==Need to be identified==
| |
| QA Cost (in persons-hours) for:
| |
| * writting test cases (the first time)
| |
| * preparing a QA cycle (update test cases, get a new QA system ready for testing)
| |
| * running all test cases
| |
| | |
| When should a QA cycle be run?<br>
| |
| What function specs/Use Cases have we got?<br>
| |
| Who can write test cases?<br>
| |
| Who can run test cases?<br>
| |
| Do we need test data?
| |
| | |
| ==Ideas/Notes==
| |
| * it would be good to identify the different kind of Moodledocs: User Manuals, Function Specifications, Technical Specifications, Requirements, ... due to the wiki collaborative aspect, it is understandable that sometimes documents are a mix of User Manual, Functional and Technical specs. Maybe could we create these specific category: User Manuals, Function Specifications, Technical Specifications, Requirements
| |
| * There is a lack of detailed functional specifications and Use Cases => at this moment only developers and highly experimented users would be able to write test cases with missing use cases.
| |
| * Moodle doesn't have deadline. It is released when it will be ready. However we have to take care about no-end QA phase.
| |
| * there are many experimented users who can test Moodle.
| |
| * some people/companies using Moodle probably have already written test plans, test cases, maybe even use cases.
| |
| * need documentation on how to write a test case and how to run test cases. Need also document for QA manager (setting a QA cycle, read the result, managing the QA cycle)
| |
| * write an easy-to-read Quality Engineering section in Moodledocs (include Code Testing as well).
| |
| * we could create a testing program as Netbeans [http://qa.netbeans.org/processes/cat/65/index.html]
| |
| * create a Moodle package including test data for people testing in local
| |
| * We dont have a long build process, so smoke testing could seem not required. However it could be good to have it if we create a testing program.
| |
| | |
| ==Jira as Test Case Management System==
| |
| The main reason to use Jira as Test Case Management System is that the Moodle community is familiar with. It will also be easy to link test case issues to bug issues.
| |
| * The search tool of Jira will provide the QA results.
| |
| * we will create a main issue containing thousand of sub-tasks. These subtasks will be the test cases. Then for every new QA cycle we will clone this main issue. Need to identify how to change subtask version quickly/easily.
| |
| * re-write the ''Jira as a Test Case Management Software documentation'' [https://docs.moodle.org/en/Jira_as_a_Test_Case_Management_Software]
| |
| | |
| ==Other QA from popular open-source projects==
| |
| * Netbeans wrote specification tests [http://wiki.netbeans.org/TestSpecifications]. They have a QA program [http://qa.netbeans.org/processes/cat/65/index.html]
| |
| * Firefox use an integrated testcase management and QA tool called Litmus [https://litmus.mozilla.org/]. An example of a test case: [https://litmus.mozilla.org/show_test.cgi?id=5036]. They've got more than 7000 test cases.
| |
| * OpenOffice has a QA page [http://qa.openoffice.org/ooQAReloaded/ooQA-ManualTesting.html]. I found it a bit a hassle to navigate into these pages. Make some succinct documentation for Moodle QA tester in order to start quickly and easily.
| |
