{{Work in progress}}
=Methodologies in the Moodle Development Process=
Moodle has an established process for developing and integrating software. This section describes relevant testing methodology within the existing process.
The full [https://docs.moodle.org/dev/Process development]
life-cycle at Moodle is documented in detail [https://docs.moodle.org/dev/Releases#General_release_calendar elsewhere].
Moodle is currently developed using Agile methodologies including
Scrum. The following stages of development take place within this
life-cycle:

* Weekly integration cycle
* 3-weekly stable sprints
* 6-monthly major releases
* 3 minor releases between every major release

==Weekly (Continuous) Integration Cycle==
A manual testing phase of integration issues occurs towards
the end of the SDLC at Moodle during the Wednesday (AWST) testing
phase. This is the last possible chance Moodle gets to capture
regressions. Relying soley on this phase to capture regressions could result in the late discovery of regressions
during unguided exploratory testing and worse still the introduction
of regressions into Master. Discovery of issues late in the SDLC
prevents issues from being dealt with in a timely manner when
integrators, or even those writing the code, are able to resolve those issues relatively easily.

The processes described here ensure the bulk of
testing and the majority of responsibility for capturing regressions
does not lie with the Wednesday testing phase. Testing is performed as early in the SDLC as possible.
==New Unit Tests==
Creating unit tests prior to coding starts
the whole SDLC off with a focus on quality. If code isn't
considered complete until unit tests pass, then the code itself will
be of a high quality by the time it is implemented. This also creates a
repository of unit regression tests in an iterative manner while not
requiring of a great deal of extra effort to write those tests. As
these tests are added, they will provide early regression testing of
new code.

Unit tests in the PHPUnit framework are a requirement when submitting code to Moodle.

==Integration Steps==
===Step 1: GIT Pull to integration CI server===
When developers are ready to submit their code into the integration process, they set the status of the tracker issue to 'Ready for integration'. This in turn triggers the Jenkins CI job "Pre-check remote branch". This job performs the following tasks:
* Pull the changes to a local branch on the Integration CI Server.
** git reset --hard origin => so it is in the same status than the official git repository
** git pull => get the changes proposed by a developer
* Run checks on the code.
* Generate a results artefact that is sent to the developer.

Please note; each job begins by resetting the local git repository to the original state so the pulls are never permanent. This mitigates the risk posed by "collisions" as the purpose of this step is not to test for "collisions" between conflicting patches but to verify that code meets the required standard to proceed to peer review.
===Step 2: Peer Review===
Any code sent back to the developer at Peer Review, after any changes have been made, must be must pass step 1 again before re-review.
===Step 3: GIT Pull to integration===
Code passing peer review can be pulled from the developers repository to integration where, among other things, the main Jenkins integration jobs run. At this stage integrators also check for "collisions" and conflicts between submitted patches.
===Step 4: GIT Pull to Nightly Build Machine, performance and browser driven automation suite===
A set of regression tests are run at 2AM AWST Daily. These tests are scheduled to run automatically from the Moodle test automation framework and run on all supported platforms e.g. all databases MySQL, Oracle, MSSQL, PostgreSQL. The following tests are performed in this order:
# Moodle unit tests, including those submitted by developers.
# PerfComp performance comparison tests.
# Selenium acceptance tests, automated browser tests based upon the existing Moodle QA test suite.
===Step 5: Issue retest===
On Wednesday (AWST) the manual issue retesting is performed at Moodle HQ. Issue test steps must be clear and concise, allowing the issue to be recreated easily. Testing steps must go in the testing instructions field to prove that the bug is resolved. Areas of Moodle to investigate with extra exploratory testing should be entered here also.

==Stable Sprints and Point Releases==
Development work performed during stable sprints and point releases are subject to the stringent integration review and testing process described above.

==Major Releases and New Development==
The Moodle QA cycle is a user acceptance testing phase at the end of every 6 month development cycle and is a chance for the community to become directly involved in road testing the latest testing release.

In reality testing starts much earlier in the development process. All code submitted to Moodle has to pass through the integration process and normally does so in an incremental manner. All testing to be undertaken for a Moodle release is tailored to what will be in that release and defined in the [https://docs.moodle.org/dev/2.4_Test_Plan current release test plan test plan ].

2012-08-29T08:22:39Z

Esqueeze:

{{Work in progress}}
=Introduction=
==Purpose==
The purpose of this document is to define a high level QA strategy
at Moodle and communicate that process to the relevant stakeholders.
This document is loosely based upon the IEEE829 standard for software
testing documentation; adaptations have been made to this standard to
make the document more lightweight, suitable for use in an agile
process and relevant to the organization.

This document is test centric, only explaining the development
process from a testing point of view. The scope of this document does
not cover the details of the entire development process at Moodle but
how testing is applied to it. This document also discusses potential
benefits to the organization of the processes discussed.
Currently the processes that are documented here are under development. Progress can be viewed via the [https://docs.moodle.org/dev/Testing_strategy/Implementation_Plan Implementation Plan]
==Scope==
<P STYLE="margin-bottom: 0cm">This document encompasses all testing
activities at Moodle and how they will be performed. This includes:</P>
<UL>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Testing_Methodology_At_Moodle Testing Methodology at Moodle]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/The_Moodle_Testing_Process The Moodle Testing Process]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Managing_Legacy_Items Managing legacy items]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Hardware_Requirements Hardware requirements]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Testing_Tools_and_Their_Use Testing Tools and Their Use]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Test_Environments Test Environments]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Implementation_Plan Implementation Plan]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Training Training]</P>
<LI><P STYLE="margin-bottom: 0cm">[https://docs.moodle.org/dev/Testing_strategy/Useful_Resources Useful Resources]</P>
</UL>

Testing strategy/The Moodle Testing Process

2012-08-29T01:59:16Z

Esqueeze:

{{Work in progress}}
<H1 CLASS="western">Methodologies in the Moodle Development Process</H1>
<P>Moodle has an established process for developing and integrating software. This section describes relevant testing methodology within the existing process.</P>
<P>The full [https://docs.moodle.org/dev/Process development]
life-cycle at Moodle is documented in detail [https://docs.moodle.org/dev/Releases#General_release_calendar elsewhere].
Moodle is currently developed using Agile methodologies including
Scrum. The following stages of development take place within this
life-cycle:</P>
<UL>
<LI><P>Weekly integration cycle</P>
<LI><P>3-weekly stable sprints</P>
<LI><P>6-monthly major releases</P>
<LI><P>3 minor releases between every major release</P>
</UL>
<H2 CLASS="western">Weekly (Continuous) Integration
Cycle</H2>
<P>A manual testing phase of integration issues occurs towards
the end of the SDLC at Moodle during the Wednesday (AWST) testing
phase. This is the last possible chance Moodle gets to capture
regressions. Relying soley on this phase to capture regressions could result in the late discovery of regressions
during unguided exploratory testing and worse still the introduction
of regressions into Master. Discovery of issues late in the SDLC
prevents issues from being dealt with in a timely manner when
integrators, or even those writing the code, are able to resolve those issues relatively easily.
</P>
<P>The processes described here ensure the bulk of
testing and the majority of responsibility for capturing regressions
does not lie with the Wednesday testing phase. Testing is performed as early in the SDLC as possible.</P>
<H2 CLASS="western">New Unit Tests</H2>
<P>Creating unit tests prior to coding starts
the whole SDLC off with a focus on quality. If code isn't
considered complete until unit tests pass, then the code itself will
be of a high quality by the time it is implemented. This also creates a
repository of unit regression tests in an iterative manner while not
requiring of a great deal of extra effort to write those tests. As
these tests are added, they will provide early regression testing of
new code.</P>
<P>Unit tests in the PHPUnit framework are a requirement when submitting code to Moodle.</P>

<H2 CLASS="western">Integration Steps</H2>
<H3 CLASS="western">Step 1: GIT Pull to integration CI server</H3>
When developers are ready to submit their code into the integration process, they set the status of the tracker issue to 'Ready for integration'. This in turn triggers the Jenkins CI job "Pre-check remote branch". This job performs the following tasks:
* Pull the changes to a local branch on the Integration CI Server.
** git reset --hard origin => so it is in the same status than the official git repository
** git pull => get the changes proposed by a developer
* Run checks on the code.
* Generate a results artefact that is sent to the developer.

Please note; each job begins by resetting the local git repository to the original state so the pulls are never permanent. This mitigates the risk posed by "collisions" as the purpose of this step is not to test for "collisions" between conflicting patches but to verify that code meets the required standard to proceed to peer review.
<H3 CLASS="western">Step 2: Peer Review</H3>
<P>Any code sent back to the developer at Peer Review, after any changes have been made, must be must pass step 1 again before re-review.</P>
<H3 CLASS="western">Step 3: GIT Pull to integration</H3>
<P>Code passing peer review can be pulled from the developers repository to integration where, among other things, the main Jenkins integration jobs run. At this stage integrators also check for "collisions" and conflicts between submitted patches.</P>
<H3 CLASS="western">Step 4: GIT Pull to Nightly Build Machine, performance and browser driven automation suite</H3>
A set of regression tests are run at 2AM AWST Daily. These tests are scheduled to run automatically from the Moodle test automation framework and run on all supported platforms e.g. all databases MySQL, Oracle, MSSQL, PostgreSQL. The following tests are performed in this order:
# Moodle unit tests, including those submitted by developers.
# PerfComp performance comparison tests.
# Selenium acceptance tests, automated browser tests based upon the existing Moodle QA test suite.
<H3 CLASS="western">Step 5: Issue retest</H3>
<P>On Wednesday (AWST) the manual issue retesting is performed at Moodle HQ. Issue test steps must be clear and concise, allowing the issue to be recreated easily. Testing steps must go in the testing instructions field to prove that the bug is resolved. Areas of Moodle to investigate with extra exploratory testing should be entered here also.</P>
<P>The format for writing test steps is:</P>
<P>'''Pre-requisites:'''</P>
<P>Everything required to allow the test to start. This can include, but is not limited to: issue specific data, test platform information: e.g. OS.</P>
<P>'''Test Steps:'''</P>
<P>A clear and concise list of test steps required to test the issue with the expected results of each step.</P>
<P>'''Actual Result:'''</P>
<P>The actual result of what happened when the test is complete.</P>
<P>'''Exploratory focus:'''</P>
<P>Areas of Moodle that may need to be investigated further to identify regressions etc.</P>
<P>An issue passes when the actual result of test no longer occurs and no regressions have been created.</P>
<H2 CLASS="western">Stable Sprints</H2>
'''TODO'''
<H2 CLASS="western">Major Releases and New
Development</H2>
'''TODO'''
<H2 CLASS="western">Point Releases</H2>
'''TODO'''