Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Testing strategy/Test Environments: Difference between revisions

From MoodleDocs
(Created page with " <H1 CLASS="western">Test Environments</H1> <P>Test environments for automation will be hosted on the VM server. These will include a CI server (the build machine), database serv...")
 
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Work in progress}}
== Test Automation Environments ==
Test environments for automation are hosted on
the Xen VM server. These will include a CI server (the build machine), The nightly build machine
database servers and web servers.


<H1 CLASS="western">Test Environments</H1>
== Requirements Gathering ==
<P>Test environments for automation will be hosted on
As a task while provisioning the VM Server, two
the VM server. These will include a CI server (the build machine),
database servers, web servers and client PCs.
</P>
<H2 CLASS="western">Requirements Gathering</H2>
<P>As a task while provisioning the VM Server, two
surveys have been run to determine popular Test Platforms. Survey 1
surveys have been run to determine popular Test Platforms. Survey 1
gathered metrics from the Moodle Community and Survey 2 gathered
gathered metrics from the Moodle Community and Survey 2 gathered
metrics from Moodle Partners. Limiting factors of these surveys are
metrics from Moodle Partners. Limiting factors of these surveys are
that both samples are relatively small. The results of the surveys
that both samples are relatively small. The results of the surveys
are as follows:</P>
are as follows:
<H3 CLASS="western">Survey
=== Survey 1 The Moodle Community:===
1 The Moodle Community:</H3>
Fig. 1 Separate Database and Web Server Yes/No:
<P>Fig. 4 Separate Database and Web Server Yes/No:</P>
 
<P>[[File:Fig._2_user_split_system.png]]</P>
[[File:Fig._2_user_split_system.png]]
<P>Fig. 5 Operating System of Separate Web Server:</P>
 
<P>[[File:fig.3_user_os_separatewebserver.gif]]
Fig. 2 Operating System of Separate Web Server:
</P>
 
<P>Fig. 6 Operating
[[File:fig.3_user_os_separatewebserver.gif]]
System of Separate Database Server:</P>
 
<P>[[File:fig.6_user_OS_SeparateDB.gif]]</P>
Fig. 3 Operating System of Separate Database Server:
<P>Fig. 7 What Databases are Used on a Split System:</P>
 
<P>[[File:Fig7_user_dbs_used_split.png]]</P>
[[File:fig.6_user_OS_SeparateDB.gif]]
<P>Fig. 8 What Databases are Used on an all in one
 
Database Server and Webserver:</P>
Fig. 4 What Databases are Used on a Split System:
<P>[[File:fig8_user_database_combined.png]]</P>
 
<P>Fig. 9 What Web
[[File:Fig7_user_dbs_used_split.png]]
Server is Used on a Split System:</P>
 
<P>[[File:user_webserver_split.png]]</P>
Fig. 5 What Databases are Used on an all in one Database Server and Webserver:
<P>Fig. 10 What Web Server is Used on an all in one
 
Database Server and Webserver:</P>
[[File:fig8_user_database_combined.png]]
<P>[[File:user_webserver_combined.png]]</P>
 
<P>As Debian and Ubuntu are similar and Red Hat and
Fig. 6 What Web Server is Used on a Split System:
Centos are similar that,based upon our sample data, our three main
 
operating used by the Moodle community Debian/Ubuntu, Red Hat/Centos
[[File:user_webserver_split.png]]
and MS Server 2008 with a weighting of almost 75% towards the two
 
Linux operating systems.</P>
Fig. 7 What Web Server is Used on an all in one Database Server and Webserver:
<P>Of the five supported databases, the majority of
 
community users prefer MySQL with PostgreSQL and MSSQL a fairly equal
[[File:user_webserver_combined.png]]
but distant second and third.</P>
 
<P>The Majority of community users also use Apache with
As Debian and Ubuntu are similar and Red Hat and Centos are similar that,based upon our sample data, our three main operating used by the Moodle community Debian/Ubuntu, Red Hat/Centos and MS Server 2008 with a weighting of almost 75% towards the two
IIS a very distant second.</P>
Linux operating systems.
<H3 CLASS="western">7.1.1Survey
 
2 The Moodle Partners:</H3>
Of the five supported databases, the majority of community users prefer MySQL with PostgreSQL and MSSQL a fairly equal
<P>Fig. 11 Separate Database and Web Server Yes/No:</P>
but distant second and third.
<P>[[File:partner_split_combine.png]]
 
</P>
The Majority of community users also use Apache with IIS a very distant second.
<P>Fig. 12 Partner OS Database Server (combining answers
 
for fig 11):</P>
=== Survey 2 The Moodle Partners: ===
<P>[[File:partner_OS_DB.gif]]
 
</P>
Fig. 8 Separate Database and Web Server Yes/No:
<P>Fig. 13 Partner OS
 
Web Server (combining answers for fig 8):</P>
[[File:partner_split_combine.png]]
<P>[[File:partner_OS_Web.gif]]
 
</P>
Fig. 9 Partner OS Database Server:
<P>Fig. 14 What Databases are Used on a Split System:</P>
 
<P>[[File:partner_db_split.png]]
[[File:partner_OS_DB.gif]]
</P>
 
<P>Fig. 15 What Databases are Used on a Combined System:</P>
Fig. 10 Partner OS Web Server (combining answers for fig 8):
<P>[[File:partner_db_combine.png]]
 
</P>
[[File:partner_OS_Web.gif]]
<P>Fig. 16 What Web
 
Servers are used on a Split System:</P>
Fig. 11 What Databases are Used on a Split System:
<P>[[File:partner_webserver_split.png]]</P>
 
<P>Fig. 17 What Web Servers are used on a Combined
[[File:partner_db_split.png]]
System:</P>
 
<P>[[File:partner_webserver_combined.png]]
Fig. 12 What Databases are Used on a Combined System:
</P>
 
<P>This suggests that the majority of partners use
[[File:partner_db_combine.png]]
Debian/Ubuntu or Centos/Rad Hat Linux, Apache and either a MySQL or
 
Fig. 13 What Web Servers are used on a Split System:
 
[[File:partner_webserver_split.png]]
 
Fig. 14 What Web Servers are used on a Combined System:
 
[[File:partner_webserver_combined.png]]
 
This suggests that the majority of partners use Debian/Ubuntu or Centos/Rad Hat Linux, Apache and either a MySQL or
PostgreSQL database.  
PostgreSQL database.  
</P>
 
<H2 CLASS="western">VM Requirements</H2>
== VM Requirements ==
<P>In true agile fashion and to promote flexibility,
VM's for test environments are modular i.e. they are separate database servers and web servers. This allows the mixing and
VM's for test environments will be modular i.e. they will be separate
matching of databases and their host OS to the web servers and their host OS as requirements change. At any time, any Web Server can
database servers and web servers. This will allow the mixing and
connect to any Database Server. It should also reduce potential numbers of running VMs required for a particular platform.
matching of databases and their host OS to the web servers and their
A master hub VM is required for Jenkins and the hub for the Selenium grid.
host OS as requirements change. At any time, any Web Server can
 
connect to any Database Server. It should also reduce potential
== CI Server ==
numbers of running VMs required for a particular platform.</P>
There are two CI Servers.
<P>A master hub VM is required for Jenkins and the hub for the Selenium
===Integration CI===
grid.</P>
This is used to run integration jobs, checks and tests when code is checked in. The unit tests run on this machine on a MySQL database.
<H1 CLASS="western" STYLE="page-break-before: always">Implementation</H1>
===Nightly Integration Build===
<P>Agile testing guru and automation expert Brian Marick
This is used to run PHPUnit tests on Postgres, performance tests against the Moodle supported databases and Browser driven Selenium tests.
coined the term “Hump of Pain” to describe the initial phase of
 
implementing test automation. The impact of this hump can be
Fig. 15 Selenium Test VM Architecture:
minimised by good planning, good coaching and a strong managerial
 
support for the new processes. As time progresses, particularly as
[[File:VM Architecture Selenium.gif]]
infrastructure becomes implemented, the test cases in the automated
suite are completed and the introduced practices become second
nature; the hump disappears.</P>
<H2 CLASS="western">Agile Implementation of Testing
Practices</H2>
<P>Modifications to the process should be made in an incremental
manner based up scrum methodology. A series of 2 weekly sprints will
be used to time-box test planning and implementation tasks.</P>
<H2 CLASS="western">Phase 1: Infrastructure and the
Integration Process</H2>
<P>The infrastructure to maintain automated testing
effort must be implemented. This process will start with the
installation of cloud based hardware at Moodle HQ and the setup of
VM's to support the required test environments.</P>
<P>The integration process affects quality for the whole
organisation as all code must go through the integration process to
become 'live'. The processes described above will will be initially
implemented at integration. Any changes to the existing processes
will take place side-by-side with existing processes to manage risk.
i.e. existing processes impacted by changes will be maintained in
their current state until implementation of changes becomes stable.</P>
<P>The setup of test environments will initially start
with a restricted number of high risk test environments so that the
automation process, particularly test cases, can be developed
incrementally.</P>

Latest revision as of 06:13, 14 September 2012

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Test Automation Environments

Test environments for automation are hosted on the Xen VM server. These will include a CI server (the build machine), The nightly build machine database servers and web servers.

Requirements Gathering

As a task while provisioning the VM Server, two surveys have been run to determine popular Test Platforms. Survey 1 gathered metrics from the Moodle Community and Survey 2 gathered metrics from Moodle Partners. Limiting factors of these surveys are that both samples are relatively small. The results of the surveys are as follows:

Survey 1 The Moodle Community:

Fig. 1 Separate Database and Web Server Yes/No:

Fig. 2 user split system.png

Fig. 2 Operating System of Separate Web Server:

fig.3 user os separatewebserver.gif

Fig. 3 Operating System of Separate Database Server:

fig.6 user OS SeparateDB.gif

Fig. 4 What Databases are Used on a Split System:

Fig7 user dbs used split.png

Fig. 5 What Databases are Used on an all in one Database Server and Webserver:

fig8 user database combined.png

Fig. 6 What Web Server is Used on a Split System:

user webserver split.png

Fig. 7 What Web Server is Used on an all in one Database Server and Webserver:

user webserver combined.png

As Debian and Ubuntu are similar and Red Hat and Centos are similar that,based upon our sample data, our three main operating used by the Moodle community Debian/Ubuntu, Red Hat/Centos and MS Server 2008 with a weighting of almost 75% towards the two Linux operating systems.

Of the five supported databases, the majority of community users prefer MySQL with PostgreSQL and MSSQL a fairly equal but distant second and third.

The Majority of community users also use Apache with IIS a very distant second.

Survey 2 The Moodle Partners:

Fig. 8 Separate Database and Web Server Yes/No:

partner split combine.png

Fig. 9 Partner OS Database Server:

partner OS DB.gif

Fig. 10 Partner OS Web Server (combining answers for fig 8):

partner OS Web.gif

Fig. 11 What Databases are Used on a Split System:

partner db split.png

Fig. 12 What Databases are Used on a Combined System:

partner db combine.png

Fig. 13 What Web Servers are used on a Split System:

partner webserver split.png

Fig. 14 What Web Servers are used on a Combined System:

partner webserver combined.png

This suggests that the majority of partners use Debian/Ubuntu or Centos/Rad Hat Linux, Apache and either a MySQL or PostgreSQL database.

VM Requirements

VM's for test environments are modular i.e. they are separate database servers and web servers. This allows the mixing and matching of databases and their host OS to the web servers and their host OS as requirements change. At any time, any Web Server can connect to any Database Server. It should also reduce potential numbers of running VMs required for a particular platform. A master hub VM is required for Jenkins and the hub for the Selenium grid.

CI Server

There are two CI Servers.

Integration CI

This is used to run integration jobs, checks and tests when code is checked in. The unit tests run on this machine on a MySQL database.

Nightly Integration Build

This is used to run PHPUnit tests on Postgres, performance tests against the Moodle supported databases and Browser driven Selenium tests.

Fig. 15 Selenium Test VM Architecture:

VM Architecture Selenium.gif

Retrieved from "https://docs.moodle.org/dev/index.php?title=Testing_strategy/Test_Environments&oldid=35544"
Powered by MediaWiki