Note:

This site is no longer used and is in read-only mode. Instead please go to our new Moodle Developer Resource site.

Testing strategy/Testing Tools and Their Use: Difference between revisions

From MoodleDocs
VisualWikitext
Tim Barker (talk | contribs)
361 edits
Created page with "<H1 CLASS="western">Testing Tools and Their Use</H1> <P>All of the tools specified below are free with most being open source and extendible. The advantages are:</P> <UL> <LI><P..."
 
This page will not be migrated to new devdocs
 
(9 intermediate revisions by one other user not shown)
Line 1: Line 1:
<H1 CLASS="western">Testing Tools and Their Use</H1>
{{Template:WillNotMigrate}}
<P>All of the tools specified below are free with most
{{Work in progress}}
being open source and extendible. The advantages are:</P>
=Testing Tools and Their Use=
<UL>
All of the tools specified below are free with most being open source and extendible. The advantages are:
<LI><P>There is no cost to Moodle for licences.</P>
* There is no cost to Moodle for licences.
<LI><P>Applications are web based with a tiny footprint
* Applications are web based with a tiny footprint and will run on low spec hardware/VMs.
and will run on low spec hardware/VMs.</P>
* Applications with databases all run on MySQL,which is free.
<LI><P>Applications with databases all run on MySQL,
* Moodle can extend the tools/create plugins, in-house, if customisations are required.
which is free.</P>
 
<LI><P>Moodle can extend the tools/create plugins,
==Continuous Integration==
in-house, if customisations are required.</P>
Continuous integration is the process of running tests frequently when code is checked into a repository. The aim of this is to keep code clean and allow the early capture of
</UL>
broken/incomplete code, early warning of conflicting changes (regressions) and a constantly available current “build” for demo
<H2 CLASS="western">Continuous Integration</H2>
and testing.
<P>Continuous integration is the process of running
Moodle's continuous integration process uses a second "Test" integration server where code must be submitted to prior to being reviewed by integrators.
tests frequently when code is checked into a repository. The aim of
===Jenkins===
this is to keep code clean and allow the early capture of
Jenkins is a fully extendible application that monitors executions of repeated jobs. Results are displayed via a web application. Jenkins is an information hub to provide Moodle with continuous feedback about the quality of Moodle software.
broken/incomplete code, early warning of conflicting changes
Jenkins can monitor any/multiple GIT repository for changes without adding hooks to the code and be used to run a set of
(regressions) and a constantly available current “build” for demo
tasks such as:
and testing.</P>
* Pull git
<H3 CLASS="western">Jenkins</H3>
* Run unit tests
<P>Jenkins is a fully extendible application that
* Create a moodle site
monitors executions of repeated jobs. Results are displayed via a web
* Add some information to it
application.</P>
* Run the selenium tests
<P>Jenkins will be used as an information hub to provide
* Report errors/warnings/failures
Moodle with continuous feedback about the quality of Moodle software.</P>
===Performance Testing Matrix===
<P>Jenkins can monitor any/multiple GIT repository for
Measuring system performance during this testing will assist with preventing degradation of performance in existing code caused by regressions.
changes without adding hooks to the code and be used to run a set of
To assist with early capture of performance degradation due to regression; a performance matrix records performance against automated user interactions generated using JMeter.
tasks such as:</P>
===PHPUnit===
<UL>
Unit testing tool for PHP; implementation is already
<LI><P>Pull git</P>
planned.
<LI><P>Run unit tests</P>
===SimpleTest===
<LI><P>Create a moodle site</P>
Moodle's legacy unit testing tool.
<LI><P>Add some information to it</P>
 
<LI><P>Run the selenium tests</P>
==Other Automated Testing==
<LI><P>Report errors/warnings/failures</P>
Methods of testing other than unit testing is automated. Selenium is used for functional testing in conjunction with the Behat behavioural driven development tool.
</UL>
===Selenium===
<H3 CLASS="western">Phing
Selenium is an extremely powerful, free and fully extendible browser automation tool. Selenium Webdriver – Selenium Server is a language independent server application that runs Selenium test cases. Selenium WebDriver is a collection of language specific bindings to drive a browser. There are unofficial language bindings for PHP and WebDriver. Selenium test cases can be written in an object oriented manner and re-usable methods can be created. Selenium is highly resistant to CSS changes and supports using XPath to locate page elements.
</H3>
Furthermore Selenium server can be run in a grid configuration so that tests can
<P>“PHing Is Not GNU make” is a build tool for PHP based upon Apache Ant. Phing commands
are specified in XML just like in Ant. Phing can run PHPUnit and
SimpleTest unit tests, execute shell commands, perform actions with
GIT and much, much more.</P>
<P STYLE="font-weight: normal">As part of continuous
integration Phing will be the engine behind Jenkins, performing tasks
like getting the latest code for a test environment for the GIT
repository and running unit tests.</P>
<H3 CLASS="western">Phing Jenkins Plugin</H3>
<P>This plugin integrates Phing with Jenkins.</P>
<H3 CLASS="western">Clover PHP Plugin</H3>
<P>To capture code coverage reports from PHPUnit.</P>
<H3 CLASS="western">PHPUnit</H3>
<P>Unit testing tool for PHP; implementation is already
planned.</P>
<H3 CLASS="western">SimpleTest</H3>
<P>Moodle's legacy unit testing tool.</P>
<H2 CLASS="western">Other Automated Testing</H2>
<P>Methods of testing other than unit testing will be
automated. This includes, but is not limited to functional testing.</P>
<H3 CLASS="western">Selenium</H3>
<P>Selenium is an extremely powerful, free and fully
extendible browser automation tool that contains the following
features:</P>
<UL>
<LI><P>Selenium IDE – Firefox plugin app for
recording browser interactions with any web application. This is
ideal for creating quick bug recreation test scripts and automation
aided exploratory testing.</P>
<LI><P>Selenium Webdriver – Selenium Server is a
server application that runs Selenium test cases. Selenium webdriver
is a collection of language specific bindings to drive a browser.
</P>
</UL>
<P>Selenium provides extensions to existing unit test
tools such as JUnit. There are unofficial language bindings for
PHPUnit and webdriver and an official backwards compatibility to
version one of Selenium through PHPUnit.</P>
<P>Selenium test cases can be written in an object
oriented manner and re-usable methods can be created.</P>
<P>Selenium is highly
resistant to CSS changes and supports using XPath to locate page
elements.</P>
<P>Furthermore
Selenium server can be run in a grid configuration so that tests can
run in parallel across multiple test environments controlled by one
run in parallel across multiple test environments controlled by one
hub:</P>
hub:
<P>Fig.2 Selenium Grid</P>
Fig.2 Selenium Grid
<P>[[File:grid1.png]]</P>
[[File:grid1.png]]
<P>When a test is
When a test is
required to run on a specific platform Selenium Grid can be
required to run on a specific platform Selenium Grid can be
made aware of the environments you wish to use. The Selenium Hub will
made aware of the environments you wish to use. The Selenium Hub will
then ensure that a test runs only on the Selenium slaves providing
then ensure that a test runs only on the Selenium slaves providing
the requested environment:</P>
the requested environment:
<P>Fig.3 Requesting Grid Environments</P>
Fig.3 Requesting Grid Environments
<P>[[File:grid2.png]]
[[File:grid2.png]]
</P>
===Behat===
<H3 CLASS="western">Selenium Jenkins Plugin</H3>
Tests that will run in Selenium are written in Behat. Behat is a PHP port of Cucumber, the popular Behavioural Driven Development (BDD) tool. Gherkin is a ubiquitous, domain specific language (DSL) that is used to document system behaviours. This DSL can be understood in the context of the business domain and acts as a layer of abstraction to automated tests. A great way of documenting requirements.
<P>A Jenkins plugin that gives Jenkins the ability to
===PHPUnit Selenium Extension===
turn a Jenkins cluster into a Selenium Cluster. The plugin installs
Selenium/Behat test results are output in PHPUnit xml format.
Selenium Grid on all of the slaves automatically and sets up the grid
 
on it's own. Great for deploying the latest code to multiple Selenium
==Hosting Test Environments==
test environments automatically.</P>
Hosting instances of Jenkins and test environments will be performed using server virtualization.
<H3 CLASS="western">Firebug</H3>
===XEN===
<P>Firebug is Firefox plugin that gives a user the
Xen is a robust, secure, high performance type 1 baremetal hypervisor, or a virtual machine monitor (VMM). Xen can securely
ability to easily inspect page source.</P>
<H3 CLASS="western">Firepath</H3>
<P>Firepath is a Firefox plugin that can generate XPath
expressions.</P>
<H3 CLASS="western">Eclipse IDE</H3>
<P>Eclipse IDE will be used to design Selenium test
cases.</P>
<H3 CLASS="western">JUnit/PHPUnit Selenium
Extension</H3>
<P>SeleniumHQ officially support JUnit extensions for
Selenium. The JUnit support is more mature than PHPUnit and is fully
updated to work with the latest version of Selenium Server.</P>
<H2 CLASS="western">Hosting Test Environments</H2>
<P>Hosting instances of Jenkins and test environments
will be performed using server virtualization.</P>
<H3 CLASS="western">XEN</H3>
<P>Xen is a robust, secure, high performance type 1 baremetal
hypervisor, or a virtual machine monitor (VMM). Xen can securely
execute multiple virtual machines, each running its own OS, on a
execute multiple virtual machines, each running its own OS, on a
single physical system with close-to-native performance. Xen is open
single physical system with close-to-native performance. Xen is open
source, and it's used as the core virtualization engine for many
source, and it's used as the core virtualization engine for many
vendors products.</P>
vendors products.
<H2 CLASS="western">Version Control</H2>
==Version Control==
<H3 CLASS="western">GIT</H3>
===GIT===
<P>Moodle use GIT for version control.</P>
Moodle use GIT for version control.
<H2 CLASS="western">Test Management and Defect
==Test Management and Defect Tracking==
Tracking</H2>
===Jira===
<H3 CLASS="western">TestLink</H3>
Moodle already use Jira for development and project
<P>TestLink is a free test management tool written in
planning and tracking issues.
PHP. The test management tool is used to specify and manage tests and
===Zephyr===
test suites, then report on test execution. A test management tool is
The Zephyr test management plugin for Jira is used to manage test documentation and execution.
useful for managing and co-ordinating organisational testing effort
and documenting testing. Currently Moodle use Jira as the test
management tool. Jira is software development management tool and not
a test management tool.</P>
<P>In TestLink we can create testing projects, test
plans and document test cases, manage test execution runs then log
and report results for manual and automated tests.</P>
<H3 CLASS="western">TestLink Jenkins Plugin</H3>
<P>This plugin integrates TestLink with Jenkins and
generates reports on automated test execution.</P>
<H3 CLASS="western">Jira</H3>
<P>Moodle already use Jira for development and project
planning and tracking issues.</P>

Latest revision as of 10:16, 27 January 2025


Warning: This page is no longer in use. The information contained on the page should NOT be seen as relevant or reliable.


Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Testing Tools and Their Use

All of the tools specified below are free with most being open source and extendible. The advantages are:

  • There is no cost to Moodle for licences.
  • Applications are web based with a tiny footprint and will run on low spec hardware/VMs.
  • Applications with databases all run on MySQL,which is free.
  • Moodle can extend the tools/create plugins, in-house, if customisations are required.

Continuous Integration

Continuous integration is the process of running tests frequently when code is checked into a repository. The aim of this is to keep code clean and allow the early capture of broken/incomplete code, early warning of conflicting changes (regressions) and a constantly available current “build” for demo and testing. Moodle's continuous integration process uses a second "Test" integration server where code must be submitted to prior to being reviewed by integrators.

Jenkins

Jenkins is a fully extendible application that monitors executions of repeated jobs. Results are displayed via a web application. Jenkins is an information hub to provide Moodle with continuous feedback about the quality of Moodle software. Jenkins can monitor any/multiple GIT repository for changes without adding hooks to the code and be used to run a set of tasks such as:

  • Pull git
  • Run unit tests
  • Create a moodle site
  • Add some information to it
  • Run the selenium tests
  • Report errors/warnings/failures

Performance Testing Matrix

Measuring system performance during this testing will assist with preventing degradation of performance in existing code caused by regressions. To assist with early capture of performance degradation due to regression; a performance matrix records performance against automated user interactions generated using JMeter.

PHPUnit

Unit testing tool for PHP; implementation is already planned.

SimpleTest

Moodle's legacy unit testing tool.

Other Automated Testing

Methods of testing other than unit testing is automated. Selenium is used for functional testing in conjunction with the Behat behavioural driven development tool.

Selenium

Selenium is an extremely powerful, free and fully extendible browser automation tool. Selenium Webdriver – Selenium Server is a language independent server application that runs Selenium test cases. Selenium WebDriver is a collection of language specific bindings to drive a browser. There are unofficial language bindings for PHP and WebDriver. Selenium test cases can be written in an object oriented manner and re-usable methods can be created. Selenium is highly resistant to CSS changes and supports using XPath to locate page elements. Furthermore Selenium server can be run in a grid configuration so that tests can run in parallel across multiple test environments controlled by one hub: Fig.2 Selenium Grid When a test is required to run on a specific platform Selenium Grid can be made aware of the environments you wish to use. The Selenium Hub will then ensure that a test runs only on the Selenium slaves providing the requested environment: Fig.3 Requesting Grid Environments

Behat

Tests that will run in Selenium are written in Behat. Behat is a PHP port of Cucumber, the popular Behavioural Driven Development (BDD) tool. Gherkin is a ubiquitous, domain specific language (DSL) that is used to document system behaviours. This DSL can be understood in the context of the business domain and acts as a layer of abstraction to automated tests. A great way of documenting requirements.

PHPUnit Selenium Extension

Selenium/Behat test results are output in PHPUnit xml format.

Hosting Test Environments

Hosting instances of Jenkins and test environments will be performed using server virtualization.

XEN

Xen is a robust, secure, high performance type 1 baremetal hypervisor, or a virtual machine monitor (VMM). Xen can securely execute multiple virtual machines, each running its own OS, on a single physical system with close-to-native performance. Xen is open source, and it's used as the core virtualization engine for many vendors products.

Version Control

GIT

Moodle use GIT for version control.

Test Management and Defect Tracking

Jira

Moodle already use Jira for development and project planning and tracking issues.

Zephyr

The Zephyr test management plugin for Jira is used to manage test documentation and execution.

Retrieved from "https://docs.moodle.org/dev/index.php?title=Testing_strategy/Testing_Tools_and_Their_Use&oldid=64318"
Powered by MediaWiki