Acceptance testing: Difference between revisions
No edit summary |
No edit summary |
||
Line 103: | Line 103: | ||
* php admin/tool/behat/cli/util.php --buildconfigfile # Updates your config file with all Moodle components .feature files and steps definitions | * php admin/tool/behat/cli/util.php --buildconfigfile # Updates your config file with all Moodle components .feature files and steps definitions | ||
* php admin/tool/behat/cli/util.php --runtests # To run the whole list of tests, you can execute only a set of tests with --tags="mod_forum" option or add extra behat commands with --extra="--format junit --out /path/to/junit/reports" | * php admin/tool/behat/cli/util.php --runtests # To run the whole list of tests, you can execute only a set of tests with --tags="mod_forum" option or add extra behat commands with --extra="--format junit --out /path/to/junit/reports" | ||
== Writting features == | |||
* Select the most appropriate Moodle component to include your test and create a COMPONENTNAME/tests/behat/an_example.feature | |||
* Write your scenarios checking the list of available steps to include (Site Administration -> Development -> Acceptance testing) and maintaining a correct indentation | |||
* Run the tests, you can specify a tag in both the .feature file and the test runner to limit the executed tests. | |||
== Links == | == Links == | ||
Technical info: https://docs.moodle.org/dev/Behat_integration | Technical info: https://docs.moodle.org/dev/Behat_integration |
Revision as of 00:12, 18 October 2012
Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.
Introduction
This page describes how we describe Moodle functionalities and how we automatically test all of this Moodle's functionalities.
Behat is a behavioural driven development (BDD) tool written in PHP, it can parse a human-readable list of sentences (called steps) and execute actions in a browser using Selenium or other tools to simulate users interactions.
(This is not in standard Moodle yet: to play a bit with it now clone https://github.com/moodlehq/moodle-behat-features and follow the installation and usage instructions.)
How it works
Behat parses and executes features files which describes Moodle features (for example Post in a forum), each feature file is composed by many scenarios (for example Add a post to a discussion or Create a new discussion), and finally each scenario is composed by steps (for example I press "Post to Forum" or I should see "My post title"). When the feature file is executed, every step internally is translated into an PHP method and is executed.
This features are executed nightly in the HQ servers with all the supported databases (MySQL, PostgreSQL, MSSQL and Oracle) and with different browsers (Firefox, Internet Explorer, Safari and Chrome) to avoid regressions and test new functionalities.
Examples
- There is a closed list of steps to use in the features, a feature written with the basic (or low-level) steps looks like this:
@auth Feature: Login In order to login As a moodle user I need to be able to validate the username and password against moodle Scenario: Login as an existing user Given I am on "login/index.php" When I fill in "username" with "admin" And I fill in "password" with "moodle" And I press "loginbtn" Then I should see "Moodle 101: Course Name" Scenario: Login as an unexisting user Given I am on "login/index.php" When I fill in "username" with "admin" And I fill in "password" with "moodle" And I press "loginbtn" Then I should see "Moodle 101: Course Name"
Note that The 3 sentences below Feature: Login are only information about what we want to test.
These are simple scenarios, but most of Moodle functionalities would require a huge list of this steps to test a scenario, imagine a Add a post to a discussion scenario; you need to login, create a course, create a user and enrol it in the course... Most of this steps is not what we intend to test in a Post in a forum feature, Moodle provides extra steps to quickly set up the context required to test a Moodle feature, for example:
@mod_forum Feature: Post in a forum In order to add contents to a forum As a moodle user I need to be able to enter the contents and verify they are correctly shown Background: Given the following "courses" exists | fullname | shortname | | Course 1 | C1 | And the following "users" exists | username | firstname | lastname | email | | student1 | First | Student | stu1@asd.com | And the following "enrolments" exists | username | courseshortname | enrolmenttype | | student1 | C1 | manual | Scenario: Create a forum Given I loggin as an "admin" And I go to "Course 1" course And I add a "forum" to section "1" And I fill the form with | Forum name | ForumTest1 | | Forum description | Test description | | Forum type | Standard forum for general use | | Any other field | Value | When I press "Save and display" Then I should see "Test1" And I should see "Add a new discussion topic" Scenario: Add a post to a discussion Given I run "Create a forum" scenario And I go to "Course 1" course And I follow "ForumTest1" When I press "Add a new discussion topic" And I fill the form with | Subject | test subject | | Message | message body | And I press "Post to forum" Then I should see "test subject"
Note that:
- Background is executed before each scenario execution
- Each scenario is executed in an isolated testing environment, so what you set up in an scenario (like the ForumTest1 forum in the example above) is cleaned up after the scenario execution
- The prefixes "Given", "When" and "Then" are only informative and they as used to define the context (Given), specify the action (When) and check the results (Then).
Requirements
- Selenium server (formerly the Selenium RC Server) You can download it from http://seleniumhq.org/download/
- PHP 5.3.3
- PHPUnit
- There are other packages and PHP settings requirements, but they will be checked by the behat-moodle composer installer
Installation
- Set up your Moodle installation and initialize your PHPUnit environment https://docs.moodle.org/23/en/Installing_Moodle and https://docs.moodle.org/dev/PHPUnit
- Clone the behat-moodle project https://github.com/dmonllao/behat-moodle and follow the README https://github.com/dmonllao/behat-moodle/blob/master/README.md
- If you have not done it, in behat-moodle/ set the Moodle site URL and the path to the config.yml file (/PATH/TO/YOUR/MOODLEDATA/behat/config.yml) in your /behat.yml file
- In Moodle, set the 'behatpath' setting pointing to your behat-moodle/ root (setting under Site Administration -> Server -> System paths)
Usage
- Run your Selenium server with "java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar"
- Open another CLI and "cd /to/your/moodle/dirroot/"
- php admin/tool/behat/cli/util.php --buildconfigfile # Updates your config file with all Moodle components .feature files and steps definitions
- php admin/tool/behat/cli/util.php --runtests # To run the whole list of tests, you can execute only a set of tests with --tags="mod_forum" option or add extra behat commands with --extra="--format junit --out /path/to/junit/reports"
Writting features
- Select the most appropriate Moodle component to include your test and create a COMPONENTNAME/tests/behat/an_example.feature
- Write your scenarios checking the list of available steps to include (Site Administration -> Development -> Acceptance testing) and maintaining a correct indentation
- Run the tests, you can specify a tag in both the .feature file and the test runner to limit the executed tests.
Links
Technical info: https://docs.moodle.org/dev/Behat_integration