MoodleDocs - User contributions [en]

PHPUnit

2017-05-24T11:24:22Z

Mackensen: /* LDAP */ clarify

{{Moodle 2.3}}

=What is PHPUnit=
PHPUnit by Sebastian Bergmann is an advanced unit testing framework for PHP. It is installed as Composer dependency and is not part of Moodle installation. To run PHPUnit tests, you have to manually install it on your development computer or test server.

Read the excellent guide at
* [http://phpunit.de/manual/current/en/index.html PHPUnit Manual]

=Installation of PHPUnit via Composer=

Windows may require additional installation steps.
On Windows go to https://getcomposer.org/download/ and download the Composer-Setup.exe file

* Execute Composer installer

cd /your/moodle/dirroot
php composer.phar install

(If that gives you connection problems try...)

php composer.phar install --prefer-source

Troubleshooting:
* On Windows if you are behind a proxy you will need to setup an environment variable called HTTP_PROXY with a value detailing your HTTP Proxy address and port before composer will correctly download files.
* You may be prompted for github credentials when installing composer dependencies.
** This is used to generate an personal access token to avoid being rate limited by github.
** If you have Two Factor Authentication enabled on your github account, or do not wish to supply your own credentials you will need to generate a token manually:
*** Visit https://github.com/settings/applications and request personal access token
*** Copy this token and add it to your [https://gist.github.com/andrewnicols/c5377ed25a9df1006ce1 ~/.composer/config.json]
** ( See [https://github.com/composer/composer/issues/2280 composer issue #2280] for further details of this bug.)

Detailed instructions:
* [http://getcomposer.org/doc/00-intro.md Composer documentation]

Detailed instructions:
* [[PHPUnit installation in Windows]]
* [[PHPUnit installation in OS X]]

== Uninstalling previous PEAR based version ==

Before using composer, this page used to suggest to install phpunit via PEAR. If you did so, you may wish to uninstall that package now. This should work:

$ pear uninstall phpunit/DbUnit
$ pear uninstall phpunit/PHPUnit

=Initialisation of test environment=

Our PHPUnit integration requires a dedicated database and dataroot. First, add a new dataroot directory and prefix into your config.php, you can find examples in config-dist.php (scroll down to 'Section 9').

$CFG->phpunit_prefix = 'phpu_';
$CFG->phpunit_dataroot = '/home/example/phpu_moodledata';

Some PHPUnit tests require a live internet connection. If your system does not have a direct connection to the Internet, you also need to specify your proxy in config.php - even though you normally specify it by using the admin settings user interface. (If you do not use a proxy, you can skip this step.) Check the settings on the relevant admin settings page, or from the mdl_config table in your database, if you are unsure of the correct values.

// Normal proxy settings
$CFG->proxyhost = 'wwwcache.example.org';
$CFG->proxyport = 80;
$CFG->proxytype = 'HTTP';
$CFG->proxybypass = 'localhost, 127.0.0.1, .example.org';
// Omit the next lines if your proxy doesn't need a username/password:
$CFG->proxyuser = 'systemusername';
$CFG->proxypassword = 'systempassword';

From Moodle 2.8.5 onwards, you can also provide specific database settings for unit testing (if these are not provided, the standard database settings will be used):

<pre>$CFG->phpunit_dbtype = 'pgsql'; // 'pgsql', 'mariadb', 'mysqli', 'mssql', 'sqlsrv' or 'oci'
$CFG->phpunit_dblibrary = 'native'; // 'native' only at the moment
$CFG->phpunit_dbhost = '127.0.0.1'; // eg 'localhost' or 'db.isp.com' or IP
$CFG->phpunit_dbname = 'mytestdb'; // database name, eg moodle
$CFG->phpunit_dbuser = 'postgres'; // your database username
$CFG->phpunit_dbpass = 'some_password'; // your database password</pre>

Then you need to initialise the test environment using following command.

cd /home/example/moodle
php admin/tool/phpunit/cli/init.php

This command has to be repeated after any upgrade, plugin (un)installation or if you have added tests to a plugin you are developing:

'''NOTE:''' make sure that your php cli executable (or the one you want to use) is correctly on your path as the individual init scripts will call it repeatedly. Also, ensure en_AU locale is installed on your server.

== LDAP ==
If you want to run LDAP unit tests you must have a working, configured LDAP environment on your test server. There must be a basic LDAP tree structure in place or tests will fail with "Search: No such object". Build an LDAP tree structure in a new shell prompt:

<pre>
$ ldapadd -H ldap://127.0.0.1 -D "cn=admin,dc=yourcomputer,dc=local" -W
dn:dc=yourcomputer,dc=local
objectClass:dcObject
objectClass:organizationalUnit
dc:yourcomputer
ou:YOURCOMPUTER
</pre>

In config.php tell Moodle where to look for your test LDAP environment:

<pre>
// Constants for auth/ldap tests
define('TEST_AUTH_LDAP_HOST_URL', 'ldap://127.0.0.1');
define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_AUTH_LDAP_BIND_PW', '*');
define('TEST_AUTH_LDAP_DOMAIN', 'dc=yourcomputer,dc=local');

// Constants for lib/ldap tests
define('TEST_LDAPLIB_HOST_URL', 'ldap://127.0.0.1');
define('TEST_LDAPLIB_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_LDAPLIB_BIND_PW', '*');
define('TEST_LDAPLIB_DOMAIN', 'dc=yourcomputer,dc=local');
</pre>

=Test execution=

To execute all test suites from main configuration file execute the <code>vendor/bin/phpunit</code> script from your <code>$CFG->dirroot</code> directory.

cd /home/example/moodle
vendor/bin/phpunit

The rest of examples uses <code>phpunit</code>, please substitute it with <code>vendor/bin/phpunit</code> or create a shortcut in your dirroot.

In IDEs, you may need to specify the path to the PHPUnit configuration file. Use the absolute path to <code>phpunit.xml</code> from your <code>$CFG->dirroot</code>.

There is an alternative script for running of tests via web interface: <code>admin/tool/phpunit/webrunner.php</code>. Use this as the last resort only when you cannot use the command-line interface on your test server. It will most probably break due to permissions problems if you try to execute it both from command-line and from webrunner. This feature is not officially supported.

===How to run only some tests===

==== Running a single test quickly ====

The fastest way to run a single test is:

vendor/bin/phpunit my_test_class_name my/tests/filename.php

This is faster than other methods because it avoids the need to search for the test file, but you should be careful because it may be possible to run tests this way which are not included in the normal run. If you use this method, do at least one full test run (or --group run, as below) to ensure the test can be found.

Please note the path name is optional if the testcase class and file names are standardised as described in [[Writing PHPUnit tests]].

==== Using the @group annotation ====

If you add annotations like

<code php>
/**
* Unit tests for {@link stack_cas_keyval}.
* @group qtype_stack
*/
class qtype_stack_cas_keyval_exception_testcase extends basic_testcase {
</code>

to all the classes in your plugin, then you can run just the tests for your plugin by doing

phpunit --group qtype_stack

Therefore, it is suggested that you annotate all your tests with the Frankenstyle name of your plugin.

==== Using multiple phpunit.xml files ====

It's easy to create alternative phpunit.xml files defining which tests must be run together. For reference, take a look to the default /phpunit.xml available in your base directory once the testing environment has been initialised. After creating the custom file you will be able to run those tests with

vendor/bin/phpunit -c path/to/your/alternative/phpunit/file.xml

Also, for commodity, you can use this command:

php admin/tool/phpunit/cli/util.php --buildcomponentconfigs

It will, automatically, create one valid phpunit.xml file within each component (plugin or subsystem) and other important directories, so later you will be able to execute tests like

vendor/bin/phpunit -c mod/forum[/phpunit.xml] // Note that it's not needed to specify the name of the file (if it is 'phpunit.xml').
vendor/bin/phpunit -c question
vendor/bin/phpunit -c question/type/calculated
vendor/bin/phpunit -c backup
vendor/bin/phpunit -c lib/dml
...

or, also

cd directory/with/phpunit.xml
phpunit

=External test resources=
{{Moodle 2.6}}
By default Moodle phpunit tests contact http://download.moodle.org server when testing curl related functionality. Optionally you may checkout a local copy of the test scripts and access it instead:

# clone https://github.com/moodlehq/moodle-exttests to web directory
# add to your config.php or modify phpunit.xml file <code php>define('TEST_EXTERNAL_FILES_HTTP_URL', 'http://localhost/moodle-exttests');</code>

=Writing new tests=
* read [http://www.phpunit.de/manual/current/en/ official PHPUnit online documentation]
* see [[Writing PHPUnit tests]]

=Conversion of existing SimpleTests=
* see [[SimpleTest conversion]]

=PHPUnit support in IDEs=

* [[Setting up Eclipse]]
* [[Setting up Netbeans]]
* [[Setting up PhpStorm]]

=Common Unit Test Problems=
[[Common unit test problems]]

=Performance=

A typical run of full PHPUnit tests for Moodle 2.7 takes 10-20 minutes depending on the machine. If tests run slowly for you:

* Ensure you are using a database and filesystem running on the same machine that is running the tests (not on a remote server).
* Apply developer-only performance settings to your database: [[Postgres Tuning For Developers]]

[[Category:Unit testing]]

Running acceptance test

2017-04-09T14:29:16Z

Mackensen: add link to preconfigured vagrant profile

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'capabilities' => array(
'platform' => 'Linux'
)
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium combination]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat

Migrating log access in reports

2016-08-02T00:46:54Z

Mackensen: unit test example

This document is aimed to assist developers in replacing SQL queries that access the '''log''' table (now referred to as the "legacy log" for reports or other plugins.

A new [[Logging 2|logging system]] is being introduced in Moodle 2.7. Writing to the legacy logging table may still be supported on some systems, however it is intended that administrators will make use of other logging plugins that can write to DB tables in the same DB as Moodle, to external DBs (SQL or non-SQL), to files, and so on. By default, a '''logstore_standard''' plugin is enabled that writes the data in the internal DB table. This table should not be queried directly. The new logging plugins define interfaces for reading data from the log stores.

Reports also have the choice of accessing the active log stores via logging plugin reader interfaces or registering their own observers and storing necessary data themselves.

== Using log readers ==

First, reports need to find available log readers.

There are three interfaces defined in core that log storage plugins may implement.
* '''\core\log\reader''' - parent for all reader interfaces;
* '''\core\log\sql_reader (\core\log\sql_select_reader in Moodle 2.7 and Moodle 2.8)''' - has methods to create simple SQL queries to the log store. The table (or equivalent storage collection) used is expected to have columns that correspond to properties of class '''\core\base\event''' and
* '''\core\log\sql_internal_table_reader (\core\log\sql_internal_reader in Moodle 2.7 and Moodle 2.8)''' - additionally has a method to return the table name which can be used to JOIN queries.

The report decides which type of readers it can work with. Assuming we need a select reader, the following example will access available interfaces of this type.
<pre>
$manager = get_log_manager();
// Remember to use '\core\log\sql_select_reader' instead of '\core\log\sql_reader' in Moodle 2.7 and Moodle 2.8.
$selectreaders = $manager->get_readers('\core\log\sql_reader');
if ($selectreaders) {
$reader = reset($selectreaders);
} else {
// No available log reader found.
}
</pre>

The method \core\log\manager::get_readers() returns an associative array of singleton instances of enabled log store plugins implementing the reader interface. When a plugin finds the necessary log reader it can query the data from it. Refer to the interfaces for the list of methods they implement.

For a transition period, a site may support both legacy logging and the new standard logging plugin. A report might need to access both log storages especially if it needs information that covers a long period of time that includes the transition from the legacy log to the new standard log. Please note that columns and event names are different in the new logging system and the legacy log, so separate queries will be needed.

Here is an example of how to use the legacy log reader (which may be present but not enabled for writing).
<pre>
$manager = get_log_manager();
$allreaders = $manager->get_readers();
if (isset($allreaders['logstore_legacy'])) {
$legacyreader = $allreaders['logstore_legacy'];
if ($legacyreader->is_logging()) {
// All events continue to be recorded in {log} table.
} else {
// The {log} table is kept for storing the old logs only. New events are not written to it and must be taken from another log storage.
}
} else {
// You are probably developing for 2.10 and table {log} does not exist any more. Or administrator uninstalled the plugin.
}
</pre>

== Registering an observer ==

Accessing logs on large Moodle instances will be slow and will cause load on the entire system. Some reports may wish to create their own mini log storages, storing only relevant events and perhaps only for a limited time. If you anticipate that such tables will be small, this can be an alternative solution to reading from logs.

In order to implement an observer in a plugin you must:
* define a database table in PLUGINDIR/db/install.xml and create it in PLUGINDIR/db/upgrade.php;
* define event observers in PLUGINDIR/db/events.php with handler functions (see [[Event_2#Event_observers|Events 2]]);
* define a cron task that truncates the old entries from the table when they are no longer needed (define $plugin->cron in your version.php);
* possibly create an upgrade script (in PLUGINDIR/db/upgrade.php) that migrates any relevant events from the legacy log table into the new observer table and
* re-write the SQL queries inside the plugin to access the new observer table instead of logs.

== See also ==
* [[Writing PHPUnit tests#Logstores]]

Writing PHPUnit tests

2016-08-02T00:46:09Z

Mackensen: /* Testing sending of emails */ add note on logstore tests

{{Moodle 2.3}}

Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).

=Testcase classes=

There are two basic test class that are supposed to used in all Moodle unit tests - basic_testcase and advanced_testcase. '''Please note it is strongly recommended to put only one testcase into each class file.'''
;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code.

There is a third testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.

== Assertions ==

The complete list of assertions can be found in the [http://www.phpunit.de/manual/3.7/en/appendixes.assertions.html phpunit manual].
==Sample plugin testcase==

PHPUnit tests are located in <code>tests/*_test.php</code> files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends <code>advanced_testcase</code>:

<code php>
class mod_myplugin_sample_testcase extends advanced_testcase {
public function test_adding() {
$this->assertEquals(2, 1+2);
}
}
</code>

See [[PHPUnit integration#Class and file naming rules]] for more information.

==Inclusion of Moodle library files==

If you want to include some Moodle library files you should always declare '''global $CFG'''. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically.

==Automatic state reset==
By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.

If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest().

<code php>
class mod_myplugin_testcase extends advanced_testcase {
public function test_deleting() {
global $DB;
$this->resetAfterTest(true);
$DB->delete_records('user');
$this->assertEmpty($DB->get_records('user'));
}
public function test_user_table_was_reset() {
global $DB;
$this->assertEquals(2, $DB->count_records('user', array()));
}
}
</code>

=Generators=

Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.

Note if you are using PHPUnit [https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers @dataProvider] functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function.

==Creating users==
At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use:
<code php>
$user = $this->getDataGenerator()->create_user();
</code>

You may also specify properties of the user account, for example:
<code php>
$user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1'));
</code>

By default no user is logged-in, use setUser() method to change current $USER value:
<code php>
$this->setUser($user1);
</code>

Guest and admin accounts have a shortcut methods:
<code php>
$this->setGuestUser();
$this->setAdminUser();
</code>

Null can be used to set current user back to not-logged-in:
<code php>
$this->setUser(null);
</code>

==Creating course categories==

<code php>
$category1 = $this->getDataGenerator()->create_category();
$category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id));
</code>

==Creating courses==

<code php>
$course1 = $this->getDataGenerator()->create_course();

$category = $this->getDataGenerator()->create_category();
$course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id));
</code>

==Creating activities==

Some activity plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.

Example of creation of new course with one page resource:

<code php>
$course = $this->getDataGenerator()->create_course();
$generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
$generator->create_instance(array('course'=>$course->id));
</code>

==Creating cohorts==
{{Moodle 2.4}}
Since 2.4 there the data generator supports creation of new cohorts.

<code php>
$cohort = $this->getDataGenerator()->create_cohort();
</code>

==Simplified user enrolments==
{{Moodle 2.4}}
Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins.

<code php>
$this->getDataGenerator()->enrol_user($userid, $courseid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');
</code>

==Creating scales==

<code php>
$this->getDataGenerator()->create_scale();
$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat));
</code>

==Creating roles==

<code php>
$this->getDataGenerator()->create_role();
$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype));
</code>

==Creating tags==

<code php>
$this->getDataGenerator()->create_tag();
$this->getDataGenerator()->create_tag(array(
'userid' => $userid,
'rawname' => $rawname,
'name' => $name,
'tagtype' => $tagtype,
'description' => $description,
'descriptionformat' => $descriptionformat,
'flag' => $flag
));
</code>

==Groups==

===Creating groups===
<code php>
$this->getDataGenerator()->create_group(array('courseid' => $courseid));
$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>

===Adding users to groups===
<code php>
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid));
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid));
</code>

===Creating groupings===
<code php>
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid));
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>

===Adding groups to groupings===
<code php>
$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid));
</code>

==Repositories==

===Creating repository instances===
{{Moodle 2.5}}
Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<code php>
$this->getDataGenerator()->create_repository($type, $record, $options);
</code>

===Creating repository types===
{{Moodle 2.5}}
Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<code php>
$this->getDataGenerator()->create_repository_type($type, $record, $options);
</code>

==Creating grades==

===Grade categories===

<code php>
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid));
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname));
</code>

===Grade items===

<code php>
$this->getDataGenerator()->create_grade_item();
$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype));
</code>

===Outcomes===

<code php>
$this->getDataGenerator()->create_grade_outcome();
$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname));
</code>

==Other types of plugin==
{{Moodle 2.5}}
Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname);

For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith.

=Long tests=

All standard test should execute as fast as possible. Tests that take a loner time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php.

=Large test data=
See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [[PHPUnit_integration#Extra_methods]]

=Testing sending of messages=
{{Moodle 2.4}}
You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages.

To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later.
<code php>
$this->preventResetByRollback();
$sink = $this->redirectMessages();
//... code that is sending messages
$messages = $sink->get_messages();
$this->assertEquals(3, count($messages));
//.. test messages were generated in correct order with appropriate content
</code>

=Testing sending of emails=
{{Moodle 2.6}}
You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails.

To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later.
<code php>
unset_config('noemailever');
$sink = $this->redirectEmails();
//... code that is sending email
$messages = $sink->get_messages();
$this->assertEquals(1, count($messages));
</code>

=Logstores=
You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute.
<code php>
$this->preventResetByRollback();
set_config('enabled_stores', 'logstore_standard', 'tool_log');
set_config('buffersize', 0, 'logstore_standard');
</code>

=Extra test settings=

Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php.

There are several ways how to inject your custom settings:
* define test setting constants in your phpunit.xml file
* define test setting constants in your config.php

These constants may be then used in your test or plugin code.

=See also=
* [[PHPUnit integration]]
* [[PHPUnit]]

[[Category:Unit testing]]

PHPUnit

2016-06-30T01:38:22Z

Mackensen:

{{Moodle 2.3}}

=What is PHPUnit=
PHPUnit by Sebastian Bergmann is an advanced unit testing framework for PHP. It is installed as Composer dependency and is not part of Moodle installation. To run PHPUnit tests, you have to manually install it on your development computer or test server.

Read the excellent guide at
* [http://phpunit.de/manual/current/en/index.html PHPUnit Manual]

=Installation of PHPUnit via Composer=

Windows may require additional installation steps.
On Windows go to https://getcomposer.org/download/ and download the Composer-Setup.exe file

* Execute Composer installer

cd /your/moodle/dirroot
php composer.phar install

(If that gives you connection problems try...)

php composer.phar install --prefer-source

Troubleshooting:
* On Windows if you are behind a proxy you will need to setup an environment variable called HTTP_PROXY with a value detailing your HTTP Proxy address and port before composer will correctly download files.
* You may be prompted for github credentials when installing composer dependencies.
** This is used to generate an personal access token to avoid being rate limited by github.
** If you have Two Factor Authentication enabled on your github account, or do not wish to supply your own credentials you will need to generate a token manually:
*** Visit https://github.com/settings/applications and request personal access token
*** Copy this token and add it to your [https://gist.github.com/andrewnicols/c5377ed25a9df1006ce1 ~/.composer/config.json]
** ( See [https://github.com/composer/composer/issues/2280 composer issue #2280] for further details of this bug.)

Detailed instructions:
* [http://getcomposer.org/doc/00-intro.md Composer documentation]

Detailed instructions:
* [[PHPUnit installation in Windows]]
* [[PHPUnit installation in OS X]]

== Uninstalling previous PEAR based version ==

Before using composer, this page used to suggest to install phpunit via PEAR. If you did so, you may wish to uninstall that package now. This should work:

$ pear uninstall phpunit/DbUnit
$ pear uninstall phpunit/PHPUnit

=Initialisation of test environment=

Our PHPUnit integration requires a dedicated database and dataroot. First, add a new dataroot directory and prefix into your config.php, you can find examples in config-dist.php (scroll down to 'Section 9').

$CFG->phpunit_prefix = 'phpu_';
$CFG->phpunit_dataroot = '/home/example/phpu_moodledata';

If your system does not have a direct connection to the Internet, you also need to specify your proxy in config.php - even though you normally specify it by using the admin settings user interface. (If you do not use a proxy, you can skip this step.) Check the settings on the relevant admin settings page, or from the mdl_config table in your database, if you are unsure of the correct values.

// Normal proxy settings
$CFG->proxyhost = 'wwwcache.example.org';
$CFG->proxyport = 80;
$CFG->proxytype = 'HTTP';
$CFG->proxybypass = 'localhost, 127.0.0.1, .example.org';
// Omit the next lines if your proxy doesn't need a username/password:
$CFG->proxyuser = 'systemusername';
$CFG->proxypassword = 'systempassword';

From Moodle 2.8.5 onwards, you can also provide specific database settings for unit testing (if these are not provided, the standard database settings will be used):

<pre>$CFG->phpunit_dbtype = 'pgsql'; // 'pgsql', 'mariadb', 'mysqli', 'mssql', 'sqlsrv' or 'oci'
$CFG->phpunit_dblibrary = 'native'; // 'native' only at the moment
$CFG->phpunit_dbhost = '127.0.0.1'; // eg 'localhost' or 'db.isp.com' or IP
$CFG->phpunit_dbname = 'mytestdb'; // database name, eg moodle
$CFG->phpunit_dbuser = 'postgres'; // your database username
$CFG->phpunit_dbpass = 'some_password'; // your database password</pre>

Then you need to initialise the test environment using following command.

cd /home/example/moodle
php admin/tool/phpunit/cli/init.php

This command has to be repeated after any upgrade, plugin (un)installation or if you have added tests to a plugin you are developing:

'''NOTE:''' make sure that your php cli executable (or the one you want to use) is correctly on your path as the individual init scripts will call it repeatedly. Also, ensure en_AU locale is installed on your server.

== LDAP ==
If you want to run LDAP unit tests you must have a working, configured LDAP environment on your test server. There must be a basic LDAP tree structure in place or tests will fail with "Search: No such object". Build an LDAP tree structure in a new shell prompt:

<pre>
$ ldapadd -H ldap://127.0.0.1 -D "cn=admin,dc=yourcomputer,dc=local" -W
dn:dc=yourcomputer,dc=local
objectClass:dcObject
objectClass:organizationalUnit
dc:yourcomputer
ou:YOURCOMPUTER
</pre>

In config.php tell Moodle where to look for your test LDAP environment:

<pre>
define('TEST_AUTH_LDAP_HOST_URL', 'ldap://127.0.0.1');
define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_AUTH_LDAP_BIND_PW', '*');
define('TEST_AUTH_LDAP_DOMAIN', 'dc=yourcomputer,dc=local');
</pre>

=Test execution=

To execute all test suites from main configuration file execute the <code>vendor/bin/phpunit</code> script from your <code>$CFG->dirroot</code> directory.

cd /home/example/moodle
vendor/bin/phpunit

The rest of examples uses <code>phpunit</code>, please substitute it with <code>vendor/bin/phpunit</code> or create a shortcut in your dirroot.

In IDEs, you may need to specify the path to the PHPUnit configuration file. Use the absolute path to <code>phpunit.xml</code> from your <code>$CFG->dirroot</code>.

There is an alternative script for running of tests via web interface: <code>admin/tool/phpunit/webrunner.php</code>. Use this as the last resort only when you cannot use the command-line interface on your test server. It will most probably break due to permissions problems if you try to execute it both from command-line and from webrunner. This feature is not officially supported.

===How to run only some tests===

==== Running a single test quickly ====

The fastest way to run a single test is:

vendor/bin/phpunit my_test_class_name my/tests/filename.php

This is faster than other methods because it avoids the need to search for the test file, but you should be careful because it may be possible to run tests this way which are not included in the normal run. If you use this method, do at least one full test run (or --group run, as below) to ensure the test can be found.

Please note the path name is optional if the testcase class and file names are standardised as described in [[Writing PHPUnit tests]].

==== Using the @group annotation ====

If you add annotations like

<code php>
/**
* Unit tests for {@link stack_cas_keyval}.
* @group qtype_stack
*/
class qtype_stack_cas_keyval_exception_testcase extends basic_testcase {
</code>

to all the classes in your plugin, then you can run just the tests for your plugin by doing

phpunit --group qtype_stack

Therefore, it is suggested that you annotate all your tests with the Frankenstyle name of your plugin.

==== Using multiple phpunit.xml files ====

It's easy to create alternative phpunit.xml files defining which tests must be run together. For reference, take a look to the default /phpunit.xml available in your base directory once the testing environment has been initialised. After creating the custom file you will be able to run those tests with

vendor/bin/phpunit -c path/to/your/alternative/phpunit/file.xml

Also, for commodity, you can use this command:

php admin/tool/phpunit/cli/util.php --buildcomponentconfigs

It will, automatically, create one valid phpunit.xml file within each component (plugin or subsystem) and other important directories, so later you will be able to execute tests like

vendor/bin/phpunit -c mod/forum[/phpunit.xml] // Note that it's not needed to specify the name of the file (if it is 'phpunit.xml').
vendor/bin/phpunit -c question
vendor/bin/phpunit -c question/type/calculated
vendor/bin/phpunit -c backup
vendor/bin/phpunit -c lib/dml
...

or, also

cd directory/with/phpunit.xml
phpunit

=External test resources=
{{Moodle 2.6}}
By default Moodle phpunit tests contact http://download.moodle.org server when testing curl related functionality. Optionally you may checkout a local copy of the test scripts and access it instead:

# clone https://github.com/moodlehq/moodle-exttests to web directory
# add to your config.php or modify phpunit.xml file <code php>define('TEST_EXTERNAL_FILES_HTTP_URL', 'http://localhost/moodle-exttests');</code>

=Writing new tests=
* read [http://www.phpunit.de/manual/current/en/ official PHPUnit online documentation]
* see [[Writing PHPUnit tests]]

=Conversion of existing SimpleTests=
* see [[SimpleTest conversion]]

=PHPUnit support in IDEs=

* [[Setting up Eclipse]]
* [[Setting up Netbeans]]
* [[Setting up PhpStorm]]

=Common Unit Test Problems=
[[Common unit test problems]]

=Performance=

A typical run of full PHPUnit tests for Moodle 2.7 takes 10-20 minutes depending on the machine. If tests run slowly for you:

* Ensure you are using a database and filesystem running on the same machine that is running the tests (not on a remote server).
* Apply developer-only performance settings to your database: [[Postgres Tuning For Developers]]

[[Category:Unit testing]]

PHPUnit

2016-06-30T01:37:48Z

Mackensen: /* Initialisation of test environment */ how to bootstrap the LDAP environment

{{Moodle 2.3}}

=What is PHPUnit=
PHPUnit by Sebastian Bergmann is an advanced unit testing framework for PHP. It is installed as Composer dependency and is not part of Moodle installation. To run PHPUnit tests, you have to manually install it on your development computer or test server.

Read the excellent guide at
* [http://phpunit.de/manual/current/en/index.html PHPUnit Manual]

=Installation of PHPUnit via Composer=

Windows may require additional installation steps.
On Windows go to https://getcomposer.org/download/ and download the Composer-Setup.exe file

* Execute Composer installer

cd /your/moodle/dirroot
php composer.phar install

(If that gives you connection problems try...)

php composer.phar install --prefer-source

Troubleshooting:
* On Windows if you are behind a proxy you will need to setup an environment variable called HTTP_PROXY with a value detailing your HTTP Proxy address and port before composer will correctly download files.
* You may be prompted for github credentials when installing composer dependencies.
** This is used to generate an personal access token to avoid being rate limited by github.
** If you have Two Factor Authentication enabled on your github account, or do not wish to supply your own credentials you will need to generate a token manually:
*** Visit https://github.com/settings/applications and request personal access token
*** Copy this token and add it to your [https://gist.github.com/andrewnicols/c5377ed25a9df1006ce1 ~/.composer/config.json]
** ( See [https://github.com/composer/composer/issues/2280 composer issue #2280] for further details of this bug.)

Detailed instructions:
* [http://getcomposer.org/doc/00-intro.md Composer documentation]

Detailed instructions:
* [[PHPUnit installation in Windows]]
* [[PHPUnit installation in OS X]]

== Uninstalling previous PEAR based version ==

Before using composer, this page used to suggest to install phpunit via PEAR. If you did so, you may wish to uninstall that package now. This should work:

$ pear uninstall phpunit/DbUnit
$ pear uninstall phpunit/PHPUnit

=Initialisation of test environment=

Our PHPUnit integration requires a dedicated database and dataroot. First, add a new dataroot directory and prefix into your config.php, you can find examples in config-dist.php (scroll down to 'Section 9').

$CFG->phpunit_prefix = 'phpu_';
$CFG->phpunit_dataroot = '/home/example/phpu_moodledata';

If your system does not have a direct connection to the Internet, you also need to specify your proxy in config.php - even though you normally specify it by using the admin settings user interface. (If you do not use a proxy, you can skip this step.) Check the settings on the relevant admin settings page, or from the mdl_config table in your database, if you are unsure of the correct values.

// Normal proxy settings
$CFG->proxyhost = 'wwwcache.example.org';
$CFG->proxyport = 80;
$CFG->proxytype = 'HTTP';
$CFG->proxybypass = 'localhost, 127.0.0.1, .example.org';
// Omit the next lines if your proxy doesn't need a username/password:
$CFG->proxyuser = 'systemusername';
$CFG->proxypassword = 'systempassword';

From Moodle 2.8.5 onwards, you can also provide specific database settings for unit testing (if these are not provided, the standard database settings will be used):

<pre>$CFG->phpunit_dbtype = 'pgsql'; // 'pgsql', 'mariadb', 'mysqli', 'mssql', 'sqlsrv' or 'oci'
$CFG->phpunit_dblibrary = 'native'; // 'native' only at the moment
$CFG->phpunit_dbhost = '127.0.0.1'; // eg 'localhost' or 'db.isp.com' or IP
$CFG->phpunit_dbname = 'mytestdb'; // database name, eg moodle
$CFG->phpunit_dbuser = 'postgres'; // your database username
$CFG->phpunit_dbpass = 'some_password'; // your database password</pre>

Then you need to initialise the test environment using following command.

cd /home/example/moodle
php admin/tool/phpunit/cli/init.php

This command has to be repeated after any upgrade, plugin (un)installation or if you have added tests to a plugin you are developing:

'''NOTE:''' make sure that your php cli executable (or the one you want to use) is correctly on your path as the individual init scripts will call it repeatedly. Also, ensure en_AU locale is installed on your server.

== LDAP ==
If you want to run LDAP unit tests you must have a working, configured LDAP environment on your test server. There must be a basic LDAP tree structure in place or tests will fail with "Search: No such object". Build an LDAP tree structure in a new shell prompt:

<pre>
$ ldapadd -H ldap://127.0.0.1 -D "cn=admin,dc=yourcomputer,dc=local" -W
dn:dc=yourcomputer,dc=local
objectClass:dcObject
objectClass:organizationalUnit
dc:yourcomputer
ou:YOURCOMPUTER
</pre>

In config.php tell Moodle where to look for your test LDAP environment:

<pre>
define('TEST_AUTH_LDAP_HOST_URL', 'ldap://127.0.0.1');
define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_AUTH_LDAP_BIND_PW', '*');
define('TEST_AUTH_LDAP_DOMAIN', 'dc=yourcomputer,dc=local');

=Test execution=

To execute all test suites from main configuration file execute the <code>vendor/bin/phpunit</code> script from your <code>$CFG->dirroot</code> directory.

cd /home/example/moodle
vendor/bin/phpunit

The rest of examples uses <code>phpunit</code>, please substitute it with <code>vendor/bin/phpunit</code> or create a shortcut in your dirroot.

In IDEs, you may need to specify the path to the PHPUnit configuration file. Use the absolute path to <code>phpunit.xml</code> from your <code>$CFG->dirroot</code>.

There is an alternative script for running of tests via web interface: <code>admin/tool/phpunit/webrunner.php</code>. Use this as the last resort only when you cannot use the command-line interface on your test server. It will most probably break due to permissions problems if you try to execute it both from command-line and from webrunner. This feature is not officially supported.

===How to run only some tests===

==== Running a single test quickly ====

The fastest way to run a single test is:

vendor/bin/phpunit my_test_class_name my/tests/filename.php

This is faster than other methods because it avoids the need to search for the test file, but you should be careful because it may be possible to run tests this way which are not included in the normal run. If you use this method, do at least one full test run (or --group run, as below) to ensure the test can be found.

Please note the path name is optional if the testcase class and file names are standardised as described in [[Writing PHPUnit tests]].

==== Using the @group annotation ====

If you add annotations like

<code php>
/**
* Unit tests for {@link stack_cas_keyval}.
* @group qtype_stack
*/
class qtype_stack_cas_keyval_exception_testcase extends basic_testcase {
</code>

to all the classes in your plugin, then you can run just the tests for your plugin by doing

phpunit --group qtype_stack

Therefore, it is suggested that you annotate all your tests with the Frankenstyle name of your plugin.

==== Using multiple phpunit.xml files ====

It's easy to create alternative phpunit.xml files defining which tests must be run together. For reference, take a look to the default /phpunit.xml available in your base directory once the testing environment has been initialised. After creating the custom file you will be able to run those tests with

vendor/bin/phpunit -c path/to/your/alternative/phpunit/file.xml

Also, for commodity, you can use this command:

php admin/tool/phpunit/cli/util.php --buildcomponentconfigs

It will, automatically, create one valid phpunit.xml file within each component (plugin or subsystem) and other important directories, so later you will be able to execute tests like

vendor/bin/phpunit -c mod/forum[/phpunit.xml] // Note that it's not needed to specify the name of the file (if it is 'phpunit.xml').
vendor/bin/phpunit -c question
vendor/bin/phpunit -c question/type/calculated
vendor/bin/phpunit -c backup
vendor/bin/phpunit -c lib/dml
...

or, also

cd directory/with/phpunit.xml
phpunit

=External test resources=
{{Moodle 2.6}}
By default Moodle phpunit tests contact http://download.moodle.org server when testing curl related functionality. Optionally you may checkout a local copy of the test scripts and access it instead:

# clone https://github.com/moodlehq/moodle-exttests to web directory
# add to your config.php or modify phpunit.xml file <code php>define('TEST_EXTERNAL_FILES_HTTP_URL', 'http://localhost/moodle-exttests');</code>

=Writing new tests=
* read [http://www.phpunit.de/manual/current/en/ official PHPUnit online documentation]
* see [[Writing PHPUnit tests]]

=Conversion of existing SimpleTests=
* see [[SimpleTest conversion]]

=PHPUnit support in IDEs=

* [[Setting up Eclipse]]
* [[Setting up Netbeans]]
* [[Setting up PhpStorm]]

=Common Unit Test Problems=
[[Common unit test problems]]

=Performance=

A typical run of full PHPUnit tests for Moodle 2.7 takes 10-20 minutes depending on the machine. If tests run slowly for you:

* Ensure you are using a database and filesystem running on the same machine that is running the tests (not on a remote server).
* Apply developer-only performance settings to your database: [[Postgres Tuning For Developers]]

[[Category:Unit testing]]

Acceptance testing

2015-06-27T21:31:13Z

Mackensen: /* See also */ +Vagrant

== Introduction ==
This page describes how we describe Moodle's functionalities and automatically test them.

Behat is a behavioural driven development (BDD) tool written in PHP, it can parse a human-readable list of sentences (called steps) and execute actions in a browser using Selenium or other tools to simulate user interactions.

For technical info: [[Behat integration]]

=== How it works ===
Behat parses and executes features files which describe Moodle's features (for example ''Post in a forum''). Each feature file is composed of many scenarios (for example ''Add a post to a discussion'' or ''Create a new discussion''), and finally each scenario is composed of steps (for example ''I press "Post to forum"'' or ''I should see "My post title"''). When the feature file is executed, every step internally is translated into a PHP method and is executed.

These features are executed nightly on the HQ servers using all the supported databases (MySQL, PostgreSQL, MSSQL and Oracle) and with different browsers (Firefox, Internet Explorer, Safari and Chrome) to avoid regressions and to test new functionalities.

=== Examples ===

Note that these snippets are only examples and may not work.

* There is a closed list of steps to use in the features, a feature written with the basic (or low-level) steps looks like this:
@auth
'''Feature''': Login
In order to login
As a moodle user
I need to be able to validate the username and password against moodle

'''Scenario''': Login as an existing user
Given I am on "login/index.php"
When I fill in "username" with "admin"
And I fill in "password" with "moodle"
And I press "loginbtn"
Then I should see "Moodle 101: Course Name"

'''Scenario''': Login as an unexisting user
Given I am on "login/index.php"
When I fill in "username" with "admin"
And I fill in "password" with "moodle"
And I press "loginbtn"
Then I should see "Moodle 101: Course Name"

Note that The 3 sentences below ''Feature: Login'' are only information about what we want to test.

These are simple scenarios, but most of Moodle's functionalities would require a huge list of this steps to test a scenario, imagine a ''Add a post to a discussion'' scenario; you need to login, create a course, create a user and enrol it in the course... Most of this steps is not what we intend to test in a ''Post in a forum'' feature, Moodle provides extra steps to quickly set up the context required to test a Moodle feature, for example:

@mod @mod_forum
'''Feature''': Add forum activities and discussions
In order to discuss topics with other users
As a moodle teacher
I need to add forum activities to moodle courses

'''Scenario''': Add a forum and a discussion
'''Given''' the following "users" exists:
| username | firstname | lastname | email |
| teacher1 | Teacher | 1 | teacher1@asd.com |
'''And''' the following "courses" exists:
| fullname | shortname | category |
| Course 1 | C1 | 0 |
'''And''' the following "course enrolments" exists:
| user | course | role |
| teacher1 | C1 | editingteacher |
'''And''' I log in as "teacher1"
'''And''' I follow "Course 1"
'''And''' I turn editing mode on
'''And''' I add a "Forum" to section "1" and I fill the form with:
| Forum name | Test forum name |
| Forum type | Standard forum for general use |
| Description | Test forum description |
'''When''' I add a new discussion to "Test forum name" forum with:
| Subject | Forum post subject |
| Message | This is the body |
'''Then''' I should see "Test forum name"

Note that:

* Each scenario is executed in an isolated testing environment, so the first step begins with an empty moodle site and what you set up in an scenario (like the ''Test forum name'' forum in the example above) is cleaned up after the scenario execution
* The prefixes "Given", "When" and "Then" are only informative and they are used to define the context (Given), specify the action (When) and check the results (Then), using them properly helps to understand what the scenario is testing.

== Quick start ==

This is a quick introduction to write a functional test (acceptance tests) using steps in a development/testing site, please DON'T USE THIS IN A PRODUCTION SITE.

To let you experience the pleasure of watching a feature file doing "your work" automatically in a real browser, this guide includes 2 optional steps to download Selenium and run it in another CLI.

# Open a command line interface (you might need to run as administrator to get curl to work - one windows, right click the command prompt and 'choose run as administrator')
# '''cd /to/your/moodle/dirroot'''
# Edit config.php adding the following lines before the lib/setup.php include
#: <code language="text">$CFG->behat_prefix = 'b_';
$CFG->behat_dataroot = '/path/to/your/behat/dataroot/directory';
$CFG->behat_wwwroot = 'http://127.0.0.1'; // must be different from wwwroot
</code>
# '''curl http://getcomposer.org/installer | php''' (If you get a curl error saying 'curl is not recognized as an internal or external command' run the command line as administrator. In case you still have problems read https://docs.moodle.org/dev/Acceptance_testing#Installation)
# '''php admin/tool/behat/cli/init.php''' (If you get a curl error saying 'curl is not recognized as an internal or external command' run the command line as administrator.)
# Download selenium-server-standalone-2.NN.N.jar from http://seleniumhq.org/download/, under "Selenium server (formerly the Selenium RC Server)"
# Open another command line interface and run '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''
# '''vendor/bin/behat --config /path/to/your/behat/dataroot/directory/behat/behat.yml'''
# You just ran the current Moodle tests, now let's add your own test, add a blog entry for example
# Browse to your $CFG->behat_wwwroot, this is an empty test site and it is reset before each test (called scenario)
# From this point follow the steps you would follow to add manually a blog entry (login credentials are admin/admin)
# When you are done go to 'Site administration' -> 'Development' -> 'Acceptance testing', you will find the list of "actions" that can be run automatically, you can filter them to find what do you need to do (more steps can be added if you need, more info in https://docs.moodle.org/dev/Acceptance_testing#Adding_steps_definitions)
# To 'add a blog entry' we need to:
## Log in the system as a valid user
## Expand 'My profile' node of the navigation block
## Expand the 'Blogs' node of the navigation block
## Follow he 'Add a new entry' link
## Fill the moodle form with values for 'Entry title' and 'Blog entry body'
## Press the 'Save changes' button
## Verify you see the values you entered in the form and verify you are not in the form page
# This translated to steps is:
#: <code language="text">Given I log in as "admin"
And I expand "My profile" node
And I expand "Blogs" node
And I follow "Add a new entry"
And I set the following fields to these values:
| Entry title | I'm the name |
| Blog entry body | I'm the description |
When I press "Save changes"
Then I should see "User Blog"
And I should see "I'm the description"
And I should not see "Required"
</code>
# We need to wrap this steps following the behavior driven development guidelines (more info in https://docs.moodle.org/dev/Acceptance_testing#Writing_features)
#: <code language="text">
@core @core_blog
Feature: Add a blog entry
In order to let the world know about me
As a user
I need to write blog entries

@javascript
Scenario: Add a blog entry with valid data
Given I log in as "admin"
And I expand "My profile" node
And I expand "Blogs" node
And I follow "Add a new entry"
And I fill the moodle form with:
| Entry title | I'm the name |
| Blog entry body | I'm the description |
When I press "Save changes"
Then I should see "View all of my entries"
And I should see "I'm a description"
And I should not see "Required"
</code>
# And save it into a file, in this case '''blog/tests/behat/add_entry.feature'''
# '''php admin/tool/behat/cli/util.php --enable''' (This will update the available tests and steps definitions)
# '''vendor/bin/behat --config /path/to/your/behat/dataroot/directory/behat/behat.yml --tags @core_blog'''
# Selenium will open a browser (firefox by default) and you will see how the steps you have been writting are executed

You can also try to expand non existing nodes or change the 'Then' assertions to get a beautiful failure.

For detailed steps and/or troubleshooting:
* https://docs.moodle.org/dev/Acceptance_testing#Running_tests
* https://docs.moodle.org/dev/Acceptance_testing#Writing_features

== Requirements ==
* PHP 5.4 (see https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage for PHP 5.3, only for non-production sites)
* Other dependencies are managed by the composer installer

== Installation ==
* Edit config.php
** Use $CFG->behat_dataroot to set the directory where behat test environment dataroot will be stored, something like '''$CFG->behat_dataroot = '/your/directory/path';'''. Ensure the directory can be created or have write permissions
** Use $CFG->behat_prefix to set the database prefix of the behat test environment database tables, something like '''$CFG->behat_prefix = 'behat_';'''
** Use $CFG->behat_wwwroot to set address to be used to access behat instance. It has to be different from $CFG->wwwroot, you can use for example localhost, 127.0.0.1 or any custom local host name specified in you /etc/hosts. If you use the built-in PHP server use "http://localhost:8000" or the value you set when you started it.
* Download composer
** '''cd /your/moodle/dirroot'''
** '''curl http://getcomposer.org/installer | php'''
*** If you don't have curl installed or you have problems running '''curl http://getcomposer.org/installer | php''':
**** Download '''http://getcomposer.org/installer'''
**** Store it in /your/moodle/dirroot/composerinstaller.php for example
**** Run it from /your/moodle/dirroot with '''php composerinstaller.php''', you can delete this file after running the next step ('''php composer.phar update --dev''')
* Install behat dependencies and enable the test environment
** '''cd /your/moodle/dirroot'''
** '''php admin/tool/behat/cli/init.php'''
* (Optional) If you want to run tests that involves Javascript (most of them) you will also need Selenium
** Download it from http://seleniumhq.org/download/, named "Selenium server (formerly the Selenium RC Server)"

== Verify your installation ==
* Test your installation by browsing your /admin/tool/behat/index.php page.
If you are using MAMP/WAMP/XAMPP and don't get the report working... then you need to ensure that your web servers knows where the PHP executable (commad line) is. To do that, edit the "envvars" file (usually under library/bin) and add to $PATH the path to your php executable. Then restart the apache server and try again.
For example, for MAMP you probably need to add:

<code php>
# Adding MAMP binaries communicate where CLI is
PATH="/Applications/MAMP/bin/php/php5.x.y/bin:$PATH"; export PATH
</code>

at the end of MAMP/Library/bin/envvars.

== Running tests ==
# (Optional) Start the PHP built-in web server if you configured the $CFG->behat_wwwroot to use it:
#* Open a command line interface and '''cd /to/your/moodle/dirroot'''
#* '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# (Optional) Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''
# Initialise Behat environment
#* To initialise behat use following command with optional parameters.
#** '''php admin/tool/behat/cli/init.php'''
#* Following options can be passed to initialisation command
#** '''-j, --parallel''' Number of parallel behat run to initialise
#** '''-m, --maxruns''' Max parallel processes to be executed at one time.
#** '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
#** '''--torun''' Execute run till (Used for parallel runs on different vms)
#** '''-h, --help''' Print out this help
#* Example to initialise behat to run 4 process (Moodle CLI only supports {option}={value}, and not spaces)
#** '''php admin/tool/behat/cli/init.php -j=4'''
# Run Behat
#* If behat site is initialise without parallel options then use following command:
#** '''vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behat/behat.yml''' (For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html)
#* If behat site is initialise with parallel options then use following command:
#** '''php admin/tool/behat/cli/run.php''' (For more options '''php admin/tool/behat/cli/run.php --help''')
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''
#* If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running, try downloading the latest Selenium release from it's website as explained above
# (Optional) If you are adding new tests or steps definitions update the tests list:
#* '''php admin/tool/behat/cli/util.php --enable'''
# (Optional) Disable test environment (if you want to prevent access to test environment)
#* '''php admin/tool/behat/cli/util.php --disable'''

'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

=== Tests filters ===
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

=== Output formats ===

If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Run behat using different browsers ==

Following these instructions you can run behat in Firefox browser through Selenium, but you can run behat using different browsers through Selenium and even use phantomjs (Webkit). [[Acceptance_testing/Browsers|More info]]

== Advanced usage ==
There are a few settings for advanced use of Behat and execution in continuous integration systems, by default all this options are disabled, use this settings only if you know what you are doing.

* '''Different test server URL'''. If for example your are interested in allowing accesses from your local network because your Jenkins server is there you can set $CFG->behat_wwwroot to '''http://my.computer.local.ip:8000'''
* '''Behat configuration''', Moodle writes a behat.yml config file with info about the available tests and steps definitions along with other Behat parameters, you can override the Behat parameters we set and add your new parameters, your parameters will be merged with the Moodle ones giving priority to your values in case of conflict. This is useful for an advanced use of Behat, with multiple profiles, output formats, integration with continuous servers...
* '''Save screenshots of failures'''. You can use $CFG->behat_faildump_path to specify a directory where behat will generate a screenshot with the browser state each time a scenario fails. This is useful to detect where the problem was and work on a solution.
* '''Running with a browser other than Firefox''', by adding the following code to your config.php you can change the selected browser that is run when behat is invoked. In this case Chrome is selected, but internet explorer, firefox, iphone, android, chrome, htmlunit should be valid options. You will need to run '''php admin/tool/behat/cli/init.php''' for changes to take effect.

<code language="php">
$CFG->behat_config = array(
'default' => array(
'extensions' => array(
'Behat\MinkExtension\Extension' => array(
'selenium2' => array(
'browser' => 'chrome'
)
)
)
)
);
</code>

:Note that for Chrome, you will need the Selenium Chrome Driver (https://code.google.com/p/selenium/wiki/ChromeDriver), and it will need to be installed in the command search path.
* '''Switch completely to test environment''' option was removed in 2.7, it is recommended to always set $CFG->behat_wwwroot even in older versions instead.
* Note that when using cloud-based systems that can make use of non-standard capabilities like Saucelabs, you might want to provide configuration attributes containing the ''''-'''' character, which is automatically converted to ''''_'''' by the Symfony configuration manager that Behat is making use of (@see Symfony\Component\Config\Definition\Processor::normalizeKeys()) a way to avoid this restriction is to, adding to the vars you set like ''''max-duration'''' add the same var replacing dashes for underscores, this way the configuration manager will maintain the attribute containing dashes.
* '''Extra allowed settings''', moodle allows users to define many settings in config.php (see config-dist.php) when running the behat test site those settings are skipped to avoid interaction with the production environment, in case you are interested in allowing some of those extra settings to run the tests using a configuration similar to the one you are using in your production environment you can whitelist them adding them to $CFG->behat_extraallowedsettings (see config-dist.php for examples).
You can find more info and examples of how to use this settings in the config-dist.php file included in the Moodle codebase.

== Contributing ==

You can contribute the effort to automatically test all of Moodle's functionalities, as described in the [[Acceptance_testing/Contributing_automated_tests| guide to contributing automated tests]]. Here you can find information about [[Acceptance_testing#Writing_features|how to write new features]] and [[Acceptance_testing#Adding_steps_definitions|how to write new step definitions]] if your changes requires a base change in the [https://github.com/moodlehq/moodle-behat-extension Moodle behat extension] you can find here how to do it following the integration workflow: [[Acceptance_testing/Contributing_to_Moodle_behat_extension]].

== Writing features ==

All Moodle components and plugins (including 3rd party plugins) can specify their tests in .feature files using all the available steps.

Once you decided which functionality you want to specify as a feature you should:
# Select the most appropriate Moodle component to include your test and create a COMPONENTNAME/tests/behat/FEATURENAME.feature file
# Add a tag with the component name in Frankenstyle format (https://docs.moodle.org/dev/Frankenstyle) on the first line along with the plugin type or @core if it's a core subsystem
# Begin writing the user story of the feature, including in the 'As a ...' statement the main beneficiary of the feature:
#: <code lang="yaml">@plugintype @plugintype_pluginname
Feature: FEATURENAME
In order to ... // Why this feature is useful
As ... // It can be 'an admin', 'a teacher', 'a student', 'a guest', 'a user', 'a tests writer' and 'a developer'
I need to ... // The feature we want</code>
# From the beneficiary point of view, think of different scenarios to ensure the feature works as expected
# For each scenario you thought:
## Think of the initial context you need, for example ''1 course with 2 students on it and an assignment'', and which steps do you need to follow (interacting with the browser) to verify the scenario works as expected
## What you are testing requires Javascript? Think only on the feature you are testing (for example if you want to test that you can view your profile you don't need Javascript to click on a link and assert against plain HTML, but if you want to test something related with the course's gradebook you might want to test it with Javascript)
## Check the steps list (more info in https://docs.moodle.org/dev/Acceptance_testing#Available_steps) and set the initial context data (see https://docs.moodle.org/dev/Acceptance_testing#Fixtures for more info) and the steps to follow to verify all works as it should work.
## The prefixes ''Given'', ''When'' and ''Then'' separates the scenario in 3 parts, the initial context setup (''Given''), the action that provokes a change in the system (''When'') and the validation of that change outcomes (''Then'') So with a quick view at the scenario you can see what it is testing as the prefixes will be something like Given -> And -> And -> When -> And -> And -> Then -> And -> And -> And. Once we begin with the first ''Then'' we can consider that we are checking the outcomes so all the steps from there should be prefixed with ''Then''
## Copy the list of steps to the .feature file with the Scenario header:
##: <code lang="yaml">Scenario: Short description of the scenario
Given step 1
And step 2
And step 3
When step 4
And step 5
Then step 6</code>
## If the steps you are using requires Javascript add the @javascript tag above the "Scenario:" headline
##: <code lang="yaml">@javascript
Scenario: Short description of the scenario
...
...</code>
# Run the tests, when creating your new features/scenarios you can specify a '@wip' (work in progress) tag in both the line above the Scenario description and the tests runner (vendor/bin/behat) to execute only the new scenario instead of running the whole set of tests.
# Add extra tags to the scenario or the feature if required according to https://docs.moodle.org/dev/Acceptance_testing#Tests_filters

=== Available steps ===

Moodle provides a interface to list and filter the steps you can use when writing features. You can access it through the Administration block, following '''Site Administration''' -> '''Development''' -> '''Acceptance testing'''. It allows filtering by keyword, by the Moodle component or by the type of step:
* Processes to set up the environment
* Actions that provokes an event
* Checkings to ensure the outcomes are the expected ones

[[File:Acceptance_testing_UI_2.5.png]]

* This interface only works on sites where Behat is installed. If you are using the local PHP server, for example, you can access it on that site at http://localhost:8000 (log in as admin/admin).

=== Tips ===
* You can use a '''Background''' section before the '''Scenario''' sections, this steps will be executed before the steps of each scenario (http://docs.behat.org/guides/1.gherkin.html#backgrounds)
* You can use '''Scenario outlines''' if your scenarios are nearly the same and depends on a few vars; check out the link for an explicative example (http://docs.behat.org/guides/1.gherkin.html#scenario-outlines)
** If your scenario outline consists of many steps it may be useful to add a comment with the number of steps. If the the test fails behat will tell you in which step overall and you will have to divide by the number of steps of the scenario to know in which example.
* Is better to test the outcomes against the given data than against language strings, which are depending on the selected language.
* In case you need to interact with popup windows you need to switch to the window you want to interact with after opening it using the '''I switch to "popupwindowname" window''', close it when you finish interacting with it and return to the main window using '''I switch to main window'''
* The format of the .feature files is YAML which finds out the data hierarchy from the indentation of it's elements, so be sure that the elements are correctly nested and the indentation is correct using spaces when necessary

=== Providing values to steps ===
Most of the steps requires values, there are five methods to provide values to steps, the method depends on the step specification, you can know when a steps requires a value because you will see a drop down menu with a closed list of options that the step accepts as argument or an upper case string between double quotes, something like '''I press "BUTTON_STRING"''' or it ends with a ''':''' . The five methods are:
* '''A string/text'''; is the most common case, the texts are wrapped between double quotes (" character) you have to replace the info about the expected value for your value; for example something like '''I press "BUTTON_STRING"''' should become '''I press "Save and return to course"'''. If you want to add a string which contains a " character, you can escape it with \", for example '''I fill the "Name" field with "Alan alias \"the legend\""'''. You can identify this steps because they ends with '''_STRING'''
* '''A number'''; some steps requires numbers as values, to be more specific an undetermined number of digits from 0 to 9 (Natural numbers + 0) you can identify them because the expected value info string ends with '''_NUMBER'''
* '''A table'''; is a relation between values, the most common use of it is to fill forms. The steps which requires tables are easily identifiable because they finish with ''':''' The steps description gives info about what the table columns must contain, for example '''Fills a moodle form with field/value data'''. Here you don't need to escape the double quotes if you want to include them as part of the value.
* '''A PyString'''; is a multiline string, most commonly used to fill out forms when a newline is required. Like steps with tables, steps which require PyStrings will end with ":"
* '''A field value'''; There are many different field types, if an argument requires a field value the expected value will depend on the field type:
** Text-based fields: It expects the text. This includes textareas, input type text, input type password...
** Checkbox: It expects 1 to check and for checked and "" to uncheck or for unchecked
** Select: It expects the option text or the option value. In case you interact with a multi-select you should specify the options separating them with commas. For example: '''option1, option2, option3'''
** Radio: The text of the radio option
* '''A selector'''; there are steps that can be used with different kinds of elements, for example '''I click on "User Name" "link"''' or '''I click on "User Name" "button"''' this is a closed list of elements, they always works together with another argument, where you specify the locator (eg. the link text in a link) In the 'Acceptance testing' interface you can see a drop-down menu to select one of these options:
** field - for searching a field by its id, name, value or label
** link - for searching a link by its href, id, title, img alt or value
** button - for searching a button by its name, id, value, img alt or title
** link_or_button - for searching for both, links and buttons
** select - for searching a select field by its id, name or label
** checkbox - for searching a checkbox by its id, name, or label
** radio - for searching a radio button by its id, name, or label
** file - for searching a file input by its id, name, or label
** optgroup - for searching optgroup by its label
** option - for searching an option by its content
** dialogue - for searching a dialogue with the specified header text
** filemanager - for searching a filemanager by it's id or label
** block - for searching a Moodle block by it's English name or it's frankenstyle name
** section - for searching for a section on a course page by it's title or its written out date (e.g. "1 January - 7 January"). Use "frontpage" "section" for the frontpage section if it has no title (default)
** activity - for searching for an activity module in a course list by it's title
** region - for searching a Moodle page region with that id, in fact it works with all the page's ids
** table_row - for searching a table row which contains the specified text
** table - for searching a table by its id or caption
** fieldset - for searching a fieldset by it's id or legend
** css_element - for searching an element by its CSS selector
** xpath_element - for searching an element by its XPath
* '''A text selector'''; similar to a selector but those are the elements that returns an area of the DOM, they are useful in steps following the format '''... in the "Community finder" "block"''' where you are clicking or looking for some text inside a specific area. In the 'Acceptance testing' interface you can see a drop-down menu to select one of these options:
** dialogue - for searching a dialogue with the specified header text
** block - for searching a Moodle block by it's English name or it's frankenstyle name
** section - for searching for a section on a course page by it's title or its written out date (e.g. "1 January - 7 January"). Use "frontpage" "section" for the frontpage section if it has no title (default)
** activity - for searching for an activity module in a course list by it's title
** region - for searching a Moodle page region with that id, in fact it works with all the page's ids
** table_row - for searching a table row which contains the specified text
** table - for searching a table by its id or caption
** fieldset - for searching a fieldset by it's id or legend
** css_element - for searching an element by its CSS selector
** xpath_element - for searching an element by its XPath

==== Uploading files ====
Note than some tests requires files to be uploaded, in this case
* The '''I upload "FILEPATH_STRING" file to "FILEPICKER_FIELD_STRING" filepicker''' step can be used when located in the form page
* The file to upload should be included along with the Moodle codebase in COMPONENTNAME/tests/fixtures/*
* The file to upload is specified by it's path, which should be relative to the codebase root ('''lib/tests/fixtures/users.csv''' for example)
* '''/''' should be used as directory separator and the file names can not include this '''/''' character as all of them would be converted to the OS-dependant directory separator to maintain the compatibility with Windows systems.
* The scenarios that includes files uploading should be tagged using the '''@_file_upload''' tag

==== Checking table values ====
You can check if specific value exists or not in a table row/column by using:
* Then "STRING_IN_ROW" row "COLUMN_HEADER" column of "TABLE_ID" table should contain "VALUE_TO_CHECK"
* Then the following should exist in the "TABLE_ID" table:
| COLUMN_HEADER1 | COLUMN_HEADER2 |
| VALUE_IN_ROW_1 | VALUE_IN_ROW_1 |
| VALUE_IN_ROW_2 | VALUE_IN_ROW_2 |

=== Fixtures ===

As seen in [[https://docs.moodle.org/dev/Acceptance_testing#Examples examples]] Moodle provides a way to quickly set up the contextual data (courses, users, enrolments...) that you need to properly test scenarios, this can be done using one of the site templates (TODO) or creating entities in the background section (common for all the steps) or in the "Given" part of your scenario. Note that this steps can only be used to set up the contextual data required to test the feature but they don't test what they are doing; for example, the "Given the following "users" exists" is not testing that Moodle is able to create a user, but to test that a user can add a blog entry you might want to use this step. For further info, acceptance tests are supposed to be black-boxed tests (the tester don't know about the internals of the application) and this steps are using internal Moodle data generators instead of running all the steps required to create a user or to create a course, which speeds up the test execution. There are other features to test that all this elements can be properly created.

==== Available elements ====
Most of the available elements can only be created in relation to other elements, to hide the complexity of the Moodle internals (references by contexts, ids...) the references can be done using more human-friendly mappings.

The examples below shows how to add elements referencing other elements, there are required fields to reference the elements, other attributes will be filled with random data if they are not specified.

* Course categories
** The required field is idnumber
** References between parent/children by their idnumber, using the "category" field
Given the following "categories" exist:
| name | category | idnumber |
| Category 1 | 0 | CAT1 |
| Category 2 | CAT1 | CAT2 |

* Courses
** The required field is shortname
** Uses the category idnumber as category reference
Given the following "courses" exist:
| fullname | shortname | category | format |
| Course 1 | COURSE1 | CAT1 | topics |
| Course 2 | COURSE2 | CAT2 | |

* Activities *(note that this step does not work with all kind of activities, only the ones that have data generators)*
** The required fields are activity, course and idnumber
** Uses activity to specify the activity type
** Uses the course shortname as course reference
** Other activity-dependant fields can be specified by it's field name
Given the following "activities" exist:
| activity | course | idnumber | name | intro |
| assign | C1 | assign1 | Test assignment name | Test assignment description |
| data | C1 | data1 | Test database name | Test database description |

* Groups
** The required fields are course and idnumber
** Uses the course shortname as course reference
Given the following "groups" exist:
| name | description | course | idnumber |
| Group 1 | Anything | COURSE1 | GROUP1 |

* Groupings
** The required fields are course and idnumber
** Uses the course shortname as course reference
Given the following "groupings" exist:
| name | course | idnumber |
| Grouping 1 | COURSE1 | GROUPING1 |
| Grouping 2 | COURSE1 | GROUPING2 |

* Users
** The required field is username (if password is not set username value will be used as password too)
Given the following "users" exist:
| username | email | firstname | lastname |
| testuser | asd@asd.com | Test | User |

* Course enrolments
** The required fields are user, course and role
** Uses the course shortname as course reference
** Uses the user username as user reference
** Uses the role shortname as role reference
** Uses the enrolment name as enrol reference
Given the following "course enrolments" exist:
| user | course | role | enrol |
| testuser | COURSE1 | editingteacher | manual |

* Roles
** The required field is shortname
** Uses a one of the following roles as archetype: manager, coursecreator, editingteacher, teacher, student, guest, user and frontpage
Given the following "roles" exist:
| shortname | name | description | archetype |
| custom1 | Custom Role 1 | | editingteacher |

* Role assigns
** The required fields are user, role, contextlevel and reference
** Uses the user username as user reference
** Uses the role shortname as role reference
** Uses contextlevel + reference to specify the context. See [[#Referencing_contexts]] for more info.
Given the following "role assigns" exist:
| user | role | contextlevel | reference |
| user1 | manager | System | |
| user2 | editingteacher | Category | CATEGORY1 |
| user3 | editingteacher | Course | COURSE1 |

* System role assigns (deprecated in favour of role assigns, see item right above this one)
** The required fields are user and role
** Uses the user username as user reference
** Uses the role shortname as role reference
Given the following "system role assigns" exist:
| user | role |
| testuser | manager |

* Permission overrides
** The required fields are capability, permissions, role, and the contextlevel + it's reference
** Uses contextlevel + reference to specify the context. See [[#Referencing_contexts]] for more info.
Given the following "permission overrides" exist:
| capability | permission | role | contextlevel | reference |
| mod/forum:editanypost | Allow | student | Course | C1 |
| mod/forum:replynews | Prevent | editingteacher | Course | C1 |
| mod/paquiro:sings | Prohibit | student | System | |

* Group members
** The required fields are user and group
** Uses the group idnumber as group reference
** Uses the user username as user reference
Given the following "group members" exist:
| user | group |
| testuser | GROUP1 |

* Grouping groups
** The required fields are grouping and group
** Uses the group idnumber as group reference
** Uses the grouping idnumber as grouping reference
Given the following "grouping groups" exist:
| grouping | group |
| GROUPING1 | GROUP1 |

* Cohorts
** The required field is idnumber
Given the following "cohorts" exist:
| name | idnumber |
| Cohort 1 | COHORT1 |

==== Referencing contexts ====

Moodle has different context levels, internally they have an identifier, but to reference them from steps we can use a more human way, using the level of the context (as specified below) and the reference, which will depend on the contextlevel we are using:
* contextlevel: '''System''', '''User''', '''Category''', '''Course''' or '''Activity module'''
* reference:
** System: Nothing, just leave the the cell empty
** User: The user '''username'''
** Category: The category '''idnumber'''
** Course: The course '''shortname'''
** Activity module: The activity '''idnumber'''

The data generators which makes use of these format are pointing to here.

=== Features check list ===
* It's a new feature or a new scenario of an existing feature
* Is using the '''Background''' section or '''Scenario Outlines''' instead of duplicating steps (only when applicable)
* Are using the appropriate Moodle component tag according to https://docs.moodle.org/dev/Acceptance_testing#Tests_filters
* The user story of the feature includes a valid stakeholder and makes sense according to https://docs.moodle.org/dev/Acceptance_testing#Writing_features
* Covers both JS and non-JS environments

== Adding steps definitions ==

Each Moodle component and plugin (including 3rd party plugins) can add new steps definitions. If you are writing tests and you notice that you are repeating the same group of steps you might want to create a new step definition that allows you to substitute the group of steps for one single step, something like ''I add a forum post with "blablabla" as description'' for example; also you can create whole new steps using the APIs provided by Behat and Mink if what you need to do is not covered by any of the available steps.

As commented in https://docs.moodle.org/dev/Acceptance_testing#Fixtures, this are black box tests, so we are not supposed to know about Moodle internals; translated to developer language it means don't use Moodle internals API calls, for example you should not try to cheat using a set_config() call, you should follow Moodle's user interface to reach the setting page and change it's value.

=== Example ===

You can use this example below or any of the existing steps definitions as a template.

* auth/tests/behat/behat_auth.php
class behat_auth extends behat_base {
/**
* Logs in the user. There should exist a user with the same value as username and password
*
* This second comment line will be not shown in the steps definitions list as it is only
* reading the first line in the comment block.
*
* @Given /^I log in as "(?P<username_string>(?:[^"]|\\")*)"$/
*/
public function i_log_in_as($username) {
return array(new Given('I am on homepage'),
new Given('I follow "' . get_string('login') . '"'),
new Given('I fill in "' . get_string('username') . '" with "' . $this->escape($username) . '"'),
new Given('I fill in "' . get_string('password') . '" with "'. $this->escape($username) . '"'),
new Given('I press "' . get_string('login') . '"')
);
}
}

=== Tips ===

If you are creating a completely new step definition there are also a few things to consider:
* Steps definitions should be compatible with both Javascript and non-Javascript tests, you can use $this->running_javascript() to deal with both
* The definition code will be executed by Behat, not by Moodle, you have to keep this in mind for example when throwing exceptions, Behat exceptions will give more info to the user about where is the problem
** You can find these exceptions in '''vendor/behat/mink/src/Behat/Mink/Exception/*'''
* Selenium is fast, sometimes it tries to interact with DOM elements or tries to execute actions that requires JS that are not loaded or ready to used; this is why, sometimes and randomly, you can see an "element not found" failure
** The quickest way to solve this problem is using behat_base::find*() methods (where the * corresponds to '''<nowiki>''</nowiki>''', '''_all''', or to a named selector preceded by '''_''', http://mink.behat.org/#named-selectors) which only requires the locator as argument. This methods will wait for the requested element to be ready or return an exception if the element is not found after the timeout value expires, you can also force the timeout value, which defaults to 6 seconds. An example of a named selector use is '''$button = $this->find_button("Save changes");''' if you are not sure about the element being available you always can wrap the find*() call in a try & catch.
** For advanced usages, the spin method is defined in '''lib/behat/behat_base::spin''', consider that all the contents of the closures passed to spin() can be executed more than once, so don't use irreversible actions that can invalidate the tests results (for example use find() methods but don't use click() methods)

If you create new steps definitions or tests you must run '''php admin/tool/behat/cli/util.php --enable''' to update the Behat config file before running '''vendor/bin/behat'''

=== Check list ===

New steps should be/have:
* Implemented as public methods of a PHP class whose name must begin with 'behat_' prefix and with '.php extension
* Using the class name as filename (adding the '.php' extension) and extending MOODLEDIRROOT/lib/behat/behat_base.php (or MOODLEDIRROOT/lib/behat/behat_files.php if it's a repository or is files-related)
* With a descriptive class name, for example the component name (it will be used when filtering steps definitions)
* Stored in COMPONENTNAME/tests/behat/ directory or lib/tests/behat/ if is not part of any other component
* Describe it's purpose in a single line inside the method doc comment, the size of the comment is not a problem
* Describe the regular expression with the most appropriate tag inside the method doc comment:
** '''@Given''' - A step to set up the initial context (for example ''the following "courses" exists'')
** '''@When''' - An action that provokes an event (for example ''I press the button "buttonname"'')
** '''@Then''' - Checkings to ensure the outcomes are the expected (for example ''I should see "whatever"'')
* Depending on the inputs your definition expects you must use a different regular expression:
** '''If you expect a number:''' "(?P<info_about_what_you_expect_number>\d+)" (note that the regular expression is quoted between '''"''')
** '''If you expect a string or a text:''' "(?P<info_about_what_you_expect_string>(?:[^"]|\\")*)" Don't use '''text_selector_string''' and '''selector_string''' as info strings, they are reserved to selector types (note that the regular expression is quoted between '''"''')
** '''If you expect a table with key/value pairs (for example to fill a form):''' Finish your regular expression with ''':''' and provide info in the description about the contents of the table
** '''If you expect a selector type:''' "(?P<selector_string>[^"]*)" or "(?P<text_selector_string>[^"]*)" depending on whether you want to use any selector or you want a text-based selector (more info about selectors in https://docs.moodle.org/dev/Acceptance_testing#Providing_values_to_steps)
* To make test writer's life better is good to include explicative info in the subexpressions of the regular expression about what the test writer is supposed to put in there (for example ''I expand "(?P<nodetext>(?:[^"]|\\")*)" node'')
* Is recommended to use the static part of the regular expression as the name of the method, using underscores instead of spaces (see current steps definitions)

== Troubleshooting ==

=== How can you tell if Selenium is running? ===

Try going to http://localhost:4444/selenium-server/. If Selenium is not running, nothing will happen. You will get a time-out. If selenium is running, you will get a 404 error page saying powered by Jetty at the bottom.

=== How can I stop Selenium? ===

Go to the URL http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer. I think when it works it outputs "OKOK".

=== How to pause behat to check browser html ===

You can use following step to pause behat execution. To resume execution, press Enter/Return key in console.
<pre>And I pause scenario execution</pre>

=== Behat tells me that lots of steps are undefined ===

You probably forgot the --config option in the behat command.

== Browser specific fixes ==

If you are interested in running behat in other browsers you might be interested on https://docs.moodle.org/dev/Acceptance_testing/Browsers

When running acceptance tests in conjunction with Selenium against Chrome, IE, or Safari there is a fix for the navigation bar that gets applied in order to avoid errors arising from a bug in the webdrivers for those browsers.

<b>The issue</b>: When an acceptance test goes to interact with an element on the page it first ensures that the element in the view-port and if not scrolls the browser to get the element into the view-port. If you are running against Chrome, IE, or Safari the browser scrolls the element only just inside the view-port. It doesn't however allow for any fixed position elements such as the navigation bar.
What happens: In some situations these browsers scroll up to reach a button, however not enough as the button ends up behind the navigation bar and cannot be interacted with.

<b>Our solution</b>: We can not change browser driver behaviour so we have integrated a work around. When running acceptance tests against one of these browsers we change the position attribute of the navigation bar from fixed to absolute. This is not ideal as it is not how the user experiences the site, however it allows us to run the full acceptance test suite against these browsers so we allowed it. A notice will be displayed when you start an acceptance test run if the browser specific fixes have been applied. See MDL-47734 / MDL-45231 for more details.

== See also ==

* Guidelines for contributors: [[Acceptance_testing/Contributing_automated_tests|Contributing automated tests]]
* Technical info: [[Behat integration]]
* Behat CLI command options: http://docs.behat.org/guides/6.cli.html
* How to use selectors to interact with the site elements: http://mink.behat.org/#traverse-the-page-selectors
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat

[[Category:Behat]][[Category:Quality Assurance]]

[[es:Prueba de aceptación]]

Common unit test problems

2015-06-26T13:25:56Z

Mackensen: /* core_phpunit_advanced_testcase::test_locale_reset */ add centos example

== General Issues ==

=== Segfaults on Oracle ===

When I was running tests on oracle I was getting phpunit sefaulting.. --[[User:Dan Poltawski|Dan Poltawski]] 00:29, 16 May 2012 (WST)

'''Solution''' Set statement_cache_size in php.ini: <code>oci8.statement_cache_size = 0</code>

(More info: [https://bugs.php.net/bug.php?id=49803 PHP Bug #49803]).

=== Execution ends with "zend_mm_heap corrupted" error ===

This was happening with a standard XAMPP stack, running Moodle against Oracle.

# Verify that you are not using multiple PHP opcode accelerators together (APC, Opcode...). Also try disabling all them (they really don't help much in a phpunit execution).
# Set output_buffering in php.ini to 65536 (from default 4096).

=== Many random errors about "Warning: rmdir dataroot/cache/.../.... Directory not empty" ===

Use a different $CFG->phpunit_dataroot , completely apart from your current [LXMW]AMP stack, for example in windows 'C:\\Windows\\Temp\\dataroot'. For some reason, unknown at the time of writing this, using any directory within the [LXMW]AMP stack lead to randomly busy/locked (aka, non deletable) files.

== Specific failures ==

=== dml_testcase::test_unique_index_collation_trouble===

<code>Unique index is accent insensitive, this may cause problems for non-ascii languages. This is usually caused by accent insensitive default collation.</code>

'''Solution''' These tests should pass if you use the <code>utf8_bin</code> collation on your database for unit tests, but we recommend <code>utf8_unicode_ci</code> for actual live Moodle sites. Work is being done to resolve this issue, read more about the [[Database collation issue]] for further info.

=== dml_testcase::test_sql_binary_equal ===

<code>SQL operator "=" is expected to be case sensitive Failed asserting that 1 matches expected 2.</code>

'''Solution''' These tests should pass if you use the <code>utf8_bin</code> collation on your database for unit tests, but we recommend <code>utf8_unicode_ci</code> for actual live Moodle sites. Work is being done to resolve this issue, read more about the [[Database collation issue]] for further info.

=== grading_manager_testcase::test_tokenize===

<code>A test using UTF-8 characters has failed. Consider updating PHP and PHP's PCRE or INTL extensions (MDL-30494) Failed asserting that false is true.</code>

'''Solution''' These tests should pass if you use the <code>utf8_bin</code> collation on your database for unit tests, but we recommend <code>utf8_unicode_ci</code> for actual live Moodle sites. Work is being done to resolve this issue, read more about the [[Database collation issue]] for further info.

=== moodlesimplepie_testcase::test_getfeed ===

<code>Failed to load the sample RSS file. Please check your proxy settings in Moodle. %s</code>

'''Solution''' Your moodle needs network connectivity, please check proxy settings.

=== moodlesimplepie_testcase::test_redirect ===

<code>Failed asserting that 'cURL Error: Operation timed out after 2000 milliseconds with 0 bytes received' is null.</code>

'''Solution''' Your moodle needs network connectivity, please check proxy settings

=== collatorlib_testcase::test_asort_objects_by_method ===

<code>Collation aware sorting not supported, PHP extension "intl" is not available.</code>

'''Solution''' Install php intl extension.

Note that, on Windows, this extension does not seem to be included in the installer, but you can get it by downloading the zip, and copying the right DLLs across. If you Google, there are more detailed instructions on Stack Exchange.

=== moodlelib_testcase::test_fix_utf8 ===

<code>Failed asserting that false is identical to 'aaabbb'.</code>

'''Solution''' This problem indicates buggy iconv. See [http://tracker.moodle.org/browse/MDL-33007] for discussion. So this is a real problem, to be fixed in a future integration cycle

=== filestoragelib_testcase::test_get_file_preview ===

<code>Failed asserting that false is an instance of class "stored_file".</code>

'''Solution''' Unfortunately, the php GD extension is a requirement for this test (see MDL-36447).

Go to your Moodle directory and type <code>php admin/tool/phpunit/cli/util.php --drop</code>, then reinitialise PHPunit and run the test again.

===core_ddl_testcase::test_row_size_limits===

<code>Row size limit reached in MySQL using InnoDB, configure server to use innodb_file_format=Barracuda and innodb_file_per_table=1</code>

""Solution"" default InnoDB database tables in MySQL cannot handle more than 10 text columns, so innodb file format should be Barracuda (see MDL-46971)

Running <code>php admin/cli/mysql_compressed_rows.php</code> will solve the problem.

=== core_phpunit_advanced_testcase::test_locale_reset ===

Failed asserting that two strings are identical. Since 2.9, en_AU.UTF-8 locale is required to run phpunit.

'''Solution''' Install the AU locale on your server if you want to run phpunit. Here are sample commands for installing the en_AU locale:

* Centos: "sudo localedef -v -c -i en_AU -f UTF-8 en_AU.UTF-8"
* Ubuntu: "sudo locale-gen en_AU.UTF-8"

=== MSSQL failures with core_course_externallib_testcase===

On the MSSQL database driver, some problems occur with core_course_externallib_testcase, for example:

core_course_externallib_testcase::test_duplicate_course

unserialize(): Error at offset 24191 of 24192 bytes

'''Solution''' Ensure that the correct configuration is set in freedts.conf (from https://docs.moodle.org/26/en/FreeTDS), e.g.

<pre>text size = 20971520
client charset = UTF8</pre>

=== PHP Warning: include_once(PHPUnit/Extensions/Database/Autoload.php): failed to open stream ===

This occurs on Mac installs from experience. The reason this occurs is because DBUnit was not installed during the PHPUnit installation via PEAR.

'''Solution''' In terminal type: <code>sudo pear install -f phpunit/DbUnit</code>

=== !!! Error reading from database !!!!! Unicode data in a Unicode-only collation or ntext data cannot be sent to clients using DB-Library (such as ISQL) or ODBC version 3.7 or earlier. ===

If running unit tests under FreeTDS, remember that without a valid FREETDS environment variable, PHP won't be able to find the freetds.conf file and will default to version 5.0 which has poor support for unicode collations.

'''Solution''' Ensure a FREETDS environment variable is set prior to running the automated tests, which points to a valid path that contains a freetds.conf file. Also ensure the tds version element is set to a recent enough version (8.0)

'''Solution2''' Under windows you can put the freetds.conf file in the disk root directory where php is being executed (for example C:\freetds.conf). That directory is always looked. Also ensure the tds version element is set to a recent enough version (8.0)

Note: Another potential solution may be to set TDSVER (http://freetds.schemamania.org/userguide/freetdsconf.htm)

[[Category:Unit testing]]

=== Errors running unit tests against Oracle, related with locales, decimal points and friends ===

Surely your PHP CLI (the one executing phpunit) is not aware of the expected Oracle NLS configuration. To enforce it, set at least these 2 env variables in your system (globally or for the current user):

- NLS_LANG to AMERICAN_AMERICA.AL32UTF8
- NLS_NUMERIC_CHARACTERS to ., (dot & comma)