MoodleDocs - User contributions [en]

PHPUnit integration

2022-05-24T10:58:30Z

Bencellis: /* Extra methods */

Moodle PHPUnit integration was created to simplify using of PHPUnit framework in Moodle. It consists of specialised bootstrap script, utility scripts that initialise testing environment and highly optimised custom test case classes that handle automatic global state resetting after test that includes global variables, database rollback and purging of dataroot. We also try to bridge the gap between the design and coding style of Moodle and PHPUnit.

Most of the documentation at http://www.phpunit.de/manual/3.6/en/index.html is relevant to Moodle PHPUnit integration, exceptions and additions are described below.
=Definitions=
These definitions may differ in each testing framework or programming language. The definitions here should be valid for PHPUnit framework with our Moodle tweaks.
;Test suite: is a collection of test cases or other suites usually related to one area of the product. It is defined in PHPUnit configuration files (phpunit.xml by default).

;Test file: is a file with *_test.php name which contains a test case. (*_Test.php in PHPUnit)

;Test case: is a class with test methods. Their name should match the name of the test file (*_test). Moodle tests extend basic_testcase or advanced_testcase. (PHPUnit\Framework\TestCase in PHPUnit)

;Test: is a public method starting with "test_" prefix defined in test case class. It is the implementation of the test procedure.

;Assertion: is the actual comparison of expected and actual behaviour of the tested code.
=Organisation of test files=
All testing related files are stored in <syntaxhighlight lang="php">/tests/</syntaxhighlight> subdirectories:
* <syntaxhighlight lang="php">.../tests/*_test.php</syntaxhighlight> are PHPUnit test files
* <syntaxhighlight lang="php">.../tests/fixtures/</syntaxhighlight> contains auxiliary files used in tests
* <syntaxhighlight lang="php">.../tests/performance/</syntaxhighlight> is reserved for performance testing scripts
* <syntaxhighlight lang="php">.../tests/generator/</syntaxhighlight> for generators
* <syntaxhighlight lang="php">.../tests/other/</syntaxhighlight> is used for everything else that does not fit
==Class and file naming rules==
=== Actual (Moodle 3.11 and up) ===
The class names of all testcases need to be unique, with namespaces being used to guarantee this.
* All test case class names should match the file name, for example: the class name for <tt>mod/forum/test/this_test.php</tt> should be <tt>this_test</tt>.
* All test case classes end with '''_test''' suffix.
* All test case classes should use the namespace they belong to, for example: the namespace for <tt>mod/forum/test/this_test.php</tt> should be <tt>mod_forum</tt>. With sub-namespaces (and corresponding sub-directories) allowed to better match what is being tested.
Read about the rules for namespaces @ [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|coding style]] (grand summary = 100% the same rules that are applied to **/classes directories).

Example:
* class '''lib_test''' (with namespace <tt>mod_forum</tt>) is expected in file '''/mod/forum/tests/lib_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit mod/forum/tests/lib_test.php</syntaxhighlight>
* class '''text_test''' (with namespace <tt>core</tt>) is expected in file '''/lib/texts/text_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit lib/texts/text_test.php</syntaxhighlight>
* class '''provider_test''' (with namespace <tt>block_comments\privacy</tt>) is expected in file '''/blocks/comments/tests/privacy/provider_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit vendor/bin/phpunit blocks/comments/tests/privacy/provider_test.php</syntaxhighlight>
=== Previous (until Moodle 3.10) ===
Frankenstyle prefix is used to guarantee this. Since 2.6 it is possible to use automatic class loader when executing individual unit tests.
* All test case classes start with Frankenstyle prefix, for example: '''mod_forum_''', '''block_html_''', '''core_'''.
* All test case classes end with '''_testcase''' suffix.
* File names are constructed from the class names - the Frankenstyle prefix is removed, suffix '''_testcase''' is replaced with '''_test'''
Finally, but not less important, test case classes can be (recommended) namespaces. Read about the rules to do it properly @ [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|coding style]] (grand summary = 100% the same rules that are applied to **/classes directories).

Example:
* class '''mod_forum_lib_testcase''' is expected in file '''/mod/forum/tests/lib_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit mod_forum_lib_testcase</syntaxhighlight>
* class '''core_text_testcase''' is expected in file '''/lib/texts/text_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit core_text_testcase</syntaxhighlight>
=basic_testcase=
''basic_testcase'' is useful for simple tests that do not modify database or global variables. If something accidentally changes global state the test fails. This test case class is nearly identical to PHPUnit_Framework_TestCase used in PHPUnit documentation.
<syntaxhighlight lang="php">
<?php // File: mod/myplugin/tests/sample_test.php

namespace mod_myplugin;

class sample_test extends \basic_testcase {
public function test_equals() {
$a = 1 + 2;
$this->assertEquals(3, $a);
}
}
</syntaxhighlight>
=advanced_testcase=
By default each test starts with fresh new moodle installation. Test may modify database content, files or global variables. It is possible to use data generators to create new course, categories, module instances and other objects, alternatively table data can be preloaded from XML or CSV files.
<syntaxhighlight lang="php">
<?php // File: mod/myplugin/tests/complex_test.php

namespace mod_myplugin;

class complex_test extends \advanced_testcase {
public function test_isadmin() {
global $DB;

$this->resetAfterTest(true); // reset all changes automatically after this test

$this->assertFalse(is_siteadmin()); // by default no user is logged-in
$this->setUser(2); // switch $USER
$this->assertTrue(is_siteadmin()); // admin is logged-in now

$DB->delete_records('user', array()); // lets do something crazy

$this->resetAllData(); // that was not a good idea, let's go back
$this->assertTrue($admin = $DB->record_exists('user', array('id'=>2)));
$this->assertFalse(is_siteadmin());
}
}
</syntaxhighlight>
==Extra methods==
; resetAfterTest(bool) :true means reset automatically after test, false means keep changes to next test method, null means detect changes - the default is true.
; resetAllData() : reset global state in the middle of a test
; setAdminUser() : set current $USER as admin
; setGuestUser() : set current $USER as guest
; setUser() : set current $USER to a specific user - use getDataGenerator() to create one
; getDataGenerator() : returns data generator instance - use if you need to add new courses, users, etc.
; preventResetByRollback() : terminates active transactions, useful only when test contains own database transaction handling
; createXXXDataSet() : creates in memory structure of database table contents, used in loadDataSet() (eg: createXMLDataSet(), createCsvDataSet(), createFlatXMLDataSet())
; loadDataSet() : bulk loading of table contents
; getDebuggingMessages() : Return debugging messages from the current test. (Moodle 2.4 and upwards)
; resetDebugging() : Clear all previous debugging messages in current test. (Moodle 2.4 and upwards)
; assertDebuggingCalled() : Assert that exactly debugging was just called once. (Moodle 2.4 and upwards)
; assertDebuggingNotCalled() : Assert no debugging happened. (Moodle 2.4 and upwards)
; assertDebuggingCalledCount() : Asserts how many times debugging has been called. (Moodle 3.1 and upwards)
; [[Writing PHPUnit tests#Testing sending of messages|redirectMessages()]]: Captures ongoing messages for later testing (Moodle 2.4 and upwards)
; [[Writing PHPUnit tests#Testing_sending_of_emails|redirectEmails()]]: Captures ongoing emails for later testing (Moodle 2.6 and upwards)
==Restrictions==
* it is not possible to modify database structure such as create new table or drop columns from advanced_testcase.
=Moodle specific features=
* detection of global state changes - helps with detection of unintended changes in database
* highly optimised global state reset
* dataset loading - this feature is copied from PHPUnit database testcases
* automatic generation of phpunit.xml - init script builds list of plugin testcases
* database driver testing class - used for functional DB tests
* SimpleTest emulation class - helps with migration of old tests
* debugging() interception - enables to control and test any expected debug output. (Moodle 2.4 and upwards)
=Limitations=
* no Selenium support
* no support for PHPUnit_Extensions_Database_TestCase - it is possible to use data set loader only
=See also=
* [[Writing PHPUnit tests]]
[[Category:Unit testing]]

Behat integration

2022-04-26T14:29:37Z

Bencellis: /* change link to tools page */

== Introduction ==
This page describes the internals of Behat and the integration with Moodle. For the functional description, how to install it, run the suite and add more features, see [[Acceptance_testing]].

Behat is a framework for behavior driven development (BDD) which allows us to specify Moodle functionalities (aka features) as a human-readable list of steps. It parses these steps to executable actions to simulate user interaction against headless browsers (without java script support, only curl-kind requests) or user simulation tools like Selenium, which interacts with browsers and allows JavaScript events simulation.
== Objective ==
The aim of this integration is to allow Moodle components to have its own set of features and step definitions. This allows us to periodically execute sets of tests to detect regressions and Moodle features in different environments (browsers, DBs engines, web servers...). The Moodle QA tests will be progressively rewritten according to this format to run automatically.
== How Behat works ==
This section aims to explain the basics about BDD and Behat and a quick view of how Behat internally works from the CLI command execution to the results output.

'''Note that all those examples are not necessarily real nor part of the suite, they are meant to help understand the concepts, see [[Acceptance_testing]] for real examples'''

Some terms used:
* '''Features''': Human-readable list of scenarios that describes a feature
@auth
'''Feature''': Login
In order to login
As a moodle user
I need to be able to validate the username and password against moodle

'''Scenario''': Login as an existing user
Given I am on "login/index.php"
When I fill in "username" with "admin"
And I fill in "password" with "admin"
And I press "loginbtn"
Then I should see "Moodle 101: Course Name"

'''Scenario''': Login as an non-existing user
Given I am on "login/index.php"
When I fill in "username" with "admin"
And I fill in "password" with "admin"
And I press "loginbtn"
Then I should see "Invalid login"
* '''Scenario''': Human-readable list of steps to describe an expected behaviour
'''Scenario''': Login as an existing user
Given I am on "login/index.php"
When I fill in "username" with "admin"
And I fill in "password" with "admin"
And I press "loginbtn"
Then I should see "Moodle 101: Course Name"
* '''Steps''': Human-readable sentences that describes an action. There are 3 types of steps, "Given" describing the initial context, "When" the event that provokes a change and "Then" where the outcomes should be asserted.
I click on the "Add user" button
* '''Steps definitions''': PHP methods referenced by steps when matching it's regular expression. The @Given, @When and @Then tags are descriptive and they are not taken into account when matching steps with steps definitions. The regular expressions placeholders are returned to the PHP method as arguments so methods can use them to tell the browser which button (for example) they want to click.
/**
* @When /^I click on the "(.*)" button$/
*/
public function i_click_on_the_button($button) {
// Simulates the user interaction (see Mink description below for more info)
$this->getSession()->getPage()->pressButton($button);
}

* '''Behat''': PHP framework and CLI application that wraps the whole process of features files loading + features files parsing + execution of actions in the browser + results output (http://behat.org/)
* '''Gherkin''': Human-readable language used to define features that can be parsed and translated into PHP methods. For more info, it's the same language used by Cucumber, the BDD Ruby framework (https://github.com/cucumber/cucumber/wiki/Gherkin)
* '''Context''': In Behat scope a context is a PHP class that groups steps definitions (as methods)
* '''Mink''': Is the component which interacts with browsers, simulating a real user interaction. It allows us to write PHP code (or use the available PHP methods) to send requests to the different browsers APIs through a common interface or extend it to allow browser-specific actions. The supported browsers includes Selenium, Selenium2, Sahi... http://mink.behat.org/
* '''Selenium 2''': Web browser automation tool, applications like Mink can communicate with it through a RESTful API (http://code.google.com/p/selenium/wiki/JsonWireProtocol) to execute actions simulating user interaction.
* '''Selector type''': Related with '''locator''', is a way to select a node inside the page DOM, more info in https://docs.moodle.org/dev/Acceptance_testing#Providing_values_to_steps
* '''Locator''': Is what we are looking for inside the page DOM, it completely depends on the associated selector type, a few examples of it:
** Selector type = "link", Locator = "Link text"
** Selector type = "field", Locator = "Field legend text"
** Selector type = "css", Locator = ".css-class #id"
** Selector type = "xpath", Locator = "//input[@id='id-value']"
All this components are written in PHP, open sourced and packaged in a single and extensible framework.
=== Quick view of the whole process ===
# Behat CLI execution
#* Behat application initialization and loading of arguments (features files to execute, output format...)
#* Reads the Behat config file (browser servers are specified here)
#* Extensions overrides management
#* Gherkin initialization
# Features files selection
#* According to the arguments Gherkin looks for .features files
#** It can use different features loaders (single file, a directory, the default directory...)
#** The framework can be extended to allow multiple folders loading
# Features parsing (Gherkin)
#* Loops through the loaded features files looking for scenarios
#* Gets the list of steps of each scenario
#* There are hooks at different levels (https://docs.behat.org/en/latest/user_guide/context/hooks.html)
# Steps parsing (Gherkin)
#* Gherkin looks in the available steps definitions for a regular expression that matches the step text
# Step definition execution
#* The step definition code is executed
#* Steps definitions most of the time uses the Mink component to communicate with the browser API sending requests like "click on that button" or "go to XXX page"
# Scenario outcomes
#* The scenario counts as failed if an exception is thrown when executing a step definition (for example trying to click a non-existing button)
#* The scenario counts as passed if no exception is thrown during it's steps execution
# Finishing CLI execution
#* A summary with all the scenario results is displayed
#* It accepts different output formats (like JUnitXML) to it's execution in continuous integration systems (http://docs.behat.org/guides/6.cli.html#format-options)
== Moodle integration ==
It follows the approach chosen with PHPUnit:
* It comes disabled by default, Behat is not included within Moodle and it has to be installed separately with the composer installer
* Moodle components (subsystems and plugins) can have a tests/behat/ folder
* The scenarios are executed in a test environment, the production database and dataroot are not affected by the tests modifications
* The scenarios specifies their own fixtures and it's execution is isolated from other scenarios and features, resetting the test database and the test dataroot before each scenario
* Moodle lists the features files and steps definitions of it's components in a behat.yml file, similar to the phpunit.xml manifest
* A basic behat.yml.dist config file has been included
=== Alternative environment ===
Acceptance testing implies interaction with the browser like real users does, so it requires the site to be accessible via URL. The Moodle integration creates a new moodle site installation in parallel to the production one to run the tests in a sandbox without affecting the production environment, switching the regular $CFG->wwwroot, $CFG->dataroot and $CFG->prefix to alternatives, which should be only accessible from localhost or internal networks. Info about how to run the tests in https://docs.moodle.org/dev/Acceptance_testing#Running_tests.

This default configuration is useful when developing in a local host, but to run the tests automatically with Jenkins, travis, other CI systems or with saucelabs we provide a few extra settings. More info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage

All the behat CLI utilities we provide within the Moodle codebase (admin/tool/behat/cli/*) are using $CFG->behat_wwwroot, $CFG->behat_prefix and $CFG->behat_dataroot instead of $CFG->wwwroot, $CFG->prefix and $CFG->dataroot, this scripts are self-contained, but as we are accessing through a browser, we also need to switch the whole Moodle instance to test mode. For this there are two requirements:
* Test mode is enabled if
** '''php admin/tool/behat/cli/init.php''' or '''php admin/tool/behat/cli/util.php --enable''' has been executed
* Test mode is requested if
** The vendor/bin/behat command is running, we know it because we hook the Behat process before the tests begins to run and we require moodle config.php after it
** We set $CFG->behat_wwwroot in config.php and we are accessing the moodle instance through it
The unique $CFG->behat_wwwroot prevents unintended execution of acceptance tests on production sites.
=== Javascript ===
There are two types of tests depending on if their scenario needs a real browser capable of execute Javascript or if they can run in a headless browser.
* Tests with Javascript requires interaction with a browser through a user simulation tool like Selenium or ZombieJS to be executed; see http://mink.behat.org/#different-browsers-drivers for all available drivers
* Test that does not requires Javascript are faster to run but can not test rich applications like Moodle
In most of the cases a Javascript test would be more appropriate because most of the users uses Javascript-capable browsers, non-Javascript tests can be useful to ensure that Moodle maintains its functionality without Javascript enabled and to ensure there are no big issues, regressions or exceptions in general.
=== Admin tool "Acceptance testing" ===
There is an admin tool to run and ease the creation of acceptance tests.
* Web interface: The web interface allows you to list and filter the available steps definitions, a non-technical user can use this interface to write new features (admin/tool/behat/index.php)
* CLI: Command to enable and disable the test environment and to update the behat.yml file with the system tests and steps definitions (admin/tool/behat/cli/util.php and admin/tool/behat/cli/init.php for a quick start)
[[File:Acceptance_testing_UI_2.5.png]]
=== Available steps to create tests ===
There are behat libraries with tons of steps definitions to run all sort of processes and interactions with the browser, some of them overlaps Moodle-specific libraries and tests writers can be confused not only by this also by the amount of steps and vague or too technical steps descriptions. Moodle provides a set of steps definitions written in a common format to make tests writers life easier. New steps definitions must follow this guidelines: https://docs.moodle.org/dev/Acceptance_testing#Adding_steps_definitions
=== Behat extension ===
A new Behat extension (https://github.com/moodlehq/moodle-behat-extension) has been created to maintain Behat and its dependencies as they comes from upstream.

The aim of this extension is:
* Load features from multiple folders (Moodle subsystems and plugins)
* Load steps definitions from multiple folders and add them as subcontexts (Moodle subsystems and plugins)
* Return the available steps definitions in a more human-readable format without regexps
* Look for exceptions, debugging() calls, PHP error messages and other backtraces in Moodle's output
* Extend the Selenium2 behat driver to allow extra Selenium capabilities
* Add a new formatter method based on progress (the moodle default one) to display info about the moodle site being tested
All the other particularities of this integration can managed playing with different Behat config parameters.
[[Category:Behat]]
[[es:Behat]]

Running acceptance test

2021-12-15T15:54:52Z

Bencellis: /* Setting up Selenium */ Addition of later version and issue with version 4 jars.

Moodle uses [https://behat.org Behat], a php framework for automated functional testing, as part of a suite of testing tools.

Behat takes a set of Features, Scenarios, and Steps, and uses these to step through actions and test results using a real web browser. This is possible thanks to a protocol implemented in most modern web browsers called Webdriver.

This documentation covers how to run Behat tests within Moodle, including requirements, setup, useful tips and tricks, and basic troubleshooting.
== Requirements ==
# Any recent OS with [[Releases|supported version of Moodle]] installed
# A recent browser (We support Firefox, Chrome as standard, but other browsers are possible)
# The WebDriver implementation for your browser
# A recent version of Selenium (Optional, but recommended)
# A recent Java Runtime Environment (Required if using Selenium)
=== Recommended extras ===
Some extra software is also recommended for testing with Behat.
==== Pre-configured browser profiles: moodle-browser-config ====
Available for [[Releases|all supported versions of Moodle]], the [https://github.com/andrewnicols/moodle-browser-config moodle-browser-config] is a recommended inclusion for Behat. This configuration tooling provides a range of standard browser profiles for testing.

Note: In future, the <tt>moodle-browser-config</tt> tool may be included as composer dependency to Moodle but this is currently not the case.
===== Installation =====
# Check out the moodle-browser-config repository:
// Change directory to your git root, for example:
$ cd ~/git
// Clone the moodle-browser-config repository
git clone https://github.com/andrewnicols/moodle-browser-config
# Open your Moodle installation's <tt>config.php</tt> in your preferred editor, and require the tool's init.php:
require_once('/path/to/your/git/dir/moodle-browser-config/init.php');
===== Provided profiles =====
The full list of profiles which are included with <tt>moodle-browser-config</tt> are provided in its [https://github.com/andrewnicols/moodle-browser-config own documentation].

The following is a summary of the profiles that most users may be interested in.

You can also provide your own custom profiles, including for remote services such as Browserstack, and Saucelabs, as
well as for other browsers supporting the W3C Webdriver specification.

Please note that <tt>Safari</tt> and <tt>Safaridriver</tt> are not currently supported as they do not meet the W3C WebDriver specification.
{| class="wikitable" border="1"
|-
! Profile name
! Description
! Uses Selenium?
! Displays GUI?
|-
| firefox
| Use Firefox via Selenium
| Yes
| Yes
|-
| headlessfirefox
| Use Firefox via Selenium, without displaying the GUI
| Yes
| No
|-
| geckodriver
| Use Firefox with Geckodriver directly
| No
| Yes
|-
| headlessgeckodriver
| Use Firefox with Geckodriver directly, without displaying the GUI
| No
| No
|-
| chrome
| Use Chrome via Selenium
| Yes
| Yes
|-
| headlesschrome
| Use Chrome via Selenium, without displaying the GUI
| Yes
| No
|-
| chromedriver
| Use Chrome with Chromedriver directly
| No
| Yes
|-
| headlesschromedriver
| Use Chrome with Chromedriver directly, without displaying the GUI
| No
| No
|-
| edge
| Use Edge via Selenium
| Yes
| Yes
|-
| headlessedge
| Use Edge via Selenium, without displaying the GUI
| Yes
| No
|-
| edgedriver
| Use Edge with Edgedriver directly
| No
| Yes
|-
| headlessedgedriver
| Use Edge with Edgedriver directly, without displaying the GUI
| No
| No
|}
==== chromedriver-wrapper ====
When using Google Chrome, you must use the correct version of the <tt>chromedriver</tt> browser driver for the version of Chrome that you use.

Since Google Chrome automatically updates on a regular basis, you will need to regularly upgrade your driver to match the version of Chrome that you are using.

To make this easier, a <tt>chromedriver-wrapper</tt> utility has been written. This inspects the version of Chrome that is in your path, and downloads the correct version of <tt>chromedriver</tt> for that version, before starting it.

Installation instructinos can be found at [https://github.com/andrewnicols/chromedriver-wrapper https://github.com/andrewnicols/chromedriver-wrapper].
== Getting started (basic) ==
This is a quick walk through to get Behat running for the first time.
=== Setting up ===
There are a number of ways of configuring Behat on Moodle. This is one of the simplest.

These notes assume that you have already installed a supported Java Runtime Environment, and the [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config]] tool.
==== Setting up Selenium ====
Generally we recommend use of Selenium, though this is not a fixed requirement. You can use the browser's driver implementation directly but this is harder to setup for the first time.

Selenium is written in Java, and requires a recent version of the JRE to run. Please ensure that you have this installed prior to starting.

Since Moodle 3.9.5 / 3.10.2 / 3.11.0 Moodle will work with any modern version of Selenium. At time of writing that is <tt>3.141.59</tt>, and 4.1.0. '''Note:''' There appears to be an issue with the version 4 jars - particually in 4.1.0 that generates a '''"Could not open connection: Capability value must be set”''' error which appears to be an issue between Selenium and the MInkExtension for JavaScript tests - see https://githubmemory.com/repo/Behat/MinkExtension/activity for more information. The quick work around is to use the earlier version (3.141.59).
# Download the Selenium Server (Grid) from [https://www.selenium.dev/downloads/ https://www.selenium.dev/downloads/]. This is a single JAR file, put it anywhere handy.
# Start Selenium
# Version 3.141.59:
$ java -jar selenium-server-standalone-3.141.59.jar

# Version 4.0.0 and later:
$ java -jar selenium-server-4.0.0-beta-3.jar standalone
You can optionally specify a number of settings, depending on the version of Selenium that you are using, including the port to run on.
See the help for the version of Selenium that you are using.
==== Setting up your browsers ====
Selenium is just an intelligient wrapper to start, and manage your browser sessions. It doesn't actually include any web browsers itself.

Moodle HQ run all behat tests against both Firefox and Chrome multiple times per day. Other combinations, including Microsoft Edge, are also supported.

To use Behat, you will need a recent version of your preferred browser, as well as a <tt>driver</tt> for that browser. The driver is responsible for communication between Selenium (or Moodle directly) and the browser.

Both the browser, and its driver, must be placed inside your <tt>$PATH</tt> - this may be somewhere like <tt>/usr/bin</tt>, <tt>/usr/local/bin</tt>, or perhaps a user bin directory like <tt>~/bin</tt> which is present in your <tt>$PATH</tt>
===== Chrome =====
You can download Google Chrome from [https://www.google.com.au/chrome https://www.google.com.au/chrome].

You will need the [https://chromedriver.chromium.org/downloads correct version of the chromedriver] as per the
documentation. Alternatively you can make use of the [[Running_acceptance_test#chromedriver-wrapper|chromedriver-wrapper utility]] noted in the Recommended extras sections.

Either the <tt>chromedriver</tt> binary must be in a directory in your <tt>$PATH</tt>, or the <tt>chromedriver-wrapper/bin</tt> folder must be in your <tt>$PATH</tt>.
===== Firefox =====
You can download Mozilla Firefox from [https://www.mozilla.org/en-US/firefox/new/ https://www.mozilla.org/en-US/firefox/new/].

You will need the [https://github.com/mozilla/geckodriver/releases correct version of geckodriver as per the [https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html documentation].

The <tt>geckodriver</tt> binary must be in a directory in your <tt>$PATH</tt>.
==== Set up Moodle ====
# Create a new 'dataroot' area for files especially for behat
# Set the following in your Moodle <tt>config.php</tt>:
$CFG->behat_dataroot = '/path/to/the/dataroot/you/created';
$CFG->behat_wwwroot = 'http://127.0.0.1/path/to/your/site';
$CFG->behat_prefix = 'beh_';
# We recommend that you also include the <tt>behat-browser-config</tt> if you have not done so already.
require_once('/path/to/moodle-browser-config/init.php');
===== Notes about the behat_wwwroot =====
You will need to set the <tt>behat_wwwroot</tt> to your Moodle site, but it ''must'' use a different value to your <tt>$CFG->wwwroot</tt>.

One common way to do this is to use <tt>127.0.0.1</tt> for behat, but <tt>localhost</tt> for standard use. Alternatively you can add an additional hostname in your <tt>/etc/hosts</tt> file and use this instead.

If you use Docker, then you may be able to use <tt>host.docker.internal</tt> where your site is hosted on the docker host
==== Configure Behat for Moodle ====
After setting your configuration, you can simply initialise behat:
$ php admin/tool/behat/cli/init.php
This will install all required Composer dependencies, install a new Moodle site, and put configuration in place

When it finishes it will give advice on how to run Behat.
==== Run Behat tests ====
Before running behat, ensure that your Selenium server is running.

The easiest way to run behat, and test that everything works is by simply running it using the command provided when you
initialised Behat. If you didn't make a note of it, you can just run the initialisation again.
$ php admin/tool/behat/cli/init.php
This will give you a command which you can then run. This command will run every behat scenario, which will take a considerable amount of time. This command will look a bit like this:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml
To make this more useful you an combine it with flags, for example to only run certain <tt>tags</tt> or for a specific Behat Feature file, or Scenario.

To run all features/scenarios which are tagged with <tt>mod_forum</tt>:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --tags=@mod_forum
To run one specific feature file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature
To run one specific scenario within a feature file:
# To run the Scenario on line 38 of the file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature:38
To run one specific scenario by name:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --name="As a teacher I can see my own response"
See the upstream documentation on Behat, and Gherkin filters for more information.
===== Running using a different browser =====
The default browser in Behat is <tt>Firefox</tt>. To specify a different browser profile, you can add the <tt>--profile</tt> argument. For example, to use Chrome in Headless mode:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
If you are using the <tt>moodle-browser-config</tt> utility, then you can use any profile listed in [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config|moodle-browser-config]]. Otherwise you can write your own browser profile configuration.
=== Advanced testing ===
==== Run tests without Selenium (chromedriver, geckodriver) ====
Historically, Behat required Selenium server, however browsers now make use of their own automation layer. For example, Firefox uses <tt>Geckodriver</tt> and Chrome uses <tt>Chromedriver</tt>. As a result the use of Selenium itself is now optional.

The moodle-browser-config tool includes standard profiles to use these drivers directly and without the use of Selenium.

To use the drivers directly, you must run the driver itself, for example to run <tt>chromedriver</tt>:
$ chromedriver
To run <tt>geckodriver</tt>:
$ geckodriver
Note: geckodriver runs on port 4444 by default. You cannot geckodriver at the same time as selenium.

After starting your preferred browser, you can then run behat and specify an alternative profile, for example:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=geckodriver
==== Headless browsers ====
There are a number of reasons that you may prefer to use a headless browser. It can be particularly helpful if you are running the tests on a remote system, for example over SSH, or if you do not want to be interrupted by browsers popping up on your machine.

The following headless profiles are some of those provided in the moodle-browser-config tool as standard:
# <tt>headlessfirefox</tt> Use Firefox via Selenium, without displaying the GUI
# <tt>headlessgeckodriver</tt> Use Firefox with Geckodriver directly, without displaying the GUI
# <tt>headlesschrome</tt> Use Chrome via Selenium, without displaying the GUI
# <tt>headlesschromedriver</tt> Use Chrome with Chromedriver directly, without displaying the GUI
These can be provided to the <tt>--profile</tt> argument to behat:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
==== Parallel runs ====
Out-of-the-box, Moodle will configure Behat to run a single Moodle installation with all tests run in series. This is great for developer use where you are running a single test. or a small suite of tests. However this can be quite slow. A lot of time is spent waiting in Behat for things to happen. This may be for a page to load, for additional content to load, or even explicit waits because some interactions must be deliberately slowed down. As a result, a system running behat will not have a particularly high load most of the time.

If you want to run a large suite of tests then it is possible to take advantage of the relatively low resource consumption by running several behat runners in parallel. This is commonly referred to as a '''Parallel run'''.

A parallel run of behat takes the same codebase and creates several installations rather than just a single Moodle installation. The behat Feature files are then grouped and allocated between each of the separate installations.

To support this, each of the parallels runs needs its own:
* <tt>behat_wwwroot</tt>
* <tt>behat_dataroot</tt>
* database
Rather than using an entirely separate database, the same database is actually used, but a different <tt>behat_prefix</tt> is used to prefix the table names in the database differently.
===== Installation =====
The Behat initialisation command is responsible for preparing Moodle to run a standard run. You'll have used this before when installing for a standard run:
$ php admin/tool/behat/cli/init.php
The same command can be used to prepare Moodle for a parallel run by specifying the <tt>--parallel</tt> or <tt>-j</tt> parameter:
// Below command will initialise moodle to run 3 parallel tests.
$ php admin/tool/behat/cli/init.php --parallel=3
This can be combined with the <tt>--add-core-features-to-theme</tt> or <tt>-a</tt> flag to prepare Behat to run with all installed themes.

A number of advanced options are also available but you are unlikely to need these:
# <tt>-m=<number></tt> or <tt>--maxruns=<number></tt> Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# <tt>--fromrun=<number></tt> Initialise site to run specified run from. Used for running acceptance tests on different vms
# <tt>--torun=<number></tt> Initialise site to run specified run till. Used for running acceptance tests on different vms
# <tt>-o</tt> or <tt>--optimize-runs</tt> This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
You can view details of all of these using the <tt>--help</tt> flag to <tt>admin/tool/behat/cli/init.php</tt>
===== Running Parallel tests =====
You can use the Moodle behat runner to run all tests, including Standard runs. It is an intelligient wrapper around the standard <tt>./vendor/bin/behat</tt> command which specifies the configuration file, and other required features.
$ php admin/tool/behat/cli/run.php
Many of the standard options and parameters that can be passed to <tt>./vendor/bin/behat</tt> can also be passed to the Moodle runner, for example:
# <tt>--tags</tt> Run tests which match the specified tags
# <tt>--name="Scenario name"</tt> Run a test matching the supplied scenario name
# <tt>--feature="/path/to/test.feature"</tt> Run a specific feature file.
# <tt>--suite</tt> Features for specified theme will be executed.
The runner also includes a number of custom parameters relating to parallel runs:
# <tt>--replace</tt> Replace args string with run process number, useful for output and reruns.
# <tt>--fromrun</tt> Execute run starting from (Used for parallel runs on different vms)
# <tt>--torun</tt> Execute run till (Used for parallel runs on different vms)
The <tt>--replace</tt> feature is particularly useful and can be used to replace a string in the command with the run number. This is useful when specifying output formats, and rerun files as noted below.

The following example demonstrates how Behat might be initialised with three parallel runs, to run on all themes:
$ php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme
And then to run all tests matching the <tt>@tool_myplugin</tt> tag, against the <tt>classic</tt> theme:
$ php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="classic"
===== Custom parameters for parallel runs =====
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
$CFG->behat_parallel_run = [
[
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
],
// ...
],
To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
$CFG->behat_parallel_run = [
['wd_host' => 'http://127.0.0.1:4444/wd/hub'],
['wd_host' => 'http://127.0.0.1:4445/wd/hub'],
['wd_host' => 'http://127.0.0.1:4446/wd/hub'],
];
==== Tests filters ====
With the <tt>--tags</tt> or the <tt>-name</tt> Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* <tt>@javascript</tt>: All the tests that runs in a browser using Javascript; they require Selenium or the browser's own automation layer, as per [[Running acceptance test#Run%20tests%20without%20Selenium%20.28chromedriver.2C%20geckodriver.29|Running acceptance test#Run tests without Selenium .28chromedriver.2C geckodriver.29,]] to be running, otherwise an exception will be thrown.
* <tt>@_file_upload</tt>: All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* <tt>@_alert</tt>: All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* <tt>@_switch_window</tt>: All the tests that are using the <tt>I switch to "NAME" window</tt> step should be tagged as not all browsers manage them properly.
* <tt>@_switch_iframe</tt>: All the tests that are using the <tt>I switch to "NAME" iframe</tt> steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* <tt>@_cross_browser</tt>: All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* <tt>@componentname</tt>: Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.
==== Output formats ====
Behat is able to output in a number of different formats, and to different locations as required.

This can be achieved by specifying the <tt>--format</tt>, and <tt>--out</tt> parameters when running behat, for example:
// Run behat, using the 'pretty' format and outputting the value to /tmp/pretty.txt
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt
It is also possible to output to multiple formats simultaneously by repeating the arguments, for example:

Since Moodle 3.1 option for output is:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std
The following output formats are supported:
# <tt>progress</tt>: Prints one character per step.
# <tt>pretty</tt>: Prints the feature as is.
# <tt>junit</tt>: Outputs the failures in JUnit compatible files.
# <tt>moodle_progress</tt>: Prints Moodle branch information and dots for each step.
# <tt>moodle_list</tt>: List all scenarios.
# <tt>moodle_stepcount</tt>: List all features with total steps in each feature file. Used for parallel run.
# <tt>moodle_screenshot</tt>: Take screenshot and core dump of each step. With following options you can dump either or both.
## <tt>--format-settings '{"formats": "image"}'</tt>: will dump image only
## <tt>--format-settings '{"formats": "html"}'</tt>: will dump html only.
## <tt>--format-settings '{"formats": "html,image"}'</tt>: will dump both.
## <tt>--format-settings '{"formats": "html", "dir_permissions": "0777"}'</tt>
Note: If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

When working with parallel runs, you may wish to have an output for each run. If you were to specify a standard path for this then each of the parallel runs would overwrite the others file. The <tt>--replace</tt> option allows this to be handled:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std
In this example, the <tt>--replace</tt> argument is provided with a value of <tt>{runprocess}</tt>. Anywhere that <tt>{runprocess}</tt> appears in the command it will be replaced with the run number. The above command will generate a set of commands like:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun1/behat/behat.yml \
--format=pretty --out=/tmp/pretty_1.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun2/behat/behat.yml \
--format=pretty --out=/tmp/pretty_2.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun3/behat/behat.yml \
--format=pretty --out=/tmp/pretty_3.txt \
--format=moodle_progress --out=std
==== Rerun failed scenarios ====
With slow systems or parallel run you may experience see some random failures. These may happen when your system is too slow, when it is too fast, or where a page depends on external dependencies.

To help with this it is possible to rerun any failed scenarios using the <tt>--rerun</tt> option to Behat.

The following example runs Behat with the rerun option:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std \
--rerun
If any single test fails then the command will return a non-zero exit code. Running the same command again will mean that only failed scenarios are run.

For a parallel run it can be called as follows:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std \
--rerun
The Moodle behat runner also includes an <tt>--auto-rerun</tt> option which will automatically rerun failed scenarios exactly once.
==== Running behat with specified theme ====
Behat can be run with any installed theme, but it must be initialised with the <tt>-a</tt> or <tt>--add-core-features-to-theme</tt> option first.

After configuring the theme can be specified using the <tt>--suite</tt> parameter.

Note: The default theme in Moodle (boost) has the suite name <tt>default</tt>.
// Initialise Behat for all themes:
$ php admin/tool/behat/cli/init.php --add-core-features-to-theme

// Run Behat against all initalised themes:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml

// Run Behat against just the default theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=default

// Run Behat against just the 'classic' theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=classic
==== Using Docker to start selenium server ====
There are a wide range of docker images available which contain a browser with Selenium. You will probably be using the official SeleniumHQ images unless you have a specific reason not to.

The complete list of available SeleniumHQ images is available at [https://hub.docker.com/u/selenium/ https://hub.docker.com/u/selenium/].

Moodle uses the ''standalone'' version and any recent version with version 3.141.59 or higher is supported.

For any test which uploads files, for example when interacting with the filepicker, you must also ensure that the Moodle directory is mounted as on your local filesystem.

An example usage is:
$ docker run -d -p 4444:4444 -v `pwd`:`pwd` selenium/standalone-firefox:latest
==== Change config.php file ====
In the Moodle <tt>config.php</tt> file you must change the <tt>$CFG->behat_wwwroot</tt> to an address that can be reached from within the docker image.

You cannot use <tt>localhost</tt> or <tt>127.0.0.1</tt> as this will be the IP address of the docker container itself.

On some more recent versions of Docker you can use <tt>http://host.docker.internal/</tt>, for example if my site is located at <tt>/sm</tt> on my development machine, I can set the following:
$CFG->behat_wwwroot = 'http://host.docker.internal/sm';

==== Manually configuring other browsers =====
If you would prefer not to use the <tt>moodle-browser-config</tt> tool but still wish to specify different browsers then you can do so using the <tt>$CFG->behat_profiles</tt> array. Each key/value pair contains a profile name, and the configuration for that profile. For example:
$CFG->behat_profiles = [
'chrome' => [
'browser' => 'chrome',
'tags' => '@javascript',
],
];
[[Acceptance_testing/Browsers|More info about alternative browsers]]
== Troubleshooting ==
=== Increasing timeouts ===
You may see errors such as:
<syntaxhighlight lang="php">
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</syntaxhighlight>
Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it may just mean that your browser did not finish generating parts of the UI before behat tried was finished.

If you find that this happens regularly on different scenarios then you may want to increase the timeout:
$CFG->behat_increasetimeout = 2;
This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

Note: This is usually an indicator that your development machine is not well tuned. A better option would be to find out where the bottleneck is. This is usually the database configuration.

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<syntaxhighlight lang="php">
php admin/tool/behat/cli/util.php --enable
</syntaxhighlight>
''' For parallel runs, all options for initialising parallel runs are valid '''
=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your browser version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.
=== The tests are failing, and the error message is completely useless ===
For example, it just says "Error writing to database" with no stack trace.

Add -vv command-line option to get very verbose output.
=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.
In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');
=== Selenium server is not running ===
==== Chrome specific ====
If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659/ MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper/ Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== Examples ==
=== Quick setup and testing using moodle-docker ===
This is a quick guide to help locally pass tests for your developments, before submitting them:
# Set up a default Moodle install using [https://github.com/moodlehq/moodle-docker moodle-docker], with the database and Moodle version of your choice. See its README for more details. This will start some docker containers.
# Initialize behat to start testing with this command, from the webserver container: <syntaxhighlight lang="php">php admin/tool/behat/cli/init.php</syntaxhighlight>
# Run the behat test of your choice, from the webserver container. For instance: <syntaxhighlight lang="php">vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --tags tool_task</syntaxhighlight>
And you'll see something like:
<syntaxhighlight lang="php">
Moodle 4.0dev (Build: 20210507), 0b47ea0a44a092f9000729ca7b15fff23111538b
Php: 7.3.26, mysqli: 5.7.21, OS: Linux 5.4.0-66-generic x86_64
Run optional tests:

Accessibility: No
Server OS "Linux", Browser: "firefox"
Started at 09-05-2021, 06:00
...................................................................... 70
...............................
12 scenarios (12 passed)
101 steps (101 passed)
2m53.27s (47.61Mb)
</syntaxhighlight>
== See also ==
* [[Acceptance testing for the mobile app]]
[[Category:Quality Assurance]][[Category:Behat]]

== Information to re-home ==
==== Override behat core context for theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php
==== Disable behat context or features to run in theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To disable specific contexts and features from being executed by a specific theme/suite you can create a <tt>/theme/{MYTHEME}/tests/behat/blacklist.json</tt> file with following format.
<syntaxhighlight lang="php">
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</syntaxhighlight>
The above will:
# disable the use of step_definitions from <tt>behat_grade</tt> and <tt>behat_navigation</tt> while running theme suite; and
# disable running of scenarios in <tt>auth/tests/behat/login.feature</tt> and <tt>grade/tests/behat/grade_hidden_items.feature</tt>.
==== Override core behat selectors ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.
==== Write new tests and behat methods ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

Running acceptance test

2021-12-07T15:39:48Z

Bencellis: /* Add in Browser's automation layers */

Moodle uses [https://behat.org Behat], a php framework for automated functional testing, as part of a suite of testing tools.

Behat takes a set of Features, Scenarios, and Steps, and uses these to step through actions and test results using a real web browser. This is possible thanks to a protocol implemented in most modern web browsers called Webdriver.

This documentation covers how to run Behat tests within Moodle, including requirements, setup, useful tips and tricks, and basic troubleshooting.
== Requirements ==
# Any recent OS with [[Releases|supported version of Moodle]] installed
# A recent browser (We support Firefox, Chrome as standard, but other browsers are possible)
# The WebDriver implementation for your browser
# A recent version of Selenium (Optional, but recommended)
# A recent Java Runtime Environment (Required if using Selenium)
=== Recommended extras ===
Some extra software is also recommended for testing with Behat.
==== Pre-configured browser profiles: moodle-browser-config ====
Available for [[Releases|all supported versions of Moodle]], the [https://github.com/andrewnicols/moodle-browser-config moodle-browser-config] is a recommended inclusion for Behat. This configuration tooling provides a range of standard browser profiles for testing.

Note: In future, the <tt>moodle-browser-config</tt> tool may be included as composer dependency to Moodle but this is currently not the case.
===== Installation =====
# Check out the moodle-browser-config repository:
// Change directory to your git root, for example:
$ cd ~/git
// Clone the moodle-browser-config repository
git clone https://github.com/andrewnicols/moodle-browser-config
# Open your Moodle installation's <tt>config.php</tt> in your preferred editor, and require the tool's init.php:
require_once('/path/to/your/git/dir/moodle-browser-config/init.php');
===== Provided profiles =====
The full list of profiles which are included with <tt>moodle-browser-config</tt> are provided in its [https://github.com/andrewnicols/moodle-browser-config own documentation].

The following is a summary of the profiles that most users may be interested in.

You can also provide your own custom profiles, including for remote services such as Browserstack, and Saucelabs, as
well as for other browsers supporting the W3C Webdriver specification.

Please note that <tt>Safari</tt> and <tt>Safaridriver</tt> are not currently supported as they do not meet the W3C WebDriver specification.
{| class="wikitable" border="1"
|-
! Profile name
! Description
! Uses Selenium?
! Displays GUI?
|-
| firefox
| Use Firefox via Selenium
| Yes
| Yes
|-
| headlessfirefox
| Use Firefox via Selenium, without displaying the GUI
| Yes
| No
|-
| geckodriver
| Use Firefox with Geckodriver directly
| No
| Yes
|-
| headlessgeckodriver
| Use Firefox with Geckodriver directly, without displaying the GUI
| No
| No
|-
| chrome
| Use Chrome via Selenium
| Yes
| Yes
|-
| headlesschrome
| Use Chrome via Selenium, without displaying the GUI
| Yes
| No
|-
| chromedriver
| Use Chrome with Chromedriver directly
| No
| Yes
|-
| headlesschromedriver
| Use Chrome with Chromedriver directly, without displaying the GUI
| No
| No
|-
| edge
| Use Edge via Selenium
| Yes
| Yes
|-
| headlessedge
| Use Edge via Selenium, without displaying the GUI
| Yes
| No
|-
| edgedriver
| Use Edge with Edgedriver directly
| No
| Yes
|-
| headlessedgedriver
| Use Edge with Edgedriver directly, without displaying the GUI
| No
| No
|}
==== chromedriver-wrapper ====
When using Google Chrome, you must use the correct version of the <tt>chromedriver</tt> browser driver for the version of Chrome that you use.

Since Google Chrome automatically updates on a regular basis, you will need to regularly upgrade your driver to match the version of Chrome that you are using.

To make this easier, a <tt>chromedriver-wrapper</tt> utility has been written. This inspects the version of Chrome that is in your path, and downloads the correct version of <tt>chromedriver</tt> for that version, before starting it.

Installation instructinos can be found at [https://github.com/andrewnicols/chromedriver-wrapper https://github.com/andrewnicols/chromedriver-wrapper].
== Getting started (basic) ==
This is a quick walk through to get Behat running for the first time.
=== Setting up ===
There are a number of ways of configuring Behat on Moodle. This is one of the simplest.

These notes assume that you have already installed a supported Java Runtime Environment, and the [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config]] tool.
==== Setting up Selenium ====
Generally we recommend use of Selenium, though this is not a fixed requirement. You can use the browser's driver implementation directly but this is harder to setup for the first time.

Selenium is written in Java, and requires a recent version of the JRE to run. Please ensure that you have this installed prior to starting.

Since Moodle 3.9.5 / 3.10.2 / 3.11.0 Moodle will work with any modern version of Selenium. At time of writing that is <tt>3.141.59</tt>, and <tt>4.0.0-beta-3</tt>.
# Download the Selenium Server (Grid) from [https://www.selenium.dev/downloads/ https://www.selenium.dev/downloads/]. This is a single JAR file, put it anywhere handy.
# Start Selenium
# Version 3.141.59:
$ java -jar selenium-server-standalone-3.141.59.jar

# Version 4.0.0 and later:
$ java -jar selenium-server-4.0.0-beta-3.jar standalone
You can optionally specify a number of settings, depending on the version of Selenium that you are using, including the port to run on.
See the help for the version of Selenium that you are using.
==== Setting up your browsers ====
Selenium is just an intelligient wrapper to start, and manage your browser sessions. It doesn't actually include any web browsers itself.

Moodle HQ run all behat tests against both Firefox and Chrome multiple times per day. Other combinations, including Microsoft Edge, are also supported.

To use Behat, you will need a recent version of your preferred browser, as well as a <tt>driver</tt> for that browser. The driver is responsible for communication between Selenium (or Moodle directly) and the browser.

Both the browser, and its driver, must be placed inside your <tt>$PATH</tt> - this may be somewhere like <tt>/usr/bin</tt>, <tt>/usr/local/bin</tt>, or perhaps a user bin directory like <tt>~/bin</tt> which is present in your <tt>$PATH</tt>
===== Chrome =====
You can download Google Chrome from [https://www.google.com.au/chrome https://www.google.com.au/chrome].

You will need the [https://chromedriver.chromium.org/downloads correct version of the chromedriver] as per the
documentation. Alternatively you can make use of the [[Running_acceptance_test#chromedriver-wrapper|chromedriver-wrapper utility]] noted in the Recommended extras sections.

Either the <tt>chromedriver</tt> binary must be in a directory in your <tt>$PATH</tt>, or the <tt>chromedriver-wrapper/bin</tt> folder must be in your <tt>$PATH</tt>.
===== Firefox =====
You can download Mozilla Firefox from [https://www.mozilla.org/en-US/firefox/new/ https://www.mozilla.org/en-US/firefox/new/].

You will need the [https://github.com/mozilla/geckodriver/releases correct version of geckodriver as per the [https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html documentation].

The <tt>geckodriver</tt> binary must be in a directory in your <tt>$PATH</tt>.
==== Set up Moodle ====
# Create a new 'dataroot' area for files especially for behat
# Set the following in your Moodle <tt>config.php</tt>:
$CFG->behat_dataroot = '/path/to/the/dataroot/you/created';
$CFG->behat_wwwroot = 'http://127.0.0.1/path/to/your/site';
$CFG->behat_prefix = 'beh_';
# We recommend that you also include the <tt>behat-browser-config</tt> if you have not done so already.
require_once('/path/to/moodle-browser-config/init.php');
===== Notes about the behat_wwwroot =====
You will need to set the <tt>behat_wwwroot</tt> to your Moodle site, but it ''must'' use a different value to your <tt>$CFG->wwwroot</tt>.

One common way to do this is to use <tt>127.0.0.1</tt> for behat, but <tt>localhost</tt> for standard use. Alternatively you can add an additional hostname in your <tt>/etc/hosts</tt> file and use this instead.

If you use Docker, then you may be able to use <tt>host.docker.internal</tt> where your site is hosted on the docker host
==== Configure Behat for Moodle ====
After setting your configuration, you can simply initialise behat:
$ php admin/tool/behat/cli/init.php
This will install all required Composer dependencies, install a new Moodle site, and put configuration in place

When it finishes it will give advice on how to run Behat.
==== Run Behat tests ====
Before running behat, ensure that your Selenium server is running.

The easiest way to run behat, and test that everything works is by simply running it using the command provided when you
initialised Behat. If you didn't make a note of it, you can just run the initialisation again.
$ php admin/tool/behat/cli/init.php
This will give you a command which you can then run. This command will run every behat scenario, which will take a considerable amount of time. This command will look a bit like this:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml
To make this more useful you an combine it with flags, for example to only run certain <tt>tags</tt> or for a specific Behat Feature file, or Scenario.

To run all features/scenarios which are tagged with <tt>mod_forum</tt>:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --tags=@mod_forum
To run one specific feature file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature
To run one specific scenario within a feature file:
# To run the Scenario on line 38 of the file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature:38
To run one specific scenario by name:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --name="As a teacher I can see my own response"
See the upstream documentation on Behat, and Gherkin filters for more information.
===== Running using a different browser =====
The default browser in Behat is <tt>Firefox</tt>. To specify a different browser profile, you can add the <tt>--profile</tt> argument. For example, to use Chrome in Headless mode:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
If you are using the <tt>moodle-browser-config</tt> utility, then you can use any profile listed in [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config|moodle-browser-config]]. Otherwise you can write your own browser profile configuration.
=== Advanced testing ===
==== Run tests without Selenium (chromedriver, geckodriver) ====
Historically, Behat required Selenium server, however browsers now make use of their own automation layer. For example, Firefox uses <tt>Geckodriver</tt> and Chrome uses <tt>Chromedriver</tt>. As a result the use of Selenium itself is now optional.

The moodle-browser-config tool includes standard profiles to use these drivers directly and without the use of Selenium.

To use the drivers directly, you must run the driver itself, for example to run <tt>chromedriver</tt>:
$ chromedriver
To run <tt>geckodriver</tt>:
$ geckodriver
Note: geckodriver runs on port 4444 by default. You cannot geckodriver at the same time as selenium.

After starting your preferred browser, you can then run behat and specify an alternative profile, for example:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=geckodriver
==== Headless browsers ====
There are a number of reasons that you may prefer to use a headless browser. It can be particularly helpful if you are running the tests on a remote system, for example over SSH, or if you do not want to be interrupted by browsers popping up on your machine.

The following headless profiles are some of those provided in the moodle-browser-config tool as standard:
# <tt>headlessfirefox</tt> Use Firefox via Selenium, without displaying the GUI
# <tt>headlessgeckodriver</tt> Use Firefox with Geckodriver directly, without displaying the GUI
# <tt>headlesschrome</tt> Use Chrome via Selenium, without displaying the GUI
# <tt>headlesschromedriver</tt> Use Chrome with Chromedriver directly, without displaying the GUI
These can be provided to the <tt>--profile</tt> argument to behat:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
==== Parallel runs ====
Out-of-the-box, Moodle will configure Behat to run a single Moodle installation with all tests run in series. This is great for developer use where you are running a single test. or a small suite of tests. However this can be quite slow. A lot of time is spent waiting in Behat for things to happen. This may be for a page to load, for additional content to load, or even explicit waits because some interactions must be deliberately slowed down. As a result, a system running behat will not have a particularly high load most of the time.

If you want to run a large suite of tests then it is possible to take advantage of the relatively low resource consumption by running several behat runners in parallel. This is commonly referred to as a '''Parallel run'''.

A parallel run of behat takes the same codebase and creates several installations rather than just a single Moodle installation. The behat Feature files are then grouped and allocated between each of the separate installations.

To support this, each of the parallels runs needs its own:
* <tt>behat_wwwroot</tt>
* <tt>behat_dataroot</tt>
* database
Rather than using an entirely separate database, the same database is actually used, but a different <tt>behat_prefix</tt> is used to prefix the table names in the database differently.
===== Installation =====
The Behat initialisation command is responsible for preparing Moodle to run a standard run. You'll have used this before when installing for a standard run:
$ php admin/tool/behat/cli/init.php
The same command can be used to prepare Moodle for a parallel run by specifying the <tt>--parallel</tt> or <tt>-j</tt> parameter:
// Below command will initialise moodle to run 3 parallel tests.
$ php admin/tool/behat/cli/init.php --parallel=3
This can be combined with the <tt>--add-core-features-to-theme</tt> or <tt>-a</tt> flag to prepare Behat to run with all installed themes.

A number of advanced options are also available but you are unlikely to need these:
# <tt>-m=<number></tt> or <tt>--maxruns=<number></tt> Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# <tt>--fromrun=<number></tt> Initialise site to run specified run from. Used for running acceptance tests on different vms
# <tt>--torun=<number></tt> Initialise site to run specified run till. Used for running acceptance tests on different vms
# <tt>-o</tt> or <tt>--optimize-runs</tt> This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
You can view details of all of these using the <tt>--help</tt> flag to <tt>admin/tool/behat/cli/init.php</tt>
===== Running Parallel tests =====
You can use the Moodle behat runner to run all tests, including Standard runs. It is an intelligient wrapper around the standard <tt>./vendor/bin/behat</tt> command which specifies the configuration file, and other required features.
$ php admin/tool/behat/cli/run.php
Many of the standard options and parameters that can be passed to <tt>./vendor/bin/behat</tt> can also be passed to the Moodle runner, for example:
# <tt>--tags</tt> Run tests which match the specified tags
# <tt>--name="Scenario name"</tt> Run a test matching the supplied scenario name
# <tt>--feature="/path/to/test.feature"</tt> Run a specific feature file.
# <tt>--suite</tt> Features for specified theme will be executed.
The runner also includes a number of custom parameters relating to parallel runs:
# <tt>--replace</tt> Replace args string with run process number, useful for output and reruns.
# <tt>--fromrun</tt> Execute run starting from (Used for parallel runs on different vms)
# <tt>--torun</tt> Execute run till (Used for parallel runs on different vms)
The <tt>--replace</tt> feature is particularly useful and can be used to replace a string in the command with the run number. This is useful when specifying output formats, and rerun files as noted below.

The following example demonstrates how Behat might be initialised with three parallel runs, to run on all themes:
$ php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme
And then to run all tests matching the <tt>@tool_myplugin</tt> tag, against the <tt>classic</tt> theme:
$ php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="classic"
===== Custom parameters for parallel runs =====
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
$CFG->behat_parallel_run = [
[
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
],
// ...
],
To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
$CFG->behat_parallel_run = [
['wd_host' => 'http://127.0.0.1:4444/wd/hub'],
['wd_host' => 'http://127.0.0.1:4445/wd/hub'],
['wd_host' => 'http://127.0.0.1:4446/wd/hub'],
];
==== Tests filters ====
With the <tt>--tags</tt> or the <tt>-name</tt> Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* <tt>@javascript</tt>: All the tests that runs in a browser using Javascript; they require Selenium or the browser's own automation layer, as per [[Running acceptance test#Run%20tests%20without%20Selenium%20.28chromedriver.2C%20geckodriver.29|Running acceptance test#Run tests without Selenium .28chromedriver.2C geckodriver.29,]] to be running, otherwise an exception will be thrown.
* <tt>@_file_upload</tt>: All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* <tt>@_alert</tt>: All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* <tt>@_switch_window</tt>: All the tests that are using the <tt>I switch to "NAME" window</tt> step should be tagged as not all browsers manage them properly.
* <tt>@_switch_iframe</tt>: All the tests that are using the <tt>I switch to "NAME" iframe</tt> steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* <tt>@_cross_browser</tt>: All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* <tt>@componentname</tt>: Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.
==== Output formats ====
Behat is able to output in a number of different formats, and to different locations as required.

This can be achieved by specifying the <tt>--format</tt>, and <tt>--out</tt> parameters when running behat, for example:
// Run behat, using the 'pretty' format and outputting the value to /tmp/pretty.txt
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt
It is also possible to output to multiple formats simultaneously by repeating the arguments, for example:

Since Moodle 3.1 option for output is:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std
The following output formats are supported:
# <tt>progress</tt>: Prints one character per step.
# <tt>pretty</tt>: Prints the feature as is.
# <tt>junit</tt>: Outputs the failures in JUnit compatible files.
# <tt>moodle_progress</tt>: Prints Moodle branch information and dots for each step.
# <tt>moodle_list</tt>: List all scenarios.
# <tt>moodle_stepcount</tt>: List all features with total steps in each feature file. Used for parallel run.
# <tt>moodle_screenshot</tt>: Take screenshot and core dump of each step. With following options you can dump either or both.
## <tt>--format-settings '{"formats": "image"}'</tt>: will dump image only
## <tt>--format-settings '{"formats": "html"}'</tt>: will dump html only.
## <tt>--format-settings '{"formats": "html,image"}'</tt>: will dump both.
## <tt>--format-settings '{"formats": "html", "dir_permissions": "0777"}'</tt>
Note: If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

When working with parallel runs, you may wish to have an output for each run. If you were to specify a standard path for this then each of the parallel runs would overwrite the others file. The <tt>--replace</tt> option allows this to be handled:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std
In this example, the <tt>--replace</tt> argument is provided with a value of <tt>{runprocess}</tt>. Anywhere that <tt>{runprocess}</tt> appears in the command it will be replaced with the run number. The above command will generate a set of commands like:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun1/behat/behat.yml \
--format=pretty --out=/tmp/pretty_1.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun2/behat/behat.yml \
--format=pretty --out=/tmp/pretty_2.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun3/behat/behat.yml \
--format=pretty --out=/tmp/pretty_3.txt \
--format=moodle_progress --out=std
==== Rerun failed scenarios ====
With slow systems or parallel run you may experience see some random failures. These may happen when your system is too slow, when it is too fast, or where a page depends on external dependencies.

To help with this it is possible to rerun any failed scenarios using the <tt>--rerun</tt> option to Behat.

The following example runs Behat with the rerun option:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std \
--rerun
If any single test fails then the command will return a non-zero exit code. Running the same command again will mean that only failed scenarios are run.

For a parallel run it can be called as follows:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std \
--rerun
The Moodle behat runner also includes an <tt>--auto-rerun</tt> option which will automatically rerun failed scenarios exactly once.
==== Running behat with specified theme ====
Behat can be run with any installed theme, but it must be initialised with the <tt>-a</tt> or <tt>--add-core-features-to-theme</tt> option first.

After configuring the theme can be specified using the <tt>--suite</tt> parameter.

Note: The default theme in Moodle (boost) has the suite name <tt>default</tt>.
// Initialise Behat for all themes:
$ php admin/tool/behat/cli/init.php --add-core-features-to-theme

// Run Behat against all initalised themes:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml

// Run Behat against just the default theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=default

// Run Behat against just the 'classic' theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=classic
==== Using Docker to start selenium server ====
There are a wide range of docker images available which contain a browser with Selenium. You will probably be using the official SeleniumHQ images unless you have a specific reason not to.

The complete list of available SeleniumHQ images is available at [https://hub.docker.com/u/selenium/ https://hub.docker.com/u/selenium/].

Moodle uses the ''standalone'' version and any recent version with version 3.141.59 or higher is supported.

For any test which uploads files, for example when interacting with the filepicker, you must also ensure that the Moodle directory is mounted as on your local filesystem.

An example usage is:
$ docker run -d -p 4444:4444 -v `pwd`:`pwd` selenium/standalone-firefox:latest
==== Change config.php file ====
In the Moodle <tt>config.php</tt> file you must change the <tt>$CFG->behat_wwwroot</tt> to an address that can be reached from within the docker image.

You cannot use <tt>localhost</tt> or <tt>127.0.0.1</tt> as this will be the IP address of the docker container itself.

On some more recent versions of Docker you can use <tt>http://host.docker.internal/</tt>, for example if my site is located at <tt>/sm</tt> on my development machine, I can set the following:
$CFG->behat_wwwroot = 'http://host.docker.internal/sm';

==== Manually configuring other browsers =====
If you would prefer not to use the <tt>moodle-browser-config</tt> tool but still wish to specify different browsers then you can do so using the <tt>$CFG->behat_profiles</tt> array. Each key/value pair contains a profile name, and the configuration for that profile. For example:
$CFG->behat_profiles = [
'chrome' => [
'browser' => 'chrome',
'tags' => '@javascript',
],
];
[[Acceptance_testing/Browsers|More info about alternative browsers]]
== Troubleshooting ==
=== Increasing timeouts ===
You may see errors such as:
<syntaxhighlight lang="php">
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</syntaxhighlight>
Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it may just mean that your browser did not finish generating parts of the UI before behat tried was finished.

If you find that this happens regularly on different scenarios then you may want to increase the timeout:
$CFG->behat_increasetimeout = 2;
This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

Note: This is usually an indicator that your development machine is not well tuned. A better option would be to find out where the bottleneck is. This is usually the database configuration.

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<syntaxhighlight lang="php">
php admin/tool/behat/cli/util.php --enable
</syntaxhighlight>
''' For parallel runs, all options for initialising parallel runs are valid '''
=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your browser version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.
=== The tests are failing, and the error message is completely useless ===
For example, it just says "Error writing to database" with no stack trace.

Add -vv command-line option to get very verbose output.
=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.
In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');
=== Selenium server is not running ===
==== Chrome specific ====
If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659/ MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper/ Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== Examples ==
=== Quick setup and testing using moodle-docker ===
This is a quick guide to help locally pass tests for your developments, before submitting them:
# Set up a default Moodle install using [https://github.com/moodlehq/moodle-docker moodle-docker], with the database and Moodle version of your choice. See its README for more details. This will start some docker containers.
# Initialize behat to start testing with this command, from the webserver container: <syntaxhighlight lang="php">php admin/tool/behat/cli/init.php</syntaxhighlight>
# Run the behat test of your choice, from the webserver container. For instance: <syntaxhighlight lang="php">vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --tags tool_task</syntaxhighlight>
And you'll see something like:
<syntaxhighlight lang="php">
Moodle 4.0dev (Build: 20210507), 0b47ea0a44a092f9000729ca7b15fff23111538b
Php: 7.3.26, mysqli: 5.7.21, OS: Linux 5.4.0-66-generic x86_64
Run optional tests:

Accessibility: No
Server OS "Linux", Browser: "firefox"
Started at 09-05-2021, 06:00
...................................................................... 70
...............................
12 scenarios (12 passed)
101 steps (101 passed)
2m53.27s (47.61Mb)
</syntaxhighlight>
== See also ==
* [[Acceptance testing for the mobile app]]
[[Category:Quality Assurance]][[Category:Behat]]

== Information to re-home ==
==== Override behat core context for theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php
==== Disable behat context or features to run in theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To disable specific contexts and features from being executed by a specific theme/suite you can create a <tt>/theme/{MYTHEME}/tests/behat/blacklist.json</tt> file with following format.
<syntaxhighlight lang="php">
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</syntaxhighlight>
The above will:
# disable the use of step_definitions from <tt>behat_grade</tt> and <tt>behat_navigation</tt> while running theme suite; and
# disable running of scenarios in <tt>auth/tests/behat/login.feature</tt> and <tt>grade/tests/behat/grade_hidden_items.feature</tt>.
==== Override core behat selectors ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.
==== Write new tests and behat methods ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

Running acceptance test

2021-12-07T15:20:04Z

Bencellis: /* Parallel runs */

Moodle uses [https://behat.org Behat], a php framework for automated functional testing, as part of a suite of testing tools.

Behat takes a set of Features, Scenarios, and Steps, and uses these to step through actions and test results using a real web browser. This is possible thanks to a protocol implemented in most modern web browsers called Webdriver.

This documentation covers how to run Behat tests within Moodle, including requirements, setup, useful tips and tricks, and basic troubleshooting.
== Requirements ==
# Any recent OS with [[Releases|supported version of Moodle]] installed
# A recent browser (We support Firefox, Chrome as standard, but other browsers are possible)
# The WebDriver implementation for your browser
# A recent version of Selenium (Optional, but recommended)
# A recent Java Runtime Environment (Required if using Selenium)
=== Recommended extras ===
Some extra software is also recommended for testing with Behat.
==== Pre-configured browser profiles: moodle-browser-config ====
Available for [[Releases|all supported versions of Moodle]], the [https://github.com/andrewnicols/moodle-browser-config moodle-browser-config] is a recommended inclusion for Behat. This configuration tooling provides a range of standard browser profiles for testing.

Note: In future, the <tt>moodle-browser-config</tt> tool may be included as composer dependency to Moodle but this is currently not the case.
===== Installation =====
# Check out the moodle-browser-config repository:
// Change directory to your git root, for example:
$ cd ~/git
// Clone the moodle-browser-config repository
git clone https://github.com/andrewnicols/moodle-browser-config
# Open your Moodle installation's <tt>config.php</tt> in your preferred editor, and require the tool's init.php:
require_once('/path/to/your/git/dir/moodle-browser-config/init.php');
===== Provided profiles =====
The full list of profiles which are included with <tt>moodle-browser-config</tt> are provided in its [https://github.com/andrewnicols/moodle-browser-config own documentation].

The following is a summary of the profiles that most users may be interested in.

You can also provide your own custom profiles, including for remote services such as Browserstack, and Saucelabs, as
well as for other browsers supporting the W3C Webdriver specification.

Please note that <tt>Safari</tt> and <tt>Safaridriver</tt> are not currently supported as they do not meet the W3C WebDriver specification.
{| class="wikitable" border="1"
|-
! Profile name
! Description
! Uses Selenium?
! Displays GUI?
|-
| firefox
| Use Firefox via Selenium
| Yes
| Yes
|-
| headlessfirefox
| Use Firefox via Selenium, without displaying the GUI
| Yes
| No
|-
| geckodriver
| Use Firefox with Geckodriver directly
| No
| Yes
|-
| headlessgeckodriver
| Use Firefox with Geckodriver directly, without displaying the GUI
| No
| No
|-
| chrome
| Use Chrome via Selenium
| Yes
| Yes
|-
| headlesschrome
| Use Chrome via Selenium, without displaying the GUI
| Yes
| No
|-
| chromedriver
| Use Chrome with Chromedriver directly
| No
| Yes
|-
| headlesschromedriver
| Use Chrome with Chromedriver directly, without displaying the GUI
| No
| No
|-
| edge
| Use Edge via Selenium
| Yes
| Yes
|-
| headlessedge
| Use Edge via Selenium, without displaying the GUI
| Yes
| No
|-
| edgedriver
| Use Edge with Edgedriver directly
| No
| Yes
|-
| headlessedgedriver
| Use Edge with Edgedriver directly, without displaying the GUI
| No
| No
|}
==== chromedriver-wrapper ====
When using Google Chrome, you must use the correct version of the <tt>chromedriver</tt> browser driver for the version of Chrome that you use.

Since Google Chrome automatically updates on a regular basis, you will need to regularly upgrade your driver to match the version of Chrome that you are using.

To make this easier, a <tt>chromedriver-wrapper</tt> utility has been written. This inspects the version of Chrome that is in your path, and downloads the correct version of <tt>chromedriver</tt> for that version, before starting it.

Installation instructinos can be found at [https://github.com/andrewnicols/chromedriver-wrapper https://github.com/andrewnicols/chromedriver-wrapper].
== Getting started (basic) ==
This is a quick walk through to get Behat running for the first time.
=== Setting up ===
There are a number of ways of configuring Behat on Moodle. This is one of the simplest.

These notes assume that you have already installed a supported Java Runtime Environment, and the [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config]] tool.
==== Setting up Selenium ====
Generally we recommend use of Selenium, though this is not a fixed requirement. You can use the browser's driver implementation directly but this is harder to setup for the first time.

Selenium is written in Java, and requires a recent version of the JRE to run. Please ensure that you have this installed prior to starting.

Since Moodle 3.9.5 / 3.10.2 / 3.11.0 Moodle will work with any modern version of Selenium. At time of writing that is <tt>3.141.59</tt>, and <tt>4.0.0-beta-3</tt>.
# Download the Selenium Server (Grid) from [https://www.selenium.dev/downloads/ https://www.selenium.dev/downloads/]. This is a single JAR file, put it anywhere handy.
# Start Selenium
# Version 3.141.59:
$ java -jar selenium-server-standalone-3.141.59.jar

# Version 4.0.0 and later:
$ java -jar selenium-server-4.0.0-beta-3.jar standalone
You can optionally specify a number of settings, depending on the version of Selenium that you are using, including the port to run on.
See the help for the version of Selenium that you are using.
==== Setting up your browsers ====
Selenium is just an intelligient wrapper to start, and manage your browser sessions. It doesn't actually include any web browsers itself.

Moodle HQ run all behat tests against both Firefox and Chrome multiple times per day. Other combinations, including Microsoft Edge, are also supported.

To use Behat, you will need a recent version of your preferred browser, as well as a <tt>driver</tt> for that browser. The driver is responsible for communication between Selenium (or Moodle directly) and the browser.

Both the browser, and its driver, must be placed inside your <tt>$PATH</tt> - this may be somewhere like <tt>/usr/bin</tt>, <tt>/usr/local/bin</tt>, or perhaps a user bin directory like <tt>~/bin</tt> which is present in your <tt>$PATH</tt>
===== Chrome =====
You can download Google Chrome from [https://www.google.com.au/chrome https://www.google.com.au/chrome].

You will need the [https://chromedriver.chromium.org/downloads correct version of the chromedriver] as per the
documentation. Alternatively you can make use of the [[Running_acceptance_test#chromedriver-wrapper|chromedriver-wrapper utility]] noted in the Recommended extras sections.

Either the <tt>chromedriver</tt> binary must be in a directory in your <tt>$PATH</tt>, or the <tt>chromedriver-wrapper/bin</tt> folder must be in your <tt>$PATH</tt>.
===== Firefox =====
You can download Mozilla Firefox from [https://www.mozilla.org/en-US/firefox/new/ https://www.mozilla.org/en-US/firefox/new/].

You will need the [https://github.com/mozilla/geckodriver/releases correct version of geckodriver as per the [https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html documentation].

The <tt>geckodriver</tt> binary must be in a directory in your <tt>$PATH</tt>.
==== Set up Moodle ====
# Create a new 'dataroot' area for files especially for behat
# Set the following in your Moodle <tt>config.php</tt>:
$CFG->behat_dataroot = '/path/to/the/dataroot/you/created';
$CFG->behat_wwwroot = 'http://127.0.0.1/path/to/your/site';
$CFG->behat_prefix = 'beh_';
# We recommend that you also include the <tt>behat-browser-config</tt> if you have not done so already.
require_once('/path/to/moodle-browser-config/init.php');
===== Notes about the behat_wwwroot =====
You will need to set the <tt>behat_wwwroot</tt> to your Moodle site, but it ''must'' use a different value to your <tt>$CFG->wwwroot</tt>.

One common way to do this is to use <tt>127.0.0.1</tt> for behat, but <tt>localhost</tt> for standard use. Alternatively you can add an additional hostname in your <tt>/etc/hosts</tt> file and use this instead.

If you use Docker, then you may be able to use <tt>host.docker.internal</tt> where your site is hosted on the docker host
==== Configure Behat for Moodle ====
After setting your configuration, you can simply initialise behat:
$ php admin/tool/behat/cli/init.php
This will install all required Composer dependencies, install a new Moodle site, and put configuration in place

When it finishes it will give advice on how to run Behat.
==== Run Behat tests ====
Before running behat, ensure that your Selenium server is running.

The easiest way to run behat, and test that everything works is by simply running it using the command provided when you
initialised Behat. If you didn't make a note of it, you can just run the initialisation again.
$ php admin/tool/behat/cli/init.php
This will give you a command which you can then run. This command will run every behat scenario, which will take a considerable amount of time. This command will look a bit like this:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml
To make this more useful you an combine it with flags, for example to only run certain <tt>tags</tt> or for a specific Behat Feature file, or Scenario.

To run all features/scenarios which are tagged with <tt>mod_forum</tt>:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --tags=@mod_forum
To run one specific feature file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature
To run one specific scenario within a feature file:
# To run the Scenario on line 38 of the file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature:38
To run one specific scenario by name:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --name="As a teacher I can see my own response"
See the upstream documentation on Behat, and Gherkin filters for more information.
===== Running using a different browser =====
The default browser in Behat is <tt>Firefox</tt>. To specify a different browser profile, you can add the <tt>--profile</tt> argument. For example, to use Chrome in Headless mode:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
If you are using the <tt>moodle-browser-config</tt> utility, then you can use any profile listed in [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config|moodle-browser-config]]. Otherwise you can write your own browser profile configuration.
=== Advanced testing ===
==== Run tests without Selenium (chromedriver, geckodriver) ====
Historically, Behat required Selenium server, however browsers now make use of their own automation layer. For example, Firefox uses <tt>Geckodriver</tt> and Chrome uses <tt>Chromedriver</tt>. As a result the use of Selenium itself is now optional.

The moodle-browser-config tool includes standard profiles to use these drivers directly and without the use of Selenium.

To use the drivers directly, you must run the driver itself, for example to run <tt>chromedriver</tt>:
$ chromedriver
To run <tt>geckodriver</tt>:
$ geckodriver
Note: geckodriver runs on port 4444 by default. You cannot geckodriver at the same time as selenium.

After starting your preferred browser, you can then run behat and specify an alternative profile, for example:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=geckodriver
==== Headless browsers ====
There are a number of reasons that you may prefer to use a headless browser. It can be particularly helpful if you are running the tests on a remote system, for example over SSH, or if you do not want to be interrupted by browsers popping up on your machine.

The following headless profiles are some of those provided in the moodle-browser-config tool as standard:
# <tt>headlessfirefox</tt> Use Firefox via Selenium, without displaying the GUI
# <tt>headlessgeckodriver</tt> Use Firefox with Geckodriver directly, without displaying the GUI
# <tt>headlesschrome</tt> Use Chrome via Selenium, without displaying the GUI
# <tt>headlesschromedriver</tt> Use Chrome with Chromedriver directly, without displaying the GUI
These can be provided to the <tt>--profile</tt> argument to behat:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
==== Parallel runs ====
Out-of-the-box, Moodle will configure Behat to run a single Moodle installation with all tests run in series. This is great for developer use where you are running a single test. or a small suite of tests. However this can be quite slow. A lot of time is spent waiting in Behat for things to happen. This may be for a page to load, for additional content to load, or even explicit waits because some interactions must be deliberately slowed down. As a result, a system running behat will not have a particularly high load most of the time.

If you want to run a large suite of tests then it is possible to take advantage of the relatively low resource consumption by running several behat runners in parallel. This is commonly referred to as a '''Parallel run'''.

A parallel run of behat takes the same codebase and creates several installations rather than just a single Moodle installation. The behat Feature files are then grouped and allocated between each of the separate installations.

To support this, each of the parallels runs needs its own:
* <tt>behat_wwwroot</tt>
* <tt>behat_dataroot</tt>
* database
Rather than using an entirely separate database, the same database is actually used, but a different <tt>behat_prefix</tt> is used to prefix the table names in the database differently.
===== Installation =====
The Behat initialisation command is responsible for preparing Moodle to run a standard run. You'll have used this before when installing for a standard run:
$ php admin/tool/behat/cli/init.php
The same command can be used to prepare Moodle for a parallel run by specifying the <tt>--parallel</tt> or <tt>-j</tt> parameter:
// Below command will initialise moodle to run 3 parallel tests.
$ php admin/tool/behat/cli/init.php --parallel=3
This can be combined with the <tt>--add-core-features-to-theme</tt> or <tt>-a</tt> flag to prepare Behat to run with all installed themes.

A number of advanced options are also available but you are unlikely to need these:
# <tt>-m=<number></tt> or <tt>--maxruns=<number></tt> Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# <tt>--fromrun=<number></tt> Initialise site to run specified run from. Used for running acceptance tests on different vms
# <tt>--torun=<number></tt> Initialise site to run specified run till. Used for running acceptance tests on different vms
# <tt>-o</tt> or <tt>--optimize-runs</tt> This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
You can view details of all of these using the <tt>--help</tt> flag to <tt>admin/tool/behat/cli/init.php</tt>
===== Running Parallel tests =====
You can use the Moodle behat runner to run all tests, including Standard runs. It is an intelligient wrapper around the standard <tt>./vendor/bin/behat</tt> command which specifies the configuration file, and other required features.
$ php admin/tool/behat/cli/run.php
Many of the standard options and parameters that can be passed to <tt>./vendor/bin/behat</tt> can also be passed to the Moodle runner, for example:
# <tt>--tags</tt> Run tests which match the specified tags
# <tt>--name="Scenario name"</tt> Run a test matching the supplied scenario name
# <tt>--feature="/path/to/test.feature"</tt> Run a specific feature file.
# <tt>--suite</tt> Features for specified theme will be executed.
The runner also includes a number of custom parameters relating to parallel runs:
# <tt>--replace</tt> Replace args string with run process number, useful for output and reruns.
# <tt>--fromrun</tt> Execute run starting from (Used for parallel runs on different vms)
# <tt>--torun</tt> Execute run till (Used for parallel runs on different vms)
The <tt>--replace</tt> feature is particularly useful and can be used to replace a string in the command with the run number. This is useful when specifying output formats, and rerun files as noted below.

The following example demonstrates how Behat might be initialised with three parallel runs, to run on all themes:
$ php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme
And then to run all tests matching the <tt>@tool_myplugin</tt> tag, against the <tt>classic</tt> theme:
$ php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="classic"
===== Custom parameters for parallel runs =====
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
$CFG->behat_parallel_run = [
[
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
],
// ...
],
To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
$CFG->behat_parallel_run = [
['wd_host' => 'http://127.0.0.1:4444/wd/hub'],
['wd_host' => 'http://127.0.0.1:4445/wd/hub'],
['wd_host' => 'http://127.0.0.1:4446/wd/hub'],
];
==== Tests filters ====
With the <tt>--tags</tt> or the <tt>-name</tt> Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* <tt>@javascript</tt>: All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* <tt>@_file_upload</tt>: All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* <tt>@_alert</tt>: All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* <tt>@_switch_window</tt>: All the tests that are using the <tt>I switch to "NAME" window</tt> step should be tagged as not all browsers manage them properly.
* <tt>@_switch_iframe</tt>: All the tests that are using the <tt>I switch to "NAME" iframe</tt> steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* <tt>@_cross_browser</tt>: All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* <tt>@componentname</tt>: Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.
==== Output formats ====
Behat is able to output in a number of different formats, and to different locations as required.

This can be achieved by specifying the <tt>--format</tt>, and <tt>--out</tt> parameters when running behat, for example:
// Run behat, using the 'pretty' format and outputting the value to /tmp/pretty.txt
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt
It is also possible to output to multiple formats simultaneously by repeating the arguments, for example:

Since Moodle 3.1 option for output is:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std
The following output formats are supported:
# <tt>progress</tt>: Prints one character per step.
# <tt>pretty</tt>: Prints the feature as is.
# <tt>junit</tt>: Outputs the failures in JUnit compatible files.
# <tt>moodle_progress</tt>: Prints Moodle branch information and dots for each step.
# <tt>moodle_list</tt>: List all scenarios.
# <tt>moodle_stepcount</tt>: List all features with total steps in each feature file. Used for parallel run.
# <tt>moodle_screenshot</tt>: Take screenshot and core dump of each step. With following options you can dump either or both.
## <tt>--format-settings '{"formats": "image"}'</tt>: will dump image only
## <tt>--format-settings '{"formats": "html"}'</tt>: will dump html only.
## <tt>--format-settings '{"formats": "html,image"}'</tt>: will dump both.
## <tt>--format-settings '{"formats": "html", "dir_permissions": "0777"}'</tt>
Note: If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

When working with parallel runs, you may wish to have an output for each run. If you were to specify a standard path for this then each of the parallel runs would overwrite the others file. The <tt>--replace</tt> option allows this to be handled:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std
In this example, the <tt>--replace</tt> argument is provided with a value of <tt>{runprocess}</tt>. Anywhere that <tt>{runprocess}</tt> appears in the command it will be replaced with the run number. The above command will generate a set of commands like:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun1/behat/behat.yml \
--format=pretty --out=/tmp/pretty_1.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun2/behat/behat.yml \
--format=pretty --out=/tmp/pretty_2.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun3/behat/behat.yml \
--format=pretty --out=/tmp/pretty_3.txt \
--format=moodle_progress --out=std
==== Rerun failed scenarios ====
With slow systems or parallel run you may experience see some random failures. These may happen when your system is too slow, when it is too fast, or where a page depends on external dependencies.

To help with this it is possible to rerun any failed scenarios using the <tt>--rerun</tt> option to Behat.

The following example runs Behat with the rerun option:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std \
--rerun
If any single test fails then the command will return a non-zero exit code. Running the same command again will mean that only failed scenarios are run.

For a parallel run it can be called as follows:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std \
--rerun
The Moodle behat runner also includes an <tt>--auto-rerun</tt> option which will automatically rerun failed scenarios exactly once.
==== Running behat with specified theme ====
Behat can be run with any installed theme, but it must be initialised with the <tt>-a</tt> or <tt>--add-core-features-to-theme</tt> option first.

After configuring the theme can be specified using the <tt>--suite</tt> parameter.

Note: The default theme in Moodle (boost) has the suite name <tt>default</tt>.
// Initialise Behat for all themes:
$ php admin/tool/behat/cli/init.php --add-core-features-to-theme

// Run Behat against all initalised themes:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml

// Run Behat against just the default theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=default

// Run Behat against just the 'classic' theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=classic
==== Using Docker to start selenium server ====
There are a wide range of docker images available which contain a browser with Selenium. You will probably be using the official SeleniumHQ images unless you have a specific reason not to.

The complete list of available SeleniumHQ images is available at [https://hub.docker.com/u/selenium/ https://hub.docker.com/u/selenium/].

Moodle uses the ''standalone'' version and any recent version with version 3.141.59 or higher is supported.

For any test which uploads files, for example when interacting with the filepicker, you must also ensure that the Moodle directory is mounted as on your local filesystem.

An example usage is:
$ docker run -d -p 4444:4444 -v `pwd`:`pwd` selenium/standalone-firefox:latest
==== Change config.php file ====
In the Moodle <tt>config.php</tt> file you must change the <tt>$CFG->behat_wwwroot</tt> to an address that can be reached from within the docker image.

You cannot use <tt>localhost</tt> or <tt>127.0.0.1</tt> as this will be the IP address of the docker container itself.

On some more recent versions of Docker you can use <tt>http://host.docker.internal/</tt>, for example if my site is located at <tt>/sm</tt> on my development machine, I can set the following:
$CFG->behat_wwwroot = 'http://host.docker.internal/sm';

==== Manually configuring other browsers =====
If you would prefer not to use the <tt>moodle-browser-config</tt> tool but still wish to specify different browsers then you can do so using the <tt>$CFG->behat_profiles</tt> array. Each key/value pair contains a profile name, and the configuration for that profile. For example:
$CFG->behat_profiles = [
'chrome' => [
'browser' => 'chrome',
'tags' => '@javascript',
],
];
[[Acceptance_testing/Browsers|More info about alternative browsers]]
== Troubleshooting ==
=== Increasing timeouts ===
You may see errors such as:
<syntaxhighlight lang="php">
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</syntaxhighlight>
Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it may just mean that your browser did not finish generating parts of the UI before behat tried was finished.

If you find that this happens regularly on different scenarios then you may want to increase the timeout:
$CFG->behat_increasetimeout = 2;
This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

Note: This is usually an indicator that your development machine is not well tuned. A better option would be to find out where the bottleneck is. This is usually the database configuration.

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<syntaxhighlight lang="php">
php admin/tool/behat/cli/util.php --enable
</syntaxhighlight>
''' For parallel runs, all options for initialising parallel runs are valid '''
=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your browser version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.
=== The tests are failing, and the error message is completely useless ===
For example, it just says "Error writing to database" with no stack trace.

Add -vv command-line option to get very verbose output.
=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.
In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');
=== Selenium server is not running ===
==== Chrome specific ====
If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659/ MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper/ Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== Examples ==
=== Quick setup and testing using moodle-docker ===
This is a quick guide to help locally pass tests for your developments, before submitting them:
# Set up a default Moodle install using [https://github.com/moodlehq/moodle-docker moodle-docker], with the database and Moodle version of your choice. See its README for more details. This will start some docker containers.
# Initialize behat to start testing with this command, from the webserver container: <syntaxhighlight lang="php">php admin/tool/behat/cli/init.php</syntaxhighlight>
# Run the behat test of your choice, from the webserver container. For instance: <syntaxhighlight lang="php">vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --tags tool_task</syntaxhighlight>
And you'll see something like:
<syntaxhighlight lang="php">
Moodle 4.0dev (Build: 20210507), 0b47ea0a44a092f9000729ca7b15fff23111538b
Php: 7.3.26, mysqli: 5.7.21, OS: Linux 5.4.0-66-generic x86_64
Run optional tests:

Accessibility: No
Server OS "Linux", Browser: "firefox"
Started at 09-05-2021, 06:00
...................................................................... 70
...............................
12 scenarios (12 passed)
101 steps (101 passed)
2m53.27s (47.61Mb)
</syntaxhighlight>
== See also ==
* [[Acceptance testing for the mobile app]]
[[Category:Quality Assurance]][[Category:Behat]]

== Information to re-home ==
==== Override behat core context for theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php
==== Disable behat context or features to run in theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To disable specific contexts and features from being executed by a specific theme/suite you can create a <tt>/theme/{MYTHEME}/tests/behat/blacklist.json</tt> file with following format.
<syntaxhighlight lang="php">
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</syntaxhighlight>
The above will:
# disable the use of step_definitions from <tt>behat_grade</tt> and <tt>behat_navigation</tt> while running theme suite; and
# disable running of scenarios in <tt>auth/tests/behat/login.feature</tt> and <tt>grade/tests/behat/grade_hidden_items.feature</tt>.
==== Override core behat selectors ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.
==== Write new tests and behat methods ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

Running acceptance test

2021-12-07T15:12:36Z

Bencellis: /* Chrome */

Moodle uses [https://behat.org Behat], a php framework for automated functional testing, as part of a suite of testing tools.

Behat takes a set of Features, Scenarios, and Steps, and uses these to step through actions and test results using a real web browser. This is possible thanks to a protocol implemented in most modern web browsers called Webdriver.

This documentation covers how to run Behat tests within Moodle, including requirements, setup, useful tips and tricks, and basic troubleshooting.
== Requirements ==
# Any recent OS with [[Releases|supported version of Moodle]] installed
# A recent browser (We support Firefox, Chrome as standard, but other browsers are possible)
# The WebDriver implementation for your browser
# A recent version of Selenium (Optional, but recommended)
# A recent Java Runtime Environment (Required if using Selenium)
=== Recommended extras ===
Some extra software is also recommended for testing with Behat.
==== Pre-configured browser profiles: moodle-browser-config ====
Available for [[Releases|all supported versions of Moodle]], the [https://github.com/andrewnicols/moodle-browser-config moodle-browser-config] is a recommended inclusion for Behat. This configuration tooling provides a range of standard browser profiles for testing.

Note: In future, the <tt>moodle-browser-config</tt> tool may be included as composer dependency to Moodle but this is currently not the case.
===== Installation =====
# Check out the moodle-browser-config repository:
// Change directory to your git root, for example:
$ cd ~/git
// Clone the moodle-browser-config repository
git clone https://github.com/andrewnicols/moodle-browser-config
# Open your Moodle installation's <tt>config.php</tt> in your preferred editor, and require the tool's init.php:
require_once('/path/to/your/git/dir/moodle-browser-config/init.php');
===== Provided profiles =====
The full list of profiles which are included with <tt>moodle-browser-config</tt> are provided in its [https://github.com/andrewnicols/moodle-browser-config own documentation].

The following is a summary of the profiles that most users may be interested in.

You can also provide your own custom profiles, including for remote services such as Browserstack, and Saucelabs, as
well as for other browsers supporting the W3C Webdriver specification.

Please note that <tt>Safari</tt> and <tt>Safaridriver</tt> are not currently supported as they do not meet the W3C WebDriver specification.
{| class="wikitable" border="1"
|-
! Profile name
! Description
! Uses Selenium?
! Displays GUI?
|-
| firefox
| Use Firefox via Selenium
| Yes
| Yes
|-
| headlessfirefox
| Use Firefox via Selenium, without displaying the GUI
| Yes
| No
|-
| geckodriver
| Use Firefox with Geckodriver directly
| No
| Yes
|-
| headlessgeckodriver
| Use Firefox with Geckodriver directly, without displaying the GUI
| No
| No
|-
| chrome
| Use Chrome via Selenium
| Yes
| Yes
|-
| headlesschrome
| Use Chrome via Selenium, without displaying the GUI
| Yes
| No
|-
| chromedriver
| Use Chrome with Chromedriver directly
| No
| Yes
|-
| headlesschromedriver
| Use Chrome with Chromedriver directly, without displaying the GUI
| No
| No
|-
| edge
| Use Edge via Selenium
| Yes
| Yes
|-
| headlessedge
| Use Edge via Selenium, without displaying the GUI
| Yes
| No
|-
| edgedriver
| Use Edge with Edgedriver directly
| No
| Yes
|-
| headlessedgedriver
| Use Edge with Edgedriver directly, without displaying the GUI
| No
| No
|}
==== chromedriver-wrapper ====
When using Google Chrome, you must use the correct version of the <tt>chromedriver</tt> browser driver for the version of Chrome that you use.

Since Google Chrome automatically updates on a regular basis, you will need to regularly upgrade your driver to match the version of Chrome that you are using.

To make this easier, a <tt>chromedriver-wrapper</tt> utility has been written. This inspects the version of Chrome that is in your path, and downloads the correct version of <tt>chromedriver</tt> for that version, before starting it.

Installation instructinos can be found at [https://github.com/andrewnicols/chromedriver-wrapper https://github.com/andrewnicols/chromedriver-wrapper].
== Getting started (basic) ==
This is a quick walk through to get Behat running for the first time.
=== Setting up ===
There are a number of ways of configuring Behat on Moodle. This is one of the simplest.

These notes assume that you have already installed a supported Java Runtime Environment, and the [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config]] tool.
==== Setting up Selenium ====
Generally we recommend use of Selenium, though this is not a fixed requirement. You can use the browser's driver implementation directly but this is harder to setup for the first time.

Selenium is written in Java, and requires a recent version of the JRE to run. Please ensure that you have this installed prior to starting.

Since Moodle 3.9.5 / 3.10.2 / 3.11.0 Moodle will work with any modern version of Selenium. At time of writing that is <tt>3.141.59</tt>, and <tt>4.0.0-beta-3</tt>.
# Download the Selenium Server (Grid) from [https://www.selenium.dev/downloads/ https://www.selenium.dev/downloads/]. This is a single JAR file, put it anywhere handy.
# Start Selenium
# Version 3.141.59:
$ java -jar selenium-server-standalone-3.141.59.jar

# Version 4.0.0 and later:
$ java -jar selenium-server-4.0.0-beta-3.jar standalone
You can optionally specify a number of settings, depending on the version of Selenium that you are using, including the port to run on.
See the help for the version of Selenium that you are using.
==== Setting up your browsers ====
Selenium is just an intelligient wrapper to start, and manage your browser sessions. It doesn't actually include any web browsers itself.

Moodle HQ run all behat tests against both Firefox and Chrome multiple times per day. Other combinations, including Microsoft Edge, are also supported.

To use Behat, you will need a recent version of your preferred browser, as well as a <tt>driver</tt> for that browser. The driver is responsible for communication between Selenium (or Moodle directly) and the browser.

Both the browser, and its driver, must be placed inside your <tt>$PATH</tt> - this may be somewhere like <tt>/usr/bin</tt>, <tt>/usr/local/bin</tt>, or perhaps a user bin directory like <tt>~/bin</tt> which is present in your <tt>$PATH</tt>
===== Chrome =====
You can download Google Chrome from [https://www.google.com.au/chrome https://www.google.com.au/chrome].

You will need the [https://chromedriver.chromium.org/downloads correct version of the chromedriver] as per the
documentation. Alternatively you can make use of the [[Running_acceptance_test#chromedriver-wrapper|chromedriver-wrapper utility]] noted in the Recommended extras sections.

Either the <tt>chromedriver</tt> binary must be in a directory in your <tt>$PATH</tt>, or the <tt>chromedriver-wrapper/bin</tt> folder must be in your <tt>$PATH</tt>.
===== Firefox =====
You can download Mozilla Firefox from [https://www.mozilla.org/en-US/firefox/new/ https://www.mozilla.org/en-US/firefox/new/].

You will need the [https://github.com/mozilla/geckodriver/releases correct version of geckodriver as per the [https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html documentation].

The <tt>geckodriver</tt> binary must be in a directory in your <tt>$PATH</tt>.
==== Set up Moodle ====
# Create a new 'dataroot' area for files especially for behat
# Set the following in your Moodle <tt>config.php</tt>:
$CFG->behat_dataroot = '/path/to/the/dataroot/you/created';
$CFG->behat_wwwroot = 'http://127.0.0.1/path/to/your/site';
$CFG->behat_prefix = 'beh_';
# We recommend that you also include the <tt>behat-browser-config</tt> if you have not done so already.
require_once('/path/to/moodle-browser-config/init.php');
===== Notes about the behat_wwwroot =====
You will need to set the <tt>behat_wwwroot</tt> to your Moodle site, but it ''must'' use a different value to your <tt>$CFG->wwwroot</tt>.

One common way to do this is to use <tt>127.0.0.1</tt> for behat, but <tt>localhost</tt> for standard use. Alternatively you can add an additional hostname in your <tt>/etc/hosts</tt> file and use this instead.

If you use Docker, then you may be able to use <tt>host.docker.internal</tt> where your site is hosted on the docker host
==== Configure Behat for Moodle ====
After setting your configuration, you can simply initialise behat:
$ php admin/tool/behat/cli/init.php
This will install all required Composer dependencies, install a new Moodle site, and put configuration in place

When it finishes it will give advice on how to run Behat.
==== Run Behat tests ====
Before running behat, ensure that your Selenium server is running.

The easiest way to run behat, and test that everything works is by simply running it using the command provided when you
initialised Behat. If you didn't make a note of it, you can just run the initialisation again.
$ php admin/tool/behat/cli/init.php
This will give you a command which you can then run. This command will run every behat scenario, which will take a considerable amount of time. This command will look a bit like this:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml
To make this more useful you an combine it with flags, for example to only run certain <tt>tags</tt> or for a specific Behat Feature file, or Scenario.

To run all features/scenarios which are tagged with <tt>mod_forum</tt>:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --tags=@mod_forum
To run one specific feature file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature
To run one specific scenario within a feature file:
# To run the Scenario on line 38 of the file:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml `pwd`/mod/forum/tests/behat/private_replies.feature:38
To run one specific scenario by name:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --name="As a teacher I can see my own response"
See the upstream documentation on Behat, and Gherkin filters for more information.
===== Running using a different browser =====
The default browser in Behat is <tt>Firefox</tt>. To specify a different browser profile, you can add the <tt>--profile</tt> argument. For example, to use Chrome in Headless mode:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
If you are using the <tt>moodle-browser-config</tt> utility, then you can use any profile listed in [[Running_acceptance_test#Pre-configured_browser_profiles:_moodle-browser-config|moodle-browser-config]]. Otherwise you can write your own browser profile configuration.
=== Advanced testing ===
==== Run tests without Selenium (chromedriver, geckodriver) ====
Historically, Behat required Selenium server, however browsers now make use of their own automation layer. For example, Firefox uses <tt>Geckodriver</tt> and Chrome uses <tt>Chromedriver</tt>. As a result the use of Selenium itself is now optional.

The moodle-browser-config tool includes standard profiles to use these drivers directly and without the use of Selenium.

To use the drivers directly, you must run the driver itself, for example to run <tt>chromedriver</tt>:
$ chromedriver
To run <tt>geckodriver</tt>:
$ geckodriver
Note: geckodriver runs on port 4444 by default. You cannot geckodriver at the same time as selenium.

After starting your preferred browser, you can then run behat and specify an alternative profile, for example:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=geckodriver
==== Headless browsers ====
There are a number of reasons that you may prefer to use a headless browser. It can be particularly helpful if you are running the tests on a remote system, for example over SSH, or if you do not want to be interrupted by browsers popping up on your machine.

The following headless profiles are some of those provided in the moodle-browser-config tool as standard:
# <tt>headlessfirefox</tt> Use Firefox via Selenium, without displaying the GUI
# <tt>headlessgeckodriver</tt> Use Firefox with Geckodriver directly, without displaying the GUI
# <tt>headlesschrome</tt> Use Chrome via Selenium, without displaying the GUI
# <tt>headlesschromedriver</tt> Use Chrome with Chromedriver directly, without displaying the GUI
These can be provided to the <tt>--profile</tt> argument to behat:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --profile=headlesschrome
==== Parallel runs ====
Out-of-the-box, Moodle will configure Behat to run a single Moodle installation with all tests run in series. This is great for developer use where you are running a single test. or a small suite of tests. However this can be quite slow. A lot of time is spent waiting in Behat for things to happen. This may be for a page to load, for additional content to load, or even explicit waits because some interactions must be deliberately slowed down. As a result, a system running behat will not have a particularly high load most of the time.

If you want to run a large suite of tests then it is possible to take advantage of the relatively low resource consumption by running several behat runners in parallel. This is commonly referredt o as a '''Parallel run'''.

A parallel run of behat takes the same codebase and creates several installations rather than just a single Moodle installation. The behat Feature files are then grouped and allocated between each of the separate installations.

To support this, each of the parallels runs needs its own:
* <tt>behat_wwwroot</tt>
* <tt>behat_dataroot</tt>
* database
Rather than using an entirely separate database, the same database is actually used, but a different <tt>behat_prefix</tt> is used to prefix the table names in the database differently.
===== Installation =====
The Behat initialisation command is responsible for preparing Moodle to run a standard run. You'll have used this before when installing for a standard run:
$ php admin/tool/behat/cli/init.php
The same command can be used to prepare Moodle for a parallel run by specifying the <tt>--parallel</tt> or <tt>-j</tt> parameter:
// Below command will initialise moodle to run 3 parallel tests.
$ php admin/tool/behat/cli/init.php --parallel=3
This can be combined with the <tt>--add-core-features-to-theme</tt> or <tt>-a</tt> flag to prepare Behat to run with all installed themes.

A number of advanced options are also available but you are unlikely to need these:
# <tt>-m=<number></tt> or <tt>--maxruns=<number></tt> Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# <tt>--fromrun=<number></tt> Initialise site to run specified run from. Used for running acceptance tests on different vms
# <tt>--torun=<number></tt> Initialise site to run specified run till. Used for running acceptance tests on different vms
# <tt>-o</tt> or <tt>--optimize-runs</tt> This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
You can view details of all of these using the <tt>--help</tt> flag to <tt>admin/tool/behat/cli/init.php</tt>
===== Running Parallel tests =====
You can use the Moodle behat runner to run all tests, including Standard runs. It is an intelligient wrapper around the standard <tt>./vendor/bin/behat</tt> command which specifies the configuration file, and other required features.
$ php admin/tool/behat/cli/run.php
Many of the standard options and parameters that can be passed to <tt>./vendor/bin/behat</tt> can also be passed to the Moodle runner, for example:
# <tt>--tags</tt> Run tests which match the specified tags
# <tt>--name="Scenario name"</tt> Run a test matching the supplied scenario name
# <tt>--feature="/path/to/test.feature"</tt> Run a specific feature file.
# <tt>--suite</tt> Features for specified theme will be executed.
The runner also includes a number of custom parameters relating to parallel runs:
# <tt>--replace</tt> Replace args string with run process number, useful for output and reruns.
# <tt>--fromrun</tt> Execute run starting from (Used for parallel runs on different vms)
# <tt>--torun</tt> Execute run till (Used for parallel runs on different vms)
The <tt>--replace</tt> feature is particularly useful and can be used to replace a string in the command with the run number. This is useful when specifying output formats, and rerun files as noted below.

The following example demonstrates how Behat might be initialised with three parallel runs, to run on all themes:
$ php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme
And then to run all tests matching the <tt>@tool_myplugin</tt> tag, against the <tt>classic</tt> theme:
$ php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="classic"
===== Custom parameters for parallel runs =====
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
$CFG->behat_parallel_run = [
[
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
],
// ...
],
To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
$CFG->behat_parallel_run = [
['wd_host' => 'http://127.0.0.1:4444/wd/hub'],
['wd_host' => 'http://127.0.0.1:4445/wd/hub'],
['wd_host' => 'http://127.0.0.1:4446/wd/hub'],
];
==== Tests filters ====
With the <tt>--tags</tt> or the <tt>-name</tt> Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* <tt>@javascript</tt>: All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* <tt>@_file_upload</tt>: All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* <tt>@_alert</tt>: All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* <tt>@_switch_window</tt>: All the tests that are using the <tt>I switch to "NAME" window</tt> step should be tagged as not all browsers manage them properly.
* <tt>@_switch_iframe</tt>: All the tests that are using the <tt>I switch to "NAME" iframe</tt> steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* <tt>@_cross_browser</tt>: All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* <tt>@componentname</tt>: Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.
==== Output formats ====
Behat is able to output in a number of different formats, and to different locations as required.

This can be achieved by specifying the <tt>--format</tt>, and <tt>--out</tt> parameters when running behat, for example:
// Run behat, using the 'pretty' format and outputting the value to /tmp/pretty.txt
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt
It is also possible to output to multiple formats simultaneously by repeating the arguments, for example:

Since Moodle 3.1 option for output is:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std
The following output formats are supported:
# <tt>progress</tt>: Prints one character per step.
# <tt>pretty</tt>: Prints the feature as is.
# <tt>junit</tt>: Outputs the failures in JUnit compatible files.
# <tt>moodle_progress</tt>: Prints Moodle branch information and dots for each step.
# <tt>moodle_list</tt>: List all scenarios.
# <tt>moodle_stepcount</tt>: List all features with total steps in each feature file. Used for parallel run.
# <tt>moodle_screenshot</tt>: Take screenshot and core dump of each step. With following options you can dump either or both.
## <tt>--format-settings '{"formats": "image"}'</tt>: will dump image only
## <tt>--format-settings '{"formats": "html"}'</tt>: will dump html only.
## <tt>--format-settings '{"formats": "html,image"}'</tt>: will dump both.
## <tt>--format-settings '{"formats": "html", "dir_permissions": "0777"}'</tt>
Note: If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

When working with parallel runs, you may wish to have an output for each run. If you were to specify a standard path for this then each of the parallel runs would overwrite the others file. The <tt>--replace</tt> option allows this to be handled:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std
In this example, the <tt>--replace</tt> argument is provided with a value of <tt>{runprocess}</tt>. Anywhere that <tt>{runprocess}</tt> appears in the command it will be replaced with the run number. The above command will generate a set of commands like:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun1/behat/behat.yml \
--format=pretty --out=/tmp/pretty_1.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun2/behat/behat.yml \
--format=pretty --out=/tmp/pretty_2.txt \
--format=moodle_progress --out=std

$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun3/behat/behat.yml \
--format=pretty --out=/tmp/pretty_3.txt \
--format=moodle_progress --out=std
==== Rerun failed scenarios ====
With slow systems or parallel run you may experience see some random failures. These may happen when your system is too slow, when it is too fast, or where a page depends on external dependencies.

To help with this it is possible to rerun any failed scenarios using the <tt>--rerun</tt> option to Behat.

The following example runs Behat with the rerun option:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml \
--format=pretty --out=/tmp/pretty.txt \
--format=moodle_progress --out=std \
--rerun
If any single test fails then the command will return a non-zero exit code. Running the same command again will mean that only failed scenarios are run.

For a parallel run it can be called as follows:
$ admin/tool/behat/cli/run.php \
--replace="{runprocess}" \
--format=pretty --out=/tmp/pretty_{runprocess}.txt \
--format=moodle_progress --out=std \
--rerun
The Moodle behat runner also includes an <tt>--auto-rerun</tt> option which will automatically rerun failed scenarios exactly once.
==== Running behat with specified theme ====
Behat can be run with any installed theme, but it must be initialised with the <tt>-a</tt> or <tt>--add-core-features-to-theme</tt> option first.

After configuring the theme can be specified using the <tt>--suite</tt> parameter.

Note: The default theme in Moodle (boost) has the suite name <tt>default</tt>.
// Initialise Behat for all themes:
$ php admin/tool/behat/cli/init.php --add-core-features-to-theme

// Run Behat against all initalised themes:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml

// Run Behat against just the default theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=default

// Run Behat against just the 'classic' theme when all themes were initialised:
$ vendor/bin/behat --config /Users/nicols/Sites/moodles/sm/moodledata_behat/behatrun/behat/behat.yml --suite=classic
==== Using Docker to start selenium server ====
There are a wide range of docker images available which contain a browser with Selenium. You will probably be using the official SeleniumHQ images unless you have a specific reason not to.

The complete list of available SeleniumHQ images is available at [https://hub.docker.com/u/selenium/ https://hub.docker.com/u/selenium/].

Moodle uses the ''standalone'' version and any recent version with version 3.141.59 or higher is supported.

For any test which uploads files, for example when interacting with the filepicker, you must also ensure that the Moodle directory is mounted as on your local filesystem.

An example usage is:
$ docker run -d -p 4444:4444 -v `pwd`:`pwd` selenium/standalone-firefox:latest
==== Change config.php file ====
In the Moodle <tt>config.php</tt> file you must change the <tt>$CFG->behat_wwwroot</tt> to an address that can be reached from within the docker image.

You cannot use <tt>localhost</tt> or <tt>127.0.0.1</tt> as this will be the IP address of the docker container itself.

On some more recent versions of Docker you can use <tt>http://host.docker.internal/</tt>, for example if my site is located at <tt>/sm</tt> on my development machine, I can set the following:
$CFG->behat_wwwroot = 'http://host.docker.internal/sm';

==== Manually configuring other browsers =====
If you would prefer not to use the <tt>moodle-browser-config</tt> tool but still wish to specify different browsers then you can do so using the <tt>$CFG->behat_profiles</tt> array. Each key/value pair contains a profile name, and the configuration for that profile. For example:
$CFG->behat_profiles = [
'chrome' => [
'browser' => 'chrome',
'tags' => '@javascript',
],
];
[[Acceptance_testing/Browsers|More info about alternative browsers]]
== Troubleshooting ==
=== Increasing timeouts ===
You may see errors such as:
<syntaxhighlight lang="php">
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</syntaxhighlight>
Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it may just mean that your browser did not finish generating parts of the UI before behat tried was finished.

If you find that this happens regularly on different scenarios then you may want to increase the timeout:
$CFG->behat_increasetimeout = 2;
This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

Note: This is usually an indicator that your development machine is not well tuned. A better option would be to find out where the bottleneck is. This is usually the database configuration.

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<syntaxhighlight lang="php">
php admin/tool/behat/cli/util.php --enable
</syntaxhighlight>
''' For parallel runs, all options for initialising parallel runs are valid '''
=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your browser version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.
=== The tests are failing, and the error message is completely useless ===
For example, it just says "Error writing to database" with no stack trace.

Add -vv command-line option to get very verbose output.
=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.
In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');
=== Selenium server is not running ===
==== Chrome specific ====
If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659/ MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper/ Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== Examples ==
=== Quick setup and testing using moodle-docker ===
This is a quick guide to help locally pass tests for your developments, before submitting them:
# Set up a default Moodle install using [https://github.com/moodlehq/moodle-docker moodle-docker], with the database and Moodle version of your choice. See its README for more details. This will start some docker containers.
# Initialize behat to start testing with this command, from the webserver container: <syntaxhighlight lang="php">php admin/tool/behat/cli/init.php</syntaxhighlight>
# Run the behat test of your choice, from the webserver container. For instance: <syntaxhighlight lang="php">vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --tags tool_task</syntaxhighlight>
And you'll see something like:
<syntaxhighlight lang="php">
Moodle 4.0dev (Build: 20210507), 0b47ea0a44a092f9000729ca7b15fff23111538b
Php: 7.3.26, mysqli: 5.7.21, OS: Linux 5.4.0-66-generic x86_64
Run optional tests:

Accessibility: No
Server OS "Linux", Browser: "firefox"
Started at 09-05-2021, 06:00
...................................................................... 70
...............................
12 scenarios (12 passed)
101 steps (101 passed)
2m53.27s (47.61Mb)
</syntaxhighlight>
== See also ==
* [[Acceptance testing for the mobile app]]
[[Category:Quality Assurance]][[Category:Behat]]

== Information to re-home ==
==== Override behat core context for theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php
==== Disable behat context or features to run in theme suite ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To disable specific contexts and features from being executed by a specific theme/suite you can create a <tt>/theme/{MYTHEME}/tests/behat/blacklist.json</tt> file with following format.
<syntaxhighlight lang="php">
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</syntaxhighlight>
The above will:
# disable the use of step_definitions from <tt>behat_grade</tt> and <tt>behat_navigation</tt> while running theme suite; and
# disable running of scenarios in <tt>auth/tests/behat/login.feature</tt> and <tt>grade/tests/behat/grade_hidden_items.feature</tt>.
==== Override core behat selectors ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.
==== Write new tests and behat methods ====
Note: This documentation needs to be rehomed to a location related to writing Behat tests.

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

PHPUnit integration

2021-12-01T12:30:09Z

Bencellis: /* Organisation of test files */

Moodle PHPUnit integration was created to simplify using of PHPUnit framework in Moodle. It consists of specialised bootstrap script, utility scripts that initialise testing environment and highly optimised custom test case classes that handle automatic global state resetting after test that includes global variables, database rollback and purging of dataroot. We also try to bridge the gap between the design and coding style of Moodle and PHPUnit.

Most of the documentation at http://www.phpunit.de/manual/3.6/en/index.html is relevant to Moodle PHPUnit integration, exceptions and additions are described below.

=Definitions=
These definitions may differ in each testing framework or programming language. The definitions here should be valid for PHPUnit framework with our Moodle tweaks.

;Test suite: is a collection of test cases or other suites usually related to one area of the product. It is defined in PHPUnit configuration files (phpunit.xml by default).

;Test file: is a file with *_test.php name which contains a test case. (*_Test.php in PHPUnit)

;Test case: is a class with test methods. Their name should match the name of the test file (*_test). Moodle tests extend basic_testcase or advanced_testcase. (PHPUnit\Framework\TestCase in PHPUnit)

;Test: is a public method starting with "test_" prefix defined in test case class. It is the implementation of the test procedure.

;Assertion: is the actual comparison of expected and actual behaviour of the tested code.

=Organisation of test files=
All testing related files are stored in <syntaxhighlight lang="php">/tests/</syntaxhighlight> subdirectories:

* <syntaxhighlight lang="php">.../tests/*_test.php</syntaxhighlight> are PHPUnit test files
* <syntaxhighlight lang="php">.../tests/fixtures/</syntaxhighlight> contains auxiliary files used in tests
* <syntaxhighlight lang="php">.../tests/performance/</syntaxhighlight> is reserved for performance testing scripts
* <syntaxhighlight lang="php">.../tests/generator/</syntaxhighlight> for generators
* <syntaxhighlight lang="php">.../tests/other/</syntaxhighlight> is used for everything else that does not fit

==Class and file naming rules==
=== Actual (Moodle 3.11 and up) ===
The class names of all testcases need to be unique, with namespaces being used to guarantee this.

* All test case class names should match the file name, for example: the class name for <tt>mod/forum/test/this_test.php</tt> should be <tt>this_test</tt>.
* All test case classes end with '''_test''' suffix.
* All test case classes should use the namespace they belong to, for example: the namespace for <tt>mod/forum/test/this_test.php</tt> should be <tt>mod_forum</tt>. With sub-namespaces allowed to better match what is being tested.

Read about the rules for namespaces @ [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|coding style]] (grand summary = 100% the same rules that are applied to **/classes directories).

Example:
* class '''lib_test''' (with namespace <tt>mod_forum</tt>) is expected in file '''/mod/forum/tests/lib_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit mod/forum/tests/lib_test.php</syntaxhighlight>
* class '''text_test''' (with namespace <tt>core</tt>) is expected in file '''/lib/texts/text_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit lib/texts/text_test.php</syntaxhighlight>

=== Previous (until Moodle 3.10) ===
Frankenstyle prefix is used to guarantee this. Since 2.6 it is possible to use automatic class loader when executing individual unit tests.

* All test case classes start with Frankenstyle prefix, for example: '''mod_forum_''', '''block_html_''', '''core_'''.
* All test case classes end with '''_testcase''' suffix.
* File names are constructed from the class names - the Frankenstyle prefix is removed, suffix '''_testcase''' is replaced with '''_test'''

Finally, but not less important, test case classes can be (recommended) namespaces. Read about the rules to do it properly @ [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|coding style]] (grand summary = 100% the same rules that are applied to **/classes directories).

Example:
* class '''mod_forum_lib_testcase''' is expected in file '''/mod/forum/tests/lib_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit mod_forum_lib_testcase</syntaxhighlight>
* class '''core_text_testcase''' is expected in file '''/lib/texts/text_test.php''', executed as <syntaxhighlight lang="php">vendor/bin/phpunit core_text_testcase</syntaxhighlight>

=basic_testcase=
''basic_testcase'' is useful for simple tests that do not modify database or global variables. If something accidentally changes global state the test fails. This test case class is nearly identical to PHPUnit_Framework_TestCase used in PHPUnit documentation.

<syntaxhighlight lang="php">
<?php // File: mod/myplugin/tests/sample_test.php

namespace mod_myplugin;

class sample_test extends \basic_testcase {
public function test_equals() {
$a = 1 + 2;
$this->assertEquals(3, $a);
}
}
</syntaxhighlight>

=advanced_testcase=
By default each test starts with fresh new moodle installation. Test may modify database content, files or global variables. It is possible to use data generators to create new course, categories, module instances and other objects, alternatively table data can be preloaded from XML or CSV files.

<syntaxhighlight lang="php">
<?php // File: mod/myplugin/tests/complex_test.php

namespace mod_myplugin;

class complex_test extends \advanced_testcase {
public function test_isadmin() {
global $DB;

$this->resetAfterTest(true); // reset all changes automatically after this test

$this->assertFalse(is_siteadmin()); // by default no user is logged-in
$this->setUser(2); // switch $USER
$this->assertTrue(is_siteadmin()); // admin is logged-in now

$DB->delete_records('user', array()); // lets do something crazy

$this->resetAllData(); // that was not a good idea, let's go back
$this->assertTrue($admin = $DB->record_exists('user', array('id'=>2)));
$this->assertFalse(is_siteadmin());
}
}
</syntaxhighlight>

==Extra methods==
; resetAfterTest(bool) : true means reset automatically after test, false means keep changes to next test method, default null means detect changes
; resetAllData() : reset global state in the middle of a test
; setAdminUser() : set current $USER as admin
; setGuestUser() : set current $USER as guest
; setUser() : set current $USER to a specific user - use getDataGenerator() to create one
; getDataGenerator() : returns data generator instance - use if you need to add new courses, users, etc.
; preventResetByRollback() : terminates active transactions, useful only when test contains own database transaction handling
; createXXXDataSet() : creates in memory structure of database table contents, used in loadDataSet() (eg: createXMLDataSet(), createCsvDataSet(), createFlatXMLDataSet())
; loadDataSet() : bulk loading of table contents
; getDebuggingMessages() : Return debugging messages from the current test. (Moodle 2.4 and upwards)
; resetDebugging() : Clear all previous debugging messages in current test. (Moodle 2.4 and upwards)
; assertDebuggingCalled() : Assert that exactly debugging was just called once. (Moodle 2.4 and upwards)
; assertDebuggingNotCalled() : Assert no debugging happened. (Moodle 2.4 and upwards)
; assertDebuggingCalledCount() : Asserts how many times debugging has been called. (Moodle 3.1 and upwards)
; [[Writing PHPUnit tests#Testing sending of messages|redirectMessages()]]: Captures ongoing messages for later testing (Moodle 2.4 and upwards)
; [[Writing PHPUnit tests#Testing_sending_of_emails|redirectEmails()]]: Captures ongoing emails for later testing (Moodle 2.6 and upwards)

==Restrictions==
* it is not possible to modify database structure such as create new table or drop columns from advanced_testcase.

=Moodle specific features=
* detection of global state changes - helps with detection of unintended changes in database
* highly optimised global state reset
* dataset loading - this feature is copied from PHPUnit database testcases
* automatic generation of phpunit.xml - init script builds list of plugin testcases
* database driver testing class - used for functional DB tests
* SimpleTest emulation class - helps with migration of old tests
* debugging() interception - enables to control and test any expected debug output. (Moodle 2.4 and upwards)

=Limitations=
* no Selenium support
* no support for PHPUnit_Extensions_Database_TestCase - it is possible to use data set loader only

=See also=
* [[Writing PHPUnit tests]]

[[Category:Unit testing]]

Writing PHPUnit tests

2021-11-30T15:52:21Z

Bencellis: /* Assertions */

{{Moodle 2.3}}

Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).

=Namespaces=

All the stuff under **/tests directories is [[Coding style#Namespaces_within_.2A.2A.2Ftests_directories|subject to some simple rules]] when using namespaces. They apply to test cases, fixtures, generators and, in general, any class within those directories. Take a look to them! (grand summary = 100% the same rules that are applied to **/classes directories).

=Testcase classes=

There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. '''Please note it is strongly recommended to put only one testcase into each class file.'''
;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code.
;provider_testcase: Enhanced testcase class, enhanced for easy testing of [[Privacy API|Privacy Providers]].

There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.

== Assertions ==

The complete list of assertions can be found in the links below.

{| class="wikitable" border="1"
|-
! Moodle version
! PHPUnit version
!Links
|-
| Moodle 3.11
| PHPUnit 9.5
|[https://phpunit.readthedocs.io/en/9.5/assertions.html Documentation]
|-
| Moodle 3.10
| PHPUnit 8.5
|[https://phpunit.readthedocs.io/en/8.5/assertions.html Documentation]
|-
| Moodle 3.7 - 3.9
| PHPUnit 7.5
|[https://phpunit.readthedocs.io/en/7.5/assertions.html Documentation]
|-
| Moodle 3.4 - 3.6
| PHPUnit 6.5
|[https://phpunit.de/manual/6.5/en/assertions.html Documentation]

|}

==Sample plugin testcase==

PHPUnit tests are located in <tt>tests/*_test.php</tt> files in your plugin, for example <tt>mod/myplugin/tests/sample_test.php</tt>, the file should contain only one class that extends <tt>advanced_testcase</tt>:

<syntaxhighlight lang="php">
namespace mod_myplugin;

class sample_test extends \advanced_testcase {
public function test_adding() {
$this->assertEquals(2, 1+2);
}
}
</syntaxhighlight>

See [[PHPUnit integration#Class and file naming rules]] for more information.

==Inclusion of Moodle library files==

If you want to include some Moodle library files you should always declare '''global $CFG'''. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically.

==Automatic state reset==
By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.

If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using <tt>$this->resetAfterTest()</tt>.

<syntaxhighlight lang="php">
namespace mod_myplugin;

class test_something extends \advanced_testcase {
public function test_deleting() {
global $DB;
$this->resetAfterTest(true);
$DB->delete_records('user');
$this->assertEmpty($DB->get_records('user'));
}
public function test_user_table_was_reset() {
global $DB;
$this->assertEquals(2, $DB->count_records('user', array()));
}
}
</syntaxhighlight>

=Generators=

Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.

Note if you are using PHPUnit [https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers @dataProvider] functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function. Data providers '''must not instantiate/create data'''. Just define it. And then, the test body can proceed with the instantiation/creation.

==Creating users==
At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use:
<syntaxhighlight lang="php">
$user = $this->getDataGenerator()->create_user();
</syntaxhighlight>

You may also specify properties of the user account, for example:
<syntaxhighlight lang="php">
$user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1'));
</syntaxhighlight>

By default no user is logged-in, use setUser() method to change current $USER value:
<syntaxhighlight lang="php">
$this->setUser($user1);
</syntaxhighlight>

Guest and admin accounts have a shortcut methods:
<syntaxhighlight lang="php">
$this->setGuestUser();
$this->setAdminUser();
</syntaxhighlight>

Null can be used to set current user back to not-logged-in:
<syntaxhighlight lang="php">
$this->setUser(null);
</syntaxhighlight>

==Creating course categories==

<syntaxhighlight lang="php">
$category1 = $this->getDataGenerator()->create_category();
$category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id));
</syntaxhighlight>

==Creating courses==

<syntaxhighlight lang="php">
$course1 = $this->getDataGenerator()->create_course();

$category = $this->getDataGenerator()->create_category();
$course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id));
</syntaxhighlight>

==Creating activities==

Some activity plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.

Example of creation of new course with one page resource:

<syntaxhighlight lang="php">
$course = $this->getDataGenerator()->create_course();
$generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
$generator->create_instance(array('course'=>$course->id));
</syntaxhighlight>

The following is functionally the same, but a bit shorter:
<syntaxhighlight lang="php">
$course = $this->getDataGenerator()->create_course();
$page = $this->getDataGenerator()->create_module('page', array('course' => $course->id));
</syntaxhighlight>

==Creating cohorts==
{{Moodle 2.4}}
Since 2.4 there the data generator supports creation of new cohorts.

<syntaxhighlight lang="php">
$cohort = $this->getDataGenerator()->create_cohort();
</syntaxhighlight>

==Simplified user enrolments==
{{Moodle 2.4}}
Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins.

<syntaxhighlight lang="php">
$this->getDataGenerator()->enrol_user($userid, $courseid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');
</syntaxhighlight>

==Creating scales==

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_scale();
$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat));
</syntaxhighlight>

==Creating roles==

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_role();
$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype));
</syntaxhighlight>

==Creating tags==

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_tag();
$this->getDataGenerator()->create_tag(array(
'userid' => $userid,
'rawname' => $rawname,
'name' => $name,
'description' => $description,
'descriptionformat' => $descriptionformat,
'flag' => $flag
));
</syntaxhighlight>

==Groups==

===Creating groups===
<syntaxhighlight lang="php">
$this->getDataGenerator()->create_group(array('courseid' => $courseid));
$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</syntaxhighlight>

===Adding users to groups===
<syntaxhighlight lang="php">
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid));
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid));
</syntaxhighlight>

===Creating groupings===
<syntaxhighlight lang="php">
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid));
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</syntaxhighlight>

===Adding groups to groupings===
<syntaxhighlight lang="php">
$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid));
</syntaxhighlight>

==Repositories==

===Creating repository instances===
{{Moodle 2.5}}
Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_repository($type, $record, $options);
</syntaxhighlight>

===Creating repository types===
{{Moodle 2.5}}
Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_repository_type($type, $record, $options);
</syntaxhighlight>

==Creating grades==

===Grade categories===

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid));
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname));
</syntaxhighlight>

===Grade items===

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_grade_item();
$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype));
</syntaxhighlight>

===Outcomes===

<syntaxhighlight lang="php">
$this->getDataGenerator()->create_grade_outcome();
$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname));
</syntaxhighlight>

==Other types of plugin==
{{Moodle 2.5}}
Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname);

For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith.

=Long tests=

All standard test should execute as fast as possible. Tests that take a loner time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php.

=Large test data=
See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [[PHPUnit_integration#Extra_methods]]

=Testing sending of messages=
{{Moodle 2.4}}
You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages.

To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later.
<syntaxhighlight lang="php">
$this->preventResetByRollback();
$sink = $this->redirectMessages();
//... code that is sending messages
$messages = $sink->get_messages();
$this->assertEquals(3, count($messages));
//.. test messages were generated in correct order with appropriate content
</syntaxhighlight>

=Testing sending of emails=
{{Moodle 2.6}}
You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails.

To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later.
<syntaxhighlight lang="php">
unset_config('noemailever');
$sink = $this->redirectEmails();
//... code that is sending email
$messages = $sink->get_messages();
$this->assertEquals(1, count($messages));
</syntaxhighlight>

=Logstores=
You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute.
<syntaxhighlight lang="php">
$this->preventResetByRollback();
set_config('enabled_stores', 'logstore_standard', 'tool_log');
set_config('buffersize', 0, 'logstore_standard');
get_log_manager(true);
</syntaxhighlight>

=Check your coverage=
{{Moodle 3.7}}
PHPUnit has the ability to generate code coverage information for your unit tests.

Prior to Moodle 3.7, this coverage would load all files and generate coverage for everything regardless of whether that file could be covered at all, or whether it was intentionally covered.

Since Moodle 3.7 the '''phpunit.xml''' configuration contains generated coverage include and exclude information for each component.

==Generating include and exclude configuration==
{{Moodle 3.11}}
You can programatically describe which files will be checked for coverage by creating a <tt>coverage.php</tt> file alongside the tests that you are writing.

The <tt>coverage.php</tt> file allows you to list include and exclude files and folders within the component being
tested. All paths specified are relative to the component being tested. For example, when working with '''mod_forum''' your
code will be in '''mod/forum''', and its unit tests will be in '''mod/forum/tests/example_test.php'''. The coverage file
for this would be in '''mod/forum/tests/coverage.php''' and all paths specified would be relative to '''mod/forum'''.

It is possible to specify a combination of included files, included folders, excluded files, and
excluded folders. This would allow you, for example, to include the entire '''classes''' directory, but exclude
a specific file or folder within it.

The following is an example <tt>coverage.php</tt> file from '''mod_forum''':

Note: For Moodle versions 3.7 to 3.10, the [https://docs.moodle.org/dev/index.php?title=Writing_PHPUnit_tests&oldid=58177#Check_your_coverage syntax used] was slightly different.

<syntaxhighlight lang="php">
return new class extends phpunit_coverage_info {
/** @var array The list of folders relative to the plugin root to include in coverage generation. */
protected $includelistfolders = [
'classes',
'externallib.php',
];

/** @var array The list of files relative to the plugin root to include in coverage generation. */
protected $includelistfiles = [];

/** @var array The list of folders relative to the plugin root to exclude from coverage generation. */
protected $excludelistfolders = [];

/** @var array The list of files relative to the plugin root to exclude from coverage generation. */
protected $excludelistfiles = [];
};
</syntaxhighlight>

Also, note that you can better define which class or function each test is effectively covering by using the <tt>@covers</tt> annotation as [https://phpunit.readthedocs.io/en/9.5/code-coverage-analysis.html#specifying-covered-code-parts described in the documention].

=Best practice=

There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below.

==Check your coverage==
PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since
Moodle 3.7. We recommend that you consider checking the coverage of your plugins when you write your code.

==Keep use of resetAfterTest to a minimum==
Although many of the examples described above use the <syntaxhighlight lang="php">resetAfterTest</syntaxhighlight> nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to.
Generally speaking you should aim to write code which is mockable, and does not require real fixtures.
Use of resetAfterTest will also slow your tests down.

==Be careful with shared setUp and instance variables==

You should be careful of how you create and use instance variables in PHPUnit tests for two main reasons:

Firstly, if you create any fixtures in the setUp, or call the resetAfterTest function, these fixtures and conditions will apply for _all_ tests in the testsuite.
You will not be able to add another test to the suite which does not require these conditions without those conditions being fulfilled anyway.
This can lead to slow tests.

Secondly, because of the way in which PHPUnit operates. it creates an instance of each testcase during its bootstrap phase. These are stored in memory until the _entire suite_ completes.
This means that any fixture which is setup and not actively discarded will not be garbage collected and lead to memory bloat.
In severe cases this can lead to memory exhaustion.

==Make use of the dataProvider functionality==

The dataProvider functionality of PHPUnit is an extremely powerful and useful feature which allows you to verify a function quickly and easily with a range of different conditions.
However, the following rules should be followed when using dataProviders:
* Keep addition of resettable data requring resetAfterTest to a minimum - this will lead to many slow tests
* Data providers '''must not instantiate/create data'''. Just define it. And then, the test body can proceed with the instantiation/creation. The dataProvider is called after the testSuite is instantiated, but before any tests are run. Each test will run a full setUp and tearDown, which will destroy any data which was created.

<syntaxhighlight lang="php">
/**
* Test function accepts parameters passed from the specified data provider.
*
* @dataProvider foobar_provider
* @param int $foor
* @param int $bar
*/
public function test_foobar(int $foo, int $bar) {
// Perform the tests here.
}

/**
* Data provider for {@see self::test_foobar()}.
*
* @return array List of data sets - (string) data set name => (array) data
*/
public function foobar_provider(): array {
return [
'Same numbers' => [
'foo' => 42,
'bar' => 42,
],
'Different numbers' => [
'foo' => 21,
'bar' => 84,
],
];
}
</syntaxhighlight>

=Extra test settings=

Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php.

There are several ways how to inject your custom settings:
* define test setting constants in your phpunit.xml file
* define test setting constants in your config.php

These constants may be then used in your test or plugin code.

=Upgrading unit tests to work with Moodle 3.11 and up (PHPUnit 9.5)=
{{Moodle 3.11}}

With Moodle 3.11, '''PHPUnit was upgraded to 9.5''' (from 8.5 being used in previous versions). This was done to '''better align''' the testing environment with PHP versions supported by Moodle 3.11 (7.3, 7.4 and [[Moodle and PHP|8.0]]) (see MDL-71036 and linked issues for more details).

While a lot of existing tests will work without modification with PHPUnit 9.5, you will get a '''good number of deprecation warnings''' ("W" in the tests output) that '''should be replaced by their new counterparts as soon as possible''', because all those warnings will become errors with next PHPUnit upgrade.

To find more information about the changes coming with PHPUnit 9.5, it's recommended to read the following resources:

* [https://thephp.cc/news/2020/02/migrating-to-phpunit-9 A good article] explaining all the main changes in the release.
* [https://phpunit.de/announcements/phpunit-9.html PHPUnit 9 release Announcement].
* These multiple detailed changelogs (because all them have included modifications requiring changes): [https://github.com/sebastianbergmann/phpunit/blob/9.0.0/ChangeLog-9.0.md 9.0], [https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md 9.1], [https://github.com/sebastianbergmann/phpunit/blob/9.3.0/ChangeLog-9.3.md 9.3], [https://github.com/sebastianbergmann/phpunit/blob/9.5.0/ChangeLog-9.5.md 9.5]
* [https://phpunit.readthedocs.io/en/9.5/ Official PHPUnit manual].

A good summary of all the '''changes and replacements to perform''' is available in the [https://github.com/moodle/moodle/blob/e3a46964dc6d8ca1558c6e1e8dfdf3c1745eeaed/lib/upgrade.txt#L5-L65 lib/upgrade.txt] file. With main points being:
* All the changes that were deprecated with PHPUnit 8.5 (see the section below) are now removed and will lead to errors.
* <tt>assertContains()</tt> now performs stricter comparison (like <tt>assertSame()</tt> does). New <tt>assertContainsEquals()</tt> has been created to provide the old behavior.
* Changes to the <tt>phpunit.xml</tt> schema, mostly internal. These only will impact if you are using custom <tt>phpunit.xml</tt> files:
** The previous <tt><filter></tt> section is now (better) called <tt><coverage></tt>. And, within it:
*** <tt><whitelist></tt> has been replaced by <tt><include></tt>.
*** <tt><exclude></tt> is not a child of <tt><whitelist></tt> anymore, but of <tt><coverage></tt>.
** But with implications when defining the [[#Check your coverage|coverage information]] because <tt>$whitelistxxx</tt> properties used by the <tt>coverage.php</tt> files have been deprecated, instead use <tt>includelistfolders</tt> and <tt>includelistfiles</tt> (to better map the elements in the xml).
* Warning: It's not possible to run individual test files any more. Use any of the alternative execution methods (filter, suite, config) to specify which tests you want to run. This will be hopefully fixed in MDL-71049 once it has been agreed which the best way to proceed is.
* Deprecations, deprecations, deprecations. Lots of them in often used assertions: file assertions, regexp assertions, exception expectations... again, note that all them will become errors with the next PHPUnit update, so '''the recommendation is to update them ASAP'''.
Finally, it's also a good idea to [https://github.com/moodle/moodle/compare/fc335f5...713722c browse the changes associated with the issue], hopefully with useful explanations in the commit messages.

=Upgrading unit tests to work with Moodle 3.10 and up (PHPUnit 8.5)=
{{Moodle 3.10}}

With Moodle 3.10, '''PHPUnit was upgraded to 8.5''' (from 7.5 being used in older version). This was done to '''better align''' the testing environment with PHP versions supported by Moodle 3.10 (7.2, 7.3 and 7.4) and also to provide an '''easier jump to PHPUnit 9.x''' that will be needed for Moodle 3.11 (php versions 7.3, 7.4 and 8.0), removing a lot of deprecated stuff in advance. (see MDL-67673 and MDL-64600 for more details).

While 99% of existing tests will work without modification with PHPUnit 8.5, you will get a '''good number of deprecation warnings''' ("W" in the tests output) that '''should be replaced by their new counterparts as soon as possible''', because all those warnings will become errors with next PHPUnit upgrade.

To find more information about the changes coming with PHPUnit 8.5, it's recommended to read the following resources:

* [https://thephp.cc/news/2019/02/help-my-tests-stopped-working A good article] explaining all the main changes in the release.
* [https://phpunit.de/announcements/phpunit-8.html PHPUnit 8 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/130104cf796a88dd1547dc5beb8bd555c2deb55e/ChangeLog-8.0.md Detailed changelog of the release].
* [https://phpunit.readthedocs.io/en/8.5/ Official PHPUnit manual].
* A good summary of all the '''changes and replacements to perform''' is available in the [https://github.com/moodle/moodle/blob/6594c54b2eef62499d304bfa0939999e3a14246e/lib/upgrade.txt#L5-L37 lib/upgrade.txt] file. With main points being:
** Support for PHP 7.0 dropped (because all template methods (<tt>setUp()</tt>, <tt>tearDown()</tt>..) now require to return void. This will mostly impact 3rd-part plugins that were still running the same tests against old branches of Moodle with PHP 7.0 support.
** <tt>PHPUnit/DBUnit</tt> has been removed are replaced by a lightweight alternative.
** Deprecations, deprecations, deprecations. Lots of them in often used assertions (<tt>assertContains()</tt>, <tt>assertEquals()</tt>...) and also <tt>@expectedExceptionXXX</tt> annotations. Again, note that all them will become errors with PHPUnit 9.
* Finally, it's also a good idea to [https://github.com/moodle/moodle/compare/5903054...b13ec3c browse the changes associated with the issue], hopefully with useful enough explanations in the commit messages.

=Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5)=
{{Moodle 3.7}}

With Moodle 3.7, '''PHPUnit was upgraded to 7.5''' (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see MDL-65204 and linked issues for more details). While internally [https://phpunit.de/announcements/phpunit-7.html a lot of things changed with PHPUnit 7] (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources:
* [https://phpunit.de/announcements/phpunit-7.html PHPUnit 7 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/520723129e2b3fc1dc4c0953e43c9d40e1ecb352/ChangeLog-7.5.md Detailed changelog of the release], paying special attention to the added, deprecated, changed and deleted sections.
* [https://phpunit.readthedocs.io/en/7.5/ Official PHPUnit manual].
* Changes performed into core, mainly: new, stricter, signatures ([https://github.com/moodle/moodle/commit/26218b7 26218b7]) and assertEquals() changes, specially important when comparing strings, now performed using strict (===) equals ([https://github.com/moodle/moodle/commit/85f47ba 85f47ba]).

=Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6)=
{{Moodle 3.4}}

With Moodle 3.4, '''PHPUnit was upgraded to 6.4''' (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see MDL-60611 and linked issues for more details). While internally [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues a lot of things changed with PHPUnit 6] (namespaced classes being the more noticeable), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

Still, in '''some cases, it will impossible to maintain compatibility''' of tests between old (pre 3.4) tests and new ones, especially '''when direct use of any phpunit class is performed'''. Luckily, both travis and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly.

To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources:
* [https://thephp.cc/news/2017/02/migrating-to-phpunit-6 A very good mini-guide] showing all the important changes.
* [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues PHPUnit 6 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/9d0c024d2099531442d862b66b0ad7cf35ed8e78/ChangeLog-6.0.md Detailed changelog of the release], paying special attention to the changed and deleted sections.
* Changes performed into core, mainly: namespace class renaming ([https://github.com/moodle/moodle/commit/801a372dadb6e11c8781547603e3f0a59ce5638f 801a372]) and deprecated stuff ([https://github.com/moodle/moodle/commit/796e48a58bf18533bdca423fff7949ab119101c4 796e48a])

=See also=
* [[PHPUnit integration]]
* [[PHPUnit]]

[[Category:Unit testing]]

PHPUnit

2021-11-30T15:43:55Z

Bencellis: /* PHPUnit versions */ Add Links to documentation

{{Moodle 2.3}}
=What is PHPUnit=
PHPUnit by Sebastian Bergmann is an advanced unit testing framework for PHP. It is installed as Composer dependency and is not part of Moodle installation. To run PHPUnit tests, you have to manually install it on your development computer or test server.

Read the excellent guide at
* [https://phpunit.de/documentation.html PHPUnit Manual]
=Installation of PHPUnit via Composer=
* Install Composer
Instructions for installing composer on all platforms are here: https://getcomposer.org/download/

Install the composer.phar file to your moodle folder.
* Execute Composer installer
cd /your/moodle/dirroot

php composer.phar install
(If that gives you connection problems try...)
php composer.phar install --prefer-source
Troubleshooting:
* On Windows if you are behind a proxy you will need to setup an environment variable called HTTP_PROXY with a value detailing your HTTP Proxy address and port before composer will correctly download files.
* You may be prompted for github credentials when installing composer dependencies.
** This is used to generate an personal access token to avoid being rate limited by github.
** If you have Two Factor Authentication enabled on your github account, or do not wish to supply your own credentials you will need to generate a token manually:
*** Visit https://github.com/settings/applications and request personal access token
*** Copy this token and add it to your [https://gist.github.com/andrewnicols/c5377ed25a9df1006ce1 ~/.composer/config.json]
** ( See [https://github.com/composer/composer/issues/2280 composer issue #2280] for further details of this bug.)

Detailed instructions:
* [http://getcomposer.org/doc/00-intro.md Composer documentation]
Detailed instructions:
* [[PHPUnit installation in Windows]]
* [[PHPUnit installation in OS X]]
== Uninstalling previous PEAR based version ==
Before using composer, this page used to suggest to install phpunit via PEAR. If you did so, you may wish to uninstall that package now. This should work:
$ pear uninstall phpunit/DbUnit
$ pear uninstall phpunit/PHPUnit
= PHPUnit versions =
The following table shows what PHPUnit version is installed in which Moodle version when using the default composer setup.
{| class="wikitable" border="1"
|-
! Moodle version
! PHPUnit version
!Links
|-
| Moodle 3.11
| PHPUnit 9.5
|[https://phpunit.readthedocs.io/en/9.5/ Documentation]
|-
| Moodle 3.10
| PHPUnit 8.5
|[https://phpunit.readthedocs.io/en/8.5/ Documentation]
|-
| Moodle 3.7 - 3.9
| PHPUnit 7.5
|[https://phpunit.readthedocs.io/en/7.5/ Documentation]
|-
| Moodle 3.4 - 3.6
| PHPUnit 6.5
|[https://phpunit.de/manual/6.5/en/ Documentation]
|-
| Moodle 3.2 - 3.3
| PHPUnit 5.5
|[https://phpunit.de/manual/5.5/en/ Documentation]
|-
| Moodle 3.1
| PHPUnit 4.8.27
|[https://phpunit.de/manual/4.8/en/ Documention 4.8]
|}
=Initialisation of test environment=
Our PHPUnit integration requires a dedicated database and dataroot. First, add a new dataroot directory and prefix into your config.php, you can find examples in config-dist.php (scroll down to 'Section 9').
$CFG->phpunit_prefix = 'phpu_';
$CFG->phpunit_dataroot = '/home/example/phpu_moodledata';
Some PHPUnit tests require a live internet connection. If your system does not have a direct connection to the Internet, you also need to specify your proxy in config.php - even though you normally specify it by using the admin settings user interface. (If you do not use a proxy, you can skip this step.) Check the settings on the relevant admin settings page, or from the mdl_config table in your database, if you are unsure of the correct values.
// Normal proxy settings
$CFG->proxyhost = 'wwwcache.example.org';
$CFG->proxyport = 80;
$CFG->proxytype = 'HTTP';
$CFG->proxybypass = 'localhost, 127.0.0.1, .example.org';
// Omit the next lines if your proxy doesn't need a username/password:
$CFG->proxyuser = 'systemusername';
$CFG->proxypassword = 'systempassword';
From Moodle 2.8.5 onwards, you can also provide specific database settings for unit testing (if these are not provided, the standard database settings will be used):
<pre>$CFG->phpunit_dbtype = 'pgsql'; // 'pgsql', 'mariadb', 'mysqli', 'mssql', 'sqlsrv' or 'oci'
$CFG->phpunit_dblibrary = 'native'; // 'native' only at the moment
$CFG->phpunit_dbhost = '127.0.0.1'; // eg 'localhost' or 'db.isp.com' or IP
$CFG->phpunit_dbname = 'mytestdb'; // database name, eg moodle
$CFG->phpunit_dbuser = 'postgres'; // your database username
$CFG->phpunit_dbpass = 'some_password'; // your database password</pre>
Then you need to initialise the test environment using following command.
cd /home/example/moodle
php admin/tool/phpunit/cli/init.php
This command has to be repeated after any upgrade, plugin (un)installation or if you have added tests to a plugin you are developing:

'''NOTE:''' make sure that your php cli executable (or the one you want to use) is correctly on your path as the individual init scripts will call it repeatedly. Also, ensure en_AU locale is installed on your server.
== LDAP ==
If you want to run LDAP unit tests you must have a working, configured LDAP environment on your test server. There must be a basic LDAP tree structure in place or tests will fail with "Search: No such object". Build an LDAP tree structure in a new shell prompt:
<pre>
$ ldapadd -H ldap://127.0.0.1 -D "cn=admin,dc=yourcomputer,dc=local" -W
dn:dc=yourcomputer,dc=local
objectClass:dcObject
objectClass:organizationalUnit
dc:yourcomputer
ou:YOURCOMPUTER
</pre>
In config.php tell Moodle where to look for your test LDAP environment:
<pre>
// Constants for auth/ldap tests.
define('TEST_AUTH_LDAP_HOST_URL', 'ldap://127.0.0.1');
define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_AUTH_LDAP_BIND_PW', '*');
define('TEST_AUTH_LDAP_DOMAIN', 'dc=yourcomputer,dc=local');

// Constants for enrol/ldap tests.
define('TEST_ENROL_LDAP_HOST_URL', 'ldap://127.0.0.1');
define('TEST_ENROL_LDAP_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_ENROL_LDAP_BIND_PW', '*');
define('TEST_ENROL_LDAP_DOMAIN', 'dc=yourcomputer,dc=local');

// Constants for lib/ldap tests.
define('TEST_LDAPLIB_HOST_URL', 'ldap://127.0.0.1');
define('TEST_LDAPLIB_BIND_DN', 'cn=admin,dc=yourcomputer,dc=local');
define('TEST_LDAPLIB_BIND_PW', '*');
define('TEST_LDAPLIB_DOMAIN', 'dc=yourcomputer,dc=local');
</pre>
=Test execution=
To execute all test suites from main configuration file execute the <syntaxhighlight lang="php">vendor/bin/phpunit</syntaxhighlight> script from your <syntaxhighlight lang="php">$CFG->dirroot</syntaxhighlight> directory.
cd /home/example/moodle
vendor/bin/phpunit
The rest of examples uses <syntaxhighlight lang="php">phpunit</syntaxhighlight>, please substitute it with <syntaxhighlight lang="php">vendor/bin/phpunit</syntaxhighlight> or create a shortcut in your dirroot.
In IDEs, you may need to specify the path to the PHPUnit configuration file. Use the absolute path to <syntaxhighlight lang="php">phpunit.xml</syntaxhighlight> from your <syntaxhighlight lang="php">$CFG->dirroot</syntaxhighlight>.

There is an alternative script for running of tests via web interface: <syntaxhighlight lang="php">admin/tool/phpunit/webrunner.php</syntaxhighlight>. Use this as the last resort only when you cannot use the command-line interface on your test server. It will most probably break due to permissions problems if you try to execute it both from command-line and from webrunner. This feature is not officially supported.

===How to run only some tests===
==== Running a single test quickly (PHPUnit 9) ====
{{Moodle 3.11}}
The fastest way to run a single test in PHPUnit 9.5 and higher (Moodle 3.11 and higher) is to use the filter argument:
<syntaxhighlight lang="bash">
vendor/bin/phpunit --filter tool_dataprivacy_metadata_registry_testcase
</syntaxhighlight>
To run all tests provided by the single component, use suite and the name it has in the phpunit.xml file. Example:
<syntaxhighlight lang="bash">
vendor/bin/phpunit --testsuite workshopform_accumulative_testsuite
</syntaxhighlight>
Alternatively if you have config files built for each component:
<syntaxhighlight lang="bash">
vendor/bin/phpunit -c mod/workshop/form/accumulative/
</syntaxhighlight>
==== Running a single test quickly (PHPUnit 8) ====
{{Moodle 3.10}}
The fastest way to run a single test in PHPUnit 8.5 and lower (Moodle 3.10 and lower):
<syntaxhighlight lang="bash">
vendor/bin/phpunit my/tests/filename.php
</syntaxhighlight>
so, run this command in the CLI to see a real test in action:
<syntaxhighlight lang="bash">
vendor/bin/phpunit cohort/tests/cohortlib_test.php
</syntaxhighlight>
You can also run a single test method inside a class:
<syntaxhighlight lang="bash">
vendor/bin/phpunit --filter test_function_name path/to/file.php
</syntaxhighlight>
Note: You should be careful because it may be possible to run tests this way which are not included in the normal run if, for example, the file is not placed in the correct location. If you use this method, do at least one full test run (or --group run, as below) to ensure the test can be found.

Filters can also be applied to capture a group of similar tests across all testsuites:
<syntaxhighlight lang="bash">
vendor/bin/phpunit --filter test_flag_user
</syntaxhighlight>
It is also possible to run all tests in a component (subsystem or plugin) by using the testsuite option:
<syntaxhighlight lang="bash">
vendor/bin/phpunit --testsuite mod_forum_testsuite
vendor/bin/phpunit --testsuite core_privacy_testsuite
vendor/bin/phpunit --testsuite core_privacy_testsuite --filter test_component_is_compliant
</syntaxhighlight>
==== Using the @group annotation ====
If you add annotations like
<syntaxhighlight lang="php">
namespace qtype_stack;
/**
* Unit tests for {@link stack_cas_keyval} @ qtype/stack/tests/cas_keyval_test.php.
* @group qtype_stack
*/
class cas_keyval_test extends \basic_testcase {
</syntaxhighlight>
to all the classes in your plugin, then you can run just the tests for your plugin by doing
phpunit --group qtype_stack
Therefore, it is suggested that you annotate all your tests with the Frankenstyle name of your plugin.
==== Using multiple phpunit.xml files ====
It's easy to create alternative phpunit.xml files defining which tests must be run together. For reference, take a look to the default /phpunit.xml available in your base directory once the testing environment has been initialised. After creating the custom file you will be able to run those tests with
vendor/bin/phpunit -c path/to/your/alternative/phpunit/file.xml
Also, for commodity, you can use this command:
php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
It will, automatically, create one valid phpunit.xml file within each component (plugin or subsystem) and other important directories, so later you will be able to execute tests like
vendor/bin/phpunit -c mod/forum[/phpunit.xml] // Note that it's not needed to specify the name of the file (if it is 'phpunit.xml').
vendor/bin/phpunit -c question
vendor/bin/phpunit -c question/type/calculated
vendor/bin/phpunit -c backup
vendor/bin/phpunit -c lib/dml
...
or, also
cd directory/with/phpunit.xml
phpunit
=External test resources=
{{Moodle 2.6}}
By default Moodle phpunit tests contact http://download.moodle.org server when testing curl related functionality. Optionally you may checkout a local copy of the test scripts and access it instead:
# clone https://github.com/moodlehq/moodle-exttests to web directory
# add to your config.php or modify phpunit.xml file <syntaxhighlight lang="php">define('TEST_EXTERNAL_FILES_HTTP_URL', 'http://localhost/moodle-exttests');</syntaxhighlight>
=Writing new tests=
* read [http://www.phpunit.de/manual/current/en/ official PHPUnit online documentation]
* see [[Writing PHPUnit tests]]
=Conversion of existing SimpleTests=
* see [[SimpleTest conversion]]
=PHPUnit support in IDEs=
* [[Setting up Eclipse]]
* [[Setting up Netbeans]]
* [[Setting up PhpStorm]]
=Common Unit Test Problems=
[[Common unit test problems]]
=Performance=
A typical run of full PHPUnit tests for Moodle 2.7 takes 10-20 minutes depending on the machine. If tests run slowly for you:
* Ensure you are using a database and filesystem running on the same machine that is running the tests (not on a remote server).
* Apply developer-only performance settings to your database: [[Postgres Tuning For Developers]]
=Command line tips=
* Run all tests for a plugin (mod_mymodule in this example): vendor/bin/phpunit --testsuite=mod_mymodule_testsuite
* Run only named test: vendor/bin/phpunit --filter=test_my_test_function_name
* Display each test name before running it (useful e.g. if it crashes before the end and you want to know which test it was running at that point) vendor/bin/phpunit --debug (you will probably want to redirect this to a file as it gets very long).
[[Category:Unit testing]]

Course Custom fields

2021-10-25T11:55:38Z

Bencellis: Complete 1st Edit session.

==Introduction==
This plugin type allows you to add course custom field types of which instances can be added to a course. For example, if you want to display radio buttons on the course edit page, then create a radio custom course field plugin, then create an instance of it.
==Default Custom Fields==
Moodle supports five default course custom fields field plugins:
#checkbox
#datetime
#menu
#text
#textarea
== Required Files ==
* version.php
* language file e.g ''lang/en/customfield_radio.php''
* classes/data_controller.php
* classes/field_controller.php
=== The Class Files ===
The two required files in the classes folder must follow the [[Automatic class loading|autoloading and namespacing]] rules e.g. '''''customfield_radio'''''.
==== Field Controller ====
This class is to define the custom field configuration when the user is creating the field. The class must be called '''''field_controller''''' (for autoloading) must extend the '''''\core_customfield\field_controller''''' class.

The class must define a TYPE constant e.g.<syntaxhighlight lang="php">
/**
* Plugin type
*/
const TYPE = 'radio';
</syntaxhighlight>

====== config_form_definition($mform) ======
The only method that must be overridden is '''''config_form_definition()''''' which has a Moodle form passed in as the parameter to which it must attached form elements. Not that for the element values to be saved, the element names must be in the format ''configdata[configname]'' e.g. ''configdata[cfgdefault].''<syntaxhighlight lang="php">
/**
* Add fields for editing a radio field in admin.
*
* @param \MoodleQuickForm $mform
*/
public function config_form_definition(\MoodleQuickForm $mform) {
$mform->addElement('header', 'header_oursettings', get_string('oursettings', 'customfield_radio'));
$mform->setExpanded('header_oursettings', true);

$mform->addElement('text', 'configdata[cfgquestion]', get_string('getquestionprompt', 'customfield_radio'));
$mform->setType('configdata[cfgquestion]', PARAM_TEXT);
$mform->addRule('configdata[cfgquestion]', null, 'maxlength', 150, 'client');
$mform->addRule('configdata[cfgquestion]', null, 'required', null, 'client');

$radioarray=array();
$radioarray[] = $mform->createElement('radio', 'configdata[cfgdefault]', '', get_string('yes'), 1);
$radioarray[] = $mform->createElement('radio', 'configdata[cfgdefault]', '', get_string('no'), 0);
$mform->addGroup($radioarray, 'cfgradioar', get_string('defaultprompt', 'customfield_radio'), array(' '), false);
$mform->setDefault('configdata[cfgdefault]', 0);
}
</syntaxhighlight>There is another function that is useful to be defined.

====== config_form_validation($formdata, $formfiles) ======
This function is used to validate the form values from '''''config_form_definition().''''' It has the submitted form data and files passed in a parameters.<syntaxhighlight lang="php">
/**
* Validate the data on the field configuration form
*
* @param array $data from the add/edit profile field form
* @param array $files
* @return array associative array of error messages
*/
public function config_form_validation(array $data, $files = array()) : array {
$errors = parent::config_form_validation($data, $files);

if ($data['configdata']['uniquevalues']) {
$errors['configdata[uniquevalues]'] = get_string('nouniqueallowed', 'customfield_radio');
}

return $errors;
}
</syntaxhighlight>Please see the '''''\core_customfield\field_controller''''' class defined in '''/customfield/classes/field_controller.php''' for other functions.

==== Data Controller ====
The data controller is the class that deals with the UI in the course edit form. It must be called '''''data_controller''''' (for autoloading) and must extend the '''''\core_customfield\data_controller''''' class. It must override the following methods:
====== datafield() ======
Should return the name of the field in the db table {customfield_data} where the data is stored and must be one of the following:
* '''intvalue''' - can store integer values, this field is indexed
* '''decvalue''' - can store decimal values
* '''shortcharvalue''' - can store character values up to 255 characters long, this field is indexed
* '''charvalue''' - can store character values up to 1333 characters long, this field is not indexed
* '''value''' - can store character values of unlimited length ("text" field in the db)
<syntaxhighlight lang="php">
/**
* Return the name of the field where the information is stored
* @return string
*/
public function datafield() : string {
return 'intvalue';
}

</syntaxhighlight>
====== instance_form_definition($mform) ======
Adds a field element to the instance course edit form. The form is passed into this function.<syntaxhighlight lang="php">
/**
* Add fields for editing a radio field.
*
* @param \MoodleQuickForm $mform
*/
public function instance_form_definition(\MoodleQuickForm $mform) {
$field = $this->get_field();
$config = $field->get('configdata');
$elementname = $this->get_form_element_name();

$radioarray=array();
$radioarray[] = $mform->createElement('radio', $elementname, '', get_string('yes'), 1);
$radioarray[] = $mform->createElement('radio', $elementname, '', get_string('no'), 0);
$mform->addGroup($radioarray, 'cfgradioar', $config['cfgquestion'], array(' '), false);
$mform->setDefault($elementname, $config['cfgdefault']);
$mform->setType($elementname, PARAM_INT);
}
</syntaxhighlight>Please see the '''''\core_customfield\data_controller''''' class defined in '''/customfield/classes/data_controller.php''' for other functions.
==See Also==
* [[User profile fields|User Profile Fields]]

Course Custom fields

2021-10-25T11:36:06Z

Bencellis: Initial Edit of this page

Plugin types

2021-10-25T10:53:42Z

Bencellis: Fixed Custom Fields text

{{Plugins development}}
The M in Moodle stands for modular. The easiest and most maintainable way to add new functionality to Moodle is by writing one of these types of plugin.
== List of Moodle plugin types ==
{| class="wikitable"
|-
! Plugin type
! Component name ([[Frankenstyle]])
! Moodle path
! Description
! Moodle versions
|-
| [[Activity modules]]
| mod
| /mod
| Activity modules are essential types of plugins in Moodle as they provide activities in courses. For example: Forum, Quiz and Assignment.
| 1.0+
|-
| [[Antivirus plugins]]
| antivirus
| /lib/antivirus
| Antivirus scanner plugins provide functionality for virus scanning user uploaded files using third-party virus scanning tools in Moodle. For example: ClamAV.
| 3.1+
|-
| [[Assign_submission_plugins|Assignment submission plugins]]
| assignsubmission
| /mod/assign/submission
| Different forms of assignment submissions
| 2.3+
|-
| [[Assign_feedback_plugins|Assignment feedback plugins]]
| assignfeedback
| /mod/assign/feedback
| Different forms of assignment feedbacks
| 2.3+
|-
| [[Book tools]]
| booktool
| /mod/book/tool
| Small information-displays or tools that can be moved around pages
| 2.1+
|-
| [[Course Custom fields]]
| customfield
| /customfield/field
| Custom field types, used e. g. in Custom course fields
| 3.7+
|-
| [[Database fields]]
| datafield
| /mod/data/field
| Different types of data that may be added to the Database activity module
| 1.6+
|-
| [[Database presets]]
| datapreset
| /mod/data/preset
| Pre-defined templates for the Database activity module
| 1.6+
|-
| [[External tool source|LTI sources]]
| ltisource
| /mod/lti/source
| LTI providers can be added to external tools easily through the external tools interface see [https://docs.moodle.org/en/External_tool Documentation on External Tools]. This type of plugin is specific to LTI providers that need a plugin that can register custom handlers to process LTI messages
| 2.7+
|-
| [[File Converters]]
| fileconverter
| /files/converter
| Allow conversion between different types of user-submitted file. For example from .doc to PDF.
| 3.2+
|-
| [[LTI services]]
| ltiservice
| /mod/lti/service
| Allows the implementation of LTI services as described by the IMS LTI specification
| 2.8+
|-
| [[Machine learning backends]]
| mlbackend
| /lib/mlbackend
| Prediction processors for analytics API
| 3.4+
|-
| [[Forum reports]]
| forumreport
| /mod/forum/report
| Display various reports in the forum activity
| 3.8+
|-
| [[Quiz reports]]
| quiz
| /mod/quiz/report
| Display and analyse the results of quizzes, or just plug miscellaneous behaviour into the quiz module
| 1.1+
|-
| [[Quiz access rules]]
| quizaccess
| /mod/quiz/accessrule
| Add conditions to when or where quizzes can be attempted, for example only from some IP addresses, or student must enter a password first
| 2.2+
|-
| [[SCORM reports]]
| scormreport
| /mod/scorm/report
| Analysis of SCORM attempts
| 2.2+
|-
| [[Workshop grading strategies]]
| workshopform
| /mod/workshop/form
| Define the type of the grading form and implement the calculation of the grade for submission in the [[Workshop]] module
| 2.0+
|-
| [[Workshop allocation methods]]
| workshopallocation
| /mod/workshop/allocation
| Define ways how submissions are assigned for assessment in the [[Workshop]] module
| 2.0+
|-
| [[Workshop evaluation methods]]
| workshopeval
| /mod/workshop/eval
| Implement the calculation of the grade for assessment (grading grade) in the [[Workshop]] module
| 2.0+
|-
| [[Blocks]]
| block
| /blocks
| Small information-displays or tools that can be moved around pages
| 2.0+
|-
| [[Question types]]
| qtype
| /question/type
| Different types of question (e.g. multiple-choice, drag-and-drop) that can be used in quizzes and other activities
| 1.6+
|-
| [[Question behaviours]]
| qbehaviour
| /question/behaviour
| Control how student interact with questions during an attempt
| 2.1+
|-
| [[Question formats|Question import/export formats]]
| qformat
| /question/format
| Import and export question definitions to/from the question bank
| 1.6+
|-
| [[Filters|Text filters]]
| filter
| /filter
| Automatically convert, highlight, and transmogrify text posted into Moodle.
| 1.4+
|-
| [[Editors]]
| editor
| /lib/editor
| Alternative text editors for editing content
| 2.0+
|-
| [[Atto|Atto editor plugins]]
| atto
| /lib/editor/atto/plugins
| Extra functionality for the Atto text editor
| 2.7+
|-
| [[TinyMCE editor plugins]]
| tinymce
| /lib/editor/tinymce/plugins
| Extra functionality for the TinyMCE text editor.
| 2.4+
|-
| [[Enrolment plugins]]
| enrol
| /enrol
| Ways to control who is enrolled in courses
| 2.0+
|-
| [[Authentication plugins]]
| auth
| /auth
| Allows connection to external sources of authentication
| 2.0+
|-
| [[Admin tools]]
| tool
| /admin/tool
| Provides utility scripts useful for various site administration and maintenance tasks
| 2.2+
|-
| [[Log stores]]
| logstore
| /admin/tool/log/store
| Event logs storage back-ends
| 2.7+
|-
| [[Availability conditions]]
| availability
| /availability/condition
| Conditions to restrict user access to activities and sections.
| 2.7+
|-
| [[Calendar types]]
| calendartype
| /calendar/type
| Defines how dates are displayed throughout Moodle
| 2.6+
|-
| [[Messaging consumers]]
| message
| /message/output
| Represent various targets where messages and notifications can be sent to (email, sms, jabber, ...)
| 2.0+
|-
| [[Course formats]]
| format
| /course/format
| Different ways of laying out the activities and blocks in a course
| 1.3+
|-
| [[Data formats]]
| dataformat
| /dataformat
| Formats for data exporting and downloading
| 3.1+
|-
| [[User profile fields]]
| profilefield
| /user/profile/field
| Add new types of data to user profiles
| 1.9+
|-
| [[Reports]]
| report
| /report
| Provides useful views of data in a Moodle site for admins and teachers
| 2.2+
|-
| [[Course reports]]
| coursereport
| /course/report
| Reports of activity within the course
| Up to 2.1 (for 2.2+ see [[Reports]])
|-
| [[Gradebook export]]
| gradeexport
| /grade/export
| Export grades in various formats
| 1.9+
|-
| [[Gradebook import]]
| gradeimport
| /grade/import
| Import grades in various formats
| 1.9+
|-
| [[Gradebook reports]]
| gradereport
| /grade/report
| Display/edit grades in various layouts and reports
| 1.9+
|-
| [[Grading methods|Advanced grading methods]]
| gradingform
| /grade/grading/form
| Interfaces for actually performing grading in activity modules (eg Rubrics)
| 2.2+
|-
| [[MNet services]]
| mnetservice
| /mnet/service
| Allows to implement remote services for the [[MNet]] environment (deprecated, use web services instead)
| 2.0+
|-
| [[Webservice protocols]]
| webservice
| /webservice
| Define new protocols for web service communication (such as SOAP, XML-RPC, JSON, REST ...)
| 2.0+
|-
| [[Repository plugins]]
| repository
| /repository
| Connect to external sources of files to use in Moodle
| 2.0+
|-
| [[Portfolio plugins]]
| portfolio
| /portfolio
| Connect external portfolio services as destinations for users to store Moodle content
| 1.9+
|-
| [[Search engines]]
| search
| /search/engine
| Search engine backends to index Moodle's contents.
| 3.1+
|-
| [[Media players]]
| media
| /media/player
| Pluggable media players
| 3.2+
|-
| [[Plagiarism plugins]]
| plagiarism
| /plagiarism
| Define external services to process submitted files and content
| 2.0+
|-
| [[Cache store]]
| cachestore
| /cache/stores
| Cache storage back-ends.
| 2.4+
|-
| [[Cache locks]]
| cachelock
| /cache/locks
| Cache lock implementations.
| 2.4+
|-
| [[Themes]]
| theme
| /theme
| Change the look of Moodle by changing the the HTML and the CSS.
| 2.0+
|-
| [[Local plugins]]
| local
| /local
| Generic plugins for local customisations
| 2.0+
|-
| [[Assignment types|Legacy assignment types]]
| assignment
| /mod/assignment/type
| Different forms of assignments to be graded by teachers
| 1.x - 2.2
|-
| [[Admin reports|Legacy admin reports]]
| report
| /admin/report
| Provides useful views of data in a Moodle site, for admins only.
| Up to 2.1 (for 2.2+ see [[Reports]])
|-
| [[Content bank content types]]
| contenttype
| /contentbank/contenttype
| Content types to upload, create or edit in the content bank and use all over the Moodle site
| 3.9+
|-
| [[H5P libraries]]
| h5plib
| /h5p/h5plib
| Plugin type for the particular versions of the H5P integration library.
| 3.9+
|}
== Obtaining the list of plugin types known to your Moodle ==
To get the most exact list of types in your version of Moodle, use the following script. Put it to a file in the root directory of your Moodle installation and execute it via command line.
<syntaxhighlight lang="php">
<?php
define('CLI_SCRIPT', true);
require('config.php');

$pluginman = core_plugin_manager::instance();

foreach ($pluginman->get_plugin_types() as $type => $dir) {
$dir = substr($dir, strlen($CFG->dirroot));
printf("%-20s %-50s %s".PHP_EOL, $type, $pluginman->plugintype_name_plural($type), $dir);
}
</syntaxhighlight>
==Things you can find in all plugins==
Although there are many different types of plugin, there are some things that work the same way in all plugin types. Please see [[Plugin files]] page that describes things that work the same in all plugin types.
== Naming conventions ==
Warning if you have to choose a plugin (directory) name. The name is validated by the method <tt>lib/classes/component.php::is_valid_plugin_name()</tt> with a regexp: <tt>/^[a-z](?:[a-z0-9_](?!__))*[a-z0-9]+$/</tt>. In particular, the minus (-) character is not considered as valid, and the plugin will be silently ignored if the name is not valid.

There is an exception for [[Activity modules|activity modules]] that can not have the underscore in their name for legacy reasons.
== See also ==
* [[Guidelines_for_contributed_code|Guidelines for contributing code]]
* [[Core APIs]]
* [[Frankenstyle]]
* [http://moodle.org/plugins Moodle Plugins directory]
* [[Tutorial]] to help you learn how to write plugins for Moodle from start to finish, while showing you how to navigate the most important developer documentation along the way.
[[Category:Coding guidelines|Plugins]]
[[Category:Plugins]]

Payment API

2021-04-28T10:32:04Z

Bencellis: /* Triger the payment modal */

{{Moodle 3.10}}

Payment API allows Moodle components to have financial interactions with users.

=== Trigger the payment modal ===

In order to trigger the payment modal, all you need to do is to place an HTML element with data-action attribute equal to core_payment/triggerPayment in the page. You can use a mustache template with a content similar to the following:

<code xml>
<button
data-action="core_payment/triggerPayment"
data-component="enrol_fee"
data-paymentarea="fee"
data-itemid="{{itemid}}"
data-cost="{{coststring}}"
data-description="{{description}}"
>
{{# str }} sendpaymentbutton, enrol_fee {{/ str }}
<button>
</code>

=== The service_provider class ===

All components that want to use the payment API have to implement the \core_payment\local\callback\service_provider interface.

==== get_payable ====

<code php>
public static function get_payable(string $paymentarea, int $itemid): \core_payment\local\entities\payable;
</code>

This is the function that the payment subsystem calls to retrieve the amount and currency of what is being sold, along with the accountid that payments should be paid to.

==== deliver_order ====

<code php>
public static function deliver_order(string $paymentarea, int $itemid, int $paymentid, int $userid): bool;
</code>

This is the function that the payment subsystem calls when a user makes a successful payment. This function should give what the user paid for to them.

==See also==

* User documentation [[:en:Payment gateways|Payment gateways]]

Javascript Modules

2021-01-28T12:01:43Z

Bencellis: /* Useful links */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

Since version 3.8, Moodle supports [https://github.com/lukehoban/es6features#readme ECMAScript 2015 features] (aka ES6) in a cross browser compatible way thanks to [https://babeljs.io/ Babel JS]. In order to achieve the compatibility with older browsers Babel will compile the newer ES6 features back into ES5 Javascript. Unfortunately this means that in order for your Javascript changes to show in the browser they must be compiled by running [[Grunt]], even with the cachejs config setting set to false (i.e. "Development mode").

Note that, for Moodle 3.10 and up (see MDLSITE-6130), any new Javascript module '''must''' be written on ES6.

== Install NVM and Node ==

The recommended way of installing NodeJS is via the [https://github.com/nvm-sh/nvm Node Version Manager], or NVM. NVM allows you to have several different versions of NodeJS installed at and in-use at any once on your computer. Supported versions of Moodle all use version {{NodeJSExactVersion}} of NodeJS.

https://github.com/nvm-sh/nvm#installing-and-updating

Confirm it is working:

$ nvm --version
0.35.3

After you have installed '''nvm''', you should install the correct version of NodeJS by running the following commands from your Moodle directory:

nvm install
nvm use

If your primary use of NodeJS is for Moodle then we recommend that you set NodeJS version {{NodeJSExactVersion}} as your default version. You can do this by running:

nvm alias default {{NodeJSExactVersion}}

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[[grunt]]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

Once this is done, you can run the the following commands from your Moodle directory:

npm install

'''This may mention vulnerabilities, that's fine and doesn't apply.'''

npm install -g grunt-cli

== Development mode (Moodle v2.9 to v3.7) ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Development mode (Moodle v3.8 and above) ==

All Javascript code is now compiled using Babel which means Moodle will only ever serve minified Javascript to the browser, even in development mode. However in development mode Moodle will also send the browser the corresponding source map files for each of the Javascript modules. The source map files will tell the browser how to map the minified source code back to the unminified original source code so that the original source files will be displayed in the sources section of the browser's development tools.

While in development mode each of the Javascript modules will appear in the browser's source tree as separate modules (no more giant first.js file!) and they will also be loaded with individual network requests (this is a compromise we had to make thanks to some browser bugs with source map files).

To enable development mode set the '''cachejs''' config value to '''false''' in the admin settings or directly in your config.php file:
<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Since all Javascript must now be compiled you must run [[Grunt]] in order for you changes to appear in the browser. However rather than running Grunt manually each time on either the whole project or each file you modified, it is recommended that you just run the '''grunt watch''' task at the root of your Moodle directory. The grunt watch task will listen for changes to the Javascript files in the Moodle directory and will automatically lint and compile only the file that is changed after each change is detected. This removes the need to manually run grunt after each change.

== Development mode (Moodle v3.10 and above) ==

All the above for Moodle 3.8 and up applies, plus (see MDLSITE-6130), any new Javascript module must be written on ES6.

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== ES6 Modules (Moodle v3.8 and above) ==

In addition to AMD module syntax Moodle now supports the [https://github.com/lukehoban/es6features#modules ES6 syntax] for writing Javascript modules. All modules (defined using either syntax) are compatible with one another. Behind the scenes the ES6 module syntax is converted into an AMD syntax as part of the Babel compiling process.

Note that, for Moodle 3.10 and up (see MDLSITE-6130), any new Javascript module must be written on ES6.

The call from a PHP file takes the same format
<code>
$PAGE->requires->js_call_amd('myplugin/myfile','init');
</code>

And a minimal ES6 file will work with
<code>
export const init = () => {
window.console.log('we have been started');
};
</code>

=== Export default ===
There is one slight difference between the ES6 definition for exporting modules and the RequireJS (AMD) definition.

ES6 allows you to export a “default” value which is actually no different to exporting a named value where the name is “default”. Unfortunately, RequireJS allows for unnamed default exports (e.g. you can do "return SomeClass;") which can be imported by just requiring them in other AMD modules.

That’s a bit confusing, get to the point! Well, it basically means that in Moodle you won’t be able to write an ES6 module that exports both a default and named exports, e.g. "export default function() {...}" and "export const FOO = 'bar'" in the same module. The export default will simply override all other exports in that module.

=== Inline Javascript ===
Another important note is that ES6 support is only for stand alone Javascript files because it relies on the compilation from Babel and Grunt. That means any inline Javascript (either in PHP or in Mustache templates) won't support the ES6 features. Instead it would be best to keep the inline Javascript as minimal as possible and only use it to load a stand alone Javascript module.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

export const init = () => {
document.addEventListener('change', e => {
const someNode = e.target.closest('.someclass');
if (someNode) {
alert('It changed!');
}
});
};
</code>

For older versions of Moodle prior to 3.8, you will need to use the llegacy ES5 format instead:
<code javascript>
define([], function() {

return {
init: function() {
document.addEventListener('change', function(e) {
var someNode = e.target.closest('.someclass');
if (someNode) {
alert('It changed!');
}
});
}
};
});
</code>

This code passes the jquery module into our function (parameter $). There are a number of other useful modules available in Moodle, some of which you'll probably need in a practical application. See [[Useful_core_Javascript_modules]]. Simply list them in both the define() first parameter and the function callback. E.g.,
<code javascript>
import jQuery from 'jquery'; // We recommend that you strongly consider whether you really need jQuery. It is typically not needed in modern code.
import Str from 'core/str';
import Ajax from 'core/ajax';

export const init = config => {
};
</code>

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
export const init = (first, last) {
window.console.log(`The first name was '${first}' and the last name was '${last}'`);
};
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

import Str from 'core/str';

/**
* Reveal all of the hidden notes.
*/
const showAllNotes = () => {
document.querySelectorAll('.note.hidden').map(note => note.removeClass('hidden'));
};

/**
* Hide all of the notes.
*/
const hideAllNotes = () => document.querySelectorAll('.note').map(note => note.addClass('hidden'));

/**
* Return a personalised, formal, greeting.
*
* @param {String} name The name of the person to greet
* @returns {Promise}
*/
export const formal = name => Str.get_string('formallygreet', 'block_overview', name);

/**
* Return a personalised, informal, greeting.
*
* @param {String} name The name of the person to greet
* @returns {Promise}
*/
export const informal = name => {
return Str.get_string('informallygreet', 'block_overview', name);
};
</code>

It's important to note tha tonly functions which are exported will be callable from outside the module. These are part of the public API.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? In a limited number of situations you may not know the modules that you need until you call them. You can make use of dynamic imports to import them when you know what they are. Note: This is not the recommended approach in most cases.

<code javascript>
export const showTheThing = thingToShow => {
// Load the module for this thing.
import(`local_examples/local/types/type_${thingToShow.modname}`)
.then(thingModule => {
window.console.log(`The ${thingToShow.modname} is now available under thingModule within this scope`);

return thingModule;
});
};
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
// DO NOT DO THIS - IT DOES NOT WORK IN MOODLE
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'init', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
import Countdown from 'theme_mytheme/jquery.countdown');
import $ from 'jquery';

export const init = params => {
$('#clock').countdown(params.targetItem, event => {
$(event.target).html(event.strftime('%D days %H:%M:%S'));
});
};
</code>

2. Call your Javascript module from your template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="theme_mytheme-clock-{{uniqid}}"></span>
{{#js}}
require(['theme_mytheme/countdowntimer'], function(myModule) {
myModule.init({
targetItem: 'theme_mytheme-clock-{{uniqid}}'
});
});
{{/js}}
</code>

Note: If you feel that you need to work around MDL-62468, then you should probably be putting the data into the DOM in your template via data Attributes, or loading it via a Web Service.

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== Troubleshooting ==

=== npm-shrinkwrap.json sha1 / sha512 changes ===

If grunt changes all the hashes in npm-shrinkwrap.json then try this:

rm -rf node_modules && npm i

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)
* [[Useful_core_Javascript_modules]]
*[[Guide_to_adding_third_party_jQuery_for_AMD]] by Patrick Thibaudeau
*[https://moodle.org/mod/forum/discuss.php?d=378112#p1524459 How to get variables from PHP into javascript AMD modules in M3.5] Justin Hunt, on Moodle forums.
*MDL-67327 JavaScript issues when not following the official documentation, since Moodle 3.8
*[https://moodle.org/mod/forum/discuss.php?d=405829 Error from grunt watch - Moodle 3.9] Forum Discussion

[[Category:AJAX]]
[[Category:Javascript]]

Form API

2020-12-12T10:23:21Z

Bencellis: /* setHelpButton() */

==Overview==
Web Forms in moodle are created using forms API. Form API supports all html elements (checkbox, radio, textbox etc), with improved accessibility and security.
==Highlights==
# Tested and optimised for use on major screen-readers Dragon and JAWS.
# Tableless layout.
# Process form data securely, with required_param, optional_param and session key.
# Supports client-side validation
# Facility to add Moodle help buttons to forms.
# Support for file repository using [[File_API]]
# Support for many custom moodle specific and non-specific form elements.
# Addition for [https://docs.moodle.org/dev/lib/formslib.php_repeat_elements repeated elements].
# Addition for form elements in advance group

==Usage==
For creating a form in moodle, you have to create class extending moodleform class and override [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#definition.28.29 definition] for including form elements.
<code php>
//moodleform is defined in formslib.php
require_once("$CFG->libdir/formslib.php");

class simplehtml_form extends moodleform {
//Add elements to form
public function definition() {
global $CFG;

$mform = $this->_form; // Don't forget the underscore!

$mform->addElement('text', 'email', get_string('email')); // Add elements to your form
$mform->setType('email', PARAM_NOTAGS); //Set type of element
$mform->setDefault('email', 'Please enter email'); //Default value
...
}
//Custom validation should be added here
function validation($data, $files) {
return array();
}
}
</code>
Then instantiate form (in this case simplehtml_form) on your page.
<code php>
//include simplehtml_form.php
require_once('PATH_TO/simplehtml_form.php');

//Instantiate simplehtml_form
$mform = new simplehtml_form();

//Form processing and displaying is done here
if ($mform->is_cancelled()) {
//Handle form cancel operation, if cancel button is present on form
} else if ($fromform = $mform->get_data()) {
//In this case you process validated data. $mform->get_data() returns data posted in form.
} else {
// this branch is executed if the form is submitted but the data doesn't validate and the form should be redisplayed
// or on the first display of the form.

//Set default data (if any)
$mform->set_data($toform);
//displays the form
$mform->display();
}
</code>

If you wish to use the form within a block then you should consider using the render method, as demonstrated below:

Note that the render method does the same as the display method, except returning the HTML rather than outputting it to the browser, as with above make sure you've included the file which contains the class for your Moodle form.

<code php>
class block_yourblock extends block_base{
public function init(){
$this->title = 'Your Block';
}
public function get_content(){

$this->content = new stdClass();
$this->content->text = '';

$mform = new simplehtml_form();

//Form processing and displaying is done here
if ($mform->is_cancelled()) {
//Handle form cancel operation, if cancel button is present on form
} else if ($fromform = $mform->get_data()) {
//In this case you process validated data. $mform->get_data() returns data posted in form.
} else {
// this branch is executed if the form is submitted but the data doesn't validate and the form should be redisplayed
// or on the first display of the form.

//Set default data (if any)
$mform->set_data($toform);

//displays the form
$this->content->text = $mform->render();
}

return $this->content;

}
}
</code>

==Form elements==
===Basic form elements===
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#button button]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#checkbox checkbox]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#radio radio]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#select select]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#multi-select multi-select]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#password password]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#hidden hidden]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#html html] - div element
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#static static] - Display a static text.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#text text]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#textarea textarea]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#Use_Fieldsets_to_group_Form_Elements header]

=== Advanced form elements===

# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#autocomplete Autocomplete] - A select box that allows you to start typing to narrow the list of options, or search for results.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#advcheckbox advcheckbox] - Advance checkbox
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#float float]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#passwordunmask passwordunmask] - A password element with option to show the password in plaintext.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#recaptcha recaptcha]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#selectyesno selectyesno]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#selectwithlink selectwithlink]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#date_selector date_selector]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#date_time_selector date_time_selector]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#duration duration]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#editor editor]
# [https://docs.moodle.org/dev/Using_the_File_API_in_Moodle_forms#filepicker filepicker] - upload single file
# [https://docs.moodle.org/dev/Using_the_File_API_in_Moodle_forms#filemanager filemanager] - upload multiple files
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#tags tags]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#addGroup addGroup]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#modgrade modgrade]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#modvisible modvisible]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#choosecoursefile choosecoursefile]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#grading grading]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#questioncategory questioncategory]

===Custom form elements===

If you need a custom form element you can dynamical register new elements and then use them like this:

<code>
MoodleQuickForm::registerElementType('course_competency_rule',
"$CFG->dirroot/$CFG->admin/tool/lp/classes/course_competency_rule_form_element.php",
'tool_lp_course_competency_rule_form_element');
// Reuse the same options.
$mform->addElement('course_competency_rule', 'competency_rule', get_string('uponcoursemodulecompletion', 'tool_lp'), $options);
</code>

See:

https://github.com/moodle/moodle/blob/master/admin/tool/lp/lib.php#L157-L161

https://github.com/moodle/moodle/blob/master/admin/tool/lp/classes/course_competency_rule_form_element.php

==Commonly used functions==

====add_action_buttons($cancel = true, $submitlabel=null);====

You will normally use this helper function which is a method of moodleform to add all the 'action' buttons to the end of your form. A boolean parameter allow you to specify whether to include a cancel button and specify the label for your submit button (pass the result of get_string). Default for the submit button label is get_string('savechanges'). Note the '''$this''' not '''$mform'''

<code php>
$this->add_action_buttons();
</code>

====[[lib/formslib.php_Form_Definition#setDefault_2|setDefault()]]====
To set the default value for an element.

====[[lib/formslib.php_Form_Definition#disabledIf|disabledIf()]]====
For any element or groups of element in a form you can conditionally disable the group or individual element depending on conditions.

====[[lib/formslib.php_Form_Definition#hideIf|hideif()]]====
For any element or groups of element in a form you can conditionally hide the group or individual element depending on conditions.
Same syntax as disabledIf. Can do a simple search and replace on disabledIf. Added in Moodle 3.4.

====[[lib/formslib.php_Form_Definition#addRule|addRule()]]====
Add rule for server/client side validation. Like text field is required element and is of type email.

====setHelpButton()====
DEPRECATED - Sets pop-up help button to a form element. Use addHelpButton() instead.

====[[lib/formslib.php_Form_Definition#addHelpButton|addHelpButton()]]====
Adds pop-up help button to a form element

====[[lib/formslib.php_Form_Definition#setType|setType()]]====
PARAM_* types are used to specify how a submitted variable should be cleaned.

====[[lib/formslib.php_Form_Definition#disable_form_change_checker|disable_form_change_checker()]]====
By default, any Moodle form will pop-up an "Are you sure?" alert if you make some changes and then try to leave the page without saving. Occasionally, that is undesirable, in which case you can call <tt>$mform->disable_form_change_checker()</tt>.

==FAQ==
[https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#Use_Fieldsets_to_group_Form_Elements How to group elements]
==See also==

* [[Core_APIs]]
* [[lib/formslib.php_Usage]]
* [[lib/formslib.php_Form_Definition]]
* [[Designing usable forms]]
* [[Fragment]]
* [[MForm_Modal]]

[[Category:API]]

Form API

2020-12-12T10:22:48Z

Bencellis: /* setHelpButton() */

==Overview==
Web Forms in moodle are created using forms API. Form API supports all html elements (checkbox, radio, textbox etc), with improved accessibility and security.
==Highlights==
# Tested and optimised for use on major screen-readers Dragon and JAWS.
# Tableless layout.
# Process form data securely, with required_param, optional_param and session key.
# Supports client-side validation
# Facility to add Moodle help buttons to forms.
# Support for file repository using [[File_API]]
# Support for many custom moodle specific and non-specific form elements.
# Addition for [https://docs.moodle.org/dev/lib/formslib.php_repeat_elements repeated elements].
# Addition for form elements in advance group

==Usage==
For creating a form in moodle, you have to create class extending moodleform class and override [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#definition.28.29 definition] for including form elements.
<code php>
//moodleform is defined in formslib.php
require_once("$CFG->libdir/formslib.php");

class simplehtml_form extends moodleform {
//Add elements to form
public function definition() {
global $CFG;

$mform = $this->_form; // Don't forget the underscore!

$mform->addElement('text', 'email', get_string('email')); // Add elements to your form
$mform->setType('email', PARAM_NOTAGS); //Set type of element
$mform->setDefault('email', 'Please enter email'); //Default value
...
}
//Custom validation should be added here
function validation($data, $files) {
return array();
}
}
</code>
Then instantiate form (in this case simplehtml_form) on your page.
<code php>
//include simplehtml_form.php
require_once('PATH_TO/simplehtml_form.php');

//Instantiate simplehtml_form
$mform = new simplehtml_form();

//Form processing and displaying is done here
if ($mform->is_cancelled()) {
//Handle form cancel operation, if cancel button is present on form
} else if ($fromform = $mform->get_data()) {
//In this case you process validated data. $mform->get_data() returns data posted in form.
} else {
// this branch is executed if the form is submitted but the data doesn't validate and the form should be redisplayed
// or on the first display of the form.

//Set default data (if any)
$mform->set_data($toform);
//displays the form
$mform->display();
}
</code>

If you wish to use the form within a block then you should consider using the render method, as demonstrated below:

Note that the render method does the same as the display method, except returning the HTML rather than outputting it to the browser, as with above make sure you've included the file which contains the class for your Moodle form.

<code php>
class block_yourblock extends block_base{
public function init(){
$this->title = 'Your Block';
}
public function get_content(){

$this->content = new stdClass();
$this->content->text = '';

$mform = new simplehtml_form();

//Form processing and displaying is done here
if ($mform->is_cancelled()) {
//Handle form cancel operation, if cancel button is present on form
} else if ($fromform = $mform->get_data()) {
//In this case you process validated data. $mform->get_data() returns data posted in form.
} else {
// this branch is executed if the form is submitted but the data doesn't validate and the form should be redisplayed
// or on the first display of the form.

//Set default data (if any)
$mform->set_data($toform);

//displays the form
$this->content->text = $mform->render();
}

return $this->content;

}
}
</code>

==Form elements==
===Basic form elements===
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#button button]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#checkbox checkbox]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#radio radio]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#select select]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#multi-select multi-select]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#password password]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#hidden hidden]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#html html] - div element
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#static static] - Display a static text.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#text text]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#textarea textarea]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#Use_Fieldsets_to_group_Form_Elements header]

=== Advanced form elements===

# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#autocomplete Autocomplete] - A select box that allows you to start typing to narrow the list of options, or search for results.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#advcheckbox advcheckbox] - Advance checkbox
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#float float]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#passwordunmask passwordunmask] - A password element with option to show the password in plaintext.
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#recaptcha recaptcha]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#selectyesno selectyesno]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#selectwithlink selectwithlink]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#date_selector date_selector]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#date_time_selector date_time_selector]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#duration duration]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#editor editor]
# [https://docs.moodle.org/dev/Using_the_File_API_in_Moodle_forms#filepicker filepicker] - upload single file
# [https://docs.moodle.org/dev/Using_the_File_API_in_Moodle_forms#filemanager filemanager] - upload multiple files
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#tags tags]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#addGroup addGroup]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#modgrade modgrade]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#modvisible modvisible]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#choosecoursefile choosecoursefile]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#grading grading]
# [https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#questioncategory questioncategory]

===Custom form elements===

If you need a custom form element you can dynamical register new elements and then use them like this:

<code>
MoodleQuickForm::registerElementType('course_competency_rule',
"$CFG->dirroot/$CFG->admin/tool/lp/classes/course_competency_rule_form_element.php",
'tool_lp_course_competency_rule_form_element');
// Reuse the same options.
$mform->addElement('course_competency_rule', 'competency_rule', get_string('uponcoursemodulecompletion', 'tool_lp'), $options);
</code>

See:

https://github.com/moodle/moodle/blob/master/admin/tool/lp/lib.php#L157-L161

https://github.com/moodle/moodle/blob/master/admin/tool/lp/classes/course_competency_rule_form_element.php

==Commonly used functions==

====add_action_buttons($cancel = true, $submitlabel=null);====

You will normally use this helper function which is a method of moodleform to add all the 'action' buttons to the end of your form. A boolean parameter allow you to specify whether to include a cancel button and specify the label for your submit button (pass the result of get_string). Default for the submit button label is get_string('savechanges'). Note the '''$this''' not '''$mform'''

<code php>
$this->add_action_buttons();
</code>

====[[lib/formslib.php_Form_Definition#setDefault_2|setDefault()]]====
To set the default value for an element.

====[[lib/formslib.php_Form_Definition#disabledIf|disabledIf()]]====
For any element or groups of element in a form you can conditionally disable the group or individual element depending on conditions.

====[[lib/formslib.php_Form_Definition#hideIf|hideif()]]====
For any element or groups of element in a form you can conditionally hide the group or individual element depending on conditions.
Same syntax as disabledIf. Can do a simple search and replace on disabledIf. Added in Moodle 3.4.

====[[lib/formslib.php_Form_Definition#addRule|addRule()]]====
Add rule for server/client side validation. Like text field is required element and is of type email.

====setHelpButton()====
DEPRECATED - Sets pop-up help button to a form element.

====[[lib/formslib.php_Form_Definition#addHelpButton|addHelpButton()]]====
Adds pop-up help button to a form element

====[[lib/formslib.php_Form_Definition#setType|setType()]]====
PARAM_* types are used to specify how a submitted variable should be cleaned.

====[[lib/formslib.php_Form_Definition#disable_form_change_checker|disable_form_change_checker()]]====
By default, any Moodle form will pop-up an "Are you sure?" alert if you make some changes and then try to leave the page without saving. Occasionally, that is undesirable, in which case you can call <tt>$mform->disable_form_change_checker()</tt>.

==FAQ==
[https://docs.moodle.org/dev/lib/formslib.php_Form_Definition#Use_Fieldsets_to_group_Form_Elements How to group elements]
==See also==

* [[Core_APIs]]
* [[lib/formslib.php_Usage]]
* [[lib/formslib.php_Form_Definition]]
* [[Designing usable forms]]
* [[Fragment]]
* [[MForm_Modal]]

[[Category:API]]

OAuth 2 API

2020-10-17T09:27:03Z

Bencellis: /* I setup an OAuth 2 Issuer - how to I use it in code? */

== OAuth 2 API ==
{{ Moodle 3.3 }}

The OAuth 2 API is a set of classes that provide OAuth 2 functionality for integrating with remote systems. They exist in the folder /lib/classes/oauth2/ and there are a few concepts to be aware of.

=== Issuers ===
An OAuth Issuer is a named external system that provides identity and API access by issuing OAuth access tokens. They are configured manually at "Site administration -> Server -> OAuth 2 Services" and common ones can be quickly created from a template (Google, Office 365 and Facebook). An Issuer has a name and icon (for display on the login page), a Client ID and Client Secret (part of the OAuth spec).

=== Endpoints ===
An OAuth issuer must have a number of endpoints defined which are the URL's used to fetch and exchange access tokens, as well as fetch identity information. These will be setup automatically for OAuth services created from a template, or OAuth services using Open ID Connect.

The 3 standard endpoints which must be defined are the "authorization endpoint", "token endpoint" and "userinfo endpoint" - these are 3 urls which are used by the OAuth protocol to "allow the user to login", "obtain tokens to access the api" and "get the logged in user information".

=== Open ID Connect ===
Open ID Connect is a protocol built on top of OAuth 2 which provides some standardisation and inter-operability for OAuth 2 based services. If a "base service url" is entered for an Issuer - Moodle will attempt to retrieve the "well known configuration" which provides all the information about the other endpoints required to complete the setup for this service. E.g. for Google - the base service url is "https://accounts.google.com/". By appending ".well-known/openid-configuration" to the url we can find the service description at https://accounts.google.com/.well-known/openid-configuration which contains all the required information for us to automatically complete the setup for this service. This will work with any Open ID connect compliant service.

=== User field mappings ===
The other information we need to know about an OAuth 2 service is how to map the user information into Moodle user fields. We do this by adding to the list of user field mappings for the Issuer. The mappings for Open ID Connect services are standard and will be automatically created when setting up an Open ID compliant service - for other services you will need to create the mappings manually. Moodle will use this information to import the user profile fields when creating new accounts. The most important user field mappings are the username and email which are used to identify the Moodle account associated with the OAuth 2 login.

== I setup an OAuth 2 Issuer - how to I use it in code? ==
Any plugin that wants to use the configuration information provided by an OAuth issuer first needs to determine which issuer they should use for authentication. This is typically done by adding a setting that lets the admin choose from the list of configured issuers. An example is the "Google Drive" repository. It's possible to have multiple, very similar OAuth Issuers configured e.g. One public Google Issuer and one that is restricted to a specific domain. Once we know the Issuer we wish to use we can use it's ID to get an \core\oauth2\client class which will let us make requests against that API.

<code php>

// Get an issuer from the id.
$issuer = \core\oauth2\api::get_issuer($issuerid);
// Get an OAuth client from the issuer.
$client = \core\oauth2\api::get_user_oauth_client($this->issuer, $returnurl, $scopes);

</code>

Despite what is said in the code documentation, it is best to check that the client is authenticated and if not, redirect the user to log in.

<code php>
if (!$client->is_logged_in()) {
redirect($client->get_login_url());
}
</code>

'''What is return URL?'''

This is a url that will take you back to the current page. The way OAuth works is that the call to get the OAuth client will check to see if we have a valid session with all of the required scopes. If we don't - the user will be redirected immediately to the OAuth login page. When they have logged in - they are redirected back to the "returnurl" in Moodle which must end up making the same call to get an OAuth client - except this time we have a valid session so the client is returned. The returnurl MUST include the sesskey as one of the parameters.

'''What are the scopes?'''

These are named "permission grants" which the user is asked to accept. E.g. "email" is a scope which allows Moodle to see the email address for your logged in account. Each of the requested permissions will be displayed to the user with a description of what they mean and the user will have to "grant" Moodle this level of access - or cancel the operation. It is best practice not to request more scopes than are needed at any one time, and to incrementally request additional scopes as they are going to be used by different features in Moodle. For example - when logging into Moodle we only need to request access to the users basic profile and email address. When accessing a repository - we will need to request read/write access to the users files. Each time we create an oauth client class, we list the scopes that we will be using with this instance of the oauth client - if the user has only approved some of the scopes we request - they will be redirected to a consent screen where they approve the additional level of access.

So - what can I do with my oauth client ?

Make requests! This class extends the moodle curl class but includes authentication information with each request automatically. This means you can use standard functions like $client->get() or $client->post() to manually call rest api functions. This class will also honour the configured Moodle security settings like ipaddress black lists and proxy settings.

How do I make it easier to call API functions?

There is an abstract class \core\oauth2\rest which contains helper functions to allow you to wrap a external rest api in an easier to use class. To use it, make a subclass in your own plugin and define the "get_api_functions()" method.

<code php>

/**
* Google Drive Rest API.
*
* @package fileconverter_googledrive
* @copyright 2017 Damyon Wiese
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/
namespace fileconverter_googledrive;

defined('MOODLE_INTERNAL') || die();

/**
* Google Drive Rest API.
*
* @copyright 2017 Damyon Wiese
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/
class rest extends \core\oauth2\rest {

/**
* Define the functions of the rest API.
*
* @return array Example:
* [ 'listFiles' => [ 'method' => 'get', 'endpoint' => 'http://...', 'args' => [ 'folder' => PARAM_STRING ] ] ]
*/
public function get_api_functions() {
return [
'create' => [
'endpoint' => 'https://www.googleapis.com/drive/v3/files',
'method' => 'post',
'args' => [
'fields' => PARAM_RAW
],
'response' => 'json'
],
'delete' => [
'endpoint' => 'https://www.googleapis.com/drive/v3/files/{fileid}',
'method' => 'delete',
'args' => [
'fileid' => PARAM_RAW
],
'response' => 'json'
],
];
}
}

</code>

This example defines 2 functions in the external API 'create' and 'delete'. It specifies the http methods to use when calling these functions as well as the list of parameters. It also specifies that these functions return 'json' which means that the rest class will automatically decode the json response into an object. The url for each function call can contain parameters in the url (marked with curly braces). These will be replaced when they are passed as arguments to the function. Any remaining arguments will be appended as query parameters.

To use this class - we pass the oauth2 client to the constructor and then use the "call" method to call functions from the api.

<code php>
$service = new \fileconverter_googledrive\rest($client);

$params = ['fileid' => $fileid];

$service->call('delete', $params);
</code>

== How do I call API functions when the user is not logged in (e.g. from a scheduled task)? ==

Moodle allows you to connect a "system account" to any of the OAuth issuers. This is optional - so before enabling functionality that relies on this level of access you should check that the system account has been connected:

<code php>
if ($issuer->is_system_account_connected()) {
// Rock and roll.
}
</code>

If a system account has been connected - we can use it to get an authenticated oauth client (no redirects involved).

<code php>
$client = \core\oauth2\api::get_system_oauth_client($issuer);
</code>

This client can now be used to access apis as the Moodle system user.

If your code is going to use additional login scopes with this API - it must list all of the scopes it will use in a callback function so that the Moodle administrator can consent and agree to all the scopes when they connect the system account.

<code php>
**
* Callback to get the required scopes for system account.
*
* @param \core\oauth2\issuer $issuer
* @return string
*/
function fileconverter_googledrive_oauth2_system_scopes(\core\oauth2\issuer $issuer) {
if ($issuer->get('id') == get_config('fileconverter_googledrive', 'issuerid')) {
return 'https://www.googleapis.com/auth/drive';
}
return '';
}
</code>

The way the system account works is that a Moodle admin "connects" the system account by logging in with this account and accepting the permissions from the "Site administration -> Server -> OAuth 2 Services" page. One of the permissions they must accept is for offline access. This means that Moodle will receive a refresh token and can store it against that issuer. When we need to get a logged in OAuth client - we can exchange the refresh token for an access token directly, without having to login. The refresh token is updated by a scheduled task to make sure it never expires.

Adding a web service to a plugin

2020-06-07T12:06:28Z

Bencellis: /* Declare the web service function */

{{Moodle_2.0}}

= Quick start =
== Example ==
Have a look to the [https://github.com/moodlehq/moodle-local_wstemplate web service plugin template]. This plugin contains a web service hello_world function. To make testing easy for you, the plugin is distributed with a test client in the folder /client.
== File structure ==
The file structure is explained in these two documents:
* [[Web_services_API|Web services API]]
* [[External_functions_API|External function API]]

= Tutorial =
We will create a web service into a local plugin. This service will contain one ''local_myplugin_create_groups($groups)'' web service function. This web service function will create a group into a Moodle course.

== Write the specification documentation ==
Before starting coding, let's identify our needs writing some short specification documents.

=== functional specification===
''local_myplugin_create_groups($groups)'' will take a list of groups as parameters and it will return the same groups with their newly created id. If ever one group creation fails, the function will throw an exception, and no creation will happen.

=== technical specification ===
* '''the core function the external function will call''': ''groups_create_group()'' from [http://cvs.moodle.org/moodle/group/ /group/lib.php].
* '''the parameter types''': a list of object. This object are groups, with id/name/courseid.
* '''the returned value types''': a list of objects (groups) with their id.
* '''the user capabilities to check''': ''moodle/course:managegroups''

== Write a simple test client ==
The first thing you should code is a web service test client. You will often discover use cases that you didn't think about. We are not showing any test client code here, see [[Creating_a_web_service_client|How to create a web service client]].

== Declare the service ==
This step is optional. You can pre-build a service including any web service functions, so the Moodle administrator doesn't need to do it. Add into /local/myplugin/db/services.php:

<code php>
$services = array(
'mypluginservice' => array( // the name of the web service
'functions' => array ('local_myplugin_create_groups'), // web service functions of this service
'requiredcapability' => '', // if set, the web service user need this capability to access
// any function of this service. For example: 'some/capability:specified'
'restrictedusers' => 0, // if enabled, the Moodle administrator must link some user to this service
// into the administration
'enabled' => 1, // if enabled, the service can be reachable on a default installation
'shortname' => '', // optional – but needed if restrictedusers is set so as to allow logins.
'downloadfiles' => 0, // allow file downloads.
'uploadfiles' => 0 // allow file uploads.
)
);
</code>

Note: it is not possible for an administrator to add/remove any function from a pre-built service.

== Declare the web service function ==
Following the [[Web_services_API|Web service API]], you must declare the web service function in the ''local/myplugin/db/services.php'' file.

<code php>
$functions = array(
'local_myplugin_create_groups' => array( //web service function name
'classname' => 'local_myplugin_external', //class containing the external function OR namespaced class in classes/external/XXXX.php
'methodname' => 'create_groups', //external function name
'classpath' => 'local/myplugin/externallib.php', //file containing the class/external function - not required if using namespaced auto-loading classes.
// defaults to the service's externalib.php
'description' => 'Creates new groups.', //human readable description of the web service function
'type' => 'write', //database rights of the web service function (read, write)
'ajax' => true, // is the service available to 'internal' ajax calls.
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE) // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
'capabilities' => array(), // capabilities required by the function.
),
);
</code>

Web service functions should match the [https://docs.moodle.org/dev/Web_services_Roadmap#Naming_convention naming convention].

== Write the external function descriptions ==
Every web service function is mapped to an external function. External function are described in the [[External_functions_API|External functions API]].
Each external function is written with two other functions describing the parameters and the return values. These description functions are used by web service servers to:
* validate the web service function parameters
* validate the web service function returned values
* build WSDL files or other protocol documents

These two description functions are located in the same file and the same class mentioned in local/myplugin/db/services.php.

Thus for the web service function '''local_myplugin_create_groups()''', we need write a class named '''local_myplugin_external''' in the file '''local/myplugin/externallib.php'''. The class will contain:
* create_groups(...)
* create_groups_parameters()
* create_groups_return()

=== create_groups_parameters() ===
<code php>
require_once("$CFG->libdir/externallib.php");

class local_myplugin_external extends external_api {

/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
)
);
}
</code>

A web service function without parameters will have a parameter description function like that:
<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function functionname_parameters() {
return new external_function_parameters(
array(
//if I had any parameters, they would be described here. But I don't have any, so this array is empty.
)
);
}
</code>

A parameter can be described as:
* a list => external_multiple_structure
* an object => external_single_structure
* a primary type => external_value

Our create_groups() function expects one parameter named ''groups'', so we will first write:

<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => ...

)
);
}
</code>

This ''groups'' parameter is a list of group. So we will write :

<code php>
'groups' => new external_multiple_structure(
...
)
</code>

An external_multiple_structure object (list) can be constructed with:
* ''external_multiple_structure'' (list)
* ''external_single_structure'' (object)
* ''external_value'' (primary type).

For our function it will be a ''external_single_structure'':

<code php>
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
</code>

Thus we obtain :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
)
</code>

Each group values is a ''external_value'' (primary type):
* ''courseid'' is an integer
* ''name'' is a string (text only, not tag)
* ''description'' is a string (can be anything)
* ''enrolmentkey'' is also a string (can be anything)

We add them to the description :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'), //the second argument is a human readable description text. This text is displayed in the automatically generated documentation.
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
</code>

=== create_groups_returns() ===

It's similar to create_groups_parameters(), but instead of describing the parameters, it describes the return values.

<code php>
public static function create_groups_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'group record id'),
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
);
}
</code>

=== Required, Optional or Default value ===
A value can be VALUE_REQUIRED, VALUE_OPTIONAL, or VALUE_DEFAULT. If not mentioned, a value is VALUE_REQUIRED by default.

<code php>
'yearofstudy' => new external_value(PARAM_INT, 'year of study',VALUE_DEFAULT, 1979),
</code>

* VALUE_REQUIRED - if the value is not supplied => the server throws an error message
* VALUE_OPTIONAL - if the value is not supplied => the value is ignored. Note that VALUE_OPTIONAL can't be used in top level parameters, it must be used only within array/objects key definition. If you need top level Optional parameters you should use VALUE_DEFAULT instead.
* VALUE_DEFAULT - if the value is not supplied => the default value is used

Note: Because some web service protocols are strict about the number and types of arguments - it is not possible to specify an optional parameter as one of the top-most parameters for a function. Examples:

Not cool:
<code php>
public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'chocolatechips' => new external_value(PARAM_BOOL, PARAM_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ERROR! top level optional parameter!!!
)
);
}

</code>

Cool:
<code php>

public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'ifeellike' => new external_single_structure(
array(
'chocolatechips' => new external_value(PARAM_BOOL, VALUE_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ALL GOOD!! We have nested the params in a external_single_structure.
)
)
)
);
}

</code>

== Implement the external function ==
We declared our web service function and we defined the external function parameters and return values. We will now implement the external function:

<code php>
/**
* Create groups
* @param array $groups array of group description arrays (with keys groupname and courseid)
* @return array of newly created groups
*/
public static function create_groups($groups) { //Don't forget to set it as static
global $CFG, $DB;
require_once("$CFG->dirroot/group/lib.php");

$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));

$transaction = $DB->start_delegated_transaction(); //If an exception is thrown in the below code, all DB queries in this code will be rollback.

$groups = array();

foreach ($params['groups'] as $group) {
$group = (object)$group;

if (trim($group->name) == '') {
throw new invalid_parameter_exception('Invalid group name');
}
if ($DB->get_record('groups', array('courseid'=>$group->courseid, 'name'=>$group->name))) {
throw new invalid_parameter_exception('Group with the same name already exists in the course');
}

// now security checks
$context = get_context_instance(CONTEXT_COURSE, $group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);

// finally create the group
$group->id = groups_create_group($group, false);
$groups[] = (array)$group;
}

$transaction->allow_commit();

return $groups;
}
</code>

=== Parameter validation ===
<code php>
$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));
</code>
This ''validate_parameters'' function validates the external function parameters against the description. It will return an exception if some required parameters are missing, if parameters are not well-formed, and check the parameters validity. It is essential that you do this call to avoid potential hack.

'''Important:''' the parameters of the external function and their declaration in the description '''must be the same order'''. In this example we have only one parameter named $groups, so we don't need to worry about the order.

=== Context and Capability checks ===

<code php>
/// now security checks
$context = context_course::instance($group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);
</code>

Note: validate_context() is required in all external functions before operating on any data belonging to a context. This function does sanity and security checks on the context that was passed to the external function - and sets up the global $PAGE and $OUTPUT for rendering return values. Do NOT use require_login(), or $PAGE->set_context() in an external function.

=== Exceptions===
You can throw exceptions. There are automatically handle by Moodle web service servers.
<code php>
//Note: it is good practice to add detailled information in $debuginfo,
// and only send back a generic exception message when Moodle DEBUG mode < NORMAL.
// It's what we do here throwing the invalid_parameter_exception($debug) exception
throw new invalid_parameter_exception('Group with the same name already exists in the course');
</code>

=== Correct return values ===
The return values will be validated by the Moodle web service servers:
* return values contain some values not described => these values will be skipped.
* return values miss some required values (VALUE_REQUIRED) => the server will return an error.
* return values types don't match the description (int != PARAM_ALPHA) => the server will return an error

'''Note:''' cast all your returned objects into arrays.

== Making web service accessible through Apache Thrift ==
This step is optional. If you wish to generate SDK for different programming languages and platforms using [http://thrift.apache.org Apache Thrift framework] then [https://bitbucket.org/hhteam/moodle_thrift_tools/wiki/Home Moodle Thrift tools] can help you.

Two steps should be performed:

* Generate .thrift files for Moodle API using thriftgenerator script.
* Generate thrift handlers for PHP and copy the php files generated to your plugin source tree.

It is also recommended to include thrift files into the distribution of your plugin in order to simplify creation of client bindings for the users of your API.

That's it. Now the web API of your plugin is accessible through Apache Thrift Framework.

== Bump the plugin version ==
Edit your local/myplugin/version.php and increase the plugin version. This should trigger a Moodle upgrade and the new web service should be available in the administration (''Administration > Plugins > Web Services > Manage services'')

== Deprecation ==
External functions deprecation process is different from the [[Deprecation|standard deprecation]]. If you are interested in deprecating any of your external functions you should create a FUNCTIONNAME_is_deprecated function in your external function class. Return true if the external function is deprecated. This is an example:

<code php>

/**
* Mark the function as deprecated.
* @return bool
*/
public static function create_groups_is_deprecated() {
return true;
}
</code>

=See also=
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating_a_web_service_client|Implement a web service client]]

Specification:
* [[External services security]]
* [[External services description]]

[[Category:Web Services]]
[[Category:Plugins]]

Adding a web service to a plugin

2020-06-07T11:58:55Z

Bencellis: /* Declare the web service function */

{{Moodle_2.0}}

= Quick start =
== Example ==
Have a look to the [https://github.com/moodlehq/moodle-local_wstemplate web service plugin template]. This plugin contains a web service hello_world function. To make testing easy for you, the plugin is distributed with a test client in the folder /client.
== File structure ==
The file structure is explained in these two documents:
* [[Web_services_API|Web services API]]
* [[External_functions_API|External function API]]

= Tutorial =
We will create a web service into a local plugin. This service will contain one ''local_myplugin_create_groups($groups)'' web service function. This web service function will create a group into a Moodle course.

== Write the specification documentation ==
Before starting coding, let's identify our needs writing some short specification documents.

=== functional specification===
''local_myplugin_create_groups($groups)'' will take a list of groups as parameters and it will return the same groups with their newly created id. If ever one group creation fails, the function will throw an exception, and no creation will happen.

=== technical specification ===
* '''the core function the external function will call''': ''groups_create_group()'' from [http://cvs.moodle.org/moodle/group/ /group/lib.php].
* '''the parameter types''': a list of object. This object are groups, with id/name/courseid.
* '''the returned value types''': a list of objects (groups) with their id.
* '''the user capabilities to check''': ''moodle/course:managegroups''

== Write a simple test client ==
The first thing you should code is a web service test client. You will often discover use cases that you didn't think about. We are not showing any test client code here, see [[Creating_a_web_service_client|How to create a web service client]].

== Declare the service ==
This step is optional. You can pre-build a service including any web service functions, so the Moodle administrator doesn't need to do it. Add into /local/myplugin/db/services.php:

<code php>
$services = array(
'mypluginservice' => array( // the name of the web service
'functions' => array ('local_myplugin_create_groups'), // web service functions of this service
'requiredcapability' => '', // if set, the web service user need this capability to access
// any function of this service. For example: 'some/capability:specified'
'restrictedusers' => 0, // if enabled, the Moodle administrator must link some user to this service
// into the administration
'enabled' => 1, // if enabled, the service can be reachable on a default installation
'shortname' => '', // optional – but needed if restrictedusers is set so as to allow logins.
'downloadfiles' => 0, // allow file downloads.
'uploadfiles' => 0 // allow file uploads.
)
);
</code>

Note: it is not possible for an administrator to add/remove any function from a pre-built service.

== Declare the web service function ==
Following the [[Web_services_API|Web service API]], you must declare the web service function in the ''local/myplugin/db/services.php'' file.

<code php>
$functions = array(
'local_myplugin_create_groups' => array( //web service function name
'classname' => 'local_myplugin_external', //class containing the external function OR namespaced class in classes/external/XXXX.php
'methodname' => 'create_groups', //external function name
'classpath' => 'local/myplugin/externallib.php', //file containing the class/external function - not required if using namespaced auto-loading classes.
// defaults to the service's externalib.php
'description' => 'Creates new groups.', //human readable description of the web service function
'type' => 'write', //database rights of the web service function (read, write)
'ajax' => true, // is the service available to 'internal' ajax calls.
'loginrequired' => true, // default true – user must be logged in.
'readonlysession' => false, // default false.
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE) // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
'capabilities' => array(), // capabilities required by the function.
),
);
</code>

Web service functions should match the [https://docs.moodle.org/dev/Web_services_Roadmap#Naming_convention naming convention].

== Write the external function descriptions ==
Every web service function is mapped to an external function. External function are described in the [[External_functions_API|External functions API]].
Each external function is written with two other functions describing the parameters and the return values. These description functions are used by web service servers to:
* validate the web service function parameters
* validate the web service function returned values
* build WSDL files or other protocol documents

These two description functions are located in the same file and the same class mentioned in local/myplugin/db/services.php.

Thus for the web service function '''local_myplugin_create_groups()''', we need write a class named '''local_myplugin_external''' in the file '''local/myplugin/externallib.php'''. The class will contain:
* create_groups(...)
* create_groups_parameters()
* create_groups_return()

=== create_groups_parameters() ===
<code php>
require_once("$CFG->libdir/externallib.php");

class local_myplugin_external extends external_api {

/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
)
);
}
</code>

A web service function without parameters will have a parameter description function like that:
<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function functionname_parameters() {
return new external_function_parameters(
array(
//if I had any parameters, they would be described here. But I don't have any, so this array is empty.
)
);
}
</code>

A parameter can be described as:
* a list => external_multiple_structure
* an object => external_single_structure
* a primary type => external_value

Our create_groups() function expects one parameter named ''groups'', so we will first write:

<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => ...

)
);
}
</code>

This ''groups'' parameter is a list of group. So we will write :

<code php>
'groups' => new external_multiple_structure(
...
)
</code>

An external_multiple_structure object (list) can be constructed with:
* ''external_multiple_structure'' (list)
* ''external_single_structure'' (object)
* ''external_value'' (primary type).

For our function it will be a ''external_single_structure'':

<code php>
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
</code>

Thus we obtain :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
)
</code>

Each group values is a ''external_value'' (primary type):
* ''courseid'' is an integer
* ''name'' is a string (text only, not tag)
* ''description'' is a string (can be anything)
* ''enrolmentkey'' is also a string (can be anything)

We add them to the description :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'), //the second argument is a human readable description text. This text is displayed in the automatically generated documentation.
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
</code>

=== create_groups_returns() ===

It's similar to create_groups_parameters(), but instead of describing the parameters, it describes the return values.

<code php>
public static function create_groups_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'group record id'),
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
);
}
</code>

=== Required, Optional or Default value ===
A value can be VALUE_REQUIRED, VALUE_OPTIONAL, or VALUE_DEFAULT. If not mentioned, a value is VALUE_REQUIRED by default.

<code php>
'yearofstudy' => new external_value(PARAM_INT, 'year of study',VALUE_DEFAULT, 1979),
</code>

* VALUE_REQUIRED - if the value is not supplied => the server throws an error message
* VALUE_OPTIONAL - if the value is not supplied => the value is ignored. Note that VALUE_OPTIONAL can't be used in top level parameters, it must be used only within array/objects key definition. If you need top level Optional parameters you should use VALUE_DEFAULT instead.
* VALUE_DEFAULT - if the value is not supplied => the default value is used

Note: Because some web service protocols are strict about the number and types of arguments - it is not possible to specify an optional parameter as one of the top-most parameters for a function. Examples:

Not cool:
<code php>
public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'chocolatechips' => new external_value(PARAM_BOOL, PARAM_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ERROR! top level optional parameter!!!
)
);
}

</code>

Cool:
<code php>

public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'ifeellike' => new external_single_structure(
array(
'chocolatechips' => new external_value(PARAM_BOOL, VALUE_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ALL GOOD!! We have nested the params in a external_single_structure.
)
)
)
);
}

</code>

== Implement the external function ==
We declared our web service function and we defined the external function parameters and return values. We will now implement the external function:

<code php>
/**
* Create groups
* @param array $groups array of group description arrays (with keys groupname and courseid)
* @return array of newly created groups
*/
public static function create_groups($groups) { //Don't forget to set it as static
global $CFG, $DB;
require_once("$CFG->dirroot/group/lib.php");

$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));

$transaction = $DB->start_delegated_transaction(); //If an exception is thrown in the below code, all DB queries in this code will be rollback.

$groups = array();

foreach ($params['groups'] as $group) {
$group = (object)$group;

if (trim($group->name) == '') {
throw new invalid_parameter_exception('Invalid group name');
}
if ($DB->get_record('groups', array('courseid'=>$group->courseid, 'name'=>$group->name))) {
throw new invalid_parameter_exception('Group with the same name already exists in the course');
}

// now security checks
$context = get_context_instance(CONTEXT_COURSE, $group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);

// finally create the group
$group->id = groups_create_group($group, false);
$groups[] = (array)$group;
}

$transaction->allow_commit();

return $groups;
}
</code>

=== Parameter validation ===
<code php>
$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));
</code>
This ''validate_parameters'' function validates the external function parameters against the description. It will return an exception if some required parameters are missing, if parameters are not well-formed, and check the parameters validity. It is essential that you do this call to avoid potential hack.

'''Important:''' the parameters of the external function and their declaration in the description '''must be the same order'''. In this example we have only one parameter named $groups, so we don't need to worry about the order.

=== Context and Capability checks ===

<code php>
/// now security checks
$context = context_course::instance($group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);
</code>

Note: validate_context() is required in all external functions before operating on any data belonging to a context. This function does sanity and security checks on the context that was passed to the external function - and sets up the global $PAGE and $OUTPUT for rendering return values. Do NOT use require_login(), or $PAGE->set_context() in an external function.

=== Exceptions===
You can throw exceptions. There are automatically handle by Moodle web service servers.
<code php>
//Note: it is good practice to add detailled information in $debuginfo,
// and only send back a generic exception message when Moodle DEBUG mode < NORMAL.
// It's what we do here throwing the invalid_parameter_exception($debug) exception
throw new invalid_parameter_exception('Group with the same name already exists in the course');
</code>

=== Correct return values ===
The return values will be validated by the Moodle web service servers:
* return values contain some values not described => these values will be skipped.
* return values miss some required values (VALUE_REQUIRED) => the server will return an error.
* return values types don't match the description (int != PARAM_ALPHA) => the server will return an error

'''Note:''' cast all your returned objects into arrays.

== Making web service accessible through Apache Thrift ==
This step is optional. If you wish to generate SDK for different programming languages and platforms using [http://thrift.apache.org Apache Thrift framework] then [https://bitbucket.org/hhteam/moodle_thrift_tools/wiki/Home Moodle Thrift tools] can help you.

Two steps should be performed:

* Generate .thrift files for Moodle API using thriftgenerator script.
* Generate thrift handlers for PHP and copy the php files generated to your plugin source tree.

It is also recommended to include thrift files into the distribution of your plugin in order to simplify creation of client bindings for the users of your API.

That's it. Now the web API of your plugin is accessible through Apache Thrift Framework.

== Bump the plugin version ==
Edit your local/myplugin/version.php and increase the plugin version. This should trigger a Moodle upgrade and the new web service should be available in the administration (''Administration > Plugins > Web Services > Manage services'')

== Deprecation ==
External functions deprecation process is different from the [[Deprecation|standard deprecation]]. If you are interested in deprecating any of your external functions you should create a FUNCTIONNAME_is_deprecated function in your external function class. Return true if the external function is deprecated. This is an example:

<code php>

/**
* Mark the function as deprecated.
* @return bool
*/
public static function create_groups_is_deprecated() {
return true;
}
</code>

=See also=
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating_a_web_service_client|Implement a web service client]]

Specification:
* [[External services security]]
* [[External services description]]

[[Category:Web Services]]
[[Category:Plugins]]

Creating a web service client

2020-06-07T11:38:43Z

Bencellis: /* How to get a user token */

{{Moodle_2.0}}
[[Image:Moodle web service function documentation.jpg|thumb]]You need to know how to [https://docs.moodle.org/en/How_to_create_and_enable_a_web_service setup a web service] first.
To see the API Documentation, connect as Admin and go to '''Administration > Plugins > Web services > API Documentation'''

== Simple REST request example ==

To quickly test the web service works you can visit the end point from the browser or via curl. For example to call a function via REST protocol:

<code>
$ curl "https://your.site.com/moodle/webservice/rest/server.php?wstoken=...&wsfunction=...&moodlewsrestformat=json"
</code>

Customise the parameters <tt>wstoken</tt> and <tt>wsfunction</tt> to match the server side setup. Append additional parameters for the function call as needed with "&parameter_name=param", or if your parameter is an array then use "&array_name[index][parameter_name]=param". With the core_user_create_users function's (required) parameters for example:

<code>
$ curl "...&moodlewsrestformat=json&wsfunction=core_user_create_users&moodlewsrestformat=json&users[0][username]=testuser&users[0][firstname]=Anne&users[0][lastname]=Example..."
</code>

The <tt>moodlewsrestformat</tt> parameter affects the response format and can be either <tt>xml</tt> (default) or <tt>json</tt>.

== Officially supported protocols ==

; REST : The Moodle REST server accepts GET/POST parameters and return XML/JSON values. This server is not RESTfull.
; SOAP : The Moodle SOAP server is based on the Zend SOAP server (itself based on the PHP SOAP server). Zend publishes [http://framework.zend.com/manual/en/zend.soap.client.html a Zend SOAP client]. The current server implementation doesn't fully work with Java/.Net because we didn't generated a fully describe WSDL yet. If you are working on a Java/.Net client, follow or participate to the tracker issues MDL-28988 / MDL-28989
; XML-RPC : The Moodle XML-RPC server is based on Zend XML-RPC server. Zend also publishes [http://framework.zend.com/manual/en/zend.xmlrpc.client.html a Zend XML-RPC client].
; AMF (Versions < Moodle 3.0) : the Moodle AMF server is based on the Zend AMF server. The test client can be found in ''Settings blocks > Site Administration > Development > Web service test client > AMF Test client''.

== Demo client examples ==

Demo client sample codes can be downloaded on [https://github.com/moodlehq/sample-ws-clients Github].

For HTML5 app creators, you can also find:
* a nice [https://github.com/jleyva/umm phonegap / Jquery mobile template]
* a proof of concept of [http://moodle.org/mod/forum/discuss.php?d=189882 javascript cross-domain with Sencha Touch 1.1]

Node.js apps can use the [https://github.com/mudrd8mz/node-moodle-client moodle-client] module.

A [http://moodle.org/mod/forum/discuss.php?d=199453 Java Library for REST] can be found on [http://sourceforge.net/projects/moodlerestjava/ Sourceforge].

A [https://github.com/llagerlof/MoodleRest PHP Library for REST] can be found on GitHub.

A [https://github.com/zaddok/moodle Go Library for REST] can be found on GitHub.

== How to get a user token ==
{{Moodle_2.2}}
Your client can call the script located in /login/token.php with a simple HTTP request. We highly recommend to do it securely with HTTPS.
The required parameters are:
* username
* password
* service shortname - The service shortname is usually hardcoded in the pre-build service (db/service.php files). Moodle administrator will be able to edit shortnames for service created on the fly: MDL-29807. If you want to use the Mobile service, its shortname is <tt>moodle_mobile_app</tt>. Also useful to know, the database shortname field can be found in the table named external_services.

Request:
<code>
https://www.yourmoodle.com/login/token.php?username=USERNAME&password=PASSWORD&service=SERVICESHORTNAME
</code>

Response: (HTTP)
<code>
{token:4ed876sd87g6d8f7g89fsg6987dfh78d}
</code>

Response: (HTTPS) - Since at least M3.9
<code>
{
"token": "9859148a89546f0efe716a58e340849b",
"privatetoken": "8RpHJevJ42W7QN23OMkeYcdOYw3YfWgWGKsak7WB3Z88wcApSCVZ9TgY6M5fEO1m"
}
</code>

=== Difference between Moodle versions ===

* Moodle 2.2 and later: the script can generate user tokens for any service shortname (of course users must be allowed on the service, see [[:en:How to create and enable a web service|How to create and enable a web service]]).
* Moodle 2.1: the script can only generate tokens for the official built-in mobile service. However the script can returns tokens for other services, they just need to have been previously generated.

=== About service shortname ===

At the moment a service can have a shortname if you:
* create the service as a built-in service (in db/services.php files)
* add the shortname manually in the DB. Note: we'll add the admin UI for shortname later (MDL-30229)

== Text formats ==
=== Moodle 2.0 to 2.2 ===
{{Moodle_2.0}}
HTML is the format sent/received by web service functions. All returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'

=== Moodle 2.3 and later ===
{{Moodle_2.3}}
Since Moodle 2.3 you can add few GET/POST parameters to your request (for devs who have a good knowledge of File API and format_text()):
* ''moodlewssettingraw'' => false by default. If true, the function will not apply format_text() to description/summary/textarea. The function will return the raw content from the DB.
* ''moodlewssettingfileurl'' => true by default, returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'. If false the raw file url content from the DB is returned (e.g. @@PLUGINFILE@@)
* ''moodlewssettingfilter'' => false by default. If true, the function will filter during format_text()

=== Moodle 3.5 and later ===
{{Moodle_3.5}}
* ''moodlewssettinglang'' => to force a session language for when retrieving information (same behaviour than using the language selector in the site)

== See also ==
* [[Web services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating a web service and a web service function | Implement a web service and a web service function]]
* [[Web_services_Roadmap|Web service Roadmap]]

[[Category:Web Services]]

Creating a web service client

2020-06-07T11:34:54Z

Bencellis: /* How to get a user token */

{{Moodle_2.0}}
[[Image:Moodle web service function documentation.jpg|thumb]]You need to know how to [https://docs.moodle.org/en/How_to_create_and_enable_a_web_service setup a web service] first.
To see the API Documentation, connect as Admin and go to '''Administration > Plugins > Web services > API Documentation'''

== Simple REST request example ==

To quickly test the web service works you can visit the end point from the browser or via curl. For example to call a function via REST protocol:

<code>
$ curl "https://your.site.com/moodle/webservice/rest/server.php?wstoken=...&wsfunction=...&moodlewsrestformat=json"
</code>

Customise the parameters <tt>wstoken</tt> and <tt>wsfunction</tt> to match the server side setup. Append additional parameters for the function call as needed with "&parameter_name=param", or if your parameter is an array then use "&array_name[index][parameter_name]=param". With the core_user_create_users function's (required) parameters for example:

<code>
$ curl "...&moodlewsrestformat=json&wsfunction=core_user_create_users&moodlewsrestformat=json&users[0][username]=testuser&users[0][firstname]=Anne&users[0][lastname]=Example..."
</code>

The <tt>moodlewsrestformat</tt> parameter affects the response format and can be either <tt>xml</tt> (default) or <tt>json</tt>.

== Officially supported protocols ==

; REST : The Moodle REST server accepts GET/POST parameters and return XML/JSON values. This server is not RESTfull.
; SOAP : The Moodle SOAP server is based on the Zend SOAP server (itself based on the PHP SOAP server). Zend publishes [http://framework.zend.com/manual/en/zend.soap.client.html a Zend SOAP client]. The current server implementation doesn't fully work with Java/.Net because we didn't generated a fully describe WSDL yet. If you are working on a Java/.Net client, follow or participate to the tracker issues MDL-28988 / MDL-28989
; XML-RPC : The Moodle XML-RPC server is based on Zend XML-RPC server. Zend also publishes [http://framework.zend.com/manual/en/zend.xmlrpc.client.html a Zend XML-RPC client].
; AMF (Versions < Moodle 3.0) : the Moodle AMF server is based on the Zend AMF server. The test client can be found in ''Settings blocks > Site Administration > Development > Web service test client > AMF Test client''.

== Demo client examples ==

Demo client sample codes can be downloaded on [https://github.com/moodlehq/sample-ws-clients Github].

For HTML5 app creators, you can also find:
* a nice [https://github.com/jleyva/umm phonegap / Jquery mobile template]
* a proof of concept of [http://moodle.org/mod/forum/discuss.php?d=189882 javascript cross-domain with Sencha Touch 1.1]

Node.js apps can use the [https://github.com/mudrd8mz/node-moodle-client moodle-client] module.

A [http://moodle.org/mod/forum/discuss.php?d=199453 Java Library for REST] can be found on [http://sourceforge.net/projects/moodlerestjava/ Sourceforge].

A [https://github.com/llagerlof/MoodleRest PHP Library for REST] can be found on GitHub.

A [https://github.com/zaddok/moodle Go Library for REST] can be found on GitHub.

== How to get a user token ==
{{Moodle_2.2}}
Your client can call the script located in /login/token.php with a simple HTTP request. We highly recommend to do it securely with HTTPS.
The required parameters are:
* username
* password
* service shortname - The service shortname is usually hardcoded in the pre-build service (db/service.php files). Moodle administrator will be able to edit shortnames for service created on the fly: MDL-29807. If you want to use the Mobile service, its shortname is <tt>moodle_mobile_app</tt>. Also useful to know, the database shortname field can be found in the table named external_services.

Request:
<code>
https://www.yourmoodle.com/login/token.php?username=USERNAME&password=PASSWORD&service=SERVICESHORTNAME
</code>

Response: (HTTP)
<code>
{token:4ed876sd87g6d8f7g89fsg6987dfh78d}
</code>

Response: (HTTPS)
<code>
{
"token": "9859148a89546f0efe716a58e340849b",
"privatetoken": "8RpHJevJ42W7QN23OMkeYcdOYw3YfWgWGKsak7WB3Z88wcApSCVZ9TgY6M5fEO1m"
}
</code>

=== Difference between Moodle versions ===

* Moodle 2.2 and later: the script can generate user tokens for any service shortname (of course users must be allowed on the service, see [[:en:How to create and enable a web service|How to create and enable a web service]]).
* Moodle 2.1: the script can only generate tokens for the official built-in mobile service. However the script can returns tokens for other services, they just need to have been previously generated.

=== About service shortname ===

At the moment a service can have a shortname if you:
* create the service as a built-in service (in db/services.php files)
* add the shortname manually in the DB. Note: we'll add the admin UI for shortname later (MDL-30229)

== Text formats ==
=== Moodle 2.0 to 2.2 ===
{{Moodle_2.0}}
HTML is the format sent/received by web service functions. All returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'

=== Moodle 2.3 and later ===
{{Moodle_2.3}}
Since Moodle 2.3 you can add few GET/POST parameters to your request (for devs who have a good knowledge of File API and format_text()):
* ''moodlewssettingraw'' => false by default. If true, the function will not apply format_text() to description/summary/textarea. The function will return the raw content from the DB.
* ''moodlewssettingfileurl'' => true by default, returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'. If false the raw file url content from the DB is returned (e.g. @@PLUGINFILE@@)
* ''moodlewssettingfilter'' => false by default. If true, the function will filter during format_text()

=== Moodle 3.5 and later ===
{{Moodle_3.5}}
* ''moodlewssettinglang'' => to force a session language for when retrieving information (same behaviour than using the language selector in the site)

== See also ==
* [[Web services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating a web service and a web service function | Implement a web service and a web service function]]
* [[Web_services_Roadmap|Web service Roadmap]]

[[Category:Web Services]]

Adding a web service to a plugin

2020-05-31T11:39:22Z

Bencellis: /* Declare the web service function */

Adding a web service to a plugin

2020-05-31T11:33:43Z

Bencellis: /* Declare the service */

{{Moodle_2.0}}

= Quick start =
== Example ==
Have a look to the [https://github.com/moodlehq/moodle-local_wstemplate web service plugin template]. This plugin contains a web service hello_world function. To make testing easy for you, the plugin is distributed with a test client in the folder /client.
== File structure ==
The file structure is explained in these two documents:
* [[Web_services_API|Web services API]]
* [[External_functions_API|External function API]]

= Tutorial =
We will create a web service into a local plugin. This service will contain one ''local_myplugin_create_groups($groups)'' web service function. This web service function will create a group into a Moodle course.

== Write the specification documentation ==
Before starting coding, let's identify our needs writing some short specification documents.

=== functional specification===
''local_myplugin_create_groups($groups)'' will take a list of groups as parameters and it will return the same groups with their newly created id. If ever one group creation fails, the function will throw an exception, and no creation will happen.

=== technical specification ===
* '''the core function the external function will call''': ''groups_create_group()'' from [http://cvs.moodle.org/moodle/group/ /group/lib.php].
* '''the parameter types''': a list of object. This object are groups, with id/name/courseid.
* '''the returned value types''': a list of objects (groups) with their id.
* '''the user capabilities to check''': ''moodle/course:managegroups''

== Write a simple test client ==
The first thing you should code is a web service test client. You will often discover use cases that you didn't think about. We are not showing any test client code here, see [[Creating_a_web_service_client|How to create a web service client]].

== Declare the service ==
This step is optional. You can pre-build a service including any web service functions, so the Moodle administrator doesn't need to do it. Add into /local/myplugin/db/services.php:

<code php>
$services = array(
'mypluginservice' => array( // the name of the web service
'functions' => array ('local_myplugin_create_groups'), // web service functions of this service
'requiredcapability' => '', // if set, the web service user need this capability to access
// any function of this service. For example: 'some/capability:specified'
'restrictedusers' => 0, // if enabled, the Moodle administrator must link some user to this service
// into the administration
'enabled' => 1, // if enabled, the service can be reachable on a default installation
'shortname' => '', // optional – but needed if restrictedusers is set so as to allow logins.
'downloadfiles' => 0, // allow file downloads.
'uploadfiles' => 0 // allow file uploads.
)
);
</code>

Note: it is not possible for an administrator to add/remove any function from a pre-built service.

== Declare the web service function ==
Following the [[Web_services_API|Web service API]], you must declare the web service function in the ''local/myplugin/db/services.php'' file.

<code php>
$functions = array(
'local_myplugin_create_groups' => array( //web service function name
'classname' => 'local_myplugin_external', //class containing the external function
'methodname' => 'create_groups', //external function name
'classpath' => 'local/myplugin/externallib.php', //file containing the class/external function
'description' => 'Creates new groups.', //human readable description of the web service function
'type' => 'write', //database rights of the web service function (read, write)
'ajax' => true, // is the service available to 'internal' ajax calls.
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE) // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
),
);
</code>

Web service functions should match the [https://docs.moodle.org/dev/Web_services_Roadmap#Naming_convention naming convention].

== Write the external function descriptions ==
Every web service function is mapped to an external function. External function are described in the [[External_functions_API|External functions API]].
Each external function is written with two other functions describing the parameters and the return values. These description functions are used by web service servers to:
* validate the web service function parameters
* validate the web service function returned values
* build WSDL files or other protocol documents

These two description functions are located in the same file and the same class mentioned in local/myplugin/db/services.php.

Thus for the web service function '''local_myplugin_create_groups()''', we need write a class named '''local_myplugin_external''' in the file '''local/myplugin/externallib.php'''. The class will contain:
* create_groups(...)
* create_groups_parameters()
* create_groups_return()

=== create_groups_parameters() ===
<code php>
require_once("$CFG->libdir/externallib.php");

class local_myplugin_external extends external_api {

/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
)
);
}
</code>

A web service function without parameters will have a parameter description function like that:
<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function functionname_parameters() {
return new external_function_parameters(
array(
//if I had any parameters, they would be described here. But I don't have any, so this array is empty.
)
);
}
</code>

A parameter can be described as:
* a list => external_multiple_structure
* an object => external_single_structure
* a primary type => external_value

Our create_groups() function expects one parameter named ''groups'', so we will first write:

<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => ...

)
);
}
</code>

This ''groups'' parameter is a list of group. So we will write :

<code php>
'groups' => new external_multiple_structure(
...
)
</code>

An external_multiple_structure object (list) can be constructed with:
* ''external_multiple_structure'' (list)
* ''external_single_structure'' (object)
* ''external_value'' (primary type).

For our function it will be a ''external_single_structure'':

<code php>
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
</code>

Thus we obtain :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
)
</code>

Each group values is a ''external_value'' (primary type):
* ''courseid'' is an integer
* ''name'' is a string (text only, not tag)
* ''description'' is a string (can be anything)
* ''enrolmentkey'' is also a string (can be anything)

We add them to the description :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'), //the second argument is a human readable description text. This text is displayed in the automatically generated documentation.
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
</code>

=== create_groups_returns() ===

It's similar to create_groups_parameters(), but instead of describing the parameters, it describes the return values.

<code php>
public static function create_groups_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'group record id'),
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
);
}
</code>

=== Required, Optional or Default value ===
A value can be VALUE_REQUIRED, VALUE_OPTIONAL, or VALUE_DEFAULT. If not mentioned, a value is VALUE_REQUIRED by default.

<code php>
'yearofstudy' => new external_value(PARAM_INT, 'year of study',VALUE_DEFAULT, 1979),
</code>

* VALUE_REQUIRED - if the value is not supplied => the server throws an error message
* VALUE_OPTIONAL - if the value is not supplied => the value is ignored. Note that VALUE_OPTIONAL can't be used in top level parameters, it must be used only within array/objects key definition. If you need top level Optional parameters you should use VALUE_DEFAULT instead.
* VALUE_DEFAULT - if the value is not supplied => the default value is used

Note: Because some web service protocols are strict about the number and types of arguments - it is not possible to specify an optional parameter as one of the top-most parameters for a function. Examples:

Not cool:
<code php>
public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'chocolatechips' => new external_value(PARAM_BOOL, PARAM_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ERROR! top level optional parameter!!!
)
);
}

</code>

Cool:
<code php>

public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'ifeellike' => new external_single_structure(
array(
'chocolatechips' => new external_value(PARAM_BOOL, VALUE_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ALL GOOD!! We have nested the params in a external_single_structure.
)
)
)
);
}

</code>

== Implement the external function ==
We declared our web service function and we defined the external function parameters and return values. We will now implement the external function:

<code php>
/**
* Create groups
* @param array $groups array of group description arrays (with keys groupname and courseid)
* @return array of newly created groups
*/
public static function create_groups($groups) { //Don't forget to set it as static
global $CFG, $DB;
require_once("$CFG->dirroot/group/lib.php");

$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));

$transaction = $DB->start_delegated_transaction(); //If an exception is thrown in the below code, all DB queries in this code will be rollback.

$groups = array();

foreach ($params['groups'] as $group) {
$group = (object)$group;

if (trim($group->name) == '') {
throw new invalid_parameter_exception('Invalid group name');
}
if ($DB->get_record('groups', array('courseid'=>$group->courseid, 'name'=>$group->name))) {
throw new invalid_parameter_exception('Group with the same name already exists in the course');
}

// now security checks
$context = get_context_instance(CONTEXT_COURSE, $group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);

// finally create the group
$group->id = groups_create_group($group, false);
$groups[] = (array)$group;
}

$transaction->allow_commit();

return $groups;
}
</code>

=== Parameter validation ===
<code php>
$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));
</code>
This ''validate_parameters'' function validates the external function parameters against the description. It will return an exception if some required parameters are missing, if parameters are not well-formed, and check the parameters validity. It is essential that you do this call to avoid potential hack.

'''Important:''' the parameters of the external function and their declaration in the description '''must be the same order'''. In this example we have only one parameter named $groups, so we don't need to worry about the order.

=== Context and Capability checks ===

<code php>
/// now security checks
$context = context_course::instance($group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);
</code>

Note: validate_context() is required in all external functions before operating on any data belonging to a context. This function does sanity and security checks on the context that was passed to the external function - and sets up the global $PAGE and $OUTPUT for rendering return values. Do NOT use require_login(), or $PAGE->set_context() in an external function.

=== Exceptions===
You can throw exceptions. There are automatically handle by Moodle web service servers.
<code php>
//Note: it is good practice to add detailled information in $debuginfo,
// and only send back a generic exception message when Moodle DEBUG mode < NORMAL.
// It's what we do here throwing the invalid_parameter_exception($debug) exception
throw new invalid_parameter_exception('Group with the same name already exists in the course');
</code>

=== Correct return values ===
The return values will be validated by the Moodle web service servers:
* return values contain some values not described => these values will be skipped.
* return values miss some required values (VALUE_REQUIRED) => the server will return an error.
* return values types don't match the description (int != PARAM_ALPHA) => the server will return an error

'''Note:''' cast all your returned objects into arrays.

== Making web service accessible through Apache Thrift ==
This step is optional. If you wish to generate SDK for different programming languages and platforms using [http://thrift.apache.org Apache Thrift framework] then [https://bitbucket.org/hhteam/moodle_thrift_tools/wiki/Home Moodle Thrift tools] can help you.

Two steps should be performed:

* Generate .thrift files for Moodle API using thriftgenerator script.
* Generate thrift handlers for PHP and copy the php files generated to your plugin source tree.

It is also recommended to include thrift files into the distribution of your plugin in order to simplify creation of client bindings for the users of your API.

That's it. Now the web API of your plugin is accessible through Apache Thrift Framework.

== Bump the plugin version ==
Edit your local/myplugin/version.php and increase the plugin version. This should trigger a Moodle upgrade and the new web service should be available in the administration (''Administration > Plugins > Web Services > Manage services'')

== Deprecation ==
External functions deprecation process is different from the [[Deprecation|standard deprecation]]. If you are interested in deprecating any of your external functions you should create a FUNCTIONNAME_is_deprecated function in your external function class. Return true if the external function is deprecated. This is an example:

<code php>

/**
* Mark the function as deprecated.
* @return bool
*/
public static function create_groups_is_deprecated() {
return true;
}
</code>

=See also=
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating_a_web_service_client|Implement a web service client]]

Specification:
* [[External services security]]
* [[External services description]]

[[Category:Web Services]]
[[Category:Plugins]]

Creating a web service client

2020-05-31T11:21:46Z

Bencellis: /* Officially supported protocols */

{{Moodle_2.0}}
[[Image:Moodle web service function documentation.jpg|thumb]]You need to know how to [https://docs.moodle.org/en/How_to_create_and_enable_a_web_service setup a web service] first.
To see the API Documentation, connect as Admin and go to '''Administration > Plugins > Web services > API Documentation'''

== Simple REST request example ==

To quickly test the web service works you can visit the end point from the browser or via curl. For example to call a function via REST protocol:

<code>
$ curl "https://your.site.com/moodle/webservice/rest/server.php?wstoken=...&wsfunction=...&moodlewsrestformat=json"
</code>

Customise the parameters <tt>wstoken</tt> and <tt>wsfunction</tt> to match the server side setup. Append additional parameters for the function call as needed with "&parameter_name=param", or if your parameter is an array then use "&array_name[index][parameter_name]=param". With the core_user_create_users function's (required) parameters for example:

<code>
$ curl "...&moodlewsrestformat=json&wsfunction=core_user_create_users&moodlewsrestformat=json&users[0][username]=testuser&users[0][firstname]=Anne&users[0][lastname]=Example..."
</code>

The <tt>moodlewsrestformat</tt> parameter affects the response format and can be either <tt>xml</tt> (default) or <tt>json</tt>.

== Officially supported protocols ==

; REST : The Moodle REST server accepts GET/POST parameters and return XML/JSON values. This server is not RESTfull.
; SOAP : The Moodle SOAP server is based on the Zend SOAP server (itself based on the PHP SOAP server). Zend publishes [http://framework.zend.com/manual/en/zend.soap.client.html a Zend SOAP client]. The current server implementation doesn't fully work with Java/.Net because we didn't generated a fully describe WSDL yet. If you are working on a Java/.Net client, follow or participate to the tracker issues MDL-28988 / MDL-28989
; XML-RPC : The Moodle XML-RPC server is based on Zend XML-RPC server. Zend also publishes [http://framework.zend.com/manual/en/zend.xmlrpc.client.html a Zend XML-RPC client].
; AMF (Versions < Moodle 3.0) : the Moodle AMF server is based on the Zend AMF server. The test client can be found in ''Settings blocks > Site Administration > Development > Web service test client > AMF Test client''.

== Demo client examples ==

Demo client sample codes can be downloaded on [https://github.com/moodlehq/sample-ws-clients Github].

For HTML5 app creators, you can also find:
* a nice [https://github.com/jleyva/umm phonegap / Jquery mobile template]
* a proof of concept of [http://moodle.org/mod/forum/discuss.php?d=189882 javascript cross-domain with Sencha Touch 1.1]

Node.js apps can use the [https://github.com/mudrd8mz/node-moodle-client moodle-client] module.

A [http://moodle.org/mod/forum/discuss.php?d=199453 Java Library for REST] can be found on [http://sourceforge.net/projects/moodlerestjava/ Sourceforge].

A [https://github.com/llagerlof/MoodleRest PHP Library for REST] can be found on GitHub.

A [https://github.com/zaddok/moodle Go Library for REST] can be found on GitHub.

== How to get a user token ==
{{Moodle_2.2}}
Your client can call the script located in /login/token.php with a simple HTTP request. We highly recommend to do it securely with HTTPS.
The required parameters are:
* username
* password
* service shortname - The service shortname is usually hardcoded in the pre-build service (db/service.php files). Moodle administrator will be able to edit shortnames for service created on the fly: MDL-29807. If you want to use the Mobile service, its shortname is <tt>moodle_mobile_app</tt>. Also useful to know, the database shortname field can be found in the table named external_services.

Request:
<code>
https://www.yourmoodle.com/login/token.php?username=USERNAME&password=PASSWORD&service=SERVICESHORTNAME
</code>

Response:
<code>
{token:4ed876sd87g6d8f7g89fsg6987dfh78d}
</code>

=== Difference between Moodle versions ===

* Moodle 2.2 and later: the script can generate user tokens for any service shortname (of course users must be allowed on the service, see [[:en:How to create and enable a web service|How to create and enable a web service]]).
* Moodle 2.1: the script can only generate tokens for the official built-in mobile service. However the script can returns tokens for other services, they just need to have been previously generated.

=== About service shortname ===

At the moment a service can have a shortname if you:
* create the service as a built-in service (in db/services.php files)
* add the shortname manually in the DB. Note: we'll add the admin UI for shortname later (MDL-30229)

== Text formats ==
=== Moodle 2.0 to 2.2 ===
{{Moodle_2.0}}
HTML is the format sent/received by web service functions. All returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'

=== Moodle 2.3 and later ===
{{Moodle_2.3}}
Since Moodle 2.3 you can add few GET/POST parameters to your request (for devs who have a good knowledge of File API and format_text()):
* ''moodlewssettingraw'' => false by default. If true, the function will not apply format_text() to description/summary/textarea. The function will return the raw content from the DB.
* ''moodlewssettingfileurl'' => true by default, returned file urls are converted to 'http://xxxx/webservice/pluginfile.php/yyyyyyyy'. If false the raw file url content from the DB is returned (e.g. @@PLUGINFILE@@)
* ''moodlewssettingfilter'' => false by default. If true, the function will filter during format_text()

=== Moodle 3.5 and later ===
{{Moodle_3.5}}
* ''moodlewssettinglang'' => to force a session language for when retrieving information (same behaviour than using the language selector in the site)

== See also ==
* [[Web services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating a web service and a web service function | Implement a web service and a web service function]]
* [[Web_services_Roadmap|Web service Roadmap]]

[[Category:Web Services]]

Web services files handling

2020-05-30T09:32:22Z

Bencellis: /* File upload */

== Summary ==

Since Moodle 2.0, we provide web service functions to upload and download files. They are:

# moodle_file_get_files (Deprecated, use core_files_get_files since moodle 2.2 onward)
# moodle_file_upload (Deprecated, use core_files_upload since moodle 2.2 onward)

File contents are encoded in base64, and for web service transmission, it's not efficient. Mobile devices don't have enough memory to decode/encode web service request/response containing large files.

So we developed some alternative solutions to upload/download files.

== File upload ==

The entry point is ''/webservice/upload.php'', simply use HTTP POST method to upload files, it requires a web service token for authentication. If the upload is successfully, the files will be saved, prior to at least Moodle 2.9 in the user private file area but since in the user draft area. Previously you could force the files to be saved in the draft area by specifying the then optional parameter: ''filearea=draft''. Since at least Moodle 2.9, only two optional parameters are used, ''itemid'' to specify the draft area id – default 0 which is a new draft area - and ''filepath'' default \‘/\’ to specify the file’s path.

It is envisaged the ''itemid'' parameter will be used when the files are uploaded singularly in separate HTTP calls and the files are required to be in the same draft file area. The client retrieves the ''itemid'' of the first uploaded file and uses it in subsequent uploads to specify the files must be saved in the same draft file area.

On every successful upload, the file/s information are returned in JSON format. If an error occurs, an error message will be sent back in JSON format too.

Once all the files are uploaded, you can call the webservice that accepts files and pass it the ''itemid'' of the draft area containing the list of files for the request. The service can identify the uploads and manipulate them as necessary. An example of a webservice that accepts files is: ''mod_assign_save_submission''.

To accept file uploads, the the service must allow "files download" (''Administration > Plugins > Web services > Manage services > Edit service > Advanced button'')

There is an oldish code example on [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

== File download ==
We serve the files through ''/webservice/pluginfile.php''. This script requires a web service token for authentication.

Look at the [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

In case of issue, think to check that:
# the service associated with the token allow "''files download''" (''Administration > Plugins > Web services > Manage services > Edit service > Advanced button'')
# the web service is valid

Note: you could notice that ''/webservice/pluginfile.php'' has the exact same stucture than ''/pluginfile.php''. We don't serve the files through ''/pluginfile.php'' for web service clients because this script requires user's login session to work. It's why it might not be an option for web service client.

== Returning files in Web Services ==

Since Moodle 3.2, you can return a complete file area list via Web Services using the static get_area_files method in external_util.

<code php>
$forum->introfiles = external_util::get_area_files($context->id, 'mod_forum', 'intro', false, false);
</code>

You can also use the external_files structure definition in combination with the method to return the most common file fields required by WS clientes.

<code php>
public static function get_forums_by_courses_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'Forum id'),
....
'introfiles' => new external_files('Files in the introduction text', VALUE_OPTIONAL),
</code>

== See also ==
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]

[[Category:Web_Services]]

Web services files handling

2020-05-30T09:32:03Z

Bencellis: /* File upload */

== Summary ==

Since Moodle 2.0, we provide web service functions to upload and download files. They are:

# moodle_file_get_files (Deprecated, use core_files_get_files since moodle 2.2 onward)
# moodle_file_upload (Deprecated, use core_files_upload since moodle 2.2 onward)

File contents are encoded in base64, and for web service transmission, it's not efficient. Mobile devices don't have enough memory to decode/encode web service request/response containing large files.

So we developed some alternative solutions to upload/download files.

== File upload ==

The entry point is ''/webservice/upload.php'', simply use HTTP POST method to upload files, it requires a web service token for authentication. If the upload is successfully, the files will be saved, prior to at least Moodle 2.9 in the user private file area but since in the user draft area. Previously you could force the files to be saved in the draft area by specifying the then optional parameter: ''filearea=draft''. Since at least Moodle 2.9, only two optional parameters are used, ''itemid'' to specify the draft area id – default 0 which is a new draft area - and ''filepath'' default \‘/\’ to specify the file’s path.

It is envisaged the ''itemid'' parameter will be used when the files are uploaded singularly in separate HTTP calls and the files are required to be in the same draft file area. The client retrieves the ''itemid'' of the first uploaded file and uses it in subsequent uploads to specify the files must be saved in the same draft file area.

On every successful upload, the file/s information are returned in JSON format. If an error occurs, an error message will be sent back in JSON format too.

Once all the files are uploaded, you can call the webservice that accepts files and pass it the ''itemid'' of the draft area containing the list of files for the request. The service can identify the uploads and manipulate them as necessary. An example of a webservice that accepts files is: ''mod_assign_save_submission''.

To accept file uploads, the the service must allow "files download" (Administration > Plugins > Web services > Manage services > Edit service > Advanced button)

There is an oldish code example on [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

== File download ==
We serve the files through ''/webservice/pluginfile.php''. This script requires a web service token for authentication.

Look at the [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

In case of issue, think to check that:
# the service associated with the token allow "''files download''" (''Administration > Plugins > Web services > Manage services > Edit service > Advanced button'')
# the web service is valid

Note: you could notice that ''/webservice/pluginfile.php'' has the exact same stucture than ''/pluginfile.php''. We don't serve the files through ''/pluginfile.php'' for web service clients because this script requires user's login session to work. It's why it might not be an option for web service client.

== Returning files in Web Services ==

Since Moodle 3.2, you can return a complete file area list via Web Services using the static get_area_files method in external_util.

<code php>
$forum->introfiles = external_util::get_area_files($context->id, 'mod_forum', 'intro', false, false);
</code>

You can also use the external_files structure definition in combination with the method to return the most common file fields required by WS clientes.

<code php>
public static function get_forums_by_courses_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'Forum id'),
....
'introfiles' => new external_files('Files in the introduction text', VALUE_OPTIONAL),
</code>

== See also ==
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]

[[Category:Web_Services]]

Web services files handling

2020-05-30T09:30:23Z

Bencellis: /* File upload */

== Summary ==

Since Moodle 2.0, we provide web service functions to upload and download files. They are:

# moodle_file_get_files (Deprecated, use core_files_get_files since moodle 2.2 onward)
# moodle_file_upload (Deprecated, use core_files_upload since moodle 2.2 onward)

File contents are encoded in base64, and for web service transmission, it's not efficient. Mobile devices don't have enough memory to decode/encode web service request/response containing large files.

So we developed some alternative solutions to upload/download files.

== File upload ==

The entry point is ''/webservice/upload.php'', simply use HTTP POST method to upload files, it requires a web service token for authentication. If the upload is successfully, the files will be saved, prior to at least Moodle 2.9 in the user private file area but since in the user draft area. Previously you could force the files to be saved in the draft area by specifying the then optional parameter: ''filearea=draft''. Since at least Moodle 2.9, only two optional parameters are used, ''itemid'' to specify the draft area id – default 0 which is a new draft area - and ''filepath'' default \‘/\’ to specify the file’s path.

It is envisaged the ''itemid'' parameter will be used when the files are uploaded singularly in separate HTTP calls and the files are required to be in the same draft file area. The client retrieves the ''itemid'' of the first uploaded file and uses it in subsequent uploads to specify the files must be saved in the same draft file area.

On every successful upload, the file/s information are returned in JSON format. If an error occurs, an error message will be sent back in JSON format too.

Once all the files are uploaded, you can call the webservice that accepts files and pass it the ''itemid'' of the draft area containing the list of files for the request. The service can identify the uploads and manipulate them as necessary. An example of a webservice that accepts files is: ''mod_assign_save_submission''.

There is an oldish code example on [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

== File download ==
We serve the files through ''/webservice/pluginfile.php''. This script requires a web service token for authentication.

Look at the [https://github.com/moodlehq/sample-ws-clients/tree/master/PHP-HTTP-filehandling code example on Github].

In case of issue, think to check that:
# the service associated with the token allow "''files download''" (''Administration > Plugins > Web services > Manage services > Edit service > Advanced button'')
# the web service is valid

Note: you could notice that ''/webservice/pluginfile.php'' has the exact same stucture than ''/pluginfile.php''. We don't serve the files through ''/pluginfile.php'' for web service clients because this script requires user's login session to work. It's why it might not be an option for web service client.

== Returning files in Web Services ==

Since Moodle 3.2, you can return a complete file area list via Web Services using the static get_area_files method in external_util.

<code php>
$forum->introfiles = external_util::get_area_files($context->id, 'mod_forum', 'intro', false, false);
</code>

You can also use the external_files structure definition in combination with the method to return the most common file fields required by WS clientes.

<code php>
public static function get_forums_by_courses_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'Forum id'),
....
'introfiles' => new external_files('Files in the introduction text', VALUE_OPTIONAL),
</code>

== See also ==
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]

[[Category:Web_Services]]

AJAX

2020-05-12T10:24:45Z

Bencellis: /* Ajax in Moodle */

'''AJAX (Asynchronous Javascript and XML)''' is a modern web design technique that allows for more interactivity by making webpages that fetch data in the background and alter themselves without reloading the entire page. This helps to make a page feel much more like an application than a web page. AJAX is a new way of working with existing technologies (including HTML, [[Javascript]], [[CSS]] and the ''XMLHttpRequest object'' amongst others) rather than a new piece of technology in itself.

Although AJAX indicates that XML is used, the term really relates to the group of technologies and in Moodle we tend to favour use of JSON rather than XML as the syntax is lighter and leads to a smaller output. It is also easier to construct from php data structures.

== Ajax in Moodle ==
{{ Moodle 2.9 }}

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. If you do not need to call web service functions, then the standard JQuery function can be used by $.ajax().

Some benefits of this system are:
# No new ajax scripts need auditing for security vulnerabilities
# Multiple requests can be chained in a single http request
# Strict type checking for all parameters and return types
# New webservice functions benefit Ajax interfaces and web service clients

So the steps required to create an ajax interaction are:

# Write or find an existing web service function to handle the ajax interaction: See [[ Web_services ]]
# White list the web service for ajax. To do this, you can define 'ajax' => true in your function's definition, in db/services.php. Only functions that are whitelisted using this mechanism will be available to the ajax script.
# Call the web service from javascript in response to a user action:

Example calling core_get_string:
<code javascript>
require(['core/ajax'], function(ajax) {
var promises = ajax.call([
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'pluginname' } },
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'changerate' } }
]);

promises[0].done(function(response) {
console.log('mod_wiki/pluginname is' + response);
}).fail(function(ex) {
// do something with the exception
});

promises[1].done(function(response) {
console.log('mod_wiki/changerate is' + response);
}).fail(function(ex) {
// do something with the exception
});
});
</code>

Note: This example chains two separate calls to the 'core_get_string' webservice in one http request

Note: Don't actually fetch strings like this, it is just an example, use the 'core/str' module instead.

If there is only a single action, a simpler form is possible (example from Assignment):

<code>
ajax.call([{
methodname: 'mod_assign_submit_grading_form',
args: {assignmentid: assignmentid, userid: this._lastUserId, jsonformdata: JSON.stringify(data)},
done: this._handleFormSubmissionResponse.bind(this, data, nextUserId),
fail: notification.exception
}]);
</code>

('notifcation' comes from the 'core/notification' javascript module and provides a useful popup error message if the call fails)

To update parts of the UI in response to Ajax changes, consider using [[ Templates ]]

For information on writing AJAX scripts for Moodle before Moodle 2.9 see: [[ AJAX pre 2.9 ]]

Watch a video about using templates with webservices and AJAX in Moodle: https://www.youtube.com/watch?v=UTePjRZqAg8

Tricky things to know about using webservices with ajax calls:
# Any call to $PAGE->get_renderer() requires the correct theme be set. If this is done in a webservice - it is likely that the theme needs to be a parameter to the webservice.
# Text returned from a webservice must be properly filtered. This means it must go through external_format_text or external_format_string (since 3.0 - see MDL-51213) with the correct context.
# The correct context for 2 is the most specific context relating to the thing being output e.g. for a user's profile desciption the context is the user context.
# After adding any dynamic content to a page, Moodle's filters need to be notified via M.core.event.FILTER_CONTENT_UPDATED (MDL-51222 makes this easier)
# After adding or changing any webservice definition in db/services.php - you must bump the version number for either the plugin or Moodle and run the upgrade. This will install the webservice in the DB tables so it can be found by ajax.

In some very rare cases - you can mark webservices as safe to call without a session. These should only be used for webservices that return 100% public information and do not consume many resources. A current example is core_get_string. To mark a webservice as safe to call without a session you need to do 2 things.
# Add 'loginrequired' => false to the service definition in db/services.php
# Pass "false" as the 3rd argument to the ajax "call" method when calling the webservice.
The benefit to marking these safe webservice is that (a) they can be called from the login page before we have a session and (b) they will perform faster because they will bypass moodles session code when responding to the webservice call.

== See also ==

* [[ AJAX pre 2.9 ]]
* [[Templates]]
* [[Javascript Modules]]
* [[Javascript FAQ]]
* [[Javascript]]
* [[Firebug#Debugging_AJAX_with_Firebug|Debugging AJAX with Firebug]]

* [https://adaptivepath.org/ideas/ajax-new-approach-web-applications ''Ajax: A New Approach to Web Applications'', the original Ajax article by Adaptive Path] (This article is also preserved on the [https://web.archive.org/web/20070225140912/http://www.adaptivepath.com/publications/essays/archives/000385.php Internet Archive])
* [http://developer.mozilla.org/en/docs/AJAX:Getting_Started ''AJAX: Getting Started'' article on developer.mozilla.org]
* [http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html ''10 places you must use AJAX'' by Adam Bosworth] (This link is now dead, but the article is preserved on the [https://web.archive.org/web/20060127015713/http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html Internet Archive copy])
* [http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype ''Considering Ajax, Part 1: Cut through the hype'' from IBM developerworks] (Also a dead link... [https://web.archive.org/web/20080602101238/http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype Internet Archive copy])
* [http://en.wikipedia.org/wiki/Ajax_%28programming%29 Wikipedia article on ''AJAX'']
* [http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ How to Make Your AJAX Applications Accessible: 40 Tutorials and Articles] (Also a dead link... [https://web.archive.org/web/20090225094656/http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ Internet Archive copy])
*[http://www.ajaxload.info/ AJAX loading icon generator]

[[Category:Javascript]]
[[Category:AJAX]]

Repository plugins

2020-04-14T13:26:01Z

Bencellis: /* supported_filetypes() */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

====Course and User Repository Instances.====
By default, a system wide instance of a repository is created when the repository is enabled. If you want to allow either or both course specific and user repository instances, you have to specify both '''enablecourseinstances''' and '''enableuserinstances''' as options. There are three ways this can be done:

# Define ''$string[' enablecourseinstances']'' and ''$string['enableuserinstances']'' in your plugin's language file.
# Add them as items in ''get_type_option_names()'' function's return array. Note, you must not define the form fields for these options in the ''type_config_form()'' function.
# Several 'core' repositories use the '''''db/install.php''''' to create the original repository instance by constructing an instance of the ''repository_type'' class. The options can be defined in the array passed as the second parameter to the constructor.

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform, $classname='repository') {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

'''NOTE''': You cannot call ''$this'' from static methods. If you need access the non static variables, you may have to store the values in the ''__construct()'' method into private static variables.

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-14T08:49:15Z

Bencellis: /* Getting / updating settings */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform, $classname='repository') {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

'''NOTE''': You cannot call ''$this'' from static methods. If you need access the non static variables, you may have to store the values in the ''__construct()'' method into private static variables.

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-14T08:39:23Z

Bencellis: /* instance_form_validation($mform, $data, $errors) */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform, $classname='repository') {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-13T10:42:42Z

Bencellis: /* type_config_form($mform, $classname='repository') */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform, $classname='repository') {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-13T10:42:29Z

Bencellis: /* type_config_form($mform, $classname='repository') */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform, , $classname='repository') {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-13T10:35:03Z

Bencellis: /* type_form_validation($mform, $data, $errors) */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform) {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
''This function must be declared static''<br>
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-13T10:33:27Z

Bencellis: /* type_config_form($mform, $classname='repository') */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public static function type_config_form($mform) {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2020-04-13T10:26:25Z

Bencellis: /* get_type_option_names() */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an array with a single item - pluginname.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public function type_config_form($mform) {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
''This function must be declared static''<br>
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public static function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Running acceptance test

2020-03-28T13:25:46Z

Bencellis: /* Chrome specific */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode was not tested.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====
First, you should have a look at [[Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium#Working_combinations_of_OS.2BBrowser.2Bselenium|working combinations of OS+Browser+selenium]].

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible). Ensure you have the right version of the Chrome driver - see [[#Trouble_shooting| Trouble shooting]].
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running" when running the behat tests). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>vendor/bin/behat</tt>'. If you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files. Use comma-separated list like '<tt>@some_thing,@some_thing_else</tt>' to run tests from multiple areas. See upstream documentation on Gherkin filters for advanced syntax and more complex examples.
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j=<number> or --parallel=<number>''' (required) Number of parallel behat run to initialise
# '''-m=<number> or --maxruns=<number>''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun=<number>''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun=<number>''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a=<name> or --add-core-features-to-theme=<name>''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>

You almost always want to limit the number of tests that are run. To run all the tests in one plugin:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml --tags mod_myplugin
</code>

To run all the tests in one .feature file:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature
</code>

To run a single scenario:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature:40
</code>

Here, '40' is the line-number of the feature file where the Scenario starts.

=== Parallel runs ===
For running parallel runs, use following command

php admin/tool/behat/cli/run.php

Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

Example: Initialise and run Behat tests for a custom plugin under a custom theme:

php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme="mytheme"
php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="mytheme"

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" iframe''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with --add-core-features-to-theme={THEME_NAME}, e.g.

<code>
php admin/tool/behat/cli/init.php --add-core-features-to-theme=clean
vendor/bin/behat --suite=clean --tags="@enrol_foobar"

</code>

That is a core theme but it will work with custom theme. No = or quotes needed around the theme name.

Make sure that <tt>$CFG->theme</tt> is '''not set''' in your config.php.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Write new tests and behat methods ===

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

Alternative way of running three Selenium servers in parallel:

$ printf %d\\n {4444..4446} | xargs -n 1 -P 3 java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

=== Increasing timeouts ===

You may see errors such as:

<code>
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</code>

Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it could just mean that the browser was not yet ready. You may find that the test works if you run it again. If you get this error frequently, it might be useful to increase the timeout.

It is possible to increase this timeout by adding a line in your config.php. (Requires Moodle versions 3.5 (from 3.5.6), 3.6 (from 3.6.4), or 3.7+.)

<code>
$CFG->behat_increasetimeout = 2;
</code>

This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== Trouble shooting ==

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.

In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');

=== Selenium server is not running ===
==== Chrome specific ====

If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659/ MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper/ Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== See also ==
* [[Acceptance testing for the mobile app]]

[[Category:Quality Assurance]][[Category:Behat]]

Running acceptance test

2020-03-28T13:02:29Z

Bencellis: /* 2. Set up Selenium */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode was not tested.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====
First, you should have a look at [[Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium#Working_combinations_of_OS.2BBrowser.2Bselenium|working combinations of OS+Browser+selenium]].

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible). Ensure you have the right version of the Chrome driver - see [[#Trouble_shooting| Trouble shooting]].
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running" when running the behat tests). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>vendor/bin/behat</tt>'. If you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files. Use comma-separated list like '<tt>@some_thing,@some_thing_else</tt>' to run tests from multiple areas. See upstream documentation on Gherkin filters for advanced syntax and more complex examples.
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j=<number> or --parallel=<number>''' (required) Number of parallel behat run to initialise
# '''-m=<number> or --maxruns=<number>''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun=<number>''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun=<number>''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a=<name> or --add-core-features-to-theme=<name>''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>

You almost always want to limit the number of tests that are run. To run all the tests in one plugin:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml --tags mod_myplugin
</code>

To run all the tests in one .feature file:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature
</code>

To run a single scenario:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature:40
</code>

Here, '40' is the line-number of the feature file where the Scenario starts.

=== Parallel runs ===
For running parallel runs, use following command

php admin/tool/behat/cli/run.php

Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

Example: Initialise and run Behat tests for a custom plugin under a custom theme:

php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme="mytheme"
php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="mytheme"

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" iframe''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with --add-core-features-to-theme={THEME_NAME}, e.g.

<code>
php admin/tool/behat/cli/init.php --add-core-features-to-theme=clean
vendor/bin/behat --suite=clean --tags="@enrol_foobar"

</code>

That is a core theme but it will work with custom theme. No = or quotes needed around the theme name.

Make sure that <tt>$CFG->theme</tt> is '''not set''' in your config.php.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Write new tests and behat methods ===

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

Alternative way of running three Selenium servers in parallel:

$ printf %d\\n {4444..4446} | xargs -n 1 -P 3 java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

=== Increasing timeouts ===

You may see errors such as:

<code>
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</code>

Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it could just mean that the browser was not yet ready. You may find that the test works if you run it again. If you get this error frequently, it might be useful to increase the timeout.

It is possible to increase this timeout by adding a line in your config.php. (Requires Moodle versions 3.5 (from 3.5.6), 3.6 (from 3.6.4), or 3.7+.)

<code>
$CFG->behat_increasetimeout = 2;
</code>

This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== Trouble shooting ==

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.

In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');

=== Selenium server is not running ===
==== Chrome specific ====

If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659| MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper| Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== See also ==
* [[Acceptance testing for the mobile app]]

[[Category:Quality Assurance]][[Category:Behat]]

Running acceptance test

2020-03-28T12:54:56Z

Bencellis: /* Trouble shooting */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode was not tested.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====
First, you should have a look at [[Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium#Working_combinations_of_OS.2BBrowser.2Bselenium|working combinations of OS+Browser+selenium]].

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running" when running the behat tests). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>vendor/bin/behat</tt>'. If you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files. Use comma-separated list like '<tt>@some_thing,@some_thing_else</tt>' to run tests from multiple areas. See upstream documentation on Gherkin filters for advanced syntax and more complex examples.
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j=<number> or --parallel=<number>''' (required) Number of parallel behat run to initialise
# '''-m=<number> or --maxruns=<number>''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun=<number>''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun=<number>''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a=<name> or --add-core-features-to-theme=<name>''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>

You almost always want to limit the number of tests that are run. To run all the tests in one plugin:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml --tags mod_myplugin
</code>

To run all the tests in one .feature file:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature
</code>

To run a single scenario:

<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml /path/to/moodle/mod/myplugin/tests/behat/testsomething.feature:40
</code>

Here, '40' is the line-number of the feature file where the Scenario starts.

=== Parallel runs ===
For running parallel runs, use following command

php admin/tool/behat/cli/run.php

Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

Example: Initialise and run Behat tests for a custom plugin under a custom theme:

php admin/tool/behat/cli/init.php --parallel=3 --add-core-features-to-theme="mytheme"
php admin/tool/behat/cli/run.php --tags="@tool_myplugin" --suite="mytheme"

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" iframe''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with --add-core-features-to-theme={THEME_NAME}, e.g.

<code>
php admin/tool/behat/cli/init.php --add-core-features-to-theme=clean
vendor/bin/behat --suite=clean --tags="@enrol_foobar"

</code>

That is a core theme but it will work with custom theme. No = or quotes needed around the theme name.

Make sure that <tt>$CFG->theme</tt> is '''not set''' in your config.php.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Write new tests and behat methods ===

If you want to write tests for your own integration, you can do so by creating new tests with format .feature. Follow instructions in [[Writing_acceptance_tests|this page]] to write new tests.

It is also possible to add new steps the moodle behat integration. In order to do so, you will have to create a new .php class with the prefix '''behat_'''. Copy the format from '''lib\behat\behat_base.php''', but set your class to extend the behat_base class instead of the MinkExtension. You can define new behat steps by declaring functions with the appropriate heading.

You will not be able to use these steps and features right away. Check [[Running_acceptance_test#New_step_definitions_or_features_are_not_executed|this section]] for instructions on how to update the behat integration.

For further information on how to create new steps definitions, check [[Acceptance testing/Custom acceptance steps]].

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

Alternative way of running three Selenium servers in parallel:

$ printf %d\\n {4444..4446} | xargs -n 1 -P 3 java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

=== Increasing timeouts ===

You may see errors such as:

<code>
Javascript code and/or AJAX requests are not ready after 10 seconds.
There is a Javascript error or the code is extremely slow.
</code>

Sometimes this indicates a genuine problem with the code, but if you are using a slow computer, it could just mean that the browser was not yet ready. You may find that the test works if you run it again. If you get this error frequently, it might be useful to increase the timeout.

It is possible to increase this timeout by adding a line in your config.php. (Requires Moodle versions 3.5 (from 3.5.6), 3.6 (from 3.6.4), or 3.7+.)

<code>
$CFG->behat_increasetimeout = 2;
</code>

This will increase all the timeouts by a factor of 2; if that isn't sufficient, you could use 3.

Increasing timeouts will make tests run a bit slower (because there are points where Behat waits up to a timeout to make sure something doesn't happen) so don't do this unless you need to.

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== Trouble shooting ==

=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Errors during setup (before test are launched) ===
Typical errors are:
* Behat requirement not satisfied: http://127.0.0.1/m/stable_master is not available, ensure you specified correct url and that the server is set up and started.
* Behat is configured but not enabled on this test site.

In order to fix those errors please check that: the behat_dataroot has correct write permissions and that the $CFG->behat* variables are placed before the lib/setup.php include:
require_once(__DIR__ . '/lib/setup.php');

=== Selenium server is not running ===
==== Chrome specific ====

If you are using chrome, you need to ensure that the driver matches the version of the installed chrome browser – which may change on OS updates/upgrades. Moodle or Selenium will not give the appropriate message – see [https://tracker.moodle.org/browse/MDL-67659| MDL-67659]. One solution is the one suggested in the issue and use Andrew Nicols’ [https://github.com/andrewnicols/chromedriver-wrapper| Chromedriver Wrapper] which will ensure you have the appropriate driver before running the tests.

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

== See also ==
* [[Acceptance testing for the mobile app]]

[[Category:Quality Assurance]][[Category:Behat]]

Acceptance testing/Browsers/Working combinations of OS+Browser+selenium

2020-03-28T12:28:48Z

Bencellis: /* Moodle 3.5 and up */

= Working combinations of OS+Browser+selenium =
As OS, Browsers and Selenium keeps updating, some combination might fail on different Moodle releases.

Following combinations have been tested at the time of release of Moodle version and will be supported for that combination.

== Moodle 3.5 and up ==
{| class="wikitable"
|OS
|Browser
|Selenium Server
|Chrome Driver
|IE Driver
|Notes
|-
|style="white-space: nowrap;"|Linux - Debian Stretch/Buster
|style="white-space: nowrap;"|Firefox 47.0.1
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| N/A
| N/A
| Requires special behat config ([[Actual Selenium with old Firefox 47.0.1|more info here]])
|-
| Linux - Debian Stretch/Buster
| Chrome 66
Chrome 76<br/>
Chrome 80
| [https://selenium-release.storage.googleapis.com/3.11/selenium-server-standalone-3.11.0.jar 3.11.0]
[https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]<br/>
[https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.38/ 2.38]
[https://chromedriver.storage.googleapis.com/index.html?path=76.0.3809.126/ 76.0.3809.126]<br/>
[https://chromedriver.storage.googleapis.com/index.html?path=80.0.3987.106/ 80.0.3987.106]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
Chrome >=76 can be used since Moodle 20190909 builds.
|-
| MacOS X
| Firefox 47.0.1
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| N/A
| N/A
| Requires special behat config ([[Actual Selenium with old Firefox 47.0.1|more info here]])
|-
| MacOS X
| Chrome 72
Chrome 77
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
[https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.46/ 2.46]
[https://chromedriver.storage.googleapis.com/index.html?path=77.0.3865.40/ 77.0.3865.40]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
Chrome >=76 can be used since Moodle 20190909 builds.
|-
| Windows
| Chrome 72
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| [https://chromedriver.storage.googleapis.com/index.html?path=72.0.3626.69/ 72.0.3626.69]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
|}
General note: Many of the combinations below, for Moodle 3.1 and up, should continue working acceptably well for Moodle 3.5 and up. Just the those listed above are actively being used now (CI infrastructure, developers...), hence, verified to be running ok. Feel free to add more relevant working combinations!

== Moodle 3.4==
{| class="wikitable"
|OS
|Browser
|Selenium Server
|Chrome Driver
|IE Driver
|Notes
|-
|style="white-space: nowrap;"|Linux - Debian Stretch
|style="white-space: nowrap;"|Firefox 47.0.1
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| N/A
| N/A
| Requires special behat config ([[Actual Selenium with old Firefox 47.0.1|more info here]])
|-
| Linux - Debian Stretch
| Chrome 66
| [https://selenium-release.storage.googleapis.com/3.11/selenium-server-standalone-3.11.0.jar 3.11.0]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.38/ 2.38]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
|-
| MacOS X
| Firefox 47.0.1
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| N/A
| N/A
| Requires special behat config ([[Actual Selenium with old Firefox 47.0.1|more info here]])
|-
| MacOS X
| Chrome 72
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.46/ 2.46]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
|-
| Windows
| Chrome 72
| [https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar 3.141.59]
| [https://chromedriver.storage.googleapis.com/index.html?path=72.0.3626.69/ 72.0.3626.69]
| N/A
| Any other valid combination should work ok, normally.<br/>(Here there is a [https://github.com/SeleniumHQ/docker-selenium/releases good list] of them, as reference)
|}

== Moodle 3.2 and 3.3 ==
{| class="wikitable"
|OS
|Browser
|Selenium Server
|Chrome Driver
|IE Driver
|-
| Linux - Ubuntu 16.04
| Firefox 47.0.1
| [http://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar 2.53.1]
| N/A
| N/A
|-
|Linux - Ubuntu 16.04
|[https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Linux_x64%2F403380%2Fchrome-linux.zip?generation=1467337264475000&alt=media Chrome 53.0]
| [http://selenium-release.storage.googleapis.com/index.html?path=3.0/ 3.0.1]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.25/ 2.25]
| N/A
|-
| Linux - Ubuntu 16.04
| Phantomjs 2.1.1
| 2.53.1
| N/A
| N/A
|-
| Windows 7/10
| Firefox 47.0.1
| 2.53.1
| N/A
| N/A
|-
| Windows 7/10
| Chrome v53.0
| [http://selenium-release.storage.googleapis.com/index.html?path=3.0/ 3.0.1]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.25/ 2.25]
| N/A
|-
| MacOS X
| Firefox 47.0.1
| 2.53.1
| N/A
| N/A
|-
| MacOS X
| [http://www.slimjet.com/chrome/google-chrome-old-version.php Chrome v53.0]
| [http://selenium-release.storage.googleapis.com/index.html?path=3.0/ 3.0.1]
| [http://chromedriver.storage.googleapis.com/index.html?path=2.25/ 2.25]
| N/A

|}

== Moodle 3.1 ==
{| class="wikitable"
|OS
|Browser
|Selenium Server
|Chrome Driver
|IE Driver
|-
| Linux - Ubuntu 16.04
| [https://download.mozilla.org/?product=firefox-47.0.1-SSL&os=linux64&lang=en-GB Firefox 47.0.1]
| [http://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar 2.53.1]
| N/A
| N/A
|-
|Linux - Ubuntu 16.04
|[https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Linux_x64%2F386249%2Fchrome-linux.zip?generation=1460160957434000&alt=media Chrome 51.0]
| 2.53.1
| 2.22
| N/A
|-
| Linux - Ubuntu 16.04
| Phantomjs 2.1.1
| 2.53.1
| N/A
| N/A
|-
| Windows 7/10
| Firefox 47.0.1
| 2.53.1
| N/A
| N/A
|-
| Windows 7/10
| Chrome v51.0
| 2.53.1
| 2.22
| N/A
|-
| MacOS X
| Firefox 47.0.1
| 2.53.1
| N/A
| N/A
|-
| MacOS X
| Chrome v51.0
| 2.53.1
| 2.22
| N/A
|-
| MacOS X
| PhantomJS 2.1.1
| 2.53.1
| N/A
| N/A
|}

== Moodle 3.0 and lower ==
{| class="wikitable"
|OS
|Browser
|Selenium Server
|Chrome Driver
|IE Driver
|-
|Linux - Ubuntu 14.10
|Firefox 42.0
|2.47.1
| N/A
| N/A
|-
|Linux - Ubuntu 14.10
|Chrome 46.0
|2.47.1
| 2.19.346067
| N/A
|-
|Linux - Ubuntu 14.10
|Phantomjs 2.0.0
|2.47.1
| N/A
| N/A
|-
|Windows 7/10
|Firefox 41.0
|2.47.1
| N/A
| N/A
|-
|Windows 7/10
|Chrome 47.0
|2.47.1
| 2.20
| N/A
|-
|MacOS X
|Firefox 41.0
|2.47.1
| N/A
| N/A
|-
|MacOS X
|Chrome 46.0
|2.47.1
| 2.20
| N/A
|-
|MacOS X
|Chrome 48.0
|2.51.0
| 2.21
| N/A
|-
|MacOS X
|PhantomJS - 2.0.0 & 2.1.1
|2.48.2
| N/A
| N/A
|}

[[Category:Behat]]

Old Events API

2012-12-11T15:22:26Z

Bencellis: /* Events wishlist */

==Overview==

The Events API is a core system in Moodle to allow communication between modules.

An '''event''' is when something "interesting" happens in Moodle that is worth alerting the system about.

Any Moodle modules can '''trigger''' new events (with attached data), and other modules can elect to '''handle''' those events with custom functions that operate on the given data.

==Example==

Let's look at an example of how events are used to implement enrollment in Moodle 2.0. In the enrollment system, lib/enrollib.php will generate a 'user_enrolled' event upon completion of a enrollment of a user.

===Triggering an event===

When a user is enrolled, a 'user_enrolled' event is built and sent out by the enrol API. In this example let's pretend someone was just enrolled.

The enrol lib (could even be a module thats creating this event) needs to create an object with the data that this event needs. This may vary completely for different types of events, it's just a data object.
<code php>
$ue = new stdClass();
$ue->enrolid = $instance->id;
$ue->status = is_null($status) ? ENROL_USER_ACTIVE : $status;
$ue->userid = $userid;
$ue->timestart = $timestart;
$ue->timeend = $timeend;
$ue->modifierid = $USER->id;
...
$ue->courseid = $courseid;
$ue->enrol = $name;

events_trigger('user_enrolled', $ue);
</code>
Then we post the object as an event and forget about it.:
<code php>
events_trigger('user_enrolled', $eventdata);
</code>
We have now broadcast an event that we think other parts of the system might need to know about.

===Handling an event===

Modules or core code can define an events.php in under its */db directory which defines events they want to be notified about, and describes which of their functions or class methods should be notified. For this case, there is this definition of a handler in mod/forum/db/events.php.

<code php>
$handlers = array (
'user_enrolled' => array (
'handlerfile' => '/mod/forum/lib.php',
'handlerfunction' => 'forum_user_enrolled',
'schedule' => 'instant',
'internal' => 1,
),

'user_unenrolled' => array (
'handlerfile' => '/mod/forum/lib.php',
'handlerfunction' => 'forum_user_unenrolled',
'schedule' => 'instant',
'internal' => 1,
),
);
</code>

These events.php files are parsed during install / upgrade and stored in a simple database table.

Now, when a '''user_enrolled''' event happens, all the registered handlers functions for that event will be called something like this (but with more error handling):
<code php>
include_once($CFG->dirroot.$handlers['user_enrolled']['handlerfile']);
call_user_func($handlers['user_enrolled']['handlerfunction'], $eventdata);
</code>
Any code can hook into any events this way.

The handler function accepts one parameter (the event data object) and should return a boolean. Returning false indicates that there was an error and the event will be left in the event queue.
<code php>
function forum_user_unenrolled($eventdata) {
// handle event
// ...
return true;
}
</code>
note: a lib/db/events.php exists as legacy and no events should be added here. Core API is not supposed to define events for consumption. Only modules and core components (non-api) should define events for consumption/handling.

==Database structure==

There are 3 core tables for events. Note that if a handler is queued, and yet to be processed or processing failed, then all subsequent calls on that handler must be queued.

===events_handlers===

This table is for storing which components requests what type of event, and the location of the responsible handler functions.

These entries are created by parsing events.php files in all the modules, and can be rebuilt any time (during an upgrade, say).

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Info'''
|-
|id
|int(10)
|auto increment identifier
|-
|eventname
|varchar(255)
|name of the event, e.g. 'message_send'
|-
|handlermodule
|varchar(255)
|e.g. moodle, mod/forum, block/rss_client
|-
|handlerfile
|varchar(255)
|path to the file of the function, eg /lib/messagelib.php
|-
|handlerfunction
|text
|serialized string or array describing function, suitable to be passed to '''call_user_func()'''
|-
|schedule
|varchar(255)
|'cron' or 'instant'.
|-
|status
|int(10)
|number of failed attempts to process this handler
|-
|internal
|int(2)
|1 means standard plugin handler, 0 indicates if event handler sends data to external systems, this is used for example to prevent immediate sending of events from pending db transactions
|}

===events_queue===

This table is for storing queued events. It stores only one copy of the eventdata here, and entries from this table are being references by the events_queue_handlers table.

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Info'''
|-
|id
|int(10)
|auto increment identifier
|-
|eventdata
|longtext
|serialized version of the data object passed to the event handler.
|-
|stackdump
|text
|serialized debug_backtrace showing where the event was fired from
|-
|userid
|int(10)
|$USER->id when the event was fired
|-
|timecreated
|int(10)
|time stamp of the first time this was added
|}

===events_queue_handlers===

This is the list of queued handlers for processing. The event object is retrieved from the events_queue table. When no further reference is made to the events_queue table, the corresponding entry in the events_queue table should be deleted. Entry should get deleted (?) after a successful event processing by the specified handler. The status field keeps track of failures, after it gets to a certain number (eg 10?) it should trigger an "event failed" event (that could result in admin being emailed etc, or perhaps even the originating module taking care of it or rolling something back etc).

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Info'''
|-
|id
|int(10)
|auto increment identifier
|-
|queuedeventid
|int(10)
|foreign key id corresponding to the id of the event_queues table
|-
|handlerid
|int(10)
|foreign key id corresponding to the id of the event_handlers table
|-
|status
|int(10)
|number of failed attempts to process this handler
|-
|errormessage
|text
|if an error happened last time we tried to process this event, record it here.
|-
|timemodified
|int(10)
|time stamp of the last attempt to run this from the queue
|}

==Standards for naming events==

All event names should follow a consistent naming pattern, such as componentname_noun_verb (See [[Frankenstyle]] about the component name)

If the event is being fired after the action has taken place (as in most cases) then use the past tense for the verb (created / deleted / updated / sent).

If the event '''is''' the action, then use the present tense (create / delete / update / send).

==Events which exist==

When we add new events to core we should always add them here too.

Under each event, list the data sent as part of the event.

===Users===
* user_created
** full new record from 'user' table
* user_deleted
** record from 'user' table before marked as deleted
* user_updated
** full new record from 'user' table
* user_enrolled
** full user enrolment record (TBC)
* user_logout
** params TBC
* user_unenrol_modified
** full user enrolment record (TBC)
* user_unenrolled
** full user enrolment record (TBC)
* course_completed (introduced in version 2.3.2)
** full course_completions table record

===Roles===
* role_assigned
** full new record from 'role_assignments' table
* role_unassigned
** record from 'role_assignments', course context only

===Courses===
* course_created
** full course record
* course_updated
** full course record
* course_deleted
** full course record
* course_category_deleted
** full category record
* course_content_removed
** full course record

===Groups===
* groups_member_added
** groupid, userid
* groups_member_removed
** groupid, userid
* groups_group_created
** id, courseid, name, description, timecreated, timemodified, picture
* groups_group_updated
** id, courseid, name, description, timecreated, timemodified, picture
* groups_group_deleted
** id, courseid, name, description, timecreated, timemodified, picture
* groups_grouping_created
** id, courseid, name, timecreated, timemodified
* groups_grouping_updated
** id, courseid, name, timecreated, timemodified
* groups_grouping_deleted
** id, courseid, name, timecreated, timemodified
* groups_members_removed ''(user deleted from all groups in a course)''
** courseid, userid
* groups_groupings_groups_removed ''(remove all groups from all groupings in a course)''
** courseid (as plain integer, not object)
* groups_groups_deleted ''(delete all groups in a course)''
** courseid (as plain integer, not object)
* groups_groupings_deleted ''(delete all groupings in a course)''
** courseid (as plain integer, not object)

===Cohorts===
* cohort_added
** full cohort record (TBC)
* cohort_deleted
** full cohort record (TBC)
* cohort_member_added
** cohort ID, user ID (TBC)
* cohort_member_removed
** cohort ID, user ID (TBC)
* cohort_updated
** full cohort record (TBC)

===Messaging===
* message_send
** component = 'mod/forum': path in Moodle
** name = 'posts': type of message from that module (as module defines it)
** userfrom = $userfrom: a user object to send from
** userto = $userto: a user object to send to
** subject = 'subject line': a short text line
** fullmessage = 'full plain text': raw text as entered by user
** fullmessageformat = FORMAT_PLAIN|FORMAT_HTML|FORMAT_MOODLE|FORMAT_MARKDOWN: the format of this text
** fullmessagehtml = 'long html text'; html rendered version (optional)
** smallmessage = 'short text': useful for plugins like sms or twitter (optional)

===Portfolio===
* portfolio_send
** id : recordid in portfolio_tempdata table, used for itemid in file storage

===Blog===
Added for 2.5
* blog_entry_added
** full blog record
* blog_entry_edited
** full blog record
* blog_entry_deleted
** full blog record

===Other===
* assessable_file_uploaded
* assessable_files_done
* mod_created
* mod_deleted
* mod_updated
* quiz_attempt_started
* quiz_attempt_submitted - ''Note: these two quiz events changed in Moodle 2.1''.

==Events wishlist==

List of events which it would be nice to have. Please add to this list if what you want is not shown here.

* mform_print_form -- this for all types of form e.g. admin settings, user profile, module updating, + some sort of standard way of discriminating between them e.g. if ($form->name == 'user_profile') {}. This would be better triggered at the end of the form generation process so that new bits can be inserted at any point, or existing bits could be removed.
* module_installed ''(= mod_created in v2?)''
* module_removed ''(= mod_deleted in v2?)''
* grade_update ''(= quiz_attempt_processed in v2?)''
* assignment_submitted
* course module completion state changed (completed / not completed)
* course_restored
* user_login
Provide event trigger hooks for the modules, similar to what is done for the cron service which checks the LOCAL directory for a cron file. It is already possible to define an event trigger but the core/module code must be modified to actually make use of it.

== See also ==

* [[Core APIs]]
* [http://moodle.org/mod/forum/discuss.php?d=69103 Original General Developer Forum thread discussing this proposal].
* [[Messaging_2.0]]

[[Category:Coding guidelines|Events]]
[[Category:Grades]]
[[Category:API]]