MoodleDocs - User contributions [en]

Grunt

2020-12-07T13:11:00Z

Howardsmiller: /* Running grunt */

{{Moodle 2.9}}

Grunt is a command line tool used to prepare our javascript and less-generated css for production usage. After making any change to javascript or less files in Moodle, you must run grunt to lint, minify and package the javascript/css properly so that it can be served by Moodle.

== Install grunt ==
javascript and CSS in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[http://gruntjs.com/ grunt]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment. You will need to install NodeJS and the Grunt tools. For documentation on this process see [[Javascript_Modules#Install_grunt]].

== Running grunt ==
Typical commands:

grunt amd # Short-cut for grunt jshint uglify, rebuild all AMD modules.
grunt shifter # Run shifter
grunt js # Short-cut for grunt amd shifter

grunt css # Run less
grunt # Try to do the right thing:
# * If you are in a folder called amd, do grunt amd
# * If you are in a folder called yui/src/something, do grunt shifter
# * Otherwise build everything (grunt css js).

grunt watch # Watch for changes and re-run grunt tasks depending on what file changes
grunt eslint --show-lint-warnings # Show pedantic lint warnings for JS

On Linux/Mac it will build everything in the current folder and below. You need to cd into the amd folder of your module root, i.e. <tt>dirroot/blocks/foo/amd</tt> before running <tt>grunt amd</tt> - this will compile only your plugins AMD source files. You can make output more verbose by adding <tt>-v</tt> parameter, if used with <tt>grunt shifter</tt> you will have to cd into the <tt>module/yui/src</tt> folder, and to show what your lint errors are you can also use the <tt>-v</tt> parameter. On Windows, you need to specify the path on the command line like <tt>--root=admin/tool/templatelibrary</tt>.

'''Note for grunt watch'''. If you get an error complaining about 'watchman', you need to install it. Ubuntu/Debian is 'sudo apt install watchman' (for example).

== Using Grunt in additional plugins ==

You may want to use Grunt for performing tasks in your custom Moodle plugins. For building AMD and YUI modules in your plugin, the standard configuration Gruntfile.js located in the Moodle root should work well. For building CSS files from LESS templates, you will have to set up a separate Grunt installation in the root of your plugin.

If you do not have it yet, create the package.json file in the root of your plugin:

{
"name": "moodle-plugintype_pluginname"
}

Install grunt, grunt less and grunt watch modules. Note that you should put the folder node_modules into your plugin's .gitignore file, too.

$ cd /path/to/your/plugin/root
$ npm install --save-dev grunt grunt-contrib-less grunt-contrib-watch grunt-load-gruntfile

Create a Gruntfile.js in the root of your plugin and configure the task for building CSS files from LESS:

<code javascript>
"use strict";

module.exports = function (grunt) {

// We need to include the core Moodle grunt file too, otherwise we can't run tasks like "amd".
require("grunt-load-gruntfile")(grunt);
grunt.loadGruntfile("../../Gruntfile.js");

// Load all grunt tasks.
grunt.loadNpmTasks("grunt-contrib-less");
grunt.loadNpmTasks("grunt-contrib-watch");
grunt.loadNpmTasks("grunt-contrib-clean");

grunt.initConfig({
watch: {
// If any .less file changes in directory "less" then run the "less" task.
files: "less/*.less",
tasks: ["less"]
},
less: {
// Production config is also available.
development: {
options: {
// Specifies directories to scan for @import directives when parsing.
// Default value is the directory of the source, which is probably what you want.
paths: ["less/"],
compress: true
},
files: {
"styles.css": "less/styles.less"
}
},
}
});
// The default task (running "grunt" in console).
grunt.registerTask("default", ["less"]);
};
</code>

Now running "grunt" or "grunt less" in your plugin root folder will compile the file less/styles.less and saves it as styles.css. Running "grunt watch" will watch the less/*.less files and if some is changed, it will immediately rebuild the CSS file.

If you are working on a custom theme, you may have multiple less/*.less files that you want to compile to their style/*.css counterparts. You can either define an explicit list all such file pairs, or let that list be created for you by making use of [http://gruntjs.com/configuring-tasks#building-the-files-object-dynamically expand:true feature] of gruntfile.js:

<code javascript>
// This dynamically creates the list of files to be processed.
files: [
{
expand: true,
cwd: "less/",
src: "*.less",
dest: "style/",
ext: ".css"
}
]
</code>

== See also ==
* [[YUI/Shifter]]
* [[Javascript Modules]]
* [[LESS]]
* [https://moodle.org/mod/forum/discuss.php?d=400229#p1614743 Bitbucket pipeline to build YUI JS]

[[Category:AJAX]]
[[Category:Javascript]]

lib/formslib.php Form Definition

2020-11-27T14:44:39Z

Howardsmiller: /* date_time_selector */

{{Formslib}}
== ''definition()'' ==

The definition of the elements to be included in the form, their 'types' (PARAM_*), helpbuttons included, etc. is all included in a function you must define in your class.

''definition()'' is used to define the elements in the form and '''this definition will be used for validating data submitted as well as for printing the form.''' For select and checkbox type elements only data that could have been selected will be allowed. And only data that corresponds to a form element in the definition will be accepted as submitted data.

The ''definition()'' should include all elements that are going to be used on form, some elements may be removed or tweaked later in ''definition_after_data()''. Please do not create conditional elements in ''definition()'', the definition() should not directly depend on the submitted data.

Note that the definition function is called when the form class is instantiated. There is no option to (say) manipulate data in the class (that may affect the rendering of the form) between instantiating the form and calling any other methods.

<code php>
require_once("$CFG->libdir/formslib.php");

class simplehtml_form extends moodleform {

function definition() {
global $CFG;

$mform = $this->_form; // Don't forget the underscore!

$mform->addElement()... // Add elements to your form
...
} // Close the function
} // Close the class
</code>
===Passing parameters to the Form===

The constructor for ''moodleform'' allows a number of parameters including one (''$customdata'') to permit an array of arbitrary data to be passed to your form.

For example, you can pass the data "$email" and "$username" to the Form's class for use inside (say) the definition.
<code php>
...
$mform_simple = new simplehtml_form( null, array('email'=>$email, 'username'=>$username ) );
...
</code>
(the first parameter is $action, ''null'' will cause the form action to be determined automatically)

Secondly, inside the form definition you can use those parameters to set the default values to some of the form's fields
<code php>
...
$mform->addElement('text', 'email', get_string('email'), 'maxlength="100" size="25" ');
$mform->setType('email', PARAM_NOTAGS);
$mform->addRule('email', get_string('missingemail'), 'required', null, 'server');
// Set default value by using a passed parameter
$mform->setDefault('email',$this->_customdata['email']);
...
</code>

==Use Fieldsets to group Form Elements==

You use code like this to open a fieldset with a ''legend''. <br />
('''Note''': Some themes turn off legends on admin setting pages by using CSS: <nowiki>#adminsettings legend {display:none;}</nowiki>.)

<code php>
$mform->addElement('header', 'nameforyourheaderelement', get_string('titleforlegened', 'modulename'));
</code>

You can't yet nest these visible fieldsets unfortunately. But in fact groups of elements are wrapped in invisible fieldsets.

{{Moodle 2.5}}
Since Moodle 2.5 fieldsets without any required fields are collapsed by default. To display these fieldsets on page load, use:
<code php>
$mform->setExpanded('foo')
</code>
To close a fieldset on page load, use:
<code php>
$mform->setExpanded('foo', false)
</code>

You close a fieldset with moodle_form's closeHeaderBefore method. You tell closeHeaderBefore the element before you wish to end the fieldset. A fieldset is automatically closed if you open a new one. You need to use this code only if you want to close a fieldset and the subsequent form elements are not to be enclosed by a visible fieldset (they are still enclosed with an invisibe one with no legend) :

<code php>
$mform->closeHeaderBefore('buttonar');
</code>

==addElement==

Use the addElement method to add an element to a form. The first few arguments are always the same. The first param is the type of the element to add. The second is the elementname to use which is normally the html name of the element in the form. The third is often the text for the label for the element.

Some examples are below :
=== button ===

<code php>
$mform->addElement('button', 'intro', get_string("buttonlabel"));
</code>

A button element. If you want a submit or cancel button see 'submit' element.

=== autocomplete ===
{{Moodle 3.1}}
Available since Moodle 3.1

<code php>
$searchareas = \core_search\manager::get_search_areas_list(true);
$areanames = array();
foreach ($searchareas as $areaid => $searcharea) {
$areanames[$areaid] = $searcharea->get_visible_name();
}
$options = array(
'multiple' => true,

'noselectionstring' => get_string('allareas', 'search'),
);
$mform->addElement('autocomplete', 'areaids', get_string('searcharea', 'search'), $areanames, $options);
</code>

The autocomplete element is an advanced form element that supports server-side searching - or simple filtering of a predefined list of options. Some benefits of using this form element are that it handles very large datasets extremely well - it has great accessibility built in - and it gives a good user experience. If you have so much data you need to build pagination into a page - you could probably come up with a better design using this. The simplest way to use it is compatible with the standard 'select' form element. You give it a list of options and some parameters to configure how it behaves. The valid parameters for this simple mode of operation are:
* multiple (boolean - default false) - Allow more than one selected item. The data coming from the form will be an array in this case.

[[image:autocomplete_multiple2.png|center|thumb|alt=autocomplete with multiple option|autocomplete with multiple option.]]

* noselectionstring (string - default <nowiki>''</nowiki>) - The text to display when nothing is selected.
* showsuggestions (boolean - default true) - Do not show the list of suggestions when the user starts typing.
* placeholder (string - default <nowiki>''</nowiki>) - The text to show in the search box when it is empty.
* casesensitive (boolean - default false) - Is the search case sensitive ?
* tags (boolean - default false) - This changes the behaviour so that the user can create new valid entries in the list by typing them and pressing enter.
* ajax (string - default <nowiki>''</nowiki>) - This string is the name of an AMD module that can fetch and format results.
* valuehtmlcallback - For use with the AJAX option, so that it can format the initial value of the form field (available since Moodle 3.5)

More explanation on the 'ajax' option. This should be the name of an AMD module that implements 2 functions:
<code javascript>
/**
* Source of data for Ajax element.
*
* @param {String} selector The selector of the auto complete element.
* @param {String} query The query string.
* @param {Function} callback A callback function receiving an array of results.
* @return {Void}
*/
transport: function(selector, query, callback) ...

/**
* Process the results for auto complete elements.
*
* @param {String} selector The selector of the auto complete element.
* @param {Array} results An array or results.
* @return {Array} New array of results.
*/
processResults: function(selector, results)...

</code>
A good example is here: [https://github.com/moodle/moodle/blob/master/admin/tool/lp/amd/src/frameworks_datasource.js admin/tool/lp/amd/src/frameworks_datasource.js]

The 'valuehtmlcallback' function is needed when an AJAX-supporting form field has an initial value that needs special rendering, similar to how the AJAX code would render it when the user changes it dynamically. For example, if the field contains user ids and its initial value is '1,2' then you want it to use the rendered HTML display for each value (probably a user's name and picture), not just display those numbers. Here is an example, from /search/classes/output/form/search.php:

<code php>
'valuehtmlcallback' => function($value) {
global $DB, $OUTPUT;
$user = $DB->get_record('user', ['id' => (int)$value], '*', IGNORE_MISSING);
if (!$user || !user_can_view_profile($user)) {
return false;
}
$details = user_get_user_details($user);
return $OUTPUT->render_from_template(
'core_search/form-user-selector-suggestion', $details);
}
</code>

The Mustache template used here is the same as the one used by the AJAX code when the field is changed dynamically.

When using the ajax option in an mform with validation etc - it is recommended to sub-class the php class "MoodleQuickForm_autocomplete" so that you can provide a list of name and
values to populate the form element if the form is re-displayed due to a validation error. An example is [https://github.com/moodle/moodle/blob/MOODLE_31_STABLE/admin/tool/lp/classes/form/framework_autocomplete.php admin/tool/lp/classes/form/framework_autocomplete.php].

We have provided several useful subclasses of this form element already that are simple to use (course and tags).

<code php>
// Course example.
// Valid options are:
// 'multiple' - boolean multi select
// 'exclude' - array or int, list of course ids to never show
// 'requiredcapabilities' - array of capabilities. Uses ANY to combine them.
// 'limittoenrolled' - boolean Limits to enrolled courses.
// 'includefrontpage' - boolean Enables the frontpage to be selected.
$options = array('multiple' => true, 'includefrontpage' => true);
$mform->addElement('course', 'mappedcourses', get_string('courses'), $options);

// Tags
// Valid options are:
// 'showstandard' - boolean One of the core_tag_tag constants to say which tags to display
// 'component' - string The component name and itemtype define the tag area
// 'itemtype' - string The component name and itemtype define the tag area
$mform->addElement('tags', 'interests', get_string('interestslist'), array('itemtype' => 'user', 'component' => 'core'));
</code>

=== checkbox ===

<code php>
$mform->addElement('checkbox', 'ratingtime', get_string('ratingtime', 'forum'));
</code>

This is a simple checkbox. The third parameter for this element is the label to display on the left side of the form. You can also supply a string as a fourth parameter to specify a label that will appear on the right of the element. Checkboxes and radio buttons can be grouped and have individual labels on their right.

You can have a 5th parameter $attributes, as on other elements.

'''BEWARE:''' Unchecked checkboxes return nothing at all (as if they didn't exist). This can surprise the unwary. You may wish to use advcheckbox instead, which does return a value when not checked. 'Advcheckbox' eliminates this problem.

==== advcheckbox ====
<code php>
$mform->addElement('advcheckbox', 'ratingtime', get_string('ratingtime', 'forum'), 'Label displayed after checkbox', array('group' => 1), array(0, 1));
</code>

Similar to the checkbox above, but with some important improvements:

# The (optional) 5th parameter is a normal $attributes array, normally used to set HTML attributes for the <input> element. However, a special value of 'group' can be given, which will add a class name to the element, and enable its grouping for a [[lib/formslib.php_add_checkbox_controller|checkbox controller]]
#The (optional) 6th parameter is an array of values that will be associated with the checked/unchecked state of the checkbox. With a normal checkbox you cannot choose that value, and in fact an unchecked checkbox will not even be sent with the form data.
#It returns a 0 value when unchecked. Compare with the ordinary checkbox which does not return anything at all.

=== choosecoursefile ===
{{Moodle 1.9}}
<code php>
$mform->addElement('choosecoursefile', 'mediafile', get_string('mediafile', 'lesson'), array('courseid'=>$COURSE->id));
</code>

Choose a file from the course files area. The fourth option is a list of options for the element.

'''Note: This has been superceded by [[#filepicker|filepicker]] in Moodle 2.'''

<code php>
array('courseid' =>null, //if it is null (default then use global $COURSE
'height' =>500, // height of the popup window
'width' =>750, // width of the popup window
'options' =>'none'); //options string for the pop up window
//eg. 'menubar=0,location=0,scrollbars,resizable'
</code>

You can also pass an optional 5th parameter of attributes, as for other elements. The most useful way of using that is something like
<code php>array('maxlength' => 255, 'size' => 48)</code>
to control the maxlength / size of the text box (note size will default to 48 if not specified)

Finally, as this element is a group containing two elements (button + value), you can add validation rules by using the '''addGroupRule()''' method in this (complex) way:

<code php>$mform->addGroupRule('elementname', array('value' => array(array(list, of, rule, params, but, fieldname))));</code>

Where: '''"elementname"''' is the name of the choosecoursefile group element, '''"value"''' is the name of the text field within the group and the '''"list, of, addrule, params, but, fieldname"''' is exactly that, the list of fields in the normal addRule() function but ommiting the first one, the fieldname.

For example, the [http://cvs.moodle.org/moodle/mod/resource/type/file/resource.class.php?view=markup file/url resource type], uses one "choosecoursefile" element, and it controls the maximum length of the field (255) with this use of addGroupRule():

<code php>$mform->addGroupRule('reference', array('value' => array(array(get_string('maximumchars', '', 255), 'maxlength', 255, 'client'))));</code>

=== date_selector ===

<code php>
$mform->addElement('date_selector', 'assesstimefinish', get_string('to'));
</code>

This is a date selector. You can select a Day, Month and Year using a group of select boxes. The fourth param here is an array of options. The defaults for the options are :

<code php>
array(
'startyear' => 1970,
'stopyear' => 2020,
'timezone' => 99,
'optional' => false
);
</code>

You can override these defaults by supplying an array as fourth param with one or more keys with a value to override the default. You can supply a fifth param of attributes here as well.

=== date_time_selector ===

<code php>
$mform->addElement('date_time_selector', 'assesstimestart', get_string('from'));
</code>

This is a group of select boxes to select a date (Day Month and Year) and time (Hour and Minute). When submitted, submitted data is processed and a timestamp is passed to $form->get_data(); the fourth param here is an array of options. The defaults for the options are:

<code php>
array(
'startyear' => 1970,
'stopyear' => 2020,
'timezone' => 99,
'step' => 5,
'optional' => false,
);
</code>

You can override these defaults by supplying an array as fourth param with one or more keys with a value to override the default. Setting 'optional' to true adds an 'enable' checkbox to the selector. You can supply a fifth param of attributes here as well.

===duration===
{{Moodle 2.0}}

<code php>
$mform->addElement('duration', 'timelimit', get_string('timelimit', 'quiz'));
</code>
This field type lets the user input an interval of time. It comprises a text field, where you can type a number, and a dropdown for selecting a unit (days, hours, minutes or seconds). When submitted the value is converted to a number of seconds.

You can add a fourth parameter to give options. At the moment the only option supported is here is an array of options. The defaults for the options is:
<code php>array('optional' => true)</code>

You can also pass an optional 5th parameter of attributes, as for other elements. The most useful way of using that is something like
<code php>array('size' => 5)</code>
to control the size of the text box.

=== editor ===

This replaces the old htmleditor field type. It allows the user to enter rich text content in a variety of formats.

<code php>
$mform->addElement('editor', 'fieldname', get_string('labeltext', 'langfile'));
$mform->setType('fieldname', PARAM_RAW);
</code>

NOTE: It won't work properly without the setType() as shown.

If you would like to let the user use the filepicker to upload images etc. that are used in the content, then see [[Using_the_File_API_in_Moodle_forms]].

You can supply a fourth param to htmleditor of an array of options that are mostly related to file handling:

<code php>
array(
'subdirs'=>0,
'maxbytes'=>0,
'maxfiles'=>0,
'changeformat'=>0,
'context'=>null,
'noclean'=>0,
'trusttext'=>0,
'enable_filemanagement' => true);
</code>

The option 'enable_filemanagement' will display the file management button on true and remove it on false.

To save the data if you don't care about files:
<code php>
$formdata = $mform->get_data();
$text = $formdata->fieldname['text'];
$format = $formdata->fieldname['format'];
</code>

Note: Because the text editor might be "Atto" (depending on user preferences) and Atto has an "autosave" feature - it requires that the combination of $PAGE->url and this elementid are unique. If not, the autosaved text for a different form may be restored into this form.

=== file ===

File upload input box with browse button. In the form definition type

<code php>
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
</code>

after form submission and validation use

<code php>
if ($data = $mform->get_data()) {
...
$mform->save_files($destination_directory);
...
}
</code>

If there is no requirement to save the file, you can read the file contents directly into a string as follows...

<code php>
$mform->get_file_content('attachment');
</code>

If you need advanced settings such as required file, different max upload size or name of uploaded file

<code php>
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0, true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->addRule('attachment', null, 'required');

</code>

<code php>
if ($data = $mform->get_data()) {
...
$mform->save_files($destination_directory);
$newfilename = $mform->get_new_filename();
...
}
</code>

When porting old code it is also possible to use the upload manager directly for processing of uploaded files.

Please note that if using set_upload_manager() it must be before addElement('file',..).

{{Moodle 2.0}}
File uploading was rewritten in 2.0. Please see inline docs for now. This page will be updated when the new API stabilises.

===filepicker===
{{Moodle 2.0}}
General replacement of ''file'' element.

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null, array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>
See also [[Using the File API in Moodle forms]]

=== hidden ===
<code php>
$mform->addElement('hidden', 'reply', 'yes');
</code>

A hidden element. Set the element name (in this case '''reply''') to the stated value (in this case '''yes''').

=== html ===
You can add arbitrary HTML to your Moodle form:

<code php>
$mform->addElement('html', '<div class="qheader">');
</code>

See [http://moodle.org/mod/forum/discuss.php?d=126935 "Question: Can I put a moodleform inside a table td?"] for a concrete example.

=== htmleditor & format ===

These elements are now deprecated. Please use the [[#editor|editor]] field type instead.

===modgrade===
<pre>
$mform->addElement('modgrade', 'scale', get_string('grade'), false);
</pre>
This is a custom element for selecting a grade for any activity module. The fourth argument is whether to include an option for no grade which has a value 0. This select box does include scales. The default is true, include no grade option.

A helpbutton is automatically added.

===modvisible===
<pre>
$mform->addElement('modvisible', 'visible', get_string('visible'));
</pre>
This is a custom element for selecting a grade visibility in an activity mod update form.

===password===
<pre>
$mform->addElement('password', 'password', get_string('label'), $attributes);
</pre>

A password element. Fourth param is an array or string of attributes.

===passwordunmask===
{{Moodle 1.9}}
<pre>
$mform->addElement('passwordunmask', 'password', get_string('label'), $attributes);
</pre>

A password element with option to show the password in plaintext. Fourth param is an array or string of attributes.

=== radio ===
{{Moodle 2.3}}
<code php>
$radioarray=array();
$radioarray[] = $mform->createElement('radio', 'yesno', '', get_string('yes'), 1, $attributes);
$radioarray[] = $mform->createElement('radio', 'yesno', '', get_string('no'), 0, $attributes);
$mform->addGroup($radioarray, 'radioar', '', array(' '), false);
</code>

Second param names the radio button and should be the same for each button in the group in order to toggle correctly. Third param would be the label for the form element but is generally ignored as this element will always be in a group which has it's own label. Fourth param is a string, a label to be displayed on the right of the element. The fifth is the value for this radio button. $attributes can be a string or an array of attributes.

It is possible to add help to individual radio buttons but this requires a custom template to be defined for the group elements. See MDL-15571.

Since 2.3 it cannot be statically called anymore, so we need to call createElement from $mform reference.

==== setDefault ====

To set the default for a radio button group as above use the following :

<code php>
$mform->setDefault('yesno', 0);
</code>

This would make the default 'no'.

===select===
<pre>
$mform->addElement('select', 'type', get_string('forumtype', 'forum'), $FORUM_TYPES, $attributes);
</pre>

The fourth param for this element is an array of options for the select box. The keys are the values for the option and the value of the array is the text for the option. The fifth param $attributes is optional, see text element for description of attributes param.

It is also possible to create a select with certain options disabled, using [http://stackoverflow.com/questions/2138089/how-can-i-use-quickform-to-add-disabled-select-options/2150275#2150275 this technique].

You can set an 'onchange' attribute when adding or creating the select element:

$form->addElement('select', 'iselTest', 'Test Select:', $arrayOfOptions, array('onchange' => 'javascript:myFunctionToDoSomething();'));

====multi-select====
<pre>
$select = $mform->addElement('select', 'colors', get_string('colors'), array('red', 'blue', 'green'), $attributes);
$select->setMultiple(true);
</pre>

=====setSelected=====

To set the default selected item in a select element, you can use the 'setSelected' method. The 'setSelected' can either get a value or an array of values.

<pre>
$options = array(
'ff0000' => 'Red',
'00ff00' => 'Green',
'0000ff' => 'Blue'
);
$select = $mform->addElement('select', 'colors', get_string('colors'), $options);
// This will select the colour blue.
$select->setSelected('0000ff');
</pre>

Or for multiple-selection:
<pre>
$skillsarray = array(
'val1' => 'Skill A',
'val2' => 'Skill B',
'val3' => 'Skill C'
);
$mform->addElement('select', 'md_skills', get_string('skills', 'metadata'), $skillsarray);
$mform->getElement('md_skills')->setMultiple(true);
// This will select the skills A and B.
$mform->getElement('md_skills')->setSelected(array('val1', 'val2'));
</pre>

However you probably don't want to do this. Instead you probably want to use setDefault, or set it using the form's setData method.

===selectyesno===
<pre>
$mform->addElement('selectyesno', 'maxbytes', get_string('maxattachmentsize', 'forum'));
</pre>

If you want a yes / no select box this one automatically translates itself and has value 1 for yes and 0 for no.

===selectwithlink===
<pre>
$options = array();
$mform->addElement('selectwithlink', 'scaleid', get_string('scale'), $options, null,
array('link' => $CFG->wwwroot.'/grade/edit/scale/edit.php?courseid='.$COURSE->id, 'label' => get_string('scalescustomcreate')));
</pre>
select type element with options containing link
===static===
<pre>
$mform->addElement('static', 'description', get_string('description', 'exercise'),
get_string('descriptionofexercise', 'exercise', $COURSE->students));
</pre>

This is a static element. It should be used with care if it is used to display a static piece of text with a label. The third param is the label and the fourth is the static text itself.

===submit, reset and cancel===

<pre>
//normally you use add_action_buttons instead of this code
$buttonarray=array();
$buttonarray[] = $mform->createElement('submit', 'submitbutton', get_string('savechanges'));
$buttonarray[] = $mform->createElement('reset', 'resetbutton', get_string('revert'));
$buttonarray[] = $mform->createElement('cancel');
$mform->addGroup($buttonarray, 'buttonar', '', ' ', false);
</pre>

A 'Submit' type element is a submit type form element which will submit the form. A 'Reset' will not submit the form but will reset any changes the user has made to form contents. A 'Cancel' element cancels form submission. You need to have a branch in your code before you check for get_data() to check if submission has been cancelled with is_cancelled(); See the example on the usage page.

You should name your submit and reset buttons 'submitbutton' and 'resetbutton' or something similar (not 'submit' and 'reset'). This avoids problems in JavaScript of collisions between form element names and names of JavaScript methods of the form object.

====add_action_buttons($cancel = true, $submitlabel=null);====

You will normally use this helper function which is a method of moodleform to add all the 'action' buttons to the end of your form. A boolean parameter allow you to specify whether to include a cancel button and specify the label for your submit button (pass the result of get_string). Default for the submit button label is get_string('savechanges'). Note the '''$this''' not '''$mform'''

<pre>
$this->add_action_buttons();
</pre>

===text===

<pre>
$mform->addElement('text', 'name', get_string('forumname', 'forum'), $attributes);
</pre>
For a simple text input element. (For text labels, use the 'static' element.) Your fourth parameter here can be a string or array of attributes for the text element. The following are equivalent :
<pre>
$attributes='size="20"';
$attributes=array('size'=>'20');
</pre>

Generally you are encouraged to use CSS instead of using attributes for styling.

A format element can be used as a format select box. It will be non-selectable if you're using an html editor.

The third param for this element is $useHtmlEditor and it defaults to null in which case an html editor is used if the browser and user profile support it.

====RTL support====
{{Moodle 3.2}}

As of Moodle 3.2, some form elements have been refined to better support right-to-left languages. In RTL, most fields should not have their direction flipped, a URL, a path to a file, a number, ... are always displayed LTR. Input fields and text areas now will best guess whether they should be forced to be displayed in LTR based on the PARAM type associated with it. You can call:
<pre>
$mform->setForceLtr('name', true/false);
</pre>
on some form fields (like 'text') to manually set the value, and for directionality.

====float====
{{Moodle 3.7}}
<pre>
$mform->addElement('float', 'defaultmark', get_string('defaultmark', 'question'), $attributes);
</pre>
Use the float element if you want a text box to get a floating point number. This element automatically supports localised decimal separators. You don't need to use setType with the float element.

===textarea===

<pre>
$mform->addElement('textarea', 'introduction', get_string("introtext", "survey"), 'wrap="virtual" rows="20" cols="50"');
</pre>

A textarea element. If you want an htmleditor use htmleditor element. Fourth element here is a string or array of attributes.

===recaptcha===
{{Moodle 1.9}}
<pre>
$mform->addElement('recaptcha', 'recaptcha_field_name', $attributes);
</pre>

Use this recaptcha element to reduce the spam risk in your forms. Third element here is a string or array of attributes. Take care to get an API key from http://recaptcha.net/api/getkey before using this element.

To check whether recaptcha is enabled at site level use:
<pre>
if (!empty($CFG->recaptchapublickey) && !empty($CFG->recaptchaprivatekey)) {
//recaptcha is enabled
}
</pre>

===tags===
{{Moodle 2.0}}

<pre>
$mform->addElement('tags', 'field_name', $lable, $options, $attributes);
</pre>

Used for editing a list of tags, for example on a blog post.

There is only one option available, 'display', which should be set to one of the contstants MoodleQuickForm_tags::ONLYOFFICIAL, NOOFFICIAL or DEFAULTUI. This controls whether the official tags are listed for easy selection, or a text area where arbitrary tags may be typed, or both. The default is both.

The value should be set/returned as an array of tags.

===grading===
<code php>
$mform->addElement('grading', 'advancedgrading', get_string('grade').':', array('gradinginstance' => $gradinginstance));
</code>

Custom element for advanced grading plugins.

When adding the 'grading' element to the form, developer must pass an object of class gradingform_instance as $attributes['gradinginstance']. Otherwise an exception will be thrown.

===questioncategory===
<code php>
$mform->addElement('questioncategory', 'category', get_string('category', 'question'),
array('contexts'=>$contexts, 'top'=>true, 'currentcat'=>$currentcat, 'nochildrenof'=>$currentcat));
</code>
Creates a drop down element to select a question category.

Options are:
'''contexts''' - (required) context in which question appears
'''currentcat''' - (optional) course category
'''top''' - (optional) if true will put top categories on top
'''nochildrenof''' - (optional) Format categories into an indented list reflecting the tree structure

=== filetypes ===
{{Moodle 3.4}}
Available since Moodle 3.4

<code php>
$mform->addElement('filetypes', 'allowedfiletypes', get_string('allowedfiletypes', 'tool_myplugin'));
</code>

Creates an input element allowing the user to specify file types for the given purpose. The typical scenario is a setting that allows the teacher define a list of allowed file types submitted by students.

The element allows the user to either type the list of filetypes manually, or select the types from the list. Also supported is selecting the whole group of file types - such as "image". The element integrates with the [[Core filetypes]] system so all default types and groups are presented, as well as those [[:en:Working with files#Site administration settings|defined locally by the admin]].

As the list can be types in manually, the form processing code should always normalize it first via the provided utility methods:

<code php>
$formdata = $mform->get_data();
$filetypesutil = new \core_form\filetypes_util();
$allowedfiletypes = $filetypesutil->normalize_file_types($formdata->allowedfiletypes);
</code>

This always returns an array of recognized valid values. The original list can be separated by whitespace, end of lines, commas, colons and semicolons. During the normalization, values are converted to lowercase, empty valies and duplicates are removed. Glob evaluation is not supported.

The normalization should also happen if the previously defined list had been saved to the database and re-read for actual usage. The normalization output value can be directly used as the accepted_types option for the filepicker.

<code php>
$filetypesutil = new \core_form\filetypes_util();
$options['accepted_types'] = $filetypesutil->normalize_file_types($allowedfiletypes);
</code>

By default, user input is validated against the list of known file types and groups. This validation can be disabled via options.

'''Supported options'''

; onlytypes : Allow selection from these file types only; for example ['onlytypes' => ['web_image']].
; allowall : Allow to select 'All file types', defaults to true. Does not apply with onlytypes are set.
; allowunknown : Skip implicit validation against the list of known file types.

Example:

<code php>
$mform->addElement('filetypes', 'doctypes', get_string('doctypes', 'tool_myplugin'), ['onlytypes' => ['document'], 'allowunknown' => true]);
</code>

==addGroup==

A 'group' in formslib is just a group of elements that will have a label and will be included on one line.

For example typical code to include a submit and cancel button on the same line :

<pre>
$buttonarray=array();
$buttonarray[] =& $mform->createElement('submit', 'submitbutton', get_string('savechanges'));
$buttonarray[] =& $mform->createElement('submit', 'cancel', get_string('cancel'));
$mform->addGroup($buttonarray, 'buttonar', '', array(' '), false);
</pre>

You use the same arguments for createElement as you do for addElement. Any label for the element in the third argument is normally ignored (but not in the case of the submit buttons above where the third argument is not for a label but is the text for the button).

Here's a bad example (don't do this for real, use the 'optional' => true option of the date element): putting a date_selector (which is itself a group of elements) and a checkbox on the same line, note that you can disable every element in the group using the group name 'availablefromgroup' but it doesn't disable the controlling element the 'availablefromenabled' checkbox:

<pre>
$availablefromgroup=array();
$availablefromgroup[] =& $mform->createElement('date_selector', 'availablefrom', '');
$availablefromgroup[] =& $mform->createElement('checkbox', 'availablefromenabled', '', get_string('enable'));
$mform->addGroup($availablefromgroup, 'availablefromgroup', get_string('availablefromdate', 'data'), ' ', false);
$mform->disabledIf('availablefromgroup', 'availablefromenabled');
</pre>

* If you want to put a group inside another array so that you can repeat items, use createElement instead of addGroup:

<pre>
$group = $mform->createElement('group', 'groupname', get_string('label'), $groupitems);
</pre>

* By default, groups modify the names of elements inside them by appending a number. This is often unhelpful, for example if you want to use disabledIf on the element. To prevent this behaviour, set the last parameter to false when creating a group.:

<pre>
$group = $mform->createElement('group', 'groupname', get_string('label'), $groupitems, null, false);
</pre>

==addRule==
<pre>
$mform->addRule('elementname', get_string('error'), 'rule type', 'extraruledata', 'server'(default), false, false);
</pre>

The first param(element) is an element name and second(message) is the error message that will be displayed to the user.
The third parameter(type) is the type of rule. The fourth param(format) is used for extra data needed with some rules such as minlength and regex. The fifth parameter(validation) validates input data on server or client side, if validation is done on client side then it will be checked on the server side as well.

<pre>
* @param string $element Form element name
* @param string $message Message to display for invalid data
* @param string $type Rule type, use getRegisteredRules() to get types
* @param string $format (optional)Required for extra rule data
* @param string $validation (optional)Where to perform validation: "server", "client"
* @param boolean $reset Client-side validation: reset the form element to its original value if there is an error?
* @param boolean $force Force the rule to be applied, even if the target form element does not exist
</pre>

'''Common Rule Types'''
* required
* maxlength
* minlength
* rangelength
* email
* regex
* lettersonly
* alphanumeric
* numeric
* nopunctuation
* nonzero
* callback
* compare

===Server side and Client side===
In case you use the ''Client side'' validation option, you can mainly check for an empty or not input field. unless you write some ''Client side'' code which will probably be JavaScript functions to verify the data inside the input fields before it is submitted to the server. It could save some time if those functions are short, simple and quick to compute.
In case you need a more complex validation checks which relay on Moodle's internal PHP libraries (or other/external PHP libraries) you better use the ''Server side'' validation checks. Where you can query the DB, write complex PHP validation functions and much much more, that are not available (easily) when using JavaScript on the client's side.

==addHelpButton==

<code php>
$mform->addHelpButton('api_key_field', 'api_key', 'block_extsearch');
</code>

The following parameters are expected:

<code php>
/**
* @param $elementname The name of the form element to add the help button for
* @param $identifier The identifier for the help string and its title (see below)
* @param $component The component name to look for the help string in
*/
</code>

# get_string($identifier, $component) // The title of the help page
# get_string("{$identifier}_help", $component) // The content of the help page

So you will need to have '''$identifier''' and '''{$identifier}_help''' defined in order for the help button to be created properly. For example the multiple choice question editing form has a button for shuffling the answers.
<code php>
$mform->addHelpButton('shuffleanswers', 'shuffleanswers', 'qtype_multichoice');
</code>
and so the language file includes the strings
<code php>
$string['shuffleanswers'] = 'Shuffle the choices?';
$string['shuffleanswers_help'] = 'If enabled,.....';
</code>
You can also add the language string like
<code php>
$string['shuffleanswers_link'] = 'question/shuffleanswers';
</code>
to add a link to more help on Moodle docs. See [[String_API]] for more information about help icons.

==setDefault==

<pre>
$mform->addElement('select', 'grade', get_string('gradeforsubmission', 'exercise'), $grades);
$mform->setHelpButton('grade', array('grade', get_string('gradeforsubmission', 'exercise'), 'exercise'));
$mform->setDefault('grade', 100);
</pre>

Set the default of the form value with setDefault($elementname, $value); where elementname is the elementname whose default you want to set and $value is the default to set. We set the defaults for the form in definition(). This default is what is used if no data is loaded into the form with set_data(); eg. on display of the form for an 'add' rather than 'update' function.

<pre>
$mform->addElement('editor', 'desc', get_string('description'));
$mform->setDefault('desc', array('text'=>$defaulttext));
</pre>

Note that when setting the default for an editor element you must use an array to define the default "text" value as shown above.

==disabledIf==

For any element or groups of element in a form you can conditionally disable the group or individual element depending on conditions.

You can use $mform->disabledIf($elementName, $dependentOn, $condition = 'notchecked', $value='1')

* elementname can be a group. If you specify a group all elements in the group will be disabled (if dependentOn is in elementname group that is ignored and not disabled). These are the element names you've used as the second argument in addElement or addGroup.
* dependentOn is the actual name of the element as it will appear in html. This can be different to the name used in addGroup particularly but also addElement where you're adding a complex element like a date_selector. Check the html of your page. You typically make the depedentOn a checkbox or select box.
* $condition will be 'notchecked', 'checked', 'noitemselected', 'eq', 'in' or, if it is anything else, we test for 'neq'.
** If $condition is 'eq' or 'neq' then we check the value of the dependentOn field and check for equality (==) or nonequality (!=) in js
** If $condition is 'checked' or 'notchecked' then we check to see if a checkbox is checked or not.
** If $condition is 'in' then we check to see if a selected item is in the given list or not. (This was introduced in Moodle 2.7+)
** If $condition is 'noitemselected' then we check to see whether nothing is selected in a dropdown list.

Examples:
// Disable my control unless a checkbox is checked.
$mform->disabledIf('mycontrol', 'somecheckbox');

// Disable my control if a checkbox '''is''' checked.
$mform->disabledIf('mycontrol', 'somecheckbox', 'checked');

// Disable my control when a dropdown has value 42.
$mform->disabledIf('mycontrol', 'someselect', 'eq', 42);

// Disable my control unless a dropdown has value 42.
$mform->disabledIf('mycontrol', 'someselect', 'neq', 42);

The possible choices here are in the dependency manager in lib/form/form.js.
===A tricky case===

You need to take care with disabledIf if you plan to use it with groups of checkboxes.

Let's say you have a group of 5 checkboxes and you want to enable a depending item such as a drop down menu only when the first and the last checkboxes are selected.

To fix ideas:

If the selection in the checkboxes group is:

mycheck_01 == 1
mycheck_02 == 0
mycheck_03 == 0
mycheck_04 == 0
mycheck_05 == 1

the depending item must be enabled while ANY OTHER COMBINATION must disable the drop down menu.

The following code will, apparently, fail:
<code php>
$mform->disabledIf('dropdownmenu', 'mycheck_01', 'neq', '1');
$mform->disabledIf('dropdownmenu', 'mycheck_02', 'neq', '0');
$mform->disabledIf('dropdownmenu', 'mycheck_03', 'neq', '0');
$mform->disabledIf('dropdownmenu', 'mycheck_04', 'neq', '0');
$mform->disabledIf('dropdownmenu', 'mycheck_05', 'neq', '1');
</code>

In fact, once you get the drop down menu enabled, you are free to unselect mycheck_01 whilst still having the depending item enabled.
This apparent bug occurs because a non-checked checkbox behaves like a non existing mform element. So the js code will not find the element "mycheck_01" and will not apply the corresponding rule.

A working solution for this kind of issue seems to be:
<code php>
$mform->disabledIf('dropdownmenu', 'mycheck_01', 'notchecked');
$mform->disabledIf('dropdownmenu', 'mycheck_02', 'checked');
$mform->disabledIf('dropdownmenu', 'mycheck_03', 'checked');
$mform->disabledIf('dropdownmenu', 'mycheck_04', 'checked');
$mform->disabledIf('dropdownmenu', 'mycheck_05', 'notchecked');
</code>
To see a failing example as the one described, try the attachments provided in MDL-38975. See also in MDL-38975 for the working solution in action with modifications suggested by Eloy.

==hideIf==
{{Moodle 3.4}}
For any element or groups of element in a form you can conditionally hide the group or individual element depending on conditions.
This uses the same syntax as disabledIf just with hideIf instead.

==setType==

PARAM_* types are used to specify how a submitted variable should be cleaned. These should be used for get parameters such as id, course etc. which are used to load a page and also with setType(); method. Every form element should have a type specified except select, radio box and checkbox elements, these elements do a good job of cleaning themselves (only specified options are allowed as user input).

===Most Commonly Used PARAM_* Types===

These are the most commonly used PARAM_* types and their proper uses. More types can be seen in moodlelib.php starting around line 100.

* PARAM_CLEAN is deprecated and you should try to use a more specific type.
* PARAM_TEXT should be used for cleaning data that is expected to contain multi-lang content. It will strip all html tags. But will still let tags for multilang support through.
* PARAM_NOTAGS should be used for cleaning data that is expected to be plain text. It will strip *all* html type tags. It will *not* let tags for multilang support through. This should be used for instance for email addresses where no multilang support is appropriate.
* PARAM_RAW means no cleaning whatsoever, it is used mostly for data from the html editor. Data from the editor is later cleaned before display using format_text() function. PARAM_RAW can also be used for data that is validated by some other way or printed by p() or s().
* PARAM_INT should be used for integers. PARAM_FLOAT is also available for decimal numbers but is not recommended for user input since it does not work for languages that use , as a decimal separator.
* PARAM_ACTION is an alias of PARAM_ALPHA and is used for hidden fields specifying form actions.

==disable_form_change_checker==

By default, any Moodle form will pop-up an "Are you sure?" alert if you make some changes and then try to leave the page without saving. Occasionally, that is undesirable, in which case you can call

$mform->disable_form_change_checker()

==References==

* [http://www.midnighthax.com/quickform.php PEAR HTML QuickForm Getting Started Guide] by Keith Edmunds of Midnighthax.com
* [http://pear.php.net/manual/en/package.html.html-quickform.php PEAR::HTML_QuickForm manual]

[[Category:Formslib]]
[[Category:Interfaces]]

Web services

2020-04-02T21:31:10Z

Howardsmiller: /* Developer documentation */

=== How it works ===
This simple example will give you an idea of how our web services infrastructure works.
#The client sends a username and password to the web service login script.
#The script returns a token for that user account.
#The client calls a particular web service function on a protocol server including the token .
#The protocol server uses the token to check that the user can call the function.
#The protocol server calls the matching external function, located in a externallib.php file inside the relevant module.
#The external function checks that the current user has_capability to do this operation.
#The external function calls the matching Moodle core function (in lib.php usually).
#The core function can return a result to the external function.
#The external function will return a result to the protocol server.
#The protocol server returns the result to the client.

== Developer documentation==
The full API can be found on any Moodle sites under ''' Administration block > Plugins > Web services > API Documentation'''.

'''Note:''' Additional services are available for uploading and downloading files which are not in the API Documentation - they are accessed in a different way. See [[Web services files handling]]

* [[How to contribute a web service function to core]]
* [[Adding a web service to a plugin| Adding a web service to your plugin]]
* [[Creating a web service client | Implement a web service client]]
* [[Web services files handling]]
* [[Web_services_Roadmap | Web service Listing & Roadmap]]

== Specification and brainstorming ==
* [[External services security | External services security]]
* [[External services description | External services description]]

== See also ==
* [[Web_service_API_functions|Web service API functions]]
* [[:en:Web_services_FAQ|Web services FAQ]]
* [[:en:How_to_create_and_enable_a_web_service|How to create and enable a web service]]
* [[:en:Enable mobile web services|How to enable the mobile web service]]
* [[:en:Web_services|Web services user documentation]]
* [http://www.slideshare.net/juanleyva/mastering-moodle-web-services-development Mastering Moodle Web Services development] - Last session of the Hackfest in the MoodleMoot UK 2016

[[Category:Web_Services]]

Web services

2020-04-02T21:30:56Z

Howardsmiller: /* Developer documentation */

=== How it works ===
This simple example will give you an idea of how our web services infrastructure works.
#The client sends a username and password to the web service login script.
#The script returns a token for that user account.
#The client calls a particular web service function on a protocol server including the token .
#The protocol server uses the token to check that the user can call the function.
#The protocol server calls the matching external function, located in a externallib.php file inside the relevant module.
#The external function checks that the current user has_capability to do this operation.
#The external function calls the matching Moodle core function (in lib.php usually).
#The core function can return a result to the external function.
#The external function will return a result to the protocol server.
#The protocol server returns the result to the client.

== Developer documentation==
The full API can be found on any Moodle sites under ''' Administration block > Plugins > Web services > API Documentation'''.

'''Note''' Additional services are available for uploading and downloading files which are not in the API Documentation - they are accessed in a different way. See [[Web services files handling]]

* [[How to contribute a web service function to core]]
* [[Adding a web service to a plugin| Adding a web service to your plugin]]
* [[Creating a web service client | Implement a web service client]]
* [[Web services files handling]]
* [[Web_services_Roadmap | Web service Listing & Roadmap]]

== Specification and brainstorming ==
* [[External services security | External services security]]
* [[External services description | External services description]]

== See also ==
* [[Web_service_API_functions|Web service API functions]]
* [[:en:Web_services_FAQ|Web services FAQ]]
* [[:en:How_to_create_and_enable_a_web_service|How to create and enable a web service]]
* [[:en:Enable mobile web services|How to enable the mobile web service]]
* [[:en:Web_services|Web services user documentation]]
* [http://www.slideshare.net/juanleyva/mastering-moodle-web-services-development Mastering Moodle Web Services development] - Last session of the Hackfest in the MoodleMoot UK 2016

[[Category:Web_Services]]

Task API

2019-08-14T10:54:32Z

Howardsmiller: /* Adhoc task usage */

{{Infobox Project
|name = Task
|state = Integrated
|tracker = MDL-25505
|discussion = https://moodle.org/mod/forum/discuss.php?d=229139
|assignee = Damyon
}}
{{Moodle 2.7}}
== Tasks ==

A task is a unit of work that needs to be done later. Good uses for tasks:

* Run a slow operation in the background
* Run a maintenance task on a regular schedule

In general any operation that takes more than a few seconds should be a candidate for a task.

== Benefits ==

* Better user experience (give them feedback immediately, that their task has been queued)
* Prevent browser timeouts
* Better performance for clusters (tasks can be run on separate, non-webserving cluster node, tasks can run in parallel)
* Failed tasks will be retried
* A better user interface will prevent users queuing multiple tasks, because they thought it had "got stuck".

== Types of task ==

=== Scheduled tasks ===
Scheduled tasks are tasks that will run on a regular schedule. A default schedule can be set, but admins have the ability to change the default schedule if required. Note: Tasks will only run as often as cron is run in Moodle. In 2.7 it is recommended to run cron once per minute to get the benefit from the new task scheduling (don't worry - it will do much less work each time it runs).

=== Adhoc tasks ===
Adhoc tasks are for when you need to queue something to run in the background immediately, they would be executed as soon as possible. Adhoc tasks can contain custom data, specific to this specific instance of the task.

== Usage ==

=== Scheduled task usage ===

Scheduled tasks are created by subclassing \core\task\scheduled_task. They also require an entry in "db/tasks.php" for your plugin.

1. Create a subclass of \core\task\scheduled_task that contains your code to run in a schedule (within "<pluginroot>/classes/task/cut_my_toe_nails.php" for the autoloading mechanism to pick up).

<code php>
namespace mod_hygene\task;

/**
* An example of a scheduled task.
*/
class cut_my_toe_nails extends \core\task\scheduled_task {

/**
* Return the task's name as shown in admin screens.
*
* @return string
*/
public function get_name() {
return get_string('cutmytoenails', 'mod_hygene');
}

/**
* Execute the task.
*/
public function execute() {
// Apply fungus cream.
// Apply chainsaw.
// Apply olive oil.
}
}
</code>
2. Create entry in db/tasks.php for your plugin (then a version bump to your plugin will install the task):
<code php>
$tasks = [
[
'classname' => 'mod_hygene\task\cut_my_toe_nails',
'blocking' => 0,
'minute' => '30',
'hour' => '17',
'day' => '*',
'month' => '1,7',
'dayofweek' => '0',
],
];
</code>

The fields required in the db/tasks.php file are:
* classname - the fully namespaced classname of your task. This needs to comply with the autoloading rules.
* blocking - if this is set to 1, no other scheduled task will run at the same time as this task. Do not set this to 1 unless you really need it as it will impact the performance of the task queue.
* minute, hour, day, dayofweek, month - This is the default schedule for running the task. The syntax matches the syntax of unix cron. Starting with Moodle 2.8, the minute and hour values accept a special 'R' syntax which causes a random value to be set in the database at install time (useful to avoid overloading web services).

3. Increase the version number in version.php for your plugin and visit the Site "Administration -> Notifications" page to install the new task.

4. (Optional - since 2.8) Run this task even when the plugin is disabled. In rare cases, you may want the scheduled tasks for a plugin to run, even when the plugin is disabled. Some of the enrolment plugins do this to clean up data. If this is the case, the scheduled task must override the "get_run_if_component_disabled()" method and return true instead of false. If they do not do this, the scheduled task will not be run while the plugin is disabled.

When called from the command line for testing purposes errors can be hidden and a misleading error about locks can be displayed. To get at the details of an error Moodle 3.2 supports a showdebugging option that will show the actual errors. Read about it here
https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks

==== Cron syntax examples ====
* day - Day of month field for task schedule.
** "*" - Every day
** "*/2" - Every 2nd day
** "1" - The first of every month
** "1,15" - The first and fifteenth of every month
* dayofweek - Day of week field for task schedule.
** "*" - Every day
** "0" - Every Sunday
** "6" - Every Saturday
** "1,5" - Every Monday and Friday
* hour - Hour field for task schedule.
** "*" - Every hour
** "*/2" - Every 2 hours
** "2-10" - Every hour from 2am until 10am (inclusive)
** "2,6,9" - 2am, 6am and 9am
** "R" - A random value between 0 and 23 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* minute - Minute field for task schedule.
** "*" - Every minute
** "*/5" - Every 5 minutes
** "2-10" - Every minute between 2 and 10 minutes past the hour (inclusive)
** "2,6,9" - 2, 6 and 9 minutes past the hour
** "R" - A random value between 0 and 59 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* month - Month field for task schedule.
** "*" - Every month
** "*/2" - Every second month
** "1" - Every January
** "1,5" - Every January and May

=== Adhoc task usage ===

This is even easier than scheduled tasks.

1. Create a subclass of \core\task\adhoc_task that contains your code to run in the background.
<pre>

class take_over_the_world extends \core\task\adhoc_task {
public function execute() {
// gain 100,000,000 friends on facebook.
// crash the stock market.
// run for president.

// Get the custom data.
$data = $this->get_custom_data();
}
}

</pre>

2. Create an instance of the task and queue it (adding custom data if required).

<pre>
// create the instance
$domination = new take_over_the_world();
// set blocking if required (it probably isn't)
// $domination->set_blocking(true);
// add custom data
$domination->set_custom_data(array(
'plansfortomorrownight' => 'The same thing we do every night, Pinky!'
));

// queue it
\core\task\manager::queue_adhoc_task($domination);

// profit
</pre>

3. There is no 3.

By default, your task will run as admin. If you would like it to run as a specific user, call set_userid().

=== Failures ===
A task, either scheduled or adhoc can sometimes fail. An example would be updating an RSS field when the network is temporarily down. This is handled by the task system automatically - all the failing task needs to do is throw an exception. The task will be retried after 1 minute. If the task keeps failing, the retry algorithm will add more time between each successive attempts up to a max of 24 hours.

=== Caches ===
There is one special case that needs to be considered with this new system. If a particular scheduled or adhoc task runs for a long time and updates many DB records - particularly something related to enrolment - the next task in the queue may suffer because various API's in moodle use static caching to speed up requests, but assume that the data will not change much between the start and end of the request. In this case, you can force the cron to exit after running a task that has done many DB updates. The next cron will be run in the next minute, and will have all static caches cleared because it's a new process. To do this call \core\task\manager::clear_static_caches();

=== Security ===
When scheduling a task to run in the background - or creating a scheduled task, the task will run in the context of the cron user (see "cron_setup_user()"). If you need to perform access checks in your background task, you should pass the userid/context in custom_data and then pass that userid to the access check functions ("require_capability('moodle/course:update', $context, $userid)").

=== Legacy cron ===
The older syntax of cron.php or modname_cron() is still supported - but is not as good as this new API. This is because:
* the legacy cron functions run serially - a long running cron in one plugin will hold up the other plugins crons
* the legacy cron functions are fragile - a failure in one cron in one plugin will prevent the cron in other functions from running at all
* the scheduling cannot be changed by admins

Note that Legacy cron has been deprecated for Moodle 3.5 (MDL-52846) and will be deleted for Moodle 3.9 (MDL-61165).

=== Generating output ===
Since Moodle 3.5 it is safe to use the [[Output_API]] in cron tasks. Prior to this there may be cases where the Output API has not been initialised.
These issues were addressed in MDL-61800 and were also backported to Moodle 3.4.3 ,and Moodle 3.3.3.

== For Admins ==
Admins have a new screen where they can adjust the schedules for any scheduled task. They can also reset any scheduled task to its default schedule.
[[File:task_admin_screenshot.png|Screenshot of admin page]]

NOTE: You can also run [https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks Scheduled tasks] via command line, individually.

== Specification ==
The specification for this feature is here: https://docs.moodle.org/dev/Scheduled_Tasks_Proposal

Writing PHPUnit tests

2019-04-12T09:14:34Z

Howardsmiller: Fix broken link

{{Moodle 2.3}}

Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset).

=Testcase classes=

There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. '''Please note it is strongly recommended to put only one testcase into each class file.'''
;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial.
;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code.
;provider_testcase: Enhanced testcase class, enhanced for easy testing of [[Privacy API|Privacy Providers]].

There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes.

== Assertions ==

The complete list of assertions can be found in the [https://phpunit.readthedocs.io/en/7.0/assertions.html].

==Sample plugin testcase==

PHPUnit tests are located in <code>tests/*_test.php</code> files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends <code>advanced_testcase</code>:

<code php>
class mod_myplugin_sample_testcase extends advanced_testcase {
public function test_adding() {
$this->assertEquals(2, 1+2);
}
}
</code>

See [[PHPUnit integration#Class and file naming rules]] for more information.

==Inclusion of Moodle library files==

If you want to include some Moodle library files you should always declare '''global $CFG'''. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically.

==Automatic state reset==
By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected.

If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest().

<code php>
class mod_myplugin_testcase extends advanced_testcase {
public function test_deleting() {
global $DB;
$this->resetAfterTest(true);
$DB->delete_records('user');
$this->assertEmpty($DB->get_records('user'));
}
public function test_user_table_was_reset() {
global $DB;
$this->assertEquals(2, $DB->count_records('user', array()));
}
}
</code>

=Generators=

Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase.

Note if you are using PHPUnit [https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers @dataProvider] functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function.

==Creating users==
At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use:
<code php>
$user = $this->getDataGenerator()->create_user();
</code>

You may also specify properties of the user account, for example:
<code php>
$user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1'));
</code>

By default no user is logged-in, use setUser() method to change current $USER value:
<code php>
$this->setUser($user1);
</code>

Guest and admin accounts have a shortcut methods:
<code php>
$this->setGuestUser();
$this->setAdminUser();
</code>

Null can be used to set current user back to not-logged-in:
<code php>
$this->setUser(null);
</code>

==Creating course categories==

<code php>
$category1 = $this->getDataGenerator()->create_category();
$category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id));
</code>

==Creating courses==

<code php>
$course1 = $this->getDataGenerator()->create_course();

$category = $this->getDataGenerator()->create_category();
$course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id));
</code>

==Creating activities==

Some activity plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.

Example of creation of new course with one page resource:

<code php>
$course = $this->getDataGenerator()->create_course();
$generator = $this->getDataGenerator()->get_plugin_generator('mod_page');
$generator->create_instance(array('course'=>$course->id));
</code>

The following is functionally the same, but a bit shorter:
<code php>
$course = $this->getDataGenerator()->create_course();
$page = $this->getDataGenerator()->create_module('page', array('course' => $course->id));
</code>

==Creating cohorts==
{{Moodle 2.4}}
Since 2.4 there the data generator supports creation of new cohorts.

<code php>
$cohort = $this->getDataGenerator()->create_cohort();
</code>

==Simplified user enrolments==
{{Moodle 2.4}}
Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins.

<code php>
$this->getDataGenerator()->enrol_user($userid, $courseid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid);
$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual');
</code>

==Creating scales==

<code php>
$this->getDataGenerator()->create_scale();
$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat));
</code>

==Creating roles==

<code php>
$this->getDataGenerator()->create_role();
$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype));
</code>

==Creating tags==

<code php>
$this->getDataGenerator()->create_tag();
$this->getDataGenerator()->create_tag(array(
'userid' => $userid,
'rawname' => $rawname,
'name' => $name,
'description' => $description,
'descriptionformat' => $descriptionformat,
'flag' => $flag
));
</code>

==Groups==

===Creating groups===
<code php>
$this->getDataGenerator()->create_group(array('courseid' => $courseid));
$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>

===Adding users to groups===
<code php>
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid));
$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid));
</code>

===Creating groupings===
<code php>
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid));
$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat));
</code>

===Adding groups to groupings===
<code php>
$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid));
</code>

==Repositories==

===Creating repository instances===
{{Moodle 2.5}}
Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<code php>
$this->getDataGenerator()->create_repository($type, $record, $options);
</code>

===Creating repository types===
{{Moodle 2.5}}
Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php..

<code php>
$this->getDataGenerator()->create_repository_type($type, $record, $options);
</code>

==Creating grades==

===Grade categories===

<code php>
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid));
$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname));
</code>

===Grade items===

<code php>
$this->getDataGenerator()->create_grade_item();
$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype));
</code>

===Outcomes===

<code php>
$this->getDataGenerator()->create_grade_outcome();
$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname));
</code>

==Other types of plugin==
{{Moodle 2.5}}
Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname);

For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith.

=Long tests=

All standard test should execute as fast as possible. Tests that take a loner time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php.

=Large test data=
See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [[PHPUnit_integration#Extra_methods]]

=Testing sending of messages=
{{Moodle 2.4}}
You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages.

To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later.
<code php>
$this->preventResetByRollback();
$sink = $this->redirectMessages();
//... code that is sending messages
$messages = $sink->get_messages();
$this->assertEquals(3, count($messages));
//.. test messages were generated in correct order with appropriate content
</code>

=Testing sending of emails=
{{Moodle 2.6}}
You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails.

To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later.
<code php>
unset_config('noemailever');
$sink = $this->redirectEmails();
//... code that is sending email
$messages = $sink->get_messages();
$this->assertEquals(1, count($messages));
</code>

=Logstores=
You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute.
<code php>
$this->preventResetByRollback();
set_config('enabled_stores', 'logstore_standard', 'tool_log');
set_config('buffersize', 0, 'logstore_standard');
</code>

=Best practice=

There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below.

==Keep use of resetAfterTest to a minimum==
Although many of the examples described above use the <code>resetAfterTest</code> nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to.
Generally speaking you should aim to write code which is mockable, and does not require real fixtures.
Use of resetAfterTest will also slow your tests down.

==Be careful with shared setUp and instance variables==

You should be careful of how you create and use instance variables in PHPUnit tests for two main reasons:

Firstly, if you create any fixtures in the setUp, or call the resetAfterTest function, these fixtures and conditions will apply for _all_ tests in the testsuite.
You will not be able to add another test to the suite which does not require these conditions without those conditions being fulfilled anyway.
This can lead to slow tests.

Secondly, because of the way in which PHPUnit operates. it creates an instance of each testcase during its bootstrap phase. These are stored in memory until the _entire suite_ completes.
This means that any fixture which is setup and not actively discarded will not be garbage collected and lead to memory bloat.
In severe cases this can lead to memory exhaustion.

==Make use of the dataProvider functionality==

The dataProvider functionality of PHPUnit is an extremely powerful and useful feature which allows you to verify a function quickly and easily with a range of different conditions.
However, the following rules should be followed when using dataProviders:
* Keep addition of resettable data requring resetAfterTest to a minimum - this will lead to many slow tests
* You can only _describe_ data in a dataProvider, you cannot create the data. The dataProvider is called after the testSuite is instantiated, but before any tests are run. Each test will run a full setUp and tearDown, which will destroy any data which was created.
=Extra test settings=

Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php.

There are several ways how to inject your custom settings:
* define test setting constants in your phpunit.xml file
* define test setting constants in your config.php

These constants may be then used in your test or plugin code.

=Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5)=
{{Moodle 3.7}}

With Moodle 3.7, '''PHPUnit was upgraded to 7.5''' (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see MDL-65204 and linked issues for more details). While internally [https://phpunit.de/announcements/phpunit-7.html a lot of things changed with PHPUnit 7] (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources:
* [https://phpunit.de/announcements/phpunit-7.html PHPUnit 7 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/7.5/ChangeLog-7.5.md Detailed changelog of the release], paying special attention to the added, deprecated, changed and deleted sections.
* [https://phpunit.readthedocs.io/en/7.5/ Official PHPUnit manual].
* Changes performed into core, mainly: new, stricter, signatures ([https://github.com/moodle/moodle/commit/26218b7 26218b7]) and assertEquals() changes, specially important when comparing strings, now performed using strict (===) equals ([https://github.com/moodle/moodle/commit/85f47ba 85f47ba]).

=Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6)=
{{Moodle 3.4}}

With Moodle 3.4, '''PHPUnit was upgraded to 6.4''' (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see MDL-60611 and linked issues for more details). While internally [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues a lot of things changed with PHPUnit 6] (namespaced classes being the more noticeable), thanks to our '''wrapping layer''' (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve.

Still, in '''some cases, it will impossible to maintain compatibility''' of tests between old (pre 3.4) tests and new ones, especially '''when direct use of any phpunit class is performed'''. Luckily, both travis and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly.

To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources:
* [https://thephp.cc/news/2017/02/migrating-to-phpunit-6 A very good mini-guide] showing all the important changes.
* [https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues PHPUnit 6 release Announcement].
* [https://github.com/sebastianbergmann/phpunit/blob/6.0/ChangeLog-6.0.md#600---2017-02-03 Detailed changelog of the release], paying special attention to the changed and deleted sections.
* Changes performed into core, mainly: namespace class renaming ([https://github.com/moodle/moodle/commit/801a372dadb6e11c8781547603e3f0a59ce5638f 801a372]) and deprecated stuff ([https://github.com/moodle/moodle/commit/796e48a58bf18533bdca423fff7949ab119101c4 796e48a])

=See also=
* [[PHPUnit integration]]
* [[PHPUnit]]

[[Category:Unit testing]]

AMD Modal

2019-03-19T14:07:26Z

Howardsmiller: /* How to manually trigger a Modal */

{{Moodle 3.2}}

This is currently (2017) the recommended way to create various sorts of pop-up dialogue boxes in JavaScript.

= Why should you use it? =
The AMD modal modules provide a simple interface for creating a modal within Moodle. The module will ensure all accessibility requirements are met including applying the correct aria roles, focus control, aria hiding background elements and locking keyboard navigation.

The modals will fire events for common actions that occur within the modal, e.g. show / hide, for other code to listen to and react accordingly.

Moodle ships with a couple of standard modal types for you to re-use including a simple cancel modal, a confirm (yes/no) modal and a save/cancel modal. Hopefully with more to come!

= How do you create a basic modal? =
If you'd simply like to display a modal with some simple content then all you'll need is the modal factory. The factory provides a create function that accepts some configuration for your modal that the factory will use to create the modal and optionally a trigger element (the element that will open the modal when activated). The create function will return a promise that is resolved with the created modal.

The configuration is provided as an object with key/value pairs. The options are:
{| class="nicetable"
|-
! '''key'''
! '''description'''
|-
| title
| the title to display in the modal header - note: this will render HTML
|-
| body
| the main content to be rendered in the modal body
|-
| footer
| the content to be rendered in the modal footer
|-
| type
| one of the modal types registered with the factory
|-
| large
| a boolean to indicate if the modal should be wider than the default size
|}

Example 1
<code javascript>
define(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

The modals will also accept a promise for the body and footer content. It is expected that the promise will be resolved with two strings, the html content and any associated javascript. This allows the modal module to work natively with the templates module, such as in the example below.

Example 2
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
// Can include JS which is run when modal is attached to DOM.
body: Templates.render('core/modal_test_3', {}),
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Since the modals aren't actually added to the DOM until they are made visible any javascript resolved with the promise is cached and only run after all of the elements are added to the DOM, so you don't have to worry about any special handling in the javascript that is being loaded from the template.

= How to manually trigger a Modal =

If your trigger is (say) a class that covers a number of actual elements, you may want to grab the actual element that caused the event. Unfortunately, ModalEvents do not capture this and you need to approach the modal a different way. Here is an example - the scenario is a list of items with delete buttons. You need to know which delete button in order to get the item id. Note that code is somewhat simplified for clarity.

<code javascript>
$('a.item-delete').on('click', function(e) {
var clickedLink = $(e.currentTarget);
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Delete item',
body: 'Do you really want to delete?',
})
.then(function(modal) {
modal.setSaveButtonText('Delete');
var root = modal.getRoot();
root.on(ModalEvents.save, function() {
var elementid = clickedLink.data('id');
// Do something to delete item
});
modal.show();
});
</code>

'Normal' javascript is used to trap the event and the required element can be captured. No 'trigger' is passed to the ModalFactory so the resulting modal cannot display itself. The modal save event (in this case clicking the delete button) is handled as normally and will still trigger as expected. 'modal.show()' is called to display the modal.

= How do you create a different type of modal? =
Moodle comes with a few specialised types of modals for common use cases. Look at lib/amd/src/modal_factory.js to see what is available. These can be created using the factory, similar to the examples above, by specifying the type of modal in the configuration provided to the create function. Each of the modal types may have different configuration options, for example the save/cancel modal doesn't allow you to set the footer content.

Example 3
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Each type of modal may fire additional events to allow your code to handle the new functionality being offered. See the modal_events.js module for a list of events that can be fired. For example, if you wanted to have a save/cancel modal that you did some form validation on before saving you could do something like the example below.

Example 4
<code javascript>
define(['jquery', 'core/modal_factory', 'core/modal_events', 'core/templates'],
function($, ModalFactory, ModalEvents, Templates) {

var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
modal.getRoot().on(ModalEvents.save, function(e) {
// Stop the default save button behaviour which is to close the modal.
e.preventDefault();
// Do your form validation here.
});
});
});
</code>

= How do you write a new type of modal? =
If you'd like to write a new type of modal to be re-used throughout your code you can extend the default modal implementation and make any customisations you require. In order to create a new modal type you'll need to create a new AMD module and import the core/modal module to extend. You can also optionally create a new modal template that builds upon the core/modal template. Finally, you can register your new type with the modal registry to allow you to create your new type using the modal factory.

For example, let's create a modal that opens a login form:
First we can create the HTML for out modal by including and extending the core modal template. All we need to do is override the block defined in that template with our new title, body and footer content.

Let's assume the file is <your_module>/templates/modal_login.mustache

<code>
{{< core/modal }}
{{$title}}{{#str}} login {{/str}}{{/title}}
{{$body}}
<div class="container">
<form>
<div class="form-group row">
<label for="inputEmail" class="col-sm-2 col-form-label">{{#str}} email {{/str}}</label>
<div class="col-sm-10">
<input type="email" class="form-control" id="inputEmail" placeholder="{{#str}} email {{/str}}">
</div>
</div>
<div class="form-group row">
<label for="inputPassword" class="col-sm-2 col-form-label">{{#str}} password {{/str}}</label>
<div class="col-sm-10">
<input type="password" class="form-control" id="inputPassword" placeholder="{{#str}} password {{/str}}">
</div>
</div>
</form>
</div>
{{/body}}
{{$footer}}
<button type="button" class="btn btn-primary" data-action="login">{{#str}} login {{/str}}</button>
<button type="button" class="btn btn-secondary" data-action="cancel">{{#str}} cancel {{/str}}</button>
{{/footer}}
{{/ core/modal }}
</code>

Next we can create the new AMD module for the login modal. You can extend the existing modal module which will give you all of the core modal functionality for free allowing you to focus on writing only the specific logic you need. In this example we would only need to write the logic for handling the login.

Let's assume the file is <your_module>/amd/src/modal_login.js

<code javascript>
define(['jquery', 'core/notification', 'core/custom_interaction_events', 'core/modal', 'core/modal_registry'],
function($, Notification, CustomEvents, Modal, ModalRegistry) {

var registered = false;
var SELECTORS = {
LOGIN_BUTTON: '[data-action="login"]',
CANCEL_BUTTON: '[data-action="cancel"]',
};

/**
* Constructor for the Modal.
*
* @param {object} root The root jQuery element for the modal
*/
var ModalLogin = function(root) {
Modal.call(this, root);

if (!this.getFooter().find(SELECTORS.LOGIN_BUTTON).length) {
Notification.exception({message: 'No login button found'});
}

if (!this.getFooter().find(SELECTORS.CANCEL_BUTTON).length) {
Notification.exception({message: 'No cancel button found'});
}
};

ModalLogin.TYPE = 'your_module-login';
ModalLogin.prototype = Object.create(Modal.prototype);
ModalLogin.prototype.constructor = ModalLogin;

/**
* Set up all of the event handling for the modal.
*
* @method registerEventListeners
*/
ModalLogin.prototype.registerEventListeners = function() {
// Apply parent event listeners.
Modal.prototype.registerEventListeners.call(this);

this.getModal().on(CustomEvents.events.activate, SELECTORS.LOGIN_BUTTON, function(e, data) {
// Add your logic for when the login button is clicked. This could include the form validation,
// loading animations, error handling etc.
}.bind(this));

this.getModal().on(CustomEvents.events.activate, SELECTORS.CANCEL_BUTTON, function(e, data) {
// Add your logic for when the cancel button is clicked.
}.bind(this));
};

// Automatically register with the modal registry the first time this module is imported so that you can create modals
// of this type using the modal factory.
if (!registered) {
ModalRegistry.register(ModalLogin.TYPE, ModalLogin, '<your_module>/modal_login');
registered = true;
}

return ModalLogin;
});
</code>

Once you have both your template and javascript ready to go then all you need to do is tie them into where ever you'd like to launch the modal. For the purpose of this example let's assume that we've got a page with a login button with id="login" then you might add this javascript to the page.

<code javascript>
define(['jquery', 'core/templates', 'core/modal_factory', '<your_module>/modal_login'], function($, Templates, ModalFactory, ModalLogin) {
var trigger = $('#login');

ModalFactory.create({type: ModalLogin.TYPE}, trigger);
});
</code>

= Related =
* AMD and [[Javascript Modules]]

[[Category:Javascript]]

AMD Modal

2019-03-19T09:57:09Z

Howardsmiller: /* How do you create a basic modal? */

{{Moodle 3.2}}

This is currently (2017) the recommended way to create various sorts of pop-up dialogue boxes in JavaScript.

= Why should you use it? =
The AMD modal modules provide a simple interface for creating a modal within Moodle. The module will ensure all accessibility requirements are met including applying the correct aria roles, focus control, aria hiding background elements and locking keyboard navigation.

The modals will fire events for common actions that occur within the modal, e.g. show / hide, for other code to listen to and react accordingly.

Moodle ships with a couple of standard modal types for you to re-use including a simple cancel modal, a confirm (yes/no) modal and a save/cancel modal. Hopefully with more to come!

= How do you create a basic modal? =
If you'd simply like to display a modal with some simple content then all you'll need is the modal factory. The factory provides a create function that accepts some configuration for your modal that the factory will use to create the modal and optionally a trigger element (the element that will open the modal when activated). The create function will return a promise that is resolved with the created modal.

The configuration is provided as an object with key/value pairs. The options are:
{| class="nicetable"
|-
! '''key'''
! '''description'''
|-
| title
| the title to display in the modal header - note: this will render HTML
|-
| body
| the main content to be rendered in the modal body
|-
| footer
| the content to be rendered in the modal footer
|-
| type
| one of the modal types registered with the factory
|-
| large
| a boolean to indicate if the modal should be wider than the default size
|}

Example 1
<code javascript>
define(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

The modals will also accept a promise for the body and footer content. It is expected that the promise will be resolved with two strings, the html content and any associated javascript. This allows the modal module to work natively with the templates module, such as in the example below.

Example 2
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
// Can include JS which is run when modal is attached to DOM.
body: Templates.render('core/modal_test_3', {}),
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Since the modals aren't actually added to the DOM until they are made visible any javascript resolved with the promise is cached and only run after all of the elements are added to the DOM, so you don't have to worry about any special handling in the javascript that is being loaded from the template.

= How to manually trigger a Modal =

If your trigger is (say) a class that covers a number of actual elements, you may want to grab the actual element that caused the event. Unfortunately, ModalEvents do not capture this and you need to approach the modal a different way. Here is an example - the scenario is a list of items with delete buttons. You need to know which delete button in order to get the item id. Note that code is somewhat simplified for clarity.

<code javascript>
$('a.item-delete').on('click', function(e) {
var clickedLink = $(e.currentTarget);
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Delete item',
body: 'Do you really want to delete?',
})
.then(function(modal) {
modal.setSaveButtonText('Delete');
var root = modal.getRoot();
root.on(ModalEvents.save, function() {
var elementid = clickedLink.data('templateid');
// Do something to delete item
});
modal.show();
});
</code>

'Normal' javascript is used to trap the event and the required element can be captured. No 'trigger' is passed to the ModalFactory so the resulting modal cannot display itself. The modal save event (in this case clicking the delete button) is handled as normally and will still trigger as expected. 'modal.show()' is called to display the modal.

= How do you create a different type of modal? =
Moodle comes with a few specialised types of modals for common use cases. Look at lib/amd/src/modal_factory.js to see what is available. These can be created using the factory, similar to the examples above, by specifying the type of modal in the configuration provided to the create function. Each of the modal types may have different configuration options, for example the save/cancel modal doesn't allow you to set the footer content.

Example 3
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Each type of modal may fire additional events to allow your code to handle the new functionality being offered. See the modal_events.js module for a list of events that can be fired. For example, if you wanted to have a save/cancel modal that you did some form validation on before saving you could do something like the example below.

Example 4
<code javascript>
define(['jquery', 'core/modal_factory', 'core/modal_events', 'core/templates'],
function($, ModalFactory, ModalEvents, Templates) {

var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
modal.getRoot().on(ModalEvents.save, function(e) {
// Stop the default save button behaviour which is to close the modal.
e.preventDefault();
// Do your form validation here.
});
});
});
</code>

= How do you write a new type of modal? =
If you'd like to write a new type of modal to be re-used throughout your code you can extend the default modal implementation and make any customisations you require. In order to create a new modal type you'll need to create a new AMD module and import the core/modal module to extend. You can also optionally create a new modal template that builds upon the core/modal template. Finally, you can register your new type with the modal registry to allow you to create your new type using the modal factory.

For example, let's create a modal that opens a login form:
First we can create the HTML for out modal by including and extending the core modal template. All we need to do is override the block defined in that template with our new title, body and footer content.

Let's assume the file is <your_module>/templates/modal_login.mustache

<code>
{{< core/modal }}
{{$title}}{{#str}} login {{/str}}{{/title}}
{{$body}}
<div class="container">
<form>
<div class="form-group row">
<label for="inputEmail" class="col-sm-2 col-form-label">{{#str}} email {{/str}}</label>
<div class="col-sm-10">
<input type="email" class="form-control" id="inputEmail" placeholder="{{#str}} email {{/str}}">
</div>
</div>
<div class="form-group row">
<label for="inputPassword" class="col-sm-2 col-form-label">{{#str}} password {{/str}}</label>
<div class="col-sm-10">
<input type="password" class="form-control" id="inputPassword" placeholder="{{#str}} password {{/str}}">
</div>
</div>
</form>
</div>
{{/body}}
{{$footer}}
<button type="button" class="btn btn-primary" data-action="login">{{#str}} login {{/str}}</button>
<button type="button" class="btn btn-secondary" data-action="cancel">{{#str}} cancel {{/str}}</button>
{{/footer}}
{{/ core/modal }}
</code>

Next we can create the new AMD module for the login modal. You can extend the existing modal module which will give you all of the core modal functionality for free allowing you to focus on writing only the specific logic you need. In this example we would only need to write the logic for handling the login.

Let's assume the file is <your_module>/amd/src/modal_login.js

<code javascript>
define(['jquery', 'core/notification', 'core/custom_interaction_events', 'core/modal', 'core/modal_registry'],
function($, Notification, CustomEvents, Modal, ModalRegistry) {

var registered = false;
var SELECTORS = {
LOGIN_BUTTON: '[data-action="login"]',
CANCEL_BUTTON: '[data-action="cancel"]',
};

/**
* Constructor for the Modal.
*
* @param {object} root The root jQuery element for the modal
*/
var ModalLogin = function(root) {
Modal.call(this, root);

if (!this.getFooter().find(SELECTORS.LOGIN_BUTTON).length) {
Notification.exception({message: 'No login button found'});
}

if (!this.getFooter().find(SELECTORS.CANCEL_BUTTON).length) {
Notification.exception({message: 'No cancel button found'});
}
};

ModalLogin.TYPE = 'your_module-login';
ModalLogin.prototype = Object.create(Modal.prototype);
ModalLogin.prototype.constructor = ModalLogin;

/**
* Set up all of the event handling for the modal.
*
* @method registerEventListeners
*/
ModalLogin.prototype.registerEventListeners = function() {
// Apply parent event listeners.
Modal.prototype.registerEventListeners.call(this);

this.getModal().on(CustomEvents.events.activate, SELECTORS.LOGIN_BUTTON, function(e, data) {
// Add your logic for when the login button is clicked. This could include the form validation,
// loading animations, error handling etc.
}.bind(this));

this.getModal().on(CustomEvents.events.activate, SELECTORS.CANCEL_BUTTON, function(e, data) {
// Add your logic for when the cancel button is clicked.
}.bind(this));
};

// Automatically register with the modal registry the first time this module is imported so that you can create modals
// of this type using the modal factory.
if (!registered) {
ModalRegistry.register(ModalLogin.TYPE, ModalLogin, '<your_module>/modal_login');
registered = true;
}

return ModalLogin;
});
</code>

Once you have both your template and javascript ready to go then all you need to do is tie them into where ever you'd like to launch the modal. For the purpose of this example let's assume that we've got a page with a login button with id="login" then you might add this javascript to the page.

<code javascript>
define(['jquery', 'core/templates', 'core/modal_factory', '<your_module>/modal_login'], function($, Templates, ModalFactory, ModalLogin) {
var trigger = $('#login');

ModalFactory.create({type: ModalLogin.TYPE}, trigger);
});
</code>

= Related =
* AMD and [[Javascript Modules]]

[[Category:Javascript]]

Useful core Javascript modules

2019-03-16T12:10:37Z

Howardsmiller: /* Language strings (core/str) */

{{Moodle 2.9}}

The source code for all these modules (and a number of others) can be found in lib/amd/src.

== Configuration settings (core/config) ==

Example of using config module:
<code javascript>
require(['core/config’], function(mdlcfg) {
console.log(mdlcfg.wwwroot); // outputs the wwwroot of moodle to console
});
</code>

== Language strings (core/str) ==

Example of using language strings module (retrieved via ajax, if the string is not yet loaded)
<code javascript>
require([‘core/str’], function(str) {
// start retrieving the localized string; store the promise that some time in the future the string will be there.
var editaPresent = str.get_string('edita', 'core', stringargument);
// as soon as the string is retrieved, i.e. the promise has been fulfilled,
// edit the text of a UI element so that it then is the localized string
// Note: $.when can be used with an arbitrary number of promised things
$.when(editaPresent).done(function(localizedEditString) {
$("someUIElementSelector").text = localizedEditString;
});
});
</code>

You can also convert multiple strings and add parameters in a single call
<code javascript>
// Real example from user/amd/src/status_field.js
var strings = [
{
key: 'unenrol',
component: 'enrol'
},
{
key: 'unenrolconfirm',
component: 'enrol',
param: {
user: parentContainer.data('fullname'),
course: parentContainer.data('coursename'),
enrolinstancename: parentContainer.data('enrolinstancename')
}
}
];
str.get_strings(strings).then(function (results) {
console.log(results);
});
</code>

The string will be retrieved via AJAX request on the first use and cached in browser local storage until purge caches or site upgrade

IT IS NOT RECOMMENDED to call:
<code php>
$PAGE->requires->string_for_js('edita', 'core'); // You do not need this!
</code>

Because:
* The list of strings used now needs to be maintained in 2 places
* The strings are always sent, and bloat the page size even if they are not used
* All strings fetched via the AJAX method above are cached in browser local storage anyway, so these strings will never be used 99% of the time

==Notifications (core/notification)==

Examples:
<code javascript>
require(['core/notification’], function(notification) {
notification.alert('Hello', 'Welcome to my site!', 'Continue');
});
</code>

Strings and notifications:
<code javascript>
require(['core/str', 'core/notification’], function(str, notification) {
str.get_strings([
{'key' : 'delete'},
{'key' : 'confirmdeletetag', component : 'tag'},
{'key' : 'yes'},
{'key' : 'no'},
]).done(function(s) {
notification.confirm(s[0], s[1], s[2], s[3], function() {
window.location.href = href;
});
}
).fail(notification.exception);
});
</code>

== URL module (core/url) ==

Useful Functions:
<code javascript>
// Generate an absolute URL by using the relative path to a file and optionally slash arguments
url.fileUrl(relativeScript, slashArgs)
// Generate an absolute URL from the given relative path, include the URL parameters and possibly the current session key
url.relativeUrl(relativePath, params, includeSessionKey)
// Generates an image url using the filename of the image and the Moodle component (core, plugin, etc.) where it can be found
url.imageUrl(imagename, component)
</code>

Example:

<code javascript>
// Prerequisites:
// - A Javascript file is present under $CFG->wwwroot."/path/to/file.js"
require(['core/url'], function(url) {
url.fileUrl("/path/to/file.js", "");
console.log("Generated URL: " + url);
});
</code>

== Ajax Module (core/ajax) ==

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. This saves having to worry about security in ajax functionality because it's all done for you.

For the full story, see [[AJAX]]

== Tree (core/tree) ==
{{Moodle 3.1}}
If you want to add a hierarchical interface, you should use the tree module. It will handle user interaction and accessibility for you.

<code javascript>
define(['jquery', 'core/tree'], function($, Tree) {
return {
init: function() {
new Tree("css/jquery selector of the tree container");
}
};
});
</code>

Read more about the [[Tree]] module

== Modal (core/modal) ==
{{Moodle 3.2}}

If you'd like to add a modal to your page the modal modules will handle all of the magic for you, all you need to bring is your content!
<code javascript>
require(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.

// Maybe... add a class to the modal dialog, to be able to style it.
modal.getRoot().addClass('mydialog');
});
});
</code>

Read more about the [[AMD_Modal]] module

== Paged content (core/paged_content_factory) ==
{{Moodle 3.6}}

If you need to create a paged content region to asynchronously load the pages as the user requests them then you can use the paged content modules.

Read more about the [[AMD_paged_content]] module

== PubSub (core/pubsub) ==
{{Moodle 3.6}}

A simple module to do event subscription and publishing in JavaScript without the need for jQuery.

Read more about the [[AMD_pubsub]] module

==See also==
* [[Javascript Modules]]
* [[Fragment]]

[[Category:AJAX]]
[[Category:Javascript]]

Useful core Javascript modules

2019-03-16T12:10:22Z

Howardsmiller: /* Language strings (core/str) */

{{Moodle 2.9}}

The source code for all these modules (and a number of others) can be found in lib/amd/src.

== Configuration settings (core/config) ==

Example of using config module:
<code javascript>
require(['core/config’], function(mdlcfg) {
console.log(mdlcfg.wwwroot); // outputs the wwwroot of moodle to console
});
</code>

== Language strings (core/str) ==

Example of using language strings module (retrieved via ajax, if the string is not yet loaded)
<code javascript>
require([‘core/str’], function(str) {
// start retrieving the localized string; store the promise that some time in the future the string will be there.
var editaPresent = str.get_string('edita', 'core', stringargument);
// as soon as the string is retrieved, i.e. the promise has been fulfilled,
// edit the text of a UI element so that it then is the localized string
// Note: $.when can be used with an arbitrary number of promised things
$.when(editaPresent).done(function(localizedEditString) {
$("someUIElementSelector").text = localizedEditString;
});
});
</code>

You can also convert multiple strings and add parameters in a single call
<code javascript>
// Real example from user/amd/src/status_field.js
var strings = [
{
key: 'unenrol',
component: 'enrol'
},
{
key: 'unenrolconfirm',
component: 'enrol',
param: {
user: parentContainer.data('fullname'),
course: parentContainer.data('coursename'),
enrolinstancename: parentContainer.data('enrolinstancename')
}
}
];
str.get_strings(strings).then(function (results) {
console.log(results);
});
/code>

The string will be retrieved via AJAX request on the first use and cached in browser local storage until purge caches or site upgrade

IT IS NOT RECOMMENDED to call:
<code php>
$PAGE->requires->string_for_js('edita', 'core'); // You do not need this!
</code>

Because:
* The list of strings used now needs to be maintained in 2 places
* The strings are always sent, and bloat the page size even if they are not used
* All strings fetched via the AJAX method above are cached in browser local storage anyway, so these strings will never be used 99% of the time

==Notifications (core/notification)==

Examples:
<code javascript>
require(['core/notification’], function(notification) {
notification.alert('Hello', 'Welcome to my site!', 'Continue');
});
</code>

Strings and notifications:
<code javascript>
require(['core/str', 'core/notification’], function(str, notification) {
str.get_strings([
{'key' : 'delete'},
{'key' : 'confirmdeletetag', component : 'tag'},
{'key' : 'yes'},
{'key' : 'no'},
]).done(function(s) {
notification.confirm(s[0], s[1], s[2], s[3], function() {
window.location.href = href;
});
}
).fail(notification.exception);
});
</code>

== URL module (core/url) ==

Useful Functions:
<code javascript>
// Generate an absolute URL by using the relative path to a file and optionally slash arguments
url.fileUrl(relativeScript, slashArgs)
// Generate an absolute URL from the given relative path, include the URL parameters and possibly the current session key
url.relativeUrl(relativePath, params, includeSessionKey)
// Generates an image url using the filename of the image and the Moodle component (core, plugin, etc.) where it can be found
url.imageUrl(imagename, component)
</code>

Example:

<code javascript>
// Prerequisites:
// - A Javascript file is present under $CFG->wwwroot."/path/to/file.js"
require(['core/url'], function(url) {
url.fileUrl("/path/to/file.js", "");
console.log("Generated URL: " + url);
});
</code>

== Ajax Module (core/ajax) ==

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. This saves having to worry about security in ajax functionality because it's all done for you.

For the full story, see [[AJAX]]

== Tree (core/tree) ==
{{Moodle 3.1}}
If you want to add a hierarchical interface, you should use the tree module. It will handle user interaction and accessibility for you.

<code javascript>
define(['jquery', 'core/tree'], function($, Tree) {
return {
init: function() {
new Tree("css/jquery selector of the tree container");
}
};
});
</code>

Read more about the [[Tree]] module

== Modal (core/modal) ==
{{Moodle 3.2}}

If you'd like to add a modal to your page the modal modules will handle all of the magic for you, all you need to bring is your content!
<code javascript>
require(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.

// Maybe... add a class to the modal dialog, to be able to style it.
modal.getRoot().addClass('mydialog');
});
});
</code>

Read more about the [[AMD_Modal]] module

== Paged content (core/paged_content_factory) ==
{{Moodle 3.6}}

If you need to create a paged content region to asynchronously load the pages as the user requests them then you can use the paged content modules.

Read more about the [[AMD_paged_content]] module

== PubSub (core/pubsub) ==
{{Moodle 3.6}}

A simple module to do event subscription and publishing in JavaScript without the need for jQuery.

Read more about the [[AMD_pubsub]] module

==See also==
* [[Javascript Modules]]
* [[Fragment]]

[[Category:AJAX]]
[[Category:Javascript]]

Adding a web service to a plugin

2019-02-01T14:40:21Z

Howardsmiller: /* Declare the web service function */

{{Moodle_2.0}}

= Quick start =
== Example ==
Have a look to the [https://github.com/moodlehq/moodle-local_wstemplate web service plugin template]. This plugin contains a web service hello_world function. To make testing easy for you, the plugin is distributed with a test client in the folder /client.
== File structure ==
The file structure is explained in these two documents:
* [[Web_services_API|Web services API]]
* [[External_functions_API|External function API]]

= Tutorial =
We will create a web service into a local plugin. This service will contain one ''local_myplugin_create_groups($groups)'' web service function. This web service function will create a group into a Moodle course.

== Write the specification documentation ==
Before starting coding, let's identify our needs writing some short specification documents.

=== functional specification===
''local_myplugin_create_groups($groups)'' will take a list of groups as parameters and it will return the same groups with their newly created id. If ever one group creation fails, the function will throw an exception, and no creation will happen.

=== technical specification ===
* '''the core function the external function will call''': ''groups_create_group()'' from [http://cvs.moodle.org/moodle/group/ /group/lib.php].
* '''the parameter types''': a list of object. This object are groups, with id/name/courseid.
* '''the returned value types''': a list of objects (groups) with their id.
* '''the user capabilities to check''': ''moodle/course:managegroups''

== Write a simple test client ==
The first thing you should code is a web service test client. You will often discover use cases that you didn't think about. We are not showing any test client code here, see [[Creating_a_web_service_client|How to create a web service client]].

== Declare the service ==
This step is optional. You can pre-build a service including any web service functions, so the Moodle administrator doesn't need to do it. Add into /local/myplugin/db/services.php:

<code php>
$services = array(
'mypluginservice' => array( //the name of the web service
'functions' => array ('local_myplugin_create_groups'), //web service functions of this service
'requiredcapability' => '', //if set, the web service user need this capability to access
//any function of this service. For example: 'some/capability:specified'
'restrictedusers' =>0, //if enabled, the Moodle administrator must link some user to this service
//into the administration
'enabled'=>1, //if enabled, the service can be reachable on a default installation
)
);
</code>

Note: it is not possible for an administrator to add/remove any function from a pre-built service.

== Declare the web service function ==
Following the [[Web_services_API|Web service API]], you must declare the web service function in the ''local/myplugin/db/services.php'' file.

<code php>
$functions = array(
'local_myplugin_create_groups' => array( //web service function name
'classname' => 'local_myplugin_external', //class containing the external function
'methodname' => 'create_groups', //external function name
'classpath' => 'local/myplugin/externallib.php', //file containing the class/external function
'description' => 'Creates new groups.', //human readable description of the web service function
'type' => 'write', //database rights of the web service function (read, write)
'ajax' => true, // is the service available to 'internal' ajax calls.
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE) // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
),
);
</code>

Web service functions should match the [https://docs.moodle.org/dev/Web_services_Roadmap#Naming_convention naming convention].

== Write the external function descriptions ==
Every web service function is mapped to an external function. External function are described in the [[External_functions_API|External functions API]].
Each external function is written with two other functions describing the parameters and the return values. These description functions are used by web service servers to:
* validate the web service function parameters
* validate the web service function returned values
* build WSDL files or other protocol documents

These two description functions are located in the same file and the same class mentioned in local/myplugin/db/services.php.

Thus for the web service function '''local_myplugin_create_groups()''', we need write a class named '''local_myplugin_external''' in the file '''local/myplugin/externallib.php'''. The class will contain:
* create_groups(...)
* create_groups_parameters()
* create_groups_return()

=== create_groups_parameters() ===
<code php>
require_once("$CFG->libdir/externallib.php");

class local_myplugin_external extends external_api {

/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
)
);
}
</code>

A web service function without parameters will have a parameter description function like that:
<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function functionname_parameters() {
return new external_function_parameters(
array(
//if I had any parameters, they would be described here. But I don't have any, so this array is empty.
)
);
}
</code>

A parameter can be described as:
* a list => external_multiple_structure
* an object => external_single_structure
* a primary type => external_value

Our create_groups() function expects one parameter named ''groups'', so we will first write:

<code php>
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => ...

)
);
}
</code>

This ''groups'' parameter is a list of group. So we will write :

<code php>
'groups' => new external_multiple_structure(
...
)
</code>

An external_multiple_structure object (list) can be constructed with:
* ''external_multiple_structure'' (list)
* ''external_single_structure'' (object)
* ''external_value'' (primary type).

For our function it will be a ''external_single_structure'':

<code php>
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
</code>

Thus we obtain :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
)
</code>

Each group values is a ''external_value'' (primary type):
* ''courseid'' is an integer
* ''name'' is a string (text only, not tag)
* ''description'' is a string (can be anything)
* ''enrolmentkey'' is also a string (can be anything)

We add them to the description :

<code php>
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'), //the second argument is a human readable description text. This text is displayed in the automatically generated documentation.
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
</code>

=== create_groups_returns() ===

It's similar to create_groups_parameters(), but instead of describing the parameters, it describes the return values.

<code php>
public static function create_groups_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'group record id'),
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
);
}
</code>

=== Required, Optional or Default value ===
A value can be VALUE_REQUIRED, VALUE_OPTIONAL, or VALUE_DEFAULT. If not mentioned, a value is VALUE_REQUIRED by default.

<code php>
'yearofstudy' => new external_value(PARAM_INT, 'year of study',VALUE_DEFAULT, 1979),
</code>

* VALUE_REQUIRED - if the value is not supplied => the server throws an error message
* VALUE_OPTIONAL - if the value is not supplied => the value is ignored. Note that VALUE_OPTIONAL can't be used in top level parameters, it must be used only within array/objects key definition. If you need top level Optional parameters you should use VALUE_DEFAULT instead.
* VALUE_DEFAULT - if the value is not supplied => the default value is used

Note: Because some web service protocols are strict about the number and types of arguments - it is not possible to specify an optional parameter as one of the top-most parameters for a function. Examples:

Not cool:
<code php>
public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'chocolatechips' => new external_value(PARAM_BOOL, PARAM_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ERROR! top level optional parameter!!!
)
);
}

</code>

Cool:
<code php>

public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'ifeellike' => new external_single_structure(
array(
'chocolatechips' => new external_value(PARAM_BOOL, VALUE_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ALL GOOD!! We have nested the params in a external_single_structure.
)
)
)
);
}

</code>

== Implement the external function ==
We declared our web service function and we defined the external function parameters and return values. We will now implement the external function:

<code php>
/**
* Create groups
* @param array $groups array of group description arrays (with keys groupname and courseid)
* @return array of newly created groups
*/
public static function create_groups($groups) { //Don't forget to set it as static
global $CFG, $DB;
require_once("$CFG->dirroot/group/lib.php");

$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));

$transaction = $DB->start_delegated_transaction(); //If an exception is thrown in the below code, all DB queries in this code will be rollback.

$groups = array();

foreach ($params['groups'] as $group) {
$group = (object)$group;

if (trim($group->name) == '') {
throw new invalid_parameter_exception('Invalid group name');
}
if ($DB->get_record('groups', array('courseid'=>$group->courseid, 'name'=>$group->name))) {
throw new invalid_parameter_exception('Group with the same name already exists in the course');
}

// now security checks
$context = get_context_instance(CONTEXT_COURSE, $group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);

// finally create the group
$group->id = groups_create_group($group, false);
$groups[] = (array)$group;
}

$transaction->allow_commit();

return $groups;
}
</code>

=== Parameter validation ===
<code php>
$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));
</code>
This ''validate_parameters'' function validates the external function parameters against the description. It will return an exception if some required parameters are missing, if parameters are not well-formed, and check the parameters validity. It is essential that you do this call to avoid potential hack.

'''Important:''' the parameters of the external function and their declaration in the description '''must be the same order'''. In this example we have only one parameter named $groups, so we don't need to worry about the order.

=== Context and Capability checks ===

<code php>
/// now security checks
$context = context_course::instance($group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);
</code>

Note: validate_context() is required in all external functions before operating on any data belonging to a context. This function does sanity and security checks on the context that was passed to the external function - and sets up the global $PAGE and $OUTPUT for rendering return values. Do NOT use require_login(), or $PAGE->set_context() in an external function.

=== Exceptions===
You can throw exceptions. There are automatically handle by Moodle web service servers.
<code php>
//Note: it is good practice to add detailled information in $debuginfo,
// and only send back a generic exception message when Moodle DEBUG mode < NORMAL.
// It's what we do here throwing the invalid_parameter_exception($debug) exception
throw new invalid_parameter_exception('Group with the same name already exists in the course');
</code>

=== Correct return values ===
The return values will be validated by the Moodle web service servers:
* return values contain some values not described => these values will be skipped.
* return values miss some required values (VALUE_REQUIRED) => the server will return an error.
* return values types don't match the description (int != PARAM_ALPHA) => the server will return an error

'''Note:''' cast all your returned objects into arrays.

== Making web service accessible through Apache Thrift ==
This step is optional. If you wish to generate SDK for different programming languages and platforms using [http://thrift.apache.org Apache Thrift framework] then [https://bitbucket.org/hhteam/moodle_thrift_tools/wiki/Home Moodle Thrift tools] can help you.

Two steps should be performed:

* Generate .thrift files for Moodle API using thriftgenerator script.
* Generate thrift handlers for PHP and copy the php files generated to your plugin source tree.

It is also recommended to include thrift files into the distribution of your plugin in order to simplify creation of client bindings for the users of your API.

That's it. Now the web API of your plugin is accessible through Apache Thrift Framework.

== Bump the plugin version ==
Edit your local/myplugin/version.php and increase the plugin version. This should trigger a Moodle upgrade and the new web service should be available in the administration (''Administration > Plugins > Web Services > Manage services'')

== Deprecation ==
External functions deprecation process is different from the [[Deprecation|standard deprecation]]. If you are interested in deprecating any of your external functions you should create a FUNCTIONNAME_is_deprecated function in your external function class. Return true if the external function is deprecated. This is an example:

<code php>

/**
* Mark the function as deprecated.
* @return bool
*/
public static function create_groups_is_deprecated() {
return true;
}
</code>

=See also=
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating_a_web_service_client|Implement a web service client]]

Specification:
* [[External services security]]
* [[External services description]]

[[Category:Web Services]]
[[Category:Plugins]]

File API

2019-01-30T09:55:22Z

Howardsmiller: /* Serving files to users */

{{Moodle 2.0}}
==Overview==

The File API is for managing all the files stored by Moodle. If you are interested in how the file API works internally, see [[File API internals]]. The page is just about what you need to know to use the file API. Related is the [[Repository API]], which lets users get files into Moodle.

If you are looking for an explanation on how to manage moodle files in moodle forms, you most likely need to read [[Using_the_File_API_in_Moodle_forms|Using the File API in Moodle forms]].

==File areas==

Files are conceptually stored in '''file areas'''. A file area is uniquely identified by:
* A context id.
* full component name (using [[Frankenstyle]]), for example 'course', 'mod_forum', 'mod_glossary', 'block_html'.
* A file area type, for example 'intro' or 'post'.
* A unique itemid. Normally, the itemid relates to something depending on the file area type. For example, for a 'course', 'intro' file area, the itemid is 0. For forum post, it is the post id.

File areas are not listed separately anywhere, they are stored implicitly in the files table. Please note that each subsystem is allowed to access only own file areas, for example only code in /mod/assignment/* may access files with component 'mod_assignment'.

===Naming file areas===

The names of the file areas are not strictly defined, but it is strongly recommended to use singulars and common names of areas if possible (intro, post, attachment, description, ...).

==Serving files to users==

You must refer to the file with a URL that includes a file-serving script, often pluginfile.php. For example

The general form of the URL is something like
<code php>
$url = $CFG->wwwroot/pluginfile.php/$contextid/$component/$filearea/arbitrary/extra/infomation.ext
</code>

A specific example might be
<code php>
$url = $CFG->wwwroot/pluginfile.php/$forumcontextid/mod_forum/post/$postid/image.jpg
</code>

Usually, you do not need to construct this URL directly - the function moodle_url::make_pluginfile_url() should be used instead:
<code php>
$url = moodle_url::make_pluginfile_url($file->get_contextid(), $file->get_component(), $file->get_filearea(), $file->get_itemid(), $file->get_filepath(), $file->get_filename(), false);
</code>
Note: If you do not need the 'itemid', then pass null in as this parameter and it will be entirely missed out from the URL - you need to take this into account when serving the file in the callback function, below.

The final parameter ('false' here) is 'forcedownload'.

The file serving script then looks at the context id, and component name, and the file area name, and based on that arranges for the file to be served, following appropriate security checks.

<strong>Note: </strong>In most cases, when developing third party plugins, pluginfile.php looks for a '''callback function''' in the appropriate plugin. These functions are stored in '''lib.php''' files and are named '''component_name_pluginfile()'''. The arbitrary/extra/infomation.ext is passed to the callback. For example, files in the mod_forum+post file area end up being served by the mod_forum_pluginfile function in mod/forum/lib.php. This function in MYPLUGIN/lib.php will usually follow a pattern like the example below, but the details will vary depending on the restrictions your plugin places on accessing different files (e.g. assignment files can only be accessed by teachers and the student who submitted the file, forum attachments require access to the discussion they are posted on):
<code php>
/**
* Serve the files from the MYPLUGIN file areas
*
* @param stdClass $course the course object
* @param stdClass $cm the course module object
* @param stdClass $context the context
* @param string $filearea the name of the file area
* @param array $args extra arguments (itemid, path)
* @param bool $forcedownload whether or not force download
* @param array $options additional options affecting the file serving
* @return bool false if the file not found, just send the file otherwise and do not return anything
*/
function MYPLUGIN_pluginfile($course, $cm, $context, $filearea, $args, $forcedownload, array $options=array()) {
// Check the contextlevel is as expected - if your plugin is a block, this becomes CONTEXT_BLOCK, etc.
if ($context->contextlevel != CONTEXT_MODULE) {
return false;
}

// Make sure the filearea is one of those used by the plugin.
if ($filearea !== 'expectedfilearea' && $filearea !== 'anotherexpectedfilearea') {
return false;
}

// Make sure the user is logged in and has access to the module (plugins that are not course modules should leave out the 'cm' part).
require_login($course, true, $cm);

// Check the relevant capabilities - these may vary depending on the filearea being accessed.
if (!has_capability('mod/MYPLUGIN:view', $context)) {
return false;
}

// Leave this line out if you set the itemid to null in make_pluginfile_url (set $itemid to 0 instead).
$itemid = array_shift($args); // The first item in the $args array.

// Use the itemid to retrieve any relevant data records and perform any security checks to see if the
// user really does have access to the file in question.

// Extract the filename / filepath from the $args array.
$filename = array_pop($args); // The last item in the $args array.
if (!$args) {
$filepath = '/'; // $args is empty => the path is '/'
} else {
$filepath = '/'.implode('/', $args).'/'; // $args contains elements of the filepath
}

// Retrieve the file from the Files API.
$fs = get_file_storage();
$file = $fs->get_file($context->id, 'mod_MYPLUGIN', $filearea, $itemid, $filepath, $filename);
if (!$file) {
return false; // The file does not exist.
}

// We can now send the file back to the browser - in this case with a cache lifetime of 1 day and no filtering.
send_stored_file($file, 86400, 0, $forcedownload, $options);
}
</code>

You normally use an API function to generate these URL automatically, most often the <tt>file_rewrite_pluginfile_urls</tt> function.

==Getting files from the user==

* See [[Using_the_File_API_in_Moodle_forms|Using the File API in Moodle forms]]

==Examples==
Please note that in reality developers outside of core will not deal with file api directly in majority of cases, instead use formslib elements which are doing all this automatically.

===Browsing files===
<code php>
$browser = get_file_browser();
$context = get_system_context();

$filearea = null;
$itemid = null;
$filename = null;
if ($fileinfo = $browser->get_file_info($context, $component, $filearea, $itemid, '/', $filename)) {
// build a Breadcrumb trail
$level = $fileinfo->get_parent();
while ($level) {
$path[] = array('name'=>$level->get_visible_name());
$level = $level->get_parent();
}
$path = array_reverse($path);
$children = $fileinfo->get_children();
foreach ($children as $child) {
if ($child->is_directory()) {
echo $child->get_visible_name();
// display contextid, itemid, component, filepath and filename
var_dump($child->get_params());
}
}
}

</code>

===Moving files around===

For example, if you have just built a file at the path
<code php>
$from_zip_file = $CFG->dataroot . '/temp/backup/' . $preferences->backup_unique_code .
'/' . $preferences->backup_name;
</code>
And you want to move it into the course_backup file area, do
<code php>
$context = get_context_instance(CONTEXT_COURSE, $preferences->backup_course);
$fs = get_file_storage();
$file_record = array('contextid'=>$context->id, 'component'=>'course', 'filearea'=>'backup',
'itemid'=>0, 'filepath'=>'/', 'filename'=>$preferences->backup_name,
'timecreated'=>time(), 'timemodified'=>time());
$fs->create_file_from_pathname($file_record, $from_zip_file);
</code>

=== List area files ===
<code php>
$fs = get_file_storage();
$files = $fs->get_area_files($contextid, 'mod_assignment', 'submission', $submission->id);
foreach ($files as $f) {
// $f is an instance of stored_file
echo $f->get_filename();
}
</code>

Or as links...

<code php>
$out = array();

$fs = get_file_storage();
$files = $fs->get_area_files($contextid, 'mod_assignment', 'submission', $submission->id);

foreach ($files as $file) {
$filename = $file->get_filename();
$url = moodle_url::make_file_url('/pluginfile.php', array($file->get_contextid(), 'mod_assignment', 'submission',
$file->get_itemid(), $file->get_filepath(), $filename));
$out[] = html_writer::link($url, $filename);
}
$br = html_writer::empty_tag('br');

return implode($br, $out);
</code>

=== Create file ===

Here's how to create a file whose contents will be a text string. This is the equivalent of the PHP function <tt>file_put_contents</tt>.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'contextid' => $context->id, // ID of context
'component' => 'mod_mymodule', // usually = table name
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Create file containing text 'hello world'
$fs->create_file_from_string($fileinfo, 'hello world');
</code>

If you want to create a file in the Moodle file area based on a 'real' file e.g. in a temporary folder, you can use <tt>create_file_from_pathname</tt> instead. Similarly, you can create a file based on some other file already in Moodle's local files by using <tt>create_file_from_storedfile</tt>. Browse through <tt>lib/filestorage/file_storage.php</tt> for details.

Unlike with ordinary files, this method will not automatically overwrite an existing file. If you wish to overwrite a file, you must first get the file and (if it exists) delete it, and only then create it again.

=== Read file ===

This is a way to read a file, equivalent to <tt>file_get_contents</tt>. '''Please note your are allowed to do this ONLY from mod/mymodule/* code, it is not acceptable to do this anywhere else.''' Other code has to use file_browser interface instead.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule', // usually = table name
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Read contents
if ($file) {
$contents = $file->get_content();
} else {
// file doesn't exist - do something
}
</code>

If you want to access the file directly on disk, this is not permitted. Instead, you need to make a copy of the file in a temporary area and use that. You can do this with <tt>$file->copy_content_to($pathname)</tt>.

=== Delete file ===

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule',
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Delete it if it exists
if ($file) {
$file->delete();
}
</code>

=== Convert between file formats (office documents) ===
{{Moodle 3.1}}

This functionality requires "unoconv" to be installed and configured on the site - so it is not available on all installations.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule',
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Try and convert it if it exists
if ($file) {
$convertedfile = $fs->get_converted_document($file, 'pdf');
}
</code>

==See also==

* [[Core APIs]]
* [[File API internals]] how the File API works internally.
* [[Using the File API in Moodle forms]]

[[Category:Files]]
[[Category:API]]

File API

2019-01-29T16:13:06Z

Howardsmiller: /* Serving files to users */

{{Moodle 2.0}}
==Overview==

The File API is for managing all the files stored by Moodle. If you are interested in how the file API works internally, see [[File API internals]]. The page is just about what you need to know to use the file API. Related is the [[Repository API]], which lets users get files into Moodle.

If you are looking for an explanation on how to manage moodle files in moodle forms, you most likely need to read [[Using_the_File_API_in_Moodle_forms|Using the File API in Moodle forms]].

==File areas==

Files are conceptually stored in '''file areas'''. A file area is uniquely identified by:
* A context id.
* full component name (using [[Frankenstyle]]), for example 'course', 'mod_forum', 'mod_glossary', 'block_html'.
* A file area type, for example 'intro' or 'post'.
* A unique itemid. Normally, the itemid relates to something depending on the file area type. For example, for a 'course', 'intro' file area, the itemid is 0. For forum post, it is the post id.

File areas are not listed separately anywhere, they are stored implicitly in the files table. Please note that each subsystem is allowed to access only own file areas, for example only code in /mod/assignment/* may access files with component 'mod_assignment'.

===Naming file areas===

The names of the file areas are not strictly defined, but it is strongly recommended to use singulars and common names of areas if possible (intro, post, attachment, description, ...).

==Serving files to users==

You must refer to the file with a URL that includes a file-serving script, often pluginfile.php. For example

The general form of the URL is something like
<code php>
$url = $CFG->wwwroot/pluginfile.php/$contextid/$component/$filearea/arbitrary/extra/infomation.ext
</code>

A specific example might be
<code php>
$url = $CFG->wwwroot/pluginfile.php/$forumcontextid/mod_forum/post/$postid/image.jpg
</code>

Usually, you do not need to construct this URL directly - the function moodle_url::make_pluginfile_url() should be used instead:
<code php>
$url = moodle_url::make_pluginfile_url($file->get_contextid(), $file->get_component(), $file->get_filearea(), $file->get_itemid(), $file->get_filepath(), $file->get_filename());
</code>
Note: If you do not need the 'itemid', then pass null in as this parameter and it will be entirely missed out from the URL - you need to take this into account when serving the file in the callback function, below.

The file serving script then looks at the context id, and component name, and the file area name, and based on that arranges for the file to be served, following appropriate security checks.

<strong>Note: </strong>In most cases, when developing third party plugins, pluginfile.php looks for a '''callback function''' in the appropriate plugin. These functions are stored in '''lib.php''' files and are named '''component_name_pluginfile()'''. The arbitrary/extra/infomation.ext is passed to the callback. For example, files in the mod_forum+post file area end up being served by the mod_forum_pluginfile function in mod/forum/lib.php. This function in MYPLUGIN/lib.php will usually follow a pattern like the example below, but the details will vary depending on the restrictions your plugin places on accessing different files (e.g. assignment files can only be accessed by teachers and the student who submitted the file, forum attachments require access to the discussion they are posted on):
<code php>
/**
* Serve the files from the MYPLUGIN file areas
*
* @param stdClass $course the course object
* @param stdClass $cm the course module object
* @param stdClass $context the context
* @param string $filearea the name of the file area
* @param array $args extra arguments (itemid, path)
* @param bool $forcedownload whether or not force download
* @param array $options additional options affecting the file serving
* @return bool false if the file not found, just send the file otherwise and do not return anything
*/
function MYPLUGIN_pluginfile($course, $cm, $context, $filearea, $args, $forcedownload, array $options=array()) {
// Check the contextlevel is as expected - if your plugin is a block, this becomes CONTEXT_BLOCK, etc.
if ($context->contextlevel != CONTEXT_MODULE) {
return false;
}

// Make sure the filearea is one of those used by the plugin.
if ($filearea !== 'expectedfilearea' && $filearea !== 'anotherexpectedfilearea') {
return false;
}

// Make sure the user is logged in and has access to the module (plugins that are not course modules should leave out the 'cm' part).
require_login($course, true, $cm);

// Check the relevant capabilities - these may vary depending on the filearea being accessed.
if (!has_capability('mod/MYPLUGIN:view', $context)) {
return false;
}

// Leave this line out if you set the itemid to null in make_pluginfile_url (set $itemid to 0 instead).
$itemid = array_shift($args); // The first item in the $args array.

// Use the itemid to retrieve any relevant data records and perform any security checks to see if the
// user really does have access to the file in question.

// Extract the filename / filepath from the $args array.
$filename = array_pop($args); // The last item in the $args array.
if (!$args) {
$filepath = '/'; // $args is empty => the path is '/'
} else {
$filepath = '/'.implode('/', $args).'/'; // $args contains elements of the filepath
}

// Retrieve the file from the Files API.
$fs = get_file_storage();
$file = $fs->get_file($context->id, 'mod_MYPLUGIN', $filearea, $itemid, $filepath, $filename);
if (!$file) {
return false; // The file does not exist.
}

// We can now send the file back to the browser - in this case with a cache lifetime of 1 day and no filtering.
send__stored_file($file, 86400, 0, $forcedownload, $options);
}
</code>

You normally use an API function to generate these URL automatically, most often the <tt>file_rewrite_pluginfile_urls</tt> function.

==Getting files from the user==

* See [[Using_the_File_API_in_Moodle_forms|Using the File API in Moodle forms]]

==Examples==
Please note that in reality developers outside of core will not deal with file api directly in majority of cases, instead use formslib elements which are doing all this automatically.

===Browsing files===
<code php>
$browser = get_file_browser();
$context = get_system_context();

$filearea = null;
$itemid = null;
$filename = null;
if ($fileinfo = $browser->get_file_info($context, $component, $filearea, $itemid, '/', $filename)) {
// build a Breadcrumb trail
$level = $fileinfo->get_parent();
while ($level) {
$path[] = array('name'=>$level->get_visible_name());
$level = $level->get_parent();
}
$path = array_reverse($path);
$children = $fileinfo->get_children();
foreach ($children as $child) {
if ($child->is_directory()) {
echo $child->get_visible_name();
// display contextid, itemid, component, filepath and filename
var_dump($child->get_params());
}
}
}

</code>

===Moving files around===

For example, if you have just built a file at the path
<code php>
$from_zip_file = $CFG->dataroot . '/temp/backup/' . $preferences->backup_unique_code .
'/' . $preferences->backup_name;
</code>
And you want to move it into the course_backup file area, do
<code php>
$context = get_context_instance(CONTEXT_COURSE, $preferences->backup_course);
$fs = get_file_storage();
$file_record = array('contextid'=>$context->id, 'component'=>'course', 'filearea'=>'backup',
'itemid'=>0, 'filepath'=>'/', 'filename'=>$preferences->backup_name,
'timecreated'=>time(), 'timemodified'=>time());
$fs->create_file_from_pathname($file_record, $from_zip_file);
</code>

=== List area files ===
<code php>
$fs = get_file_storage();
$files = $fs->get_area_files($contextid, 'mod_assignment', 'submission', $submission->id);
foreach ($files as $f) {
// $f is an instance of stored_file
echo $f->get_filename();
}
</code>

Or as links...

<code php>
$out = array();

$fs = get_file_storage();
$files = $fs->get_area_files($contextid, 'mod_assignment', 'submission', $submission->id);

foreach ($files as $file) {
$filename = $file->get_filename();
$url = moodle_url::make_file_url('/pluginfile.php', array($file->get_contextid(), 'mod_assignment', 'submission',
$file->get_itemid(), $file->get_filepath(), $filename));
$out[] = html_writer::link($url, $filename);
}
$br = html_writer::empty_tag('br');

return implode($br, $out);
</code>

=== Create file ===

Here's how to create a file whose contents will be a text string. This is the equivalent of the PHP function <tt>file_put_contents</tt>.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'contextid' => $context->id, // ID of context
'component' => 'mod_mymodule', // usually = table name
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Create file containing text 'hello world'
$fs->create_file_from_string($fileinfo, 'hello world');
</code>

If you want to create a file in the Moodle file area based on a 'real' file e.g. in a temporary folder, you can use <tt>create_file_from_pathname</tt> instead. Similarly, you can create a file based on some other file already in Moodle's local files by using <tt>create_file_from_storedfile</tt>. Browse through <tt>lib/filestorage/file_storage.php</tt> for details.

Unlike with ordinary files, this method will not automatically overwrite an existing file. If you wish to overwrite a file, you must first get the file and (if it exists) delete it, and only then create it again.

=== Read file ===

This is a way to read a file, equivalent to <tt>file_get_contents</tt>. '''Please note your are allowed to do this ONLY from mod/mymodule/* code, it is not acceptable to do this anywhere else.''' Other code has to use file_browser interface instead.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule', // usually = table name
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Read contents
if ($file) {
$contents = $file->get_content();
} else {
// file doesn't exist - do something
}
</code>

If you want to access the file directly on disk, this is not permitted. Instead, you need to make a copy of the file in a temporary area and use that. You can do this with <tt>$file->copy_content_to($pathname)</tt>.

=== Delete file ===

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule',
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Delete it if it exists
if ($file) {
$file->delete();
}
</code>

=== Convert between file formats (office documents) ===
{{Moodle 3.1}}

This functionality requires "unoconv" to be installed and configured on the site - so it is not available on all installations.

<code php>
$fs = get_file_storage();

// Prepare file record object
$fileinfo = array(
'component' => 'mod_mymodule',
'filearea' => 'myarea', // usually = table name
'itemid' => 0, // usually = ID of row in table
'contextid' => $context->id, // ID of context
'filepath' => '/', // any path beginning and ending in /
'filename' => 'myfile.txt'); // any filename

// Get file
$file = $fs->get_file($fileinfo['contextid'], $fileinfo['component'], $fileinfo['filearea'],
$fileinfo['itemid'], $fileinfo['filepath'], $fileinfo['filename']);

// Try and convert it if it exists
if ($file) {
$convertedfile = $fs->get_converted_document($file, 'pdf');
}
</code>

==See also==

* [[Core APIs]]
* [[File API internals]] how the File API works internally.
* [[Using the File API in Moodle forms]]

[[Category:Files]]
[[Category:API]]

Using the File API in Moodle forms

2019-01-18T11:44:28Z

Howardsmiller: /* Element preparation */

{{Moodle 2.0}}

==Overview==

In Moodle 2.0 all files are stored in a central database accessible via the [[File API|File API]], and every file is associated with a component and a "file area" in Moodle, such as a particular module.

A common use case is to provide a form (using Moodle's [[lib/formslib.php|Forms API]]) which allows users to upload or import files as attachments or media embedded into HTML.

Normally this works like this:
# User starts creation or re-edits an existing item in Moodle (eg forum post, resource, glossary entry etc)
# User presses some sort of button to browse for new files to attach or embed
# User sees our "Choose file..." dialog, which contains one or more repository instances.
# User chooses a file, the corresponding [[Repository plugins|Repository plugin]] takes care of copying the file into a "draft file area" within Moodle
# File appears in the text or as an attachment in the form.
# When the user hits save, the [[File API|File API]] is invoked to move the file from the draft file area into a permanent file area associated with that data

This document shows you exactly how to use Moodle forms to interact with users in a standard and secure way.

If you just want to write code to manipulate Moodle files internally (without user input) then see [[File API]].

==Form elements==

In Moodle 2.0 there are three file-related form elements for interacting with users:

# filemanager - the way to attach one or more files as a set
# editor - the way to specify a textarea with a HTML editor, and all the handling of images and movies within that HTML
# filepicker - a way to specify one file for the case when you want to process the file and throw it away

In Moodle 1.9 there were two other types which are now '''deprecated''' (they work, but please do not use these anymore)
# file - used to just allow a normal file upload from the desktop only.
# htmleditor - this old method of embedding a HTML editor in a textarea is not able to support repositories etc.

===filepicker===

File picker (''filepicker'') is a direct replacement of the older ''file'' formslib element.

It is intended for situations when you want the user to upload '''one''' file so you can process it and delete it, such as when you are importing data from a CSV file.
If you want a file that remains part of the Moodle storage and will reappear when you reopen the form, then you should use a filemanager instead (and restrict it to a single file, if necessary).

==== Using the filepicker element ====

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null,
array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>

==== Obtain the chosen file ====

To get the contents of the file:

<code php>
$content = $mform->get_file_content('userfile');
</code>

To get the name of the chosen file:

<code php>
$name = $mform->get_new_filename('userfile');
</code>

To save the chosen file to the server filesystem (such as to moodledata folder):

<code php>
$success = $mform->save_file('userfile', $fullpath, $override);
</code>

To store the chosen file in the Moodle files pool:

<code php>
$storedfile = $mform->save_stored_file('userfile', ...);
</code>

=== filemanager ===

The File Manager element improves on file picker by allowing you to manage more than one file. It is expected that the files will be stored permanently for future use (such as forum and glossary attachments).

==== Add file manager element ====

Example:
<code php>
$mform->addElement('filemanager', 'attachments', get_string('attachment', 'moodle'), null,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'areamaxbytes' => 10485760, 'maxfiles' => 50,
'accepted_types' => array('document'), 'return_types'=> FILE_INTERNAL | FILE_EXTERNAL));
</code>

Here are the fields for filemanager:

;'filemanager':This is a filemanager element :)
;elementname:The unique name of the element in the form
;elementlabel:The label string that users see
;attributes:(leave it as null)
;options: an array of further options for the filepicker (see below)

The options array can contain:

;subdirs:(Default 1) Are subdirectories allowed? (true or false)
;maxbytes:(Default 0) Restricts the size of each individual file.
;areamaxbytes:(Default 0) Restricts the total size of all the files.
;maxfiles:(Default -1) Restricts the total number of files.
;accepted_types:(Default *) You can specify what file types are accepted by filemanager. All current file types are listed in this file: [http://cvs.moodle.org/moodle/lib/filestorage/file_types.mm moodle/lib/filestorage/file_types.mm]. This is a [http://freemind.sourceforge.net/wiki/index.php/Main_Page freemind] file: if it is edited the changes will be immediately reflected in Moodle.

As of 2.3 file_types.mm is no longer used, instead the file types are listed in <code>get_mimetypes_array()</code> in lib/classes/filetypes.php (https://github.com/moodle/moodle/blob/master/lib/classes/filetypes.php).

Example usage: '''array('audio', 'video', 'document')''', you can include file extensions as well, for example: '''array('.txt', '.jpg', 'audio')'''.

==== Load existing files into draft area ====

<code php>
if (empty($entry->id)) {
$entry = new stdClass;
$entry->id = null;
}

$draftitemid = file_get_submitted_draft_itemid('attachments');

file_prepare_draft_area($draftitemid, $context->id, 'mod_glossary', 'attachment', $entry->id,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));

$entry->attachments = $draftitemid;

$mform->set_data($entry);

</code>

==== Store updated set of files ====

<code php>
if ($data = $mform->get_data()) {
// ... store or update $entry
file_save_draft_area_files($data->attachments, $context->id, 'mod_glossary', 'attachment',
$entry->id, array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));
}
</code>

===editor===
There are two ways of using the editor element in code, the first one is easier but expects some standardized fields. The second method is more low level.

====Simple use====
# name database fields: ''textfield'', ''textfieldformat'' (and ''textfieldtrust'' if required)
# create options array. note that context is the best, most local context you have available.
#:<code php>$textfieldoptions = array('trusttext'=>true, 'subdirs'=>true, 'maxfiles'=>$maxfiles, 'maxbytes'=>$maxbytes, 'context'=>$context);</code>
# add editor ''textfield_editor'' to moodle form, pass options through custom data in form constructor, set $data->id to null if data not exist yet
#:<code php>$mform->addElement('editor', 'textfield_editor', get_string('fieldname', 'somemodule'), null, $textfieldoptions);</code>
# prepare data
#:<code php>$data = file_prepare_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>
# get submitted data and after inserting/updating of data
#:<code php>$data = file_postupdate_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>

Real world examples are in mod/glossary/edit.php and mod/glossary/comment.php

====Low level use====

When using editor element you need to preprocess and postprocess the data:
# detect if form was already submitted (usually means draft is area already exists) - ''file_get_submitted_draft_itemid()''
# prepare draft file area, temporary storage of all files attached to the text - ''file_prepare_draft_area()''
# convert encoded relative links to absolute links - ''file_prepare_draft_area()''
# create form and set current data
# after submission the changed files must be merged back into original area - ''file_save_draft_area_files()''
# absolute links have to be replaced by relative links - ''file_save_draft_area_files()''

=====Replace old htmleditor with editor=====

The file picker has been integrated with with TinyMCE to make the editor element. This new element should support all types on editors and should be able to switch them on-the-fly. Instances of the old htmleditor element in your forms should be replaced by the new editor element, this may need adding of new format and trusttext columns. For example:
<code php>
$mform->addElement('editor', 'entry', get_string('definition', 'glossary'), null,
array('maxfiles' => EDITOR_UNLIMITED_FILES));
</code>
The editor element can take following options: maxfiles, maxbytes, subdirs and changeformat. Please note that the embedded files is optional feature and is not expected be used everywhere.

'''Note''': the editor element now includes text format option. You should no longer use the separate format element type.

=====Prepare current data - text and files=====

<code php>
if (empty($entry->id)) {
$entry = new object();
$entry->id = null;
$entry->definition = '';
$entry->format = FORMAT_HTML;
}

$draftid_editor = file_get_submitted_draft_itemid('entry');
$currenttext = file_prepare_draft_area($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $entry->definition);
$entry->entry = array('text'=>$currenttext, 'format'=>$entry->format, 'itemid'=>$draftid_editor);

$mform->set_data($entry);
</code>

If there are multiple files, they will share the same itemid.

=====Obtain text, format and save draft files=====

To retrieve editor content, you need to use following code:

<code php>
if ($fromform = $mform->get_data()) {
// content of editor
$messagetext = $fromform->entry['text'];
// format of content
$messageformat = $fromform->entry['format'];
}
</code>

When a user selects a file using the file picker, the file is initially stored in a draft file area, and a URL is inserted into the HTML in the editor that lets the person editing the content (but no one else) see the file.

When the user submits the form, we then need to save the draft files to the correct place in permanent storage. (Just like you have to call $DB->update_record('tablename', $data); to have the other parts of the form submission stored correctly.)

The save_files_from_draft_area function and replace absolute links with internal relative links do:
<code php>
$messagetext = file_save_draft_area_files($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $messagetext);
</code>
; $context->id, 'component', 'proper_file_area' and $entry->id : correspond to the contextid, filearea and itemid columns in the [[File_API#Table:_files|files table]].
; $messagetext : this is the message text. As the files are saved to the real file area, the URLs in this content are rewritten.

All URLs in content that point to files managed to the File API are converted to a form that starts '@@PLUGINFILE@@/' before the content is stored in the database. That is what we mean by rewriting.

== File serving==

=== Convert internal relative links to absolute links ===

Before text content is displayed to the user, any URLs in the '@@PLUGINFILE@@/' form in the content need to be rewritten to the real URL where the user can access the files.
<code php>
$messagetext = file_rewrite_pluginfile_urls($messagetext, 'pluginfile.php',
$context->id, 'mod_mymodule', 'proper_file_area', $itemid);
</code>
; $messagetext : is the content containing the @@PLUGINFILE@@ URLs from the database.
; 'pluginfile.php' : there are a number of different scripts that can serve files with different permissions checks. You need to specify which one to use.
; $context->id, 'mod_mymodule', 'proper_file_area', $itemid : uniquely identifies the file area, as before.

=== Implement file serving access control ===

Attachments and embedded images should have the same access control like the text itself, in majority of cases these files are served using pluginfile.php. Access control is defined in ''module/lib.php'' file in function ''module_pluginfile()''.

== File browsing support ==
Only owner of each file area is allowed to use low level File API function to access files, other parts of Moodle should use file browsing API.

Activities may specify browsing support in own module/lib.php file by implementing functions module_get_file_areas() and module_get_file_info().

== Upgrading your code ==
Here I will attempt to describe some simple steps you can take to upgrade your file-handling form elements from pre-2.0 code to 2.0. We will use the example of glossary, since it has been used above.

=== Preparing your options ===
Unless you are happy with the defaults, you will need to define an array of options for each file-handling form element. You could define it at different places, but it's best to put it in one place and make the array(s) available to other files if they need it. In the majority of cases, this will be in a file like edit.php

Previous code in mod/glossary/edit.php:
<code php>
$mform =& new mod_glossary_entry_form(null, compact('cm', 'glossary', 'hook', 'mode', 'e', 'context'));
</code>
New code:
<code php>
$maxbytes = $course->maxbytes;
// Could also use $CFG->maxbytes if you are not coding within a course context

$definitionoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes, 'trusttext'=>true,
'context'=>$context);
$attachmentoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Note that the data being passed to the form constructor have changed also, but this is not part of the file API changes, I just include them to avoid confusion.

These options are for the htmleditor (definition field) and the filemanager (attachment field). They are used by a file called edit_form.php.

=== Element preparation ===
Before we look at this, however, we need to "prepare" the elements so that they can correctly display existing embedded images and attached files when you are editing a record instead of just creating one. So, let's take the code we've got so far in edit.php and add to it:

Currently upgraded code in edit.php:
<code php>
$mform = new mod_glossary_entry_form(null, array(
'current'=>$entry,
'cm'=>$cm,
'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
New code with element preparation:
<code php>
$entry = new stdClass();
$entry->id = 0; \\ See notes!
$entry = file_prepare_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_prepare_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
$mform->set_data($entry);
</code>
Things to note:
* $entry in this case is simply a stdClass object which may either represent a new glossary entry or an existing one.
* $entry->id must be the unique identifier for the current object. If we are creating a new entry, it will be null, but in all cases it must be defined.
* These two functions (file_prepare_standard_editor and file_prepare_standard_filemanager) are shortcuts functions that take care of some of the tedious setting up for you, but they make a couple of assumptions:
*# You '''must''' name the form element as {element}_editor or {element}_filemanager (see next section)
*# You '''must''' have at least the following fields in the database: {element} and {element}summary, as described earlier in this documentation
*# $entry will be populated with a field (in this case) 'definition_filemanager' having the draftfileid of your files. This needs to get passed to the 'value' of the filemanager control in the form. If you get this wrong, existing file will not appear in the control when it is reloaded.

We can now look at the upgrades needed in the form definition file.

=== Form definition ===
Previous code in mod/glossary/edit_form.php:
<code php>
$mform->addElement('htmleditor', 'definition', get_string('definition', 'glossary'),
array('rows'=>20));
$mform->setType('definition', PARAM_RAW);
$mform->addRule('definition', null, 'required', null, 'client');
$mform->setHelpButton('definition', array('writing', 'richtext'), false, 'editorhelpbutton');
$mform->addElement('format');
// a bit further...
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0,
true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->setHelpButton('attachment', array('attachment', get_string('attachment', 'glossary'),
'glossary'));
</code>
New code:
<code php>
$definitionoptions = $this->_customdata['definitionoptions'];
$attachmentoptions = $this->_customdata['attachmentoptions'];
// a bit further...
$mform->addElement('editor', 'definition_editor', get_string('definition', 'glossary'), null,
$definitionoptions);
$mform->setType('definition_editor', PARAM_RAW);
$mform->addRule('definition_editor', get_string('required'), 'required', null, 'client');
// a bit further...
$mform->addElement('filemanager', 'attachment_filemanager', get_string('attachment', 'glossary'),
null, $attachmentoptions);
$mform->setHelpButton('attachment_filemanager', array('attachment2',
get_string('attachment', 'glossary'), 'glossary'));
</code>

Note the following:
* The format element and the help button are no longer required for the HTML editor element
* The name of the form element needs to be changed by adding '_editor' or '_manager' to the original name. This is a naming convention that is used by a couple of functions we will look at shortly
* Make sure $definitionoptions has context parameter, else system context is used and editor will not respect filter settings.

=== Handling submitted data ===
The final step is to handle the submitted data properly, i.e. retrieve the files and save them to disk, associating them with the record we have just created (a glossary entry in our example). This happens in edit.php:

Previous code in edit.php:
<code php>
// Section that updates an entry:
$todb->id = $e;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
$todb->attachment = $newfilename;
}

// Section that adds an entry:
if ($todb->id = insert_record("glossary_entries", $todb)) {
$e = $todb->id;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
set_field("glossary_entries", "attachment", $newfilename, "id", $todb->id);
}
}
</code>
New code:
<code php>
// $todb was renamed to $entry, and the code was refactored
// so that the file-handling code is only used once for either an add or an update action.
// If an entry is being added, $DB->insert() has already been called, so we have a valid $entry->id
$entry = file_postupdate_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_postupdate_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
// store the updated value values
$DB->update_record('glossary_entries', $entry);
</code>
Things to note:
* If you are adding a new record, you will still need to call update_record after calling the file_postupdate* functions

=== Gotchas ===
A few things to keep in mind:
* Make sure that you instantiate the moodle form before any call to $OUTPUT->header()

== See also ==

* [[File API]]
* [[Using the file API]]
* [[Repository plugins]]
* [[Portfolio API]]
* MDL-14589 - File API Meta issue
* [https://moodle.org/mod/forum/discuss.php?d=207748 Adding a text editor to a Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=157953#p692822 Button actions in Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=351013#p1416554 File Picker]
* [https://github.com/CARLOSEDUARDOVIEIRA/moodle-uploadfile Example of filemanager in an activity module]

[[Category:Files]]
[[Category:Repositories]]

Using the File API in Moodle forms

2019-01-18T11:41:54Z

Howardsmiller: /* Element preparation */

{{Moodle 2.0}}

==Overview==

In Moodle 2.0 all files are stored in a central database accessible via the [[File API|File API]], and every file is associated with a component and a "file area" in Moodle, such as a particular module.

A common use case is to provide a form (using Moodle's [[lib/formslib.php|Forms API]]) which allows users to upload or import files as attachments or media embedded into HTML.

Normally this works like this:
# User starts creation or re-edits an existing item in Moodle (eg forum post, resource, glossary entry etc)
# User presses some sort of button to browse for new files to attach or embed
# User sees our "Choose file..." dialog, which contains one or more repository instances.
# User chooses a file, the corresponding [[Repository plugins|Repository plugin]] takes care of copying the file into a "draft file area" within Moodle
# File appears in the text or as an attachment in the form.
# When the user hits save, the [[File API|File API]] is invoked to move the file from the draft file area into a permanent file area associated with that data

This document shows you exactly how to use Moodle forms to interact with users in a standard and secure way.

If you just want to write code to manipulate Moodle files internally (without user input) then see [[File API]].

==Form elements==

In Moodle 2.0 there are three file-related form elements for interacting with users:

# filemanager - the way to attach one or more files as a set
# editor - the way to specify a textarea with a HTML editor, and all the handling of images and movies within that HTML
# filepicker - a way to specify one file for the case when you want to process the file and throw it away

In Moodle 1.9 there were two other types which are now '''deprecated''' (they work, but please do not use these anymore)
# file - used to just allow a normal file upload from the desktop only.
# htmleditor - this old method of embedding a HTML editor in a textarea is not able to support repositories etc.

===filepicker===

File picker (''filepicker'') is a direct replacement of the older ''file'' formslib element.

It is intended for situations when you want the user to upload '''one''' file so you can process it and delete it, such as when you are importing data from a CSV file.
If you want a file that remains part of the Moodle storage and will reappear when you reopen the form, then you should use a filemanager instead (and restrict it to a single file, if necessary).

==== Using the filepicker element ====

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null,
array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>

==== Obtain the chosen file ====

To get the contents of the file:

<code php>
$content = $mform->get_file_content('userfile');
</code>

To get the name of the chosen file:

<code php>
$name = $mform->get_new_filename('userfile');
</code>

To save the chosen file to the server filesystem (such as to moodledata folder):

<code php>
$success = $mform->save_file('userfile', $fullpath, $override);
</code>

To store the chosen file in the Moodle files pool:

<code php>
$storedfile = $mform->save_stored_file('userfile', ...);
</code>

=== filemanager ===

The File Manager element improves on file picker by allowing you to manage more than one file. It is expected that the files will be stored permanently for future use (such as forum and glossary attachments).

==== Add file manager element ====

Example:
<code php>
$mform->addElement('filemanager', 'attachments', get_string('attachment', 'moodle'), null,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'areamaxbytes' => 10485760, 'maxfiles' => 50,
'accepted_types' => array('document'), 'return_types'=> FILE_INTERNAL | FILE_EXTERNAL));
</code>

Here are the fields for filemanager:

;'filemanager':This is a filemanager element :)
;elementname:The unique name of the element in the form
;elementlabel:The label string that users see
;attributes:(leave it as null)
;options: an array of further options for the filepicker (see below)

The options array can contain:

;subdirs:(Default 1) Are subdirectories allowed? (true or false)
;maxbytes:(Default 0) Restricts the size of each individual file.
;areamaxbytes:(Default 0) Restricts the total size of all the files.
;maxfiles:(Default -1) Restricts the total number of files.
;accepted_types:(Default *) You can specify what file types are accepted by filemanager. All current file types are listed in this file: [http://cvs.moodle.org/moodle/lib/filestorage/file_types.mm moodle/lib/filestorage/file_types.mm]. This is a [http://freemind.sourceforge.net/wiki/index.php/Main_Page freemind] file: if it is edited the changes will be immediately reflected in Moodle.

As of 2.3 file_types.mm is no longer used, instead the file types are listed in <code>get_mimetypes_array()</code> in lib/classes/filetypes.php (https://github.com/moodle/moodle/blob/master/lib/classes/filetypes.php).

Example usage: '''array('audio', 'video', 'document')''', you can include file extensions as well, for example: '''array('.txt', '.jpg', 'audio')'''.

==== Load existing files into draft area ====

<code php>
if (empty($entry->id)) {
$entry = new stdClass;
$entry->id = null;
}

$draftitemid = file_get_submitted_draft_itemid('attachments');

file_prepare_draft_area($draftitemid, $context->id, 'mod_glossary', 'attachment', $entry->id,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));

$entry->attachments = $draftitemid;

$mform->set_data($entry);

</code>

==== Store updated set of files ====

<code php>
if ($data = $mform->get_data()) {
// ... store or update $entry
file_save_draft_area_files($data->attachments, $context->id, 'mod_glossary', 'attachment',
$entry->id, array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));
}
</code>

===editor===
There are two ways of using the editor element in code, the first one is easier but expects some standardized fields. The second method is more low level.

====Simple use====
# name database fields: ''textfield'', ''textfieldformat'' (and ''textfieldtrust'' if required)
# create options array. note that context is the best, most local context you have available.
#:<code php>$textfieldoptions = array('trusttext'=>true, 'subdirs'=>true, 'maxfiles'=>$maxfiles, 'maxbytes'=>$maxbytes, 'context'=>$context);</code>
# add editor ''textfield_editor'' to moodle form, pass options through custom data in form constructor, set $data->id to null if data not exist yet
#:<code php>$mform->addElement('editor', 'textfield_editor', get_string('fieldname', 'somemodule'), null, $textfieldoptions);</code>
# prepare data
#:<code php>$data = file_prepare_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>
# get submitted data and after inserting/updating of data
#:<code php>$data = file_postupdate_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>

Real world examples are in mod/glossary/edit.php and mod/glossary/comment.php

====Low level use====

When using editor element you need to preprocess and postprocess the data:
# detect if form was already submitted (usually means draft is area already exists) - ''file_get_submitted_draft_itemid()''
# prepare draft file area, temporary storage of all files attached to the text - ''file_prepare_draft_area()''
# convert encoded relative links to absolute links - ''file_prepare_draft_area()''
# create form and set current data
# after submission the changed files must be merged back into original area - ''file_save_draft_area_files()''
# absolute links have to be replaced by relative links - ''file_save_draft_area_files()''

=====Replace old htmleditor with editor=====

The file picker has been integrated with with TinyMCE to make the editor element. This new element should support all types on editors and should be able to switch them on-the-fly. Instances of the old htmleditor element in your forms should be replaced by the new editor element, this may need adding of new format and trusttext columns. For example:
<code php>
$mform->addElement('editor', 'entry', get_string('definition', 'glossary'), null,
array('maxfiles' => EDITOR_UNLIMITED_FILES));
</code>
The editor element can take following options: maxfiles, maxbytes, subdirs and changeformat. Please note that the embedded files is optional feature and is not expected be used everywhere.

'''Note''': the editor element now includes text format option. You should no longer use the separate format element type.

=====Prepare current data - text and files=====

<code php>
if (empty($entry->id)) {
$entry = new object();
$entry->id = null;
$entry->definition = '';
$entry->format = FORMAT_HTML;
}

$draftid_editor = file_get_submitted_draft_itemid('entry');
$currenttext = file_prepare_draft_area($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $entry->definition);
$entry->entry = array('text'=>$currenttext, 'format'=>$entry->format, 'itemid'=>$draftid_editor);

$mform->set_data($entry);
</code>

If there are multiple files, they will share the same itemid.

=====Obtain text, format and save draft files=====

To retrieve editor content, you need to use following code:

<code php>
if ($fromform = $mform->get_data()) {
// content of editor
$messagetext = $fromform->entry['text'];
// format of content
$messageformat = $fromform->entry['format'];
}
</code>

When a user selects a file using the file picker, the file is initially stored in a draft file area, and a URL is inserted into the HTML in the editor that lets the person editing the content (but no one else) see the file.

When the user submits the form, we then need to save the draft files to the correct place in permanent storage. (Just like you have to call $DB->update_record('tablename', $data); to have the other parts of the form submission stored correctly.)

The save_files_from_draft_area function and replace absolute links with internal relative links do:
<code php>
$messagetext = file_save_draft_area_files($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $messagetext);
</code>
; $context->id, 'component', 'proper_file_area' and $entry->id : correspond to the contextid, filearea and itemid columns in the [[File_API#Table:_files|files table]].
; $messagetext : this is the message text. As the files are saved to the real file area, the URLs in this content are rewritten.

All URLs in content that point to files managed to the File API are converted to a form that starts '@@PLUGINFILE@@/' before the content is stored in the database. That is what we mean by rewriting.

== File serving==

=== Convert internal relative links to absolute links ===

Before text content is displayed to the user, any URLs in the '@@PLUGINFILE@@/' form in the content need to be rewritten to the real URL where the user can access the files.
<code php>
$messagetext = file_rewrite_pluginfile_urls($messagetext, 'pluginfile.php',
$context->id, 'mod_mymodule', 'proper_file_area', $itemid);
</code>
; $messagetext : is the content containing the @@PLUGINFILE@@ URLs from the database.
; 'pluginfile.php' : there are a number of different scripts that can serve files with different permissions checks. You need to specify which one to use.
; $context->id, 'mod_mymodule', 'proper_file_area', $itemid : uniquely identifies the file area, as before.

=== Implement file serving access control ===

Attachments and embedded images should have the same access control like the text itself, in majority of cases these files are served using pluginfile.php. Access control is defined in ''module/lib.php'' file in function ''module_pluginfile()''.

== File browsing support ==
Only owner of each file area is allowed to use low level File API function to access files, other parts of Moodle should use file browsing API.

Activities may specify browsing support in own module/lib.php file by implementing functions module_get_file_areas() and module_get_file_info().

== Upgrading your code ==
Here I will attempt to describe some simple steps you can take to upgrade your file-handling form elements from pre-2.0 code to 2.0. We will use the example of glossary, since it has been used above.

=== Preparing your options ===
Unless you are happy with the defaults, you will need to define an array of options for each file-handling form element. You could define it at different places, but it's best to put it in one place and make the array(s) available to other files if they need it. In the majority of cases, this will be in a file like edit.php

Previous code in mod/glossary/edit.php:
<code php>
$mform =& new mod_glossary_entry_form(null, compact('cm', 'glossary', 'hook', 'mode', 'e', 'context'));
</code>
New code:
<code php>
$maxbytes = $course->maxbytes;
// Could also use $CFG->maxbytes if you are not coding within a course context

$definitionoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes, 'trusttext'=>true,
'context'=>$context);
$attachmentoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Note that the data being passed to the form constructor have changed also, but this is not part of the file API changes, I just include them to avoid confusion.

These options are for the htmleditor (definition field) and the filemanager (attachment field). They are used by a file called edit_form.php.

=== Element preparation ===
Before we look at this, however, we need to "prepare" the elements so that they can correctly display existing embedded images and attached files when you are editing a record instead of just creating one. So, let's take the code we've got so far in edit.php and add to it:

Currently upgraded code in edit.php:
<code php>
$mform = new mod_glossary_entry_form(null, array(
'current'=>$entry,
'cm'=>$cm,
'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
New code with element preparation:
<code php>
$entry = new stdClass();
$entry->id = 0; \\ See notes!
$entry = file_prepare_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_prepare_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
$mform->set_data($entry);
</code>
Things to note:
* $entry in this case is simply a stdClass object which may either represent a new glossary entry or an existing one.
* $entry->id must be the unique identifier for the current object. If we are creating a new entry, it will be null, but in all cases it must be defined.
* These two functions (file_prepare_standard_editor and file_prepare_standard_filemanager) are shortcuts functions that take care of some of the tedious setting up for you, but they make a couple of assumptions:
*# You '''must''' name the form element as {element}_editor or {element}_filemanager (see next section)
*# You '''must''' have at least the following fields in the database: {element} and {element}summary, as described earlier in this documentation

We can now look at the upgrades needed in the form definition file.

=== Form definition ===
Previous code in mod/glossary/edit_form.php:
<code php>
$mform->addElement('htmleditor', 'definition', get_string('definition', 'glossary'),
array('rows'=>20));
$mform->setType('definition', PARAM_RAW);
$mform->addRule('definition', null, 'required', null, 'client');
$mform->setHelpButton('definition', array('writing', 'richtext'), false, 'editorhelpbutton');
$mform->addElement('format');
// a bit further...
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0,
true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->setHelpButton('attachment', array('attachment', get_string('attachment', 'glossary'),
'glossary'));
</code>
New code:
<code php>
$definitionoptions = $this->_customdata['definitionoptions'];
$attachmentoptions = $this->_customdata['attachmentoptions'];
// a bit further...
$mform->addElement('editor', 'definition_editor', get_string('definition', 'glossary'), null,
$definitionoptions);
$mform->setType('definition_editor', PARAM_RAW);
$mform->addRule('definition_editor', get_string('required'), 'required', null, 'client');
// a bit further...
$mform->addElement('filemanager', 'attachment_filemanager', get_string('attachment', 'glossary'),
null, $attachmentoptions);
$mform->setHelpButton('attachment_filemanager', array('attachment2',
get_string('attachment', 'glossary'), 'glossary'));
</code>

Note the following:
* The format element and the help button are no longer required for the HTML editor element
* The name of the form element needs to be changed by adding '_editor' or '_manager' to the original name. This is a naming convention that is used by a couple of functions we will look at shortly
* Make sure $definitionoptions has context parameter, else system context is used and editor will not respect filter settings.

=== Handling submitted data ===
The final step is to handle the submitted data properly, i.e. retrieve the files and save them to disk, associating them with the record we have just created (a glossary entry in our example). This happens in edit.php:

Previous code in edit.php:
<code php>
// Section that updates an entry:
$todb->id = $e;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
$todb->attachment = $newfilename;
}

// Section that adds an entry:
if ($todb->id = insert_record("glossary_entries", $todb)) {
$e = $todb->id;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
set_field("glossary_entries", "attachment", $newfilename, "id", $todb->id);
}
}
</code>
New code:
<code php>
// $todb was renamed to $entry, and the code was refactored
// so that the file-handling code is only used once for either an add or an update action.
// If an entry is being added, $DB->insert() has already been called, so we have a valid $entry->id
$entry = file_postupdate_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_postupdate_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
// store the updated value values
$DB->update_record('glossary_entries', $entry);
</code>
Things to note:
* If you are adding a new record, you will still need to call update_record after calling the file_postupdate* functions

=== Gotchas ===
A few things to keep in mind:
* Make sure that you instantiate the moodle form before any call to $OUTPUT->header()

== See also ==

* [[File API]]
* [[Using the file API]]
* [[Repository plugins]]
* [[Portfolio API]]
* MDL-14589 - File API Meta issue
* [https://moodle.org/mod/forum/discuss.php?d=207748 Adding a text editor to a Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=157953#p692822 Button actions in Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=351013#p1416554 File Picker]
* [https://github.com/CARLOSEDUARDOVIEIRA/moodle-uploadfile Example of filemanager in an activity module]

[[Category:Files]]
[[Category:Repositories]]

Using the File API in Moodle forms

2019-01-18T11:03:17Z

Howardsmiller: /* Element preparation */

{{Moodle 2.0}}

==Overview==

In Moodle 2.0 all files are stored in a central database accessible via the [[File API|File API]], and every file is associated with a component and a "file area" in Moodle, such as a particular module.

A common use case is to provide a form (using Moodle's [[lib/formslib.php|Forms API]]) which allows users to upload or import files as attachments or media embedded into HTML.

Normally this works like this:
# User starts creation or re-edits an existing item in Moodle (eg forum post, resource, glossary entry etc)
# User presses some sort of button to browse for new files to attach or embed
# User sees our "Choose file..." dialog, which contains one or more repository instances.
# User chooses a file, the corresponding [[Repository plugins|Repository plugin]] takes care of copying the file into a "draft file area" within Moodle
# File appears in the text or as an attachment in the form.
# When the user hits save, the [[File API|File API]] is invoked to move the file from the draft file area into a permanent file area associated with that data

This document shows you exactly how to use Moodle forms to interact with users in a standard and secure way.

If you just want to write code to manipulate Moodle files internally (without user input) then see [[File API]].

==Form elements==

In Moodle 2.0 there are three file-related form elements for interacting with users:

# filemanager - the way to attach one or more files as a set
# editor - the way to specify a textarea with a HTML editor, and all the handling of images and movies within that HTML
# filepicker - a way to specify one file for the case when you want to process the file and throw it away

In Moodle 1.9 there were two other types which are now '''deprecated''' (they work, but please do not use these anymore)
# file - used to just allow a normal file upload from the desktop only.
# htmleditor - this old method of embedding a HTML editor in a textarea is not able to support repositories etc.

===filepicker===

File picker (''filepicker'') is a direct replacement of the older ''file'' formslib element.

It is intended for situations when you want the user to upload '''one''' file so you can process it and delete it, such as when you are importing data from a CSV file.
If you want a file that remains part of the Moodle storage and will reappear when you reopen the form, then you should use a filemanager instead (and restrict it to a single file, if necessary).

==== Using the filepicker element ====

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null,
array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>

==== Obtain the chosen file ====

To get the contents of the file:

<code php>
$content = $mform->get_file_content('userfile');
</code>

To get the name of the chosen file:

<code php>
$name = $mform->get_new_filename('userfile');
</code>

To save the chosen file to the server filesystem (such as to moodledata folder):

<code php>
$success = $mform->save_file('userfile', $fullpath, $override);
</code>

To store the chosen file in the Moodle files pool:

<code php>
$storedfile = $mform->save_stored_file('userfile', ...);
</code>

=== filemanager ===

The File Manager element improves on file picker by allowing you to manage more than one file. It is expected that the files will be stored permanently for future use (such as forum and glossary attachments).

==== Add file manager element ====

Example:
<code php>
$mform->addElement('filemanager', 'attachments', get_string('attachment', 'moodle'), null,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'areamaxbytes' => 10485760, 'maxfiles' => 50,
'accepted_types' => array('document'), 'return_types'=> FILE_INTERNAL | FILE_EXTERNAL));
</code>

Here are the fields for filemanager:

;'filemanager':This is a filemanager element :)
;elementname:The unique name of the element in the form
;elementlabel:The label string that users see
;attributes:(leave it as null)
;options: an array of further options for the filepicker (see below)

The options array can contain:

;subdirs:(Default 1) Are subdirectories allowed? (true or false)
;maxbytes:(Default 0) Restricts the size of each individual file.
;areamaxbytes:(Default 0) Restricts the total size of all the files.
;maxfiles:(Default -1) Restricts the total number of files.
;accepted_types:(Default *) You can specify what file types are accepted by filemanager. All current file types are listed in this file: [http://cvs.moodle.org/moodle/lib/filestorage/file_types.mm moodle/lib/filestorage/file_types.mm]. This is a [http://freemind.sourceforge.net/wiki/index.php/Main_Page freemind] file: if it is edited the changes will be immediately reflected in Moodle.

As of 2.3 file_types.mm is no longer used, instead the file types are listed in <code>get_mimetypes_array()</code> in lib/classes/filetypes.php (https://github.com/moodle/moodle/blob/master/lib/classes/filetypes.php).

Example usage: '''array('audio', 'video', 'document')''', you can include file extensions as well, for example: '''array('.txt', '.jpg', 'audio')'''.

==== Load existing files into draft area ====

<code php>
if (empty($entry->id)) {
$entry = new stdClass;
$entry->id = null;
}

$draftitemid = file_get_submitted_draft_itemid('attachments');

file_prepare_draft_area($draftitemid, $context->id, 'mod_glossary', 'attachment', $entry->id,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));

$entry->attachments = $draftitemid;

$mform->set_data($entry);

</code>

==== Store updated set of files ====

<code php>
if ($data = $mform->get_data()) {
// ... store or update $entry
file_save_draft_area_files($data->attachments, $context->id, 'mod_glossary', 'attachment',
$entry->id, array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));
}
</code>

===editor===
There are two ways of using the editor element in code, the first one is easier but expects some standardized fields. The second method is more low level.

====Simple use====
# name database fields: ''textfield'', ''textfieldformat'' (and ''textfieldtrust'' if required)
# create options array. note that context is the best, most local context you have available.
#:<code php>$textfieldoptions = array('trusttext'=>true, 'subdirs'=>true, 'maxfiles'=>$maxfiles, 'maxbytes'=>$maxbytes, 'context'=>$context);</code>
# add editor ''textfield_editor'' to moodle form, pass options through custom data in form constructor, set $data->id to null if data not exist yet
#:<code php>$mform->addElement('editor', 'textfield_editor', get_string('fieldname', 'somemodule'), null, $textfieldoptions);</code>
# prepare data
#:<code php>$data = file_prepare_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>
# get submitted data and after inserting/updating of data
#:<code php>$data = file_postupdate_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>

Real world examples are in mod/glossary/edit.php and mod/glossary/comment.php

====Low level use====

When using editor element you need to preprocess and postprocess the data:
# detect if form was already submitted (usually means draft is area already exists) - ''file_get_submitted_draft_itemid()''
# prepare draft file area, temporary storage of all files attached to the text - ''file_prepare_draft_area()''
# convert encoded relative links to absolute links - ''file_prepare_draft_area()''
# create form and set current data
# after submission the changed files must be merged back into original area - ''file_save_draft_area_files()''
# absolute links have to be replaced by relative links - ''file_save_draft_area_files()''

=====Replace old htmleditor with editor=====

The file picker has been integrated with with TinyMCE to make the editor element. This new element should support all types on editors and should be able to switch them on-the-fly. Instances of the old htmleditor element in your forms should be replaced by the new editor element, this may need adding of new format and trusttext columns. For example:
<code php>
$mform->addElement('editor', 'entry', get_string('definition', 'glossary'), null,
array('maxfiles' => EDITOR_UNLIMITED_FILES));
</code>
The editor element can take following options: maxfiles, maxbytes, subdirs and changeformat. Please note that the embedded files is optional feature and is not expected be used everywhere.

'''Note''': the editor element now includes text format option. You should no longer use the separate format element type.

=====Prepare current data - text and files=====

<code php>
if (empty($entry->id)) {
$entry = new object();
$entry->id = null;
$entry->definition = '';
$entry->format = FORMAT_HTML;
}

$draftid_editor = file_get_submitted_draft_itemid('entry');
$currenttext = file_prepare_draft_area($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $entry->definition);
$entry->entry = array('text'=>$currenttext, 'format'=>$entry->format, 'itemid'=>$draftid_editor);

$mform->set_data($entry);
</code>

If there are multiple files, they will share the same itemid.

=====Obtain text, format and save draft files=====

To retrieve editor content, you need to use following code:

<code php>
if ($fromform = $mform->get_data()) {
// content of editor
$messagetext = $fromform->entry['text'];
// format of content
$messageformat = $fromform->entry['format'];
}
</code>

When a user selects a file using the file picker, the file is initially stored in a draft file area, and a URL is inserted into the HTML in the editor that lets the person editing the content (but no one else) see the file.

When the user submits the form, we then need to save the draft files to the correct place in permanent storage. (Just like you have to call $DB->update_record('tablename', $data); to have the other parts of the form submission stored correctly.)

The save_files_from_draft_area function and replace absolute links with internal relative links do:
<code php>
$messagetext = file_save_draft_area_files($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $messagetext);
</code>
; $context->id, 'component', 'proper_file_area' and $entry->id : correspond to the contextid, filearea and itemid columns in the [[File_API#Table:_files|files table]].
; $messagetext : this is the message text. As the files are saved to the real file area, the URLs in this content are rewritten.

All URLs in content that point to files managed to the File API are converted to a form that starts '@@PLUGINFILE@@/' before the content is stored in the database. That is what we mean by rewriting.

== File serving==

=== Convert internal relative links to absolute links ===

Before text content is displayed to the user, any URLs in the '@@PLUGINFILE@@/' form in the content need to be rewritten to the real URL where the user can access the files.
<code php>
$messagetext = file_rewrite_pluginfile_urls($messagetext, 'pluginfile.php',
$context->id, 'mod_mymodule', 'proper_file_area', $itemid);
</code>
; $messagetext : is the content containing the @@PLUGINFILE@@ URLs from the database.
; 'pluginfile.php' : there are a number of different scripts that can serve files with different permissions checks. You need to specify which one to use.
; $context->id, 'mod_mymodule', 'proper_file_area', $itemid : uniquely identifies the file area, as before.

=== Implement file serving access control ===

Attachments and embedded images should have the same access control like the text itself, in majority of cases these files are served using pluginfile.php. Access control is defined in ''module/lib.php'' file in function ''module_pluginfile()''.

== File browsing support ==
Only owner of each file area is allowed to use low level File API function to access files, other parts of Moodle should use file browsing API.

Activities may specify browsing support in own module/lib.php file by implementing functions module_get_file_areas() and module_get_file_info().

== Upgrading your code ==
Here I will attempt to describe some simple steps you can take to upgrade your file-handling form elements from pre-2.0 code to 2.0. We will use the example of glossary, since it has been used above.

=== Preparing your options ===
Unless you are happy with the defaults, you will need to define an array of options for each file-handling form element. You could define it at different places, but it's best to put it in one place and make the array(s) available to other files if they need it. In the majority of cases, this will be in a file like edit.php

Previous code in mod/glossary/edit.php:
<code php>
$mform =& new mod_glossary_entry_form(null, compact('cm', 'glossary', 'hook', 'mode', 'e', 'context'));
</code>
New code:
<code php>
$maxbytes = $course->maxbytes;
// Could also use $CFG->maxbytes if you are not coding within a course context

$definitionoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes, 'trusttext'=>true,
'context'=>$context);
$attachmentoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Note that the data being passed to the form constructor have changed also, but this is not part of the file API changes, I just include them to avoid confusion.

These options are for the htmleditor (definition field) and the filemanager (attachment field). They are used by a file called edit_form.php.

=== Element preparation ===
Before we look at this, however, we need to "prepare" the elements so that they can correctly display existing embedded images and attached files when you are editing a record instead of just creating one. So, let's take the code we've got so far in edit.php and add to it:

Currently upgraded code in edit.php:
<code php>
$mform = new mod_glossary_entry_form(null, array(
'current'=>$entry,
'cm'=>$cm,
'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
New code with element preparation:
<code php>
$entry = new stdClass();
$entry->id = 0; \\ See notes!
$entry = file_prepare_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_prepare_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Things to note:
* $entry in this case is simply a stdClass object which may either represent a new glossary entry or an existing one.
* $entry->id must be the unique identifier for the current object. If we are creating a new entry, it will be null, but in all cases it must be defined.
* These two functions (file_prepare_standard_editor and file_prepare_standard_filemanager) are shortcuts functions that take care of some of the tedious setting up for you, but they make a couple of assumptions:
*# You '''must''' name the form element as {element}_editor or {element}_filemanager (see next section)
*# You '''must''' have at least the following fields in the database: {element} and {element}summary, as described earlier in this documentation

We can now look at the upgrades needed in the form definition file.

=== Form definition ===
Previous code in mod/glossary/edit_form.php:
<code php>
$mform->addElement('htmleditor', 'definition', get_string('definition', 'glossary'),
array('rows'=>20));
$mform->setType('definition', PARAM_RAW);
$mform->addRule('definition', null, 'required', null, 'client');
$mform->setHelpButton('definition', array('writing', 'richtext'), false, 'editorhelpbutton');
$mform->addElement('format');
// a bit further...
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0,
true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->setHelpButton('attachment', array('attachment', get_string('attachment', 'glossary'),
'glossary'));
</code>
New code:
<code php>
$definitionoptions = $this->_customdata['definitionoptions'];
$attachmentoptions = $this->_customdata['attachmentoptions'];
// a bit further...
$mform->addElement('editor', 'definition_editor', get_string('definition', 'glossary'), null,
$definitionoptions);
$mform->setType('definition_editor', PARAM_RAW);
$mform->addRule('definition_editor', get_string('required'), 'required', null, 'client');
// a bit further...
$mform->addElement('filemanager', 'attachment_filemanager', get_string('attachment', 'glossary'),
null, $attachmentoptions);
$mform->setHelpButton('attachment_filemanager', array('attachment2',
get_string('attachment', 'glossary'), 'glossary'));
</code>

Note the following:
* The format element and the help button are no longer required for the HTML editor element
* The name of the form element needs to be changed by adding '_editor' or '_manager' to the original name. This is a naming convention that is used by a couple of functions we will look at shortly
* Make sure $definitionoptions has context parameter, else system context is used and editor will not respect filter settings.

=== Handling submitted data ===
The final step is to handle the submitted data properly, i.e. retrieve the files and save them to disk, associating them with the record we have just created (a glossary entry in our example). This happens in edit.php:

Previous code in edit.php:
<code php>
// Section that updates an entry:
$todb->id = $e;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
$todb->attachment = $newfilename;
}

// Section that adds an entry:
if ($todb->id = insert_record("glossary_entries", $todb)) {
$e = $todb->id;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
set_field("glossary_entries", "attachment", $newfilename, "id", $todb->id);
}
}
</code>
New code:
<code php>
// $todb was renamed to $entry, and the code was refactored
// so that the file-handling code is only used once for either an add or an update action.
// If an entry is being added, $DB->insert() has already been called, so we have a valid $entry->id
$entry = file_postupdate_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_postupdate_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
// store the updated value values
$DB->update_record('glossary_entries', $entry);
</code>
Things to note:
* If you are adding a new record, you will still need to call update_record after calling the file_postupdate* functions

=== Gotchas ===
A few things to keep in mind:
* Make sure that you instantiate the moodle form before any call to $OUTPUT->header()

== See also ==

* [[File API]]
* [[Using the file API]]
* [[Repository plugins]]
* [[Portfolio API]]
* MDL-14589 - File API Meta issue
* [https://moodle.org/mod/forum/discuss.php?d=207748 Adding a text editor to a Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=157953#p692822 Button actions in Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=351013#p1416554 File Picker]
* [https://github.com/CARLOSEDUARDOVIEIRA/moodle-uploadfile Example of filemanager in an activity module]

[[Category:Files]]
[[Category:Repositories]]

Using the File API in Moodle forms

2019-01-18T11:02:01Z

Howardsmiller: /* Element preparation */

{{Moodle 2.0}}

==Overview==

In Moodle 2.0 all files are stored in a central database accessible via the [[File API|File API]], and every file is associated with a component and a "file area" in Moodle, such as a particular module.

A common use case is to provide a form (using Moodle's [[lib/formslib.php|Forms API]]) which allows users to upload or import files as attachments or media embedded into HTML.

Normally this works like this:
# User starts creation or re-edits an existing item in Moodle (eg forum post, resource, glossary entry etc)
# User presses some sort of button to browse for new files to attach or embed
# User sees our "Choose file..." dialog, which contains one or more repository instances.
# User chooses a file, the corresponding [[Repository plugins|Repository plugin]] takes care of copying the file into a "draft file area" within Moodle
# File appears in the text or as an attachment in the form.
# When the user hits save, the [[File API|File API]] is invoked to move the file from the draft file area into a permanent file area associated with that data

This document shows you exactly how to use Moodle forms to interact with users in a standard and secure way.

If you just want to write code to manipulate Moodle files internally (without user input) then see [[File API]].

==Form elements==

In Moodle 2.0 there are three file-related form elements for interacting with users:

# filemanager - the way to attach one or more files as a set
# editor - the way to specify a textarea with a HTML editor, and all the handling of images and movies within that HTML
# filepicker - a way to specify one file for the case when you want to process the file and throw it away

In Moodle 1.9 there were two other types which are now '''deprecated''' (they work, but please do not use these anymore)
# file - used to just allow a normal file upload from the desktop only.
# htmleditor - this old method of embedding a HTML editor in a textarea is not able to support repositories etc.

===filepicker===

File picker (''filepicker'') is a direct replacement of the older ''file'' formslib element.

It is intended for situations when you want the user to upload '''one''' file so you can process it and delete it, such as when you are importing data from a CSV file.
If you want a file that remains part of the Moodle storage and will reappear when you reopen the form, then you should use a filemanager instead (and restrict it to a single file, if necessary).

==== Using the filepicker element ====

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null,
array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>

==== Obtain the chosen file ====

To get the contents of the file:

<code php>
$content = $mform->get_file_content('userfile');
</code>

To get the name of the chosen file:

<code php>
$name = $mform->get_new_filename('userfile');
</code>

To save the chosen file to the server filesystem (such as to moodledata folder):

<code php>
$success = $mform->save_file('userfile', $fullpath, $override);
</code>

To store the chosen file in the Moodle files pool:

<code php>
$storedfile = $mform->save_stored_file('userfile', ...);
</code>

=== filemanager ===

The File Manager element improves on file picker by allowing you to manage more than one file. It is expected that the files will be stored permanently for future use (such as forum and glossary attachments).

==== Add file manager element ====

Example:
<code php>
$mform->addElement('filemanager', 'attachments', get_string('attachment', 'moodle'), null,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'areamaxbytes' => 10485760, 'maxfiles' => 50,
'accepted_types' => array('document'), 'return_types'=> FILE_INTERNAL | FILE_EXTERNAL));
</code>

Here are the fields for filemanager:

;'filemanager':This is a filemanager element :)
;elementname:The unique name of the element in the form
;elementlabel:The label string that users see
;attributes:(leave it as null)
;options: an array of further options for the filepicker (see below)

The options array can contain:

;subdirs:(Default 1) Are subdirectories allowed? (true or false)
;maxbytes:(Default 0) Restricts the size of each individual file.
;areamaxbytes:(Default 0) Restricts the total size of all the files.
;maxfiles:(Default -1) Restricts the total number of files.
;accepted_types:(Default *) You can specify what file types are accepted by filemanager. All current file types are listed in this file: [http://cvs.moodle.org/moodle/lib/filestorage/file_types.mm moodle/lib/filestorage/file_types.mm]. This is a [http://freemind.sourceforge.net/wiki/index.php/Main_Page freemind] file: if it is edited the changes will be immediately reflected in Moodle.

As of 2.3 file_types.mm is no longer used, instead the file types are listed in <code>get_mimetypes_array()</code> in lib/classes/filetypes.php (https://github.com/moodle/moodle/blob/master/lib/classes/filetypes.php).

Example usage: '''array('audio', 'video', 'document')''', you can include file extensions as well, for example: '''array('.txt', '.jpg', 'audio')'''.

==== Load existing files into draft area ====

<code php>
if (empty($entry->id)) {
$entry = new stdClass;
$entry->id = null;
}

$draftitemid = file_get_submitted_draft_itemid('attachments');

file_prepare_draft_area($draftitemid, $context->id, 'mod_glossary', 'attachment', $entry->id,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));

$entry->attachments = $draftitemid;

$mform->set_data($entry);

</code>

==== Store updated set of files ====

<code php>
if ($data = $mform->get_data()) {
// ... store or update $entry
file_save_draft_area_files($data->attachments, $context->id, 'mod_glossary', 'attachment',
$entry->id, array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));
}
</code>

===editor===
There are two ways of using the editor element in code, the first one is easier but expects some standardized fields. The second method is more low level.

====Simple use====
# name database fields: ''textfield'', ''textfieldformat'' (and ''textfieldtrust'' if required)
# create options array. note that context is the best, most local context you have available.
#:<code php>$textfieldoptions = array('trusttext'=>true, 'subdirs'=>true, 'maxfiles'=>$maxfiles, 'maxbytes'=>$maxbytes, 'context'=>$context);</code>
# add editor ''textfield_editor'' to moodle form, pass options through custom data in form constructor, set $data->id to null if data not exist yet
#:<code php>$mform->addElement('editor', 'textfield_editor', get_string('fieldname', 'somemodule'), null, $textfieldoptions);</code>
# prepare data
#:<code php>$data = file_prepare_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>
# get submitted data and after inserting/updating of data
#:<code php>$data = file_postupdate_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>

Real world examples are in mod/glossary/edit.php and mod/glossary/comment.php

====Low level use====

When using editor element you need to preprocess and postprocess the data:
# detect if form was already submitted (usually means draft is area already exists) - ''file_get_submitted_draft_itemid()''
# prepare draft file area, temporary storage of all files attached to the text - ''file_prepare_draft_area()''
# convert encoded relative links to absolute links - ''file_prepare_draft_area()''
# create form and set current data
# after submission the changed files must be merged back into original area - ''file_save_draft_area_files()''
# absolute links have to be replaced by relative links - ''file_save_draft_area_files()''

=====Replace old htmleditor with editor=====

The file picker has been integrated with with TinyMCE to make the editor element. This new element should support all types on editors and should be able to switch them on-the-fly. Instances of the old htmleditor element in your forms should be replaced by the new editor element, this may need adding of new format and trusttext columns. For example:
<code php>
$mform->addElement('editor', 'entry', get_string('definition', 'glossary'), null,
array('maxfiles' => EDITOR_UNLIMITED_FILES));
</code>
The editor element can take following options: maxfiles, maxbytes, subdirs and changeformat. Please note that the embedded files is optional feature and is not expected be used everywhere.

'''Note''': the editor element now includes text format option. You should no longer use the separate format element type.

=====Prepare current data - text and files=====

<code php>
if (empty($entry->id)) {
$entry = new object();
$entry->id = null;
$entry->definition = '';
$entry->format = FORMAT_HTML;
}

$draftid_editor = file_get_submitted_draft_itemid('entry');
$currenttext = file_prepare_draft_area($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $entry->definition);
$entry->entry = array('text'=>$currenttext, 'format'=>$entry->format, 'itemid'=>$draftid_editor);

$mform->set_data($entry);
</code>

If there are multiple files, they will share the same itemid.

=====Obtain text, format and save draft files=====

To retrieve editor content, you need to use following code:

<code php>
if ($fromform = $mform->get_data()) {
// content of editor
$messagetext = $fromform->entry['text'];
// format of content
$messageformat = $fromform->entry['format'];
}
</code>

When a user selects a file using the file picker, the file is initially stored in a draft file area, and a URL is inserted into the HTML in the editor that lets the person editing the content (but no one else) see the file.

When the user submits the form, we then need to save the draft files to the correct place in permanent storage. (Just like you have to call $DB->update_record('tablename', $data); to have the other parts of the form submission stored correctly.)

The save_files_from_draft_area function and replace absolute links with internal relative links do:
<code php>
$messagetext = file_save_draft_area_files($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $messagetext);
</code>
; $context->id, 'component', 'proper_file_area' and $entry->id : correspond to the contextid, filearea and itemid columns in the [[File_API#Table:_files|files table]].
; $messagetext : this is the message text. As the files are saved to the real file area, the URLs in this content are rewritten.

All URLs in content that point to files managed to the File API are converted to a form that starts '@@PLUGINFILE@@/' before the content is stored in the database. That is what we mean by rewriting.

== File serving==

=== Convert internal relative links to absolute links ===

Before text content is displayed to the user, any URLs in the '@@PLUGINFILE@@/' form in the content need to be rewritten to the real URL where the user can access the files.
<code php>
$messagetext = file_rewrite_pluginfile_urls($messagetext, 'pluginfile.php',
$context->id, 'mod_mymodule', 'proper_file_area', $itemid);
</code>
; $messagetext : is the content containing the @@PLUGINFILE@@ URLs from the database.
; 'pluginfile.php' : there are a number of different scripts that can serve files with different permissions checks. You need to specify which one to use.
; $context->id, 'mod_mymodule', 'proper_file_area', $itemid : uniquely identifies the file area, as before.

=== Implement file serving access control ===

Attachments and embedded images should have the same access control like the text itself, in majority of cases these files are served using pluginfile.php. Access control is defined in ''module/lib.php'' file in function ''module_pluginfile()''.

== File browsing support ==
Only owner of each file area is allowed to use low level File API function to access files, other parts of Moodle should use file browsing API.

Activities may specify browsing support in own module/lib.php file by implementing functions module_get_file_areas() and module_get_file_info().

== Upgrading your code ==
Here I will attempt to describe some simple steps you can take to upgrade your file-handling form elements from pre-2.0 code to 2.0. We will use the example of glossary, since it has been used above.

=== Preparing your options ===
Unless you are happy with the defaults, you will need to define an array of options for each file-handling form element. You could define it at different places, but it's best to put it in one place and make the array(s) available to other files if they need it. In the majority of cases, this will be in a file like edit.php

Previous code in mod/glossary/edit.php:
<code php>
$mform =& new mod_glossary_entry_form(null, compact('cm', 'glossary', 'hook', 'mode', 'e', 'context'));
</code>
New code:
<code php>
$maxbytes = $course->maxbytes;
// Could also use $CFG->maxbytes if you are not coding within a course context

$definitionoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes, 'trusttext'=>true,
'context'=>$context);
$attachmentoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Note that the data being passed to the form constructor have changed also, but this is not part of the file API changes, I just include them to avoid confusion.

These options are for the htmleditor (definition field) and the filemanager (attachment field). They are used by a file called edit_form.php.

=== Element preparation ===
Before we look at this, however, we need to "prepare" the elements so that they can correctly display existing embedded images and attached files when you are editing a record instead of just creating one. So, let's take the code we've got so far in edit.php and add to it:

Currently upgraded code in edit.php:
<code php>
$mform = new mod_glossary_entry_form(null, array(
'current'=>$entry,
'cm'=>$cm,
'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
New code with element preparation:
<code php>
$entry = new stdClass();
$entry = file_prepare_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_prepare_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Things to note:
* $entry in this case is simply a stdClass object which may either represent a new glossary entry or an existing one.
* $entry->id must be the unique identifier for the current object. If we are creating a new entry, it will be null, but in all cases it must be defined.
* These two functions (file_prepare_standard_editor and file_prepare_standard_filemanager) are shortcuts functions that take care of some of the tedious setting up for you, but they make a couple of assumptions:
*# You '''must''' name the form element as {element}_editor or {element}_filemanager (see next section)
*# You '''must''' have at least the following fields in the database: {element} and {element}summary, as described earlier in this documentation

We can now look at the upgrades needed in the form definition file.

=== Form definition ===
Previous code in mod/glossary/edit_form.php:
<code php>
$mform->addElement('htmleditor', 'definition', get_string('definition', 'glossary'),
array('rows'=>20));
$mform->setType('definition', PARAM_RAW);
$mform->addRule('definition', null, 'required', null, 'client');
$mform->setHelpButton('definition', array('writing', 'richtext'), false, 'editorhelpbutton');
$mform->addElement('format');
// a bit further...
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0,
true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->setHelpButton('attachment', array('attachment', get_string('attachment', 'glossary'),
'glossary'));
</code>
New code:
<code php>
$definitionoptions = $this->_customdata['definitionoptions'];
$attachmentoptions = $this->_customdata['attachmentoptions'];
// a bit further...
$mform->addElement('editor', 'definition_editor', get_string('definition', 'glossary'), null,
$definitionoptions);
$mform->setType('definition_editor', PARAM_RAW);
$mform->addRule('definition_editor', get_string('required'), 'required', null, 'client');
// a bit further...
$mform->addElement('filemanager', 'attachment_filemanager', get_string('attachment', 'glossary'),
null, $attachmentoptions);
$mform->setHelpButton('attachment_filemanager', array('attachment2',
get_string('attachment', 'glossary'), 'glossary'));
</code>

Note the following:
* The format element and the help button are no longer required for the HTML editor element
* The name of the form element needs to be changed by adding '_editor' or '_manager' to the original name. This is a naming convention that is used by a couple of functions we will look at shortly
* Make sure $definitionoptions has context parameter, else system context is used and editor will not respect filter settings.

=== Handling submitted data ===
The final step is to handle the submitted data properly, i.e. retrieve the files and save them to disk, associating them with the record we have just created (a glossary entry in our example). This happens in edit.php:

Previous code in edit.php:
<code php>
// Section that updates an entry:
$todb->id = $e;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
$todb->attachment = $newfilename;
}

// Section that adds an entry:
if ($todb->id = insert_record("glossary_entries", $todb)) {
$e = $todb->id;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
set_field("glossary_entries", "attachment", $newfilename, "id", $todb->id);
}
}
</code>
New code:
<code php>
// $todb was renamed to $entry, and the code was refactored
// so that the file-handling code is only used once for either an add or an update action.
// If an entry is being added, $DB->insert() has already been called, so we have a valid $entry->id
$entry = file_postupdate_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_postupdate_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
// store the updated value values
$DB->update_record('glossary_entries', $entry);
</code>
Things to note:
* If you are adding a new record, you will still need to call update_record after calling the file_postupdate* functions

=== Gotchas ===
A few things to keep in mind:
* Make sure that you instantiate the moodle form before any call to $OUTPUT->header()

== See also ==

* [[File API]]
* [[Using the file API]]
* [[Repository plugins]]
* [[Portfolio API]]
* MDL-14589 - File API Meta issue
* [https://moodle.org/mod/forum/discuss.php?d=207748 Adding a text editor to a Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=157953#p692822 Button actions in Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=351013#p1416554 File Picker]
* [https://github.com/CARLOSEDUARDOVIEIRA/moodle-uploadfile Example of filemanager in an activity module]

[[Category:Files]]
[[Category:Repositories]]

Moodle 3.6 release notes

2018-12-04T15:38:30Z

Howardsmiller: /* For administrators */

[[Releases]] > {{FULLPAGENAME}}

Release date: 3 December 2018

Here is [https://tracker.moodle.org/secure/IssueNavigator!executeAdvanced.jspa?jqlQuery=project+%3D+mdl+AND+resolution+%3D+fixed+AND+fixVersion+in+%28%223.6%22%29+ORDER+BY+priority+DESC&runQuery=true&clear=true the full list of fixed issues in 3.6].

See our [https://docs.moodle.org/36/en/New_features New Features page] for a more user-friendly introduction to Moodle 3.6 with screenshots.

{{Note|If you are upgrading from a previous version, please see [[:en:Upgrading|Upgrading]] in the user docs. ''In particular, for sites using a custom theme or login form, from 3.6 onwards, the login form must include a new login token field. See [[Login token]] for details.}}

==Server requirements==

These are just the minimum supported versions. We recommend keeping all of your software and operating systems up-to-date.

* Moodle upgrade: Moodle 3.1 or later
* PHP version: minimum PHP 7.0.0 ''Note: minimum PHP version has increased since Moodle 3.3''. PHP 7.1.x and 7.2.x are supported too. [[Moodle and PHP7|PHP 7.3.x support]] is being implemented (@ MDL-63420) and '''not ready for production''' with this release.
* PHP extension '''intl''' is required since Moodle 3.4 (it was recommended in 2.0 onwards)

=== Database requirements ===

Moodle supports the following database servers. Again, version numbers are just the minimum supported version. We recommend running the latest stable version of any software.

{| class="nicetable"
|-
! Database
! Minimum version
! Recommended
|-
| [http://www.postgresql.org/ PostgreSQL]
| 9.4
| Latest
|-
| [http://www.mysql.com/ MySQL]
| 5.6
| Latest
|-
| [https://mariadb.org/ MariaDB]
| 5.5.31
| Latest
|-
| [http://www.microsoft.com/en-us/server-cloud/products/sql-server/ Microsoft SQL Server]
| 2008
| Latest
|-
| [http://www.oracle.com/us/products/database/overview/index.html Oracle Database]
| 11.2
| Latest
|}

==Client requirements==

=== Browser support ===
Moodle is compatible with any standards compliant web browser. We regularly test Moodle with the following browsers:

Desktop:
* Chrome
* Firefox
* Safari
* Edge
* Internet Explorer

Mobile:
* MobileSafari
* Google Chrome

For the best experience and optimum security, we recommend that you keep your browser up to date. https://whatbrowser.org

Note: Legacy browsers with known compatibility issues with Moodle 3.6:
* Internet Explorer 10 and below
* Safari 7 and below

==Major features==

===Dashboard and Course overview===
* MDL-63044 and MDL-63337 - New [https://docs.moodle.org/36/en/Course_overview Course overview] and [https://docs.moodle.org/36/en/Timeline_block Timeline block]
* MDL-63062 - New [https://docs.moodle.org/36/en/Recently_accessed_courses_block Recently accessed courses block]
* MDL-63063 - New [https://docs.moodle.org/36/en/Recently_accessed_items_block Recently accessed items block]
* MDL-63457 - Option to hide courses in the course overview block
* MDL-63058 - Option to star/unstar courses in the course overview block
* MDL-63064 - New [https://docs.moodle.org/36/en/Starred_courses_block Starred courses block]
* MDL-63352 - Dashboard retains user preferences for view options
* MDL-63793 - Course overview block retains user preferences for the number of courses to show
* MDL-61161 - Grace period when displaying "In progress" courses in course overview block
* MDL-63040 - Removal of Dashboard page header

===GDPR and Privacy===
Note that some of these GDPR improvements have also been backported to Moodle 3.5.3, 3.4.6 and 3.3.9.

* MDL-63116 - Data requests bulk actions
* MDL-62309 - Option to make site policies required or optional
* MDL-61652 - Capabilities for controlling who can download SAR data
* MDL-62563 - Data deletion of existing deleted users
* MDL-63897 - Pre-processing stage removed from data requests process
* MDL-62558 - Data retention summary (read-only)
* MDL-63726 - Option to remove the "Data retention summary" link in the footer
* MDL-62491 - HTML data request export format
* MDL-63401 - User expiry improvements
* MDL-63619 - Data purpose and category inheritance improvements
* MDL-62560 - Different data retention strategies for different roles in a purpose
* MDL-62554 - Ability to configure data registry to use module type defaults
* MDL-63009 - Site mentioned in email notifications of data requests
* MDL-6074 - Option to hide your name in the [https://docs.moodle.org/36/en/Online_users_block online users block]

===Messaging===
* '''MDL-57272 and MDL-63280 - Group messaging'''
* MDL-63303 - New messaging UI with [https://docs.moodle.org/36/en/Messaging messaging drawer]
* MDL-63279 - Option to [https://docs.moodle.org/36/en/Messaging_settings disable site-wide messaging]
* MDL-63214 - Privacy setting for restricting who can message you
* MDL-63213 - Option to star messaging conversations
* MDL-63283 - Notifications not sent for group conversations
* MDL-63281 - Group members synchronised with messaging conversations members

===Assignment===
* MDL-27520 - Assignment feedback can include media or other files

===Quiz===
* MDL-62610 - Improved quiz statistics report usability for randomized questions
* MDL-62708 - Option to add ID numbers to questions and question categories
* MDL-63738 - Single questions can be exported from the question bank

===Workshop===
* MDL-60820 - Teachers can specify workshop submission types

===Repositories===
* MDL-58943 - Nextcloud integration, with a [https://docs.moodle.org/36/en/Nextcloud_repository Nextcloud repository] and [https://docs.moodle.org/36/en/OAuth_2_Nextcloud_service OAuth 2 Nextcloud service]

===Open Badges===
* MDL-58454 - Support for Open Badges v2.0

===Performance===
* MDL-54035 - Performance improvements to cache flags
* MDL-47962 - Glossary auto-linking filter performance improvements

===Usability improvements===
* MDL-51177 - atto_htmlplus implemented to improve Atto editor HTML indenting
* MDL-45170 - Copy and paste of images from one WYSIWYG window to another
* MDL-61388 - Forum actions announced by screen reader when completed
* MDL-62899 - Global search displays a relevant icon next to link in results
* MDL-46415 - SVG/high resolution emoticons
* MDL-58000 - Larger badge images are used

===Experimental===
* MDL-53566 - [https://docs.moodle.org/36/en/Context_freezing Context freezing] - setting read-only access for categories, courses, activities and their content

==Other highlights==

===Functional changes===
* MDL-17943 - 'Resend confirmation email' button on login page
* MDL-14274 - IF conditions in grade calculations
* MDL-37624 - Calendar entries location support
* MDL-36754 - Images are displayed in forum notification emails
* MDL-59259 - Course format options may be specified in upload courses CSV file
* MDL-41265 - Page resource option to show/hide "Last modified"
* MDL-61378 - Forum post HTML structure improvements
* MDL-59454 - Option to download the list of course participants
* MDL-60520 - Analytics models can use different machine learning backends
* MDL-61573 - User menu: customusermenuitems map Font Awesome icons for non pix/t folders
* MDL-62320 - JSON added to the default MIME types list
* MDL-63431 - Atto media plugin title global attribute support
* MDL-60435 - Shibboleth authentication identity providers
* MDL-59169 - Grader report saves after edit with multiple tabs
* MDL-62960 - Drag and drop of course events respects the course start date

===Security issues===
* [https://moodle.org/mod/forum/discuss.php?d=378731 MSA-18-0020] Login CSRF vulnerability in login form. Note that this fix has previously been disclosed following the release of Moodle 3.5.3, 3.4.6, 3.3.9 and 3.1.15.

===For administrators===
* MDL-62334 - 'Add a new course' link in Site administration
* MDL-63253 - Admin search results provide location of the found matching page
* MDL-63772 - Capability to control use of Atto Record RTC
* MDL-63708 - New blocks supported by the mobile app can be disabled
* MDL-52953 - Legacy log store deprecation
* MDL-59429 - Log changes to site administrators
* MDL-62651 - adhoc task runner
* MDL-62777 - Site upgrades via CLI display new default settings
* MDL-63603 - Indian Rupee added to [https://docs.moodle.org/36/en/PayPal_enrolment PayPal enrolment] currencies
* MDL-60514 - Set Path to PHP CLI in order to display 'Run now' for [https://docs.moodle.org/36/en/Scheduled_tasks Scheduled tasks]

==For developers==
* MDL-55188 - Old Events API final deprecation
* MDL-54741 - Phase 2 of deprecation of functions in lib/deprecatedlib.php
* MDL-51803 - Reusable element for drag and drop sortable table or list
* MDL-63329 - memcache session handler removal
* MDL-63658 - New Favourites subsystem
* MDL-63729 - Badges web services return new fields and data added by the Open Badges v2.0 specification
* MDL-50812 - core_useragent::get_browser_version_classes distinguishes between different browsers

===Privacy API update===

In addition to existing requirements, any plugin which implements the plugin provider interface must also implement the <tt>\core_privacy\local\request\core_userlist_provider</tt> interface. Two new methods need to be implemented:

* [[Privacy API#Retrieving_the_users_in_a_context|get_users_in_context()]]
* [[Privacy API#Delete_personal_information_for_several_users_in_a_specific_context|delete_data_for_users()]]

However, the two above methods are not required for plugins that [[Privacy_API#Plugins_which_do_not_store_personal_data|implement the <tt>null_provider</tt>]] only (i.e. which do not store personal data).

Note that these changes are also required for latest Moodle 3.4.6 and 3.5.3 versions.

===Behat scenario files===

* MDL-57281 - The behat step <code>I navigate to "ITEM" node in "MAINNODE > PATH"</code> has been deprecated and throws an exception with details on how to replace it. The recommended replacement steps work in all recent Moodle versions. The updated Behat will pass with Moodle 3.4 too.

===Login token===

If your plugin provides an alternative login form (e.g. it is a theme replacing the default login form template / renderer), the login form must include a new login token field. For details of required changes, see [[Login token]]. Note that this also affects latest stable branches too.

===New core functions===

* <tt>userdate_htmltime()</tt>

===Component APIs upgrades===

Please refer to the upgrade.txt files in the relevant component directory for changes in this particular Moodle release.

* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/log/upgrade.txt admin/tool/log/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/upgrade.txt admin/tool/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/shibboleth/upgrade.txt auth/shibboleth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/upgrade.txt auth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/badges/upgrade.txt badges/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/blocks/upgrade.txt blocks/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/cache/upgrade.txt cache/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/calendar/upgrade.txt calendar/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/format/upgrade.txt course/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/upgrade.txt course/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/enrol/upgrade.txt enrol/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/filter/upgrade.txt filter/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/grading/form/upgrade.txt grade/grading/form/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/report/upgrade.txt grade/report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/upgrade.txt grade/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/lib/upgrade.txt lib/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/media/upgrade.txt media/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/message/upgrade.txt message/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/assign/upgrade.txt mod/assign/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/feedback/upgrade.txt mod/feedback/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/forum/upgrade.txt mod/forum/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/quiz/upgrade.txt mod/quiz/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/report/basic/upgrade.txt mod/scorm/report/basic/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/upgrade.txt mod/scorm/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/upgrade.txt mod/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/workshop/upgrade.txt mod/workshop/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/question/format/upgrade.txt question/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/report/upgrade.txt report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/tag/upgrade.txt tag/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/theme/upgrade.txt theme/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/user/upgrade.txt user/upgrade.txt]

==See also==
*[[Moodle 3.5 release notes]]

[[Category:Release notes]]
[[Category:Moodle 3.6]]

[[fr:Notes de mise à jour de Moodle 3.6]]
[[es:Notas de Moodle 3.6]]

Moodle 3.6 release notes

2018-12-04T15:37:29Z

Howardsmiller: /* For administrators */

[[Releases]] > {{FULLPAGENAME}}

Release date: 3 December 2018

Here is [https://tracker.moodle.org/secure/IssueNavigator!executeAdvanced.jspa?jqlQuery=project+%3D+mdl+AND+resolution+%3D+fixed+AND+fixVersion+in+%28%223.6%22%29+ORDER+BY+priority+DESC&runQuery=true&clear=true the full list of fixed issues in 3.6].

See our [https://docs.moodle.org/36/en/New_features New Features page] for a more user-friendly introduction to Moodle 3.6 with screenshots.

{{Note|If you are upgrading from a previous version, please see [[:en:Upgrading|Upgrading]] in the user docs. ''In particular, for sites using a custom theme or login form, from 3.6 onwards, the login form must include a new login token field. See [[Login token]] for details.}}

==Server requirements==

These are just the minimum supported versions. We recommend keeping all of your software and operating systems up-to-date.

* Moodle upgrade: Moodle 3.1 or later
* PHP version: minimum PHP 7.0.0 ''Note: minimum PHP version has increased since Moodle 3.3''. PHP 7.1.x and 7.2.x are supported too. [[Moodle and PHP7|PHP 7.3.x support]] is being implemented (@ MDL-63420) and '''not ready for production''' with this release.
* PHP extension '''intl''' is required since Moodle 3.4 (it was recommended in 2.0 onwards)

=== Database requirements ===

Moodle supports the following database servers. Again, version numbers are just the minimum supported version. We recommend running the latest stable version of any software.

{| class="nicetable"
|-
! Database
! Minimum version
! Recommended
|-
| [http://www.postgresql.org/ PostgreSQL]
| 9.4
| Latest
|-
| [http://www.mysql.com/ MySQL]
| 5.6
| Latest
|-
| [https://mariadb.org/ MariaDB]
| 5.5.31
| Latest
|-
| [http://www.microsoft.com/en-us/server-cloud/products/sql-server/ Microsoft SQL Server]
| 2008
| Latest
|-
| [http://www.oracle.com/us/products/database/overview/index.html Oracle Database]
| 11.2
| Latest
|}

==Client requirements==

=== Browser support ===
Moodle is compatible with any standards compliant web browser. We regularly test Moodle with the following browsers:

Desktop:
* Chrome
* Firefox
* Safari
* Edge
* Internet Explorer

Mobile:
* MobileSafari
* Google Chrome

For the best experience and optimum security, we recommend that you keep your browser up to date. https://whatbrowser.org

Note: Legacy browsers with known compatibility issues with Moodle 3.6:
* Internet Explorer 10 and below
* Safari 7 and below

==Major features==

===Dashboard and Course overview===
* MDL-63044 and MDL-63337 - New [https://docs.moodle.org/36/en/Course_overview Course overview] and [https://docs.moodle.org/36/en/Timeline_block Timeline block]
* MDL-63062 - New [https://docs.moodle.org/36/en/Recently_accessed_courses_block Recently accessed courses block]
* MDL-63063 - New [https://docs.moodle.org/36/en/Recently_accessed_items_block Recently accessed items block]
* MDL-63457 - Option to hide courses in the course overview block
* MDL-63058 - Option to star/unstar courses in the course overview block
* MDL-63064 - New [https://docs.moodle.org/36/en/Starred_courses_block Starred courses block]
* MDL-63352 - Dashboard retains user preferences for view options
* MDL-63793 - Course overview block retains user preferences for the number of courses to show
* MDL-61161 - Grace period when displaying "In progress" courses in course overview block
* MDL-63040 - Removal of Dashboard page header

===GDPR and Privacy===
Note that some of these GDPR improvements have also been backported to Moodle 3.5.3, 3.4.6 and 3.3.9.

* MDL-63116 - Data requests bulk actions
* MDL-62309 - Option to make site policies required or optional
* MDL-61652 - Capabilities for controlling who can download SAR data
* MDL-62563 - Data deletion of existing deleted users
* MDL-63897 - Pre-processing stage removed from data requests process
* MDL-62558 - Data retention summary (read-only)
* MDL-63726 - Option to remove the "Data retention summary" link in the footer
* MDL-62491 - HTML data request export format
* MDL-63401 - User expiry improvements
* MDL-63619 - Data purpose and category inheritance improvements
* MDL-62560 - Different data retention strategies for different roles in a purpose
* MDL-62554 - Ability to configure data registry to use module type defaults
* MDL-63009 - Site mentioned in email notifications of data requests
* MDL-6074 - Option to hide your name in the [https://docs.moodle.org/36/en/Online_users_block online users block]

===Messaging===
* '''MDL-57272 and MDL-63280 - Group messaging'''
* MDL-63303 - New messaging UI with [https://docs.moodle.org/36/en/Messaging messaging drawer]
* MDL-63279 - Option to [https://docs.moodle.org/36/en/Messaging_settings disable site-wide messaging]
* MDL-63214 - Privacy setting for restricting who can message you
* MDL-63213 - Option to star messaging conversations
* MDL-63283 - Notifications not sent for group conversations
* MDL-63281 - Group members synchronised with messaging conversations members

===Assignment===
* MDL-27520 - Assignment feedback can include media or other files

===Quiz===
* MDL-62610 - Improved quiz statistics report usability for randomized questions
* MDL-62708 - Option to add ID numbers to questions and question categories
* MDL-63738 - Single questions can be exported from the question bank

===Workshop===
* MDL-60820 - Teachers can specify workshop submission types

===Repositories===
* MDL-58943 - Nextcloud integration, with a [https://docs.moodle.org/36/en/Nextcloud_repository Nextcloud repository] and [https://docs.moodle.org/36/en/OAuth_2_Nextcloud_service OAuth 2 Nextcloud service]

===Open Badges===
* MDL-58454 - Support for Open Badges v2.0

===Performance===
* MDL-54035 - Performance improvements to cache flags
* MDL-47962 - Glossary auto-linking filter performance improvements

===Usability improvements===
* MDL-51177 - atto_htmlplus implemented to improve Atto editor HTML indenting
* MDL-45170 - Copy and paste of images from one WYSIWYG window to another
* MDL-61388 - Forum actions announced by screen reader when completed
* MDL-62899 - Global search displays a relevant icon next to link in results
* MDL-46415 - SVG/high resolution emoticons
* MDL-58000 - Larger badge images are used

===Experimental===
* MDL-53566 - [https://docs.moodle.org/36/en/Context_freezing Context freezing] - setting read-only access for categories, courses, activities and their content

==Other highlights==

===Functional changes===
* MDL-17943 - 'Resend confirmation email' button on login page
* MDL-14274 - IF conditions in grade calculations
* MDL-37624 - Calendar entries location support
* MDL-36754 - Images are displayed in forum notification emails
* MDL-59259 - Course format options may be specified in upload courses CSV file
* MDL-41265 - Page resource option to show/hide "Last modified"
* MDL-61378 - Forum post HTML structure improvements
* MDL-59454 - Option to download the list of course participants
* MDL-60520 - Analytics models can use different machine learning backends
* MDL-61573 - User menu: customusermenuitems map Font Awesome icons for non pix/t folders
* MDL-62320 - JSON added to the default MIME types list
* MDL-63431 - Atto media plugin title global attribute support
* MDL-60435 - Shibboleth authentication identity providers
* MDL-59169 - Grader report saves after edit with multiple tabs
* MDL-62960 - Drag and drop of course events respects the course start date

===Security issues===
* [https://moodle.org/mod/forum/discuss.php?d=378731 MSA-18-0020] Login CSRF vulnerability in login form. Note that this fix has previously been disclosed following the release of Moodle 3.5.3, 3.4.6, 3.3.9 and 3.1.15.

===For administrators===
* MDL-62334 - 'Add a new course' link in Site administration
* MDL-63253 - Admin search results provide location of the found matching page
* MDL-63772 - Capability to control use of Atto Record RTC
* MDL-63708 - New blocks supported by the mobile app can be disabled
* MDL-52953 - Legacy log store deprecation
* MDL-59429 - Log changes to site administrators
* MDL-62651 - adhoc task runner
* MDL-62777 - Site upgrades via CLI display new default settings
* MDL-63603 - Indian Rupee added to [https://docs.moodle.org/36/en/PayPal_enrolment PayPal enrolment] currencies
* MDL-60514 - Set Path to PHP CLI in order to display 'Run now' for [Scheduled_tasks]

==For developers==
* MDL-55188 - Old Events API final deprecation
* MDL-54741 - Phase 2 of deprecation of functions in lib/deprecatedlib.php
* MDL-51803 - Reusable element for drag and drop sortable table or list
* MDL-63329 - memcache session handler removal
* MDL-63658 - New Favourites subsystem
* MDL-63729 - Badges web services return new fields and data added by the Open Badges v2.0 specification
* MDL-50812 - core_useragent::get_browser_version_classes distinguishes between different browsers

===Privacy API update===

In addition to existing requirements, any plugin which implements the plugin provider interface must also implement the <tt>\core_privacy\local\request\core_userlist_provider</tt> interface. Two new methods need to be implemented:

* [[Privacy API#Retrieving_the_users_in_a_context|get_users_in_context()]]
* [[Privacy API#Delete_personal_information_for_several_users_in_a_specific_context|delete_data_for_users()]]

However, the two above methods are not required for plugins that [[Privacy_API#Plugins_which_do_not_store_personal_data|implement the <tt>null_provider</tt>]] only (i.e. which do not store personal data).

Note that these changes are also required for latest Moodle 3.4.6 and 3.5.3 versions.

===Behat scenario files===

* MDL-57281 - The behat step <code>I navigate to "ITEM" node in "MAINNODE > PATH"</code> has been deprecated and throws an exception with details on how to replace it. The recommended replacement steps work in all recent Moodle versions. The updated Behat will pass with Moodle 3.4 too.

===Login token===

If your plugin provides an alternative login form (e.g. it is a theme replacing the default login form template / renderer), the login form must include a new login token field. For details of required changes, see [[Login token]]. Note that this also affects latest stable branches too.

===New core functions===

* <tt>userdate_htmltime()</tt>

===Component APIs upgrades===

Please refer to the upgrade.txt files in the relevant component directory for changes in this particular Moodle release.

* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/log/upgrade.txt admin/tool/log/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/upgrade.txt admin/tool/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/shibboleth/upgrade.txt auth/shibboleth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/upgrade.txt auth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/badges/upgrade.txt badges/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/blocks/upgrade.txt blocks/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/cache/upgrade.txt cache/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/calendar/upgrade.txt calendar/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/format/upgrade.txt course/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/upgrade.txt course/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/enrol/upgrade.txt enrol/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/filter/upgrade.txt filter/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/grading/form/upgrade.txt grade/grading/form/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/report/upgrade.txt grade/report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/upgrade.txt grade/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/lib/upgrade.txt lib/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/media/upgrade.txt media/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/message/upgrade.txt message/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/assign/upgrade.txt mod/assign/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/feedback/upgrade.txt mod/feedback/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/forum/upgrade.txt mod/forum/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/quiz/upgrade.txt mod/quiz/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/report/basic/upgrade.txt mod/scorm/report/basic/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/upgrade.txt mod/scorm/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/upgrade.txt mod/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/workshop/upgrade.txt mod/workshop/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/question/format/upgrade.txt question/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/report/upgrade.txt report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/tag/upgrade.txt tag/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/theme/upgrade.txt theme/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/user/upgrade.txt user/upgrade.txt]

==See also==
*[[Moodle 3.5 release notes]]

[[Category:Release notes]]
[[Category:Moodle 3.6]]

[[fr:Notes de mise à jour de Moodle 3.6]]
[[es:Notas de Moodle 3.6]]

Moodle 3.6 release notes

2018-12-04T15:36:21Z

Howardsmiller: /* For administrators */

[[Releases]] > {{FULLPAGENAME}}

Release date: 3 December 2018

Here is [https://tracker.moodle.org/secure/IssueNavigator!executeAdvanced.jspa?jqlQuery=project+%3D+mdl+AND+resolution+%3D+fixed+AND+fixVersion+in+%28%223.6%22%29+ORDER+BY+priority+DESC&runQuery=true&clear=true the full list of fixed issues in 3.6].

See our [https://docs.moodle.org/36/en/New_features New Features page] for a more user-friendly introduction to Moodle 3.6 with screenshots.

{{Note|If you are upgrading from a previous version, please see [[:en:Upgrading|Upgrading]] in the user docs. ''In particular, for sites using a custom theme or login form, from 3.6 onwards, the login form must include a new login token field. See [[Login token]] for details.}}

==Server requirements==

These are just the minimum supported versions. We recommend keeping all of your software and operating systems up-to-date.

* Moodle upgrade: Moodle 3.1 or later
* PHP version: minimum PHP 7.0.0 ''Note: minimum PHP version has increased since Moodle 3.3''. PHP 7.1.x and 7.2.x are supported too. [[Moodle and PHP7|PHP 7.3.x support]] is being implemented (@ MDL-63420) and '''not ready for production''' with this release.
* PHP extension '''intl''' is required since Moodle 3.4 (it was recommended in 2.0 onwards)

=== Database requirements ===

Moodle supports the following database servers. Again, version numbers are just the minimum supported version. We recommend running the latest stable version of any software.

{| class="nicetable"
|-
! Database
! Minimum version
! Recommended
|-
| [http://www.postgresql.org/ PostgreSQL]
| 9.4
| Latest
|-
| [http://www.mysql.com/ MySQL]
| 5.6
| Latest
|-
| [https://mariadb.org/ MariaDB]
| 5.5.31
| Latest
|-
| [http://www.microsoft.com/en-us/server-cloud/products/sql-server/ Microsoft SQL Server]
| 2008
| Latest
|-
| [http://www.oracle.com/us/products/database/overview/index.html Oracle Database]
| 11.2
| Latest
|}

==Client requirements==

=== Browser support ===
Moodle is compatible with any standards compliant web browser. We regularly test Moodle with the following browsers:

Desktop:
* Chrome
* Firefox
* Safari
* Edge
* Internet Explorer

Mobile:
* MobileSafari
* Google Chrome

For the best experience and optimum security, we recommend that you keep your browser up to date. https://whatbrowser.org

Note: Legacy browsers with known compatibility issues with Moodle 3.6:
* Internet Explorer 10 and below
* Safari 7 and below

==Major features==

===Dashboard and Course overview===
* MDL-63044 and MDL-63337 - New [https://docs.moodle.org/36/en/Course_overview Course overview] and [https://docs.moodle.org/36/en/Timeline_block Timeline block]
* MDL-63062 - New [https://docs.moodle.org/36/en/Recently_accessed_courses_block Recently accessed courses block]
* MDL-63063 - New [https://docs.moodle.org/36/en/Recently_accessed_items_block Recently accessed items block]
* MDL-63457 - Option to hide courses in the course overview block
* MDL-63058 - Option to star/unstar courses in the course overview block
* MDL-63064 - New [https://docs.moodle.org/36/en/Starred_courses_block Starred courses block]
* MDL-63352 - Dashboard retains user preferences for view options
* MDL-63793 - Course overview block retains user preferences for the number of courses to show
* MDL-61161 - Grace period when displaying "In progress" courses in course overview block
* MDL-63040 - Removal of Dashboard page header

===GDPR and Privacy===
Note that some of these GDPR improvements have also been backported to Moodle 3.5.3, 3.4.6 and 3.3.9.

* MDL-63116 - Data requests bulk actions
* MDL-62309 - Option to make site policies required or optional
* MDL-61652 - Capabilities for controlling who can download SAR data
* MDL-62563 - Data deletion of existing deleted users
* MDL-63897 - Pre-processing stage removed from data requests process
* MDL-62558 - Data retention summary (read-only)
* MDL-63726 - Option to remove the "Data retention summary" link in the footer
* MDL-62491 - HTML data request export format
* MDL-63401 - User expiry improvements
* MDL-63619 - Data purpose and category inheritance improvements
* MDL-62560 - Different data retention strategies for different roles in a purpose
* MDL-62554 - Ability to configure data registry to use module type defaults
* MDL-63009 - Site mentioned in email notifications of data requests
* MDL-6074 - Option to hide your name in the [https://docs.moodle.org/36/en/Online_users_block online users block]

===Messaging===
* '''MDL-57272 and MDL-63280 - Group messaging'''
* MDL-63303 - New messaging UI with [https://docs.moodle.org/36/en/Messaging messaging drawer]
* MDL-63279 - Option to [https://docs.moodle.org/36/en/Messaging_settings disable site-wide messaging]
* MDL-63214 - Privacy setting for restricting who can message you
* MDL-63213 - Option to star messaging conversations
* MDL-63283 - Notifications not sent for group conversations
* MDL-63281 - Group members synchronised with messaging conversations members

===Assignment===
* MDL-27520 - Assignment feedback can include media or other files

===Quiz===
* MDL-62610 - Improved quiz statistics report usability for randomized questions
* MDL-62708 - Option to add ID numbers to questions and question categories
* MDL-63738 - Single questions can be exported from the question bank

===Workshop===
* MDL-60820 - Teachers can specify workshop submission types

===Repositories===
* MDL-58943 - Nextcloud integration, with a [https://docs.moodle.org/36/en/Nextcloud_repository Nextcloud repository] and [https://docs.moodle.org/36/en/OAuth_2_Nextcloud_service OAuth 2 Nextcloud service]

===Open Badges===
* MDL-58454 - Support for Open Badges v2.0

===Performance===
* MDL-54035 - Performance improvements to cache flags
* MDL-47962 - Glossary auto-linking filter performance improvements

===Usability improvements===
* MDL-51177 - atto_htmlplus implemented to improve Atto editor HTML indenting
* MDL-45170 - Copy and paste of images from one WYSIWYG window to another
* MDL-61388 - Forum actions announced by screen reader when completed
* MDL-62899 - Global search displays a relevant icon next to link in results
* MDL-46415 - SVG/high resolution emoticons
* MDL-58000 - Larger badge images are used

===Experimental===
* MDL-53566 - [https://docs.moodle.org/36/en/Context_freezing Context freezing] - setting read-only access for categories, courses, activities and their content

==Other highlights==

===Functional changes===
* MDL-17943 - 'Resend confirmation email' button on login page
* MDL-14274 - IF conditions in grade calculations
* MDL-37624 - Calendar entries location support
* MDL-36754 - Images are displayed in forum notification emails
* MDL-59259 - Course format options may be specified in upload courses CSV file
* MDL-41265 - Page resource option to show/hide "Last modified"
* MDL-61378 - Forum post HTML structure improvements
* MDL-59454 - Option to download the list of course participants
* MDL-60520 - Analytics models can use different machine learning backends
* MDL-61573 - User menu: customusermenuitems map Font Awesome icons for non pix/t folders
* MDL-62320 - JSON added to the default MIME types list
* MDL-63431 - Atto media plugin title global attribute support
* MDL-60435 - Shibboleth authentication identity providers
* MDL-59169 - Grader report saves after edit with multiple tabs
* MDL-62960 - Drag and drop of course events respects the course start date

===Security issues===
* [https://moodle.org/mod/forum/discuss.php?d=378731 MSA-18-0020] Login CSRF vulnerability in login form. Note that this fix has previously been disclosed following the release of Moodle 3.5.3, 3.4.6, 3.3.9 and 3.1.15.

===For administrators===
* MDL-62334 - 'Add a new course' link in Site administration
* MDL-63253 - Admin search results provide location of the found matching page
* MDL-63772 - Capability to control use of Atto Record RTC
* MDL-63708 - New blocks supported by the mobile app can be disabled
* MDL-52953 - Legacy log store deprecation
* MDL-59429 - Log changes to site administrators
* MDL-62651 - adhoc task runner
* MDL-62777 - Site upgrades via CLI display new default settings
* MDL-63603 - Indian Rupee added to [https://docs.moodle.org/36/en/PayPal_enrolment PayPal enrolment] currencies
* MDL-60514 - Set Path to PHP CLI in order to display 'Run now' for [[Scheduled_tasks]]

==For developers==
* MDL-55188 - Old Events API final deprecation
* MDL-54741 - Phase 2 of deprecation of functions in lib/deprecatedlib.php
* MDL-51803 - Reusable element for drag and drop sortable table or list
* MDL-63329 - memcache session handler removal
* MDL-63658 - New Favourites subsystem
* MDL-63729 - Badges web services return new fields and data added by the Open Badges v2.0 specification
* MDL-50812 - core_useragent::get_browser_version_classes distinguishes between different browsers

===Privacy API update===

In addition to existing requirements, any plugin which implements the plugin provider interface must also implement the <tt>\core_privacy\local\request\core_userlist_provider</tt> interface. Two new methods need to be implemented:

* [[Privacy API#Retrieving_the_users_in_a_context|get_users_in_context()]]
* [[Privacy API#Delete_personal_information_for_several_users_in_a_specific_context|delete_data_for_users()]]

However, the two above methods are not required for plugins that [[Privacy_API#Plugins_which_do_not_store_personal_data|implement the <tt>null_provider</tt>]] only (i.e. which do not store personal data).

Note that these changes are also required for latest Moodle 3.4.6 and 3.5.3 versions.

===Behat scenario files===

* MDL-57281 - The behat step <code>I navigate to "ITEM" node in "MAINNODE > PATH"</code> has been deprecated and throws an exception with details on how to replace it. The recommended replacement steps work in all recent Moodle versions. The updated Behat will pass with Moodle 3.4 too.

===Login token===

If your plugin provides an alternative login form (e.g. it is a theme replacing the default login form template / renderer), the login form must include a new login token field. For details of required changes, see [[Login token]]. Note that this also affects latest stable branches too.

===New core functions===

* <tt>userdate_htmltime()</tt>

===Component APIs upgrades===

Please refer to the upgrade.txt files in the relevant component directory for changes in this particular Moodle release.

* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/log/upgrade.txt admin/tool/log/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/admin/tool/upgrade.txt admin/tool/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/shibboleth/upgrade.txt auth/shibboleth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/auth/upgrade.txt auth/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/badges/upgrade.txt badges/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/blocks/upgrade.txt blocks/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/cache/upgrade.txt cache/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/calendar/upgrade.txt calendar/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/format/upgrade.txt course/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/course/upgrade.txt course/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/enrol/upgrade.txt enrol/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/filter/upgrade.txt filter/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/grading/form/upgrade.txt grade/grading/form/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/report/upgrade.txt grade/report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/grade/upgrade.txt grade/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/lib/upgrade.txt lib/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/media/upgrade.txt media/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/message/upgrade.txt message/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/assign/upgrade.txt mod/assign/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/feedback/upgrade.txt mod/feedback/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/forum/upgrade.txt mod/forum/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/quiz/upgrade.txt mod/quiz/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/report/basic/upgrade.txt mod/scorm/report/basic/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/scorm/upgrade.txt mod/scorm/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/upgrade.txt mod/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/mod/workshop/upgrade.txt mod/workshop/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/question/format/upgrade.txt question/format/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/report/upgrade.txt report/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/tag/upgrade.txt tag/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/theme/upgrade.txt theme/upgrade.txt]
* [https://git.in.moodle.com/moodle/moodle/blob/master/user/upgrade.txt user/upgrade.txt]

==See also==
*[[Moodle 3.5 release notes]]

[[Category:Release notes]]
[[Category:Moodle 3.6]]

[[fr:Notes de mise à jour de Moodle 3.6]]
[[es:Notas de Moodle 3.6]]

Git for developers

2018-11-09T14:43:34Z

Howardsmiller: /* See also */

This document is for helping you get started on Moodle development with Git. For further details of Git, see [[:Category:Git]].

== General workflow ==

'''A reasonable knowledge of the Git basics is a good idea before you start to use it for development. If you are new to Git, you are encouraged to go to 'See also' for some more general reading.'''

[[image:git-pushpull-model.png|right|thumb|400px|Moodle development workflow with Git]]
Detailed explanation of the workflow can be found in the [[Process]] page. In short, the Moodle development with Git looks like this:

* You as the contributor commit changes into your personal repository at your computer
* You push the changes into your public repository and publish links to your changes in the Moodle Tracker
* You request a peer review of your code from another developer
* When peer reviewer is happy they submit issue for integration
* Moodle integrators pull the changes from your public repository and if they like them, they put them into Moodle integration repository
* The integrated change is tested and finally pushed into Moodle production repository
* You update your local repository with all changes from the production repository and the next cycle may start again

This workflow runs in roughly weekly cycles. The integration happens on Monday and Tuesday and the testing on Wednesday. On Thursday (or Friday if testing takes too long), the production repository moodle.git is usually updated with changes from the last development week.

Most Moodle developers have their public repositories hosted at [http://github.com/ Github]. Alternatively you may want to try [http://gitorious.org Gitorious] or the legendary [http://repo.or.cz repo.or.cz]. In the examples in this guide we assume you'll set up your public repository at Github.

When you first register on tracker you can not assign issues to yourself or send them for peer review. You will be added to the developers group after your first bug fix is integrated. Before that just comment on the issue with a link to your branch and component lead or another developer will send issue for peer review for you.

== Installing Git on your computer ==

Install Git on your computer. Most Linux distributions have Git available as a package to install. On Debian/Ubuntu, type ''''sudo apt-get install git'''' on the terminal. If you are on Mac, [http://code.google.com/p/git-osx-installer/ git-osx-installer] installs it in a few clicks.

Immediately after the installation, set your name and contact e-mail. The name and e-mail will become part of your commits and they can't be changed later once your commits are accepted into the Moodle code. Therefore we ask contributors to use their real names written in capital letters, eg "John Smith" and not "john smith" or even "john5677".

git config --global user.name "Your Name"
git config --global user.email yourmail@domain.tld

Unless you are the repository maintainer, it is wise to set your Git to not push changes in file permissions:

git config --global core.filemode false

Also, it's recommended to verify that the your git installation is not performing any transformation between LFs and CRLFs. All Moodle '''uses only LFs''' and you should '''fetch/edit and push''' it that way (may need to configure your editor/IDE too). Note that having any "magic" enabled is known to cause [[Common unit test problems#The_test_file_.22evolution.test.22_should_not_contain_section_named_.22.5Blots_of_content.5D.22|problems with unit tests]] execution. So we recommend you to set:

git config --global core.autocrlf false

== Setting-up the public repository ==

1. Go to [http://github.com/ Github] and create an account.

2. Go to the [http://github.com/moodle/moodle official Moodle Github repository] and click on the Fork button. You now have your own Github Moodle repository.

3. Now you need to set up your SSH public key, so you can push to your Github Moodle repository from your local Moodle repository. On Mac you can go on this [http://help.github.com/mac-key-setup/ Github help page]. If you are on another system, go to your Github administration page, to the section SSH Public Keys, and you should see a link to a help page. Done? Good! That was the most difficult part!

== Setting-up the local repository at your computer ==

Create a local clone repository of your Github repository. In a terminal:

git clone git://github.com/YOUR_GITHUB_USERNAME/moodle.git LOCALDIR

(or: git clone git@github.com:YOUR_GITHUB_USERNAME/moodle.git LOCALDIR)

This command does several jobs for you. It creates a new folder, initializes an empty Git repository in it, sets your Github repository as the remote repository called 'origin' and makes a local checkout of the branch 'master' from it. The important point to remember now is that your Github repository is aliased as 'origin' for your local clone.

Note that the format of the URL here is important. In the first example, the URL starts "git://github.com" and this will give read-only access to the repository at github.com. If you use this URL, the "git push origin" command that appears later in this document will not work. Therefore, if you want to be able to update the "origin" repository, you should use the URL that starts "git@github.com", i.e. the second of the two "git clone" commands given above. This will give you read and write access to the repository on github.com.

== Keeping your public repository up-to-date ==

[[image:git-sync-github.png|right|thumb|400px|Fetching changes from upstream and pushing them to github]]
Your fork at Github is not updated automatically. To keep it synced with the upstream Moodle repository, you have to fetch the recent changes from the official moodle.git and push them to your public repository. To avoid problems with this it is strongly recommended that you never modify the standard Moodle branches. ''Remember: never commit directly into master and MOODLE_xx_STABLE branches.'' In other words, always create topic branches to work on. In Gitspeak, the master branch and MOODLE_xx_STABLE branches should be always fast-forwardable.

To keep your public repository up-to-date, we will register remote repository git://git.moodle.org/moodle.git under 'upstream' alias. Then we create a script to be run regularly that fetches changes from the upstream repository and pushes them to your public repository. Note that this procedure will not affect your local working directory.

To register the upstream remote:

cd moodle
git remote add upstream git://git.moodle.org/moodle.git

The following commands can be used to keep the standard Moodle branches at your Github repository synced with the upstream repository. You may wish to store them in a script so that you can run it every week after the upstream repository is updated.

#!/bin/bash
git fetch upstream
for BRANCH in MOODLE_{19..35}_STABLE master; do
git push origin refs/remotes/upstream/$BRANCH:refs/heads/$BRANCH
done

=== How it works ===

The git-fetch command does not modify your current working dir (your checkout). It just downloads all recent changes from a remote repository and stores them into so called remote-tracking branches. The git-push command takes these remote-tracking branches from upstream and pushes them to Github under the same name. Understanding this fully requires a bit knowledge of Git internals - see gitrevisions(7) man page.

Note there is no need to switch the local branch during this. You can even execute this via cron at your machine. Just note that the upstream repository updates typically just once a week.

=== New branches ===

Occasionally, moodle.org will create a new branch that does not exist in your public (e.g. Github.com) repository. If you try to push this new branch, you will see an error such as the following:

error: unable to push to unqualified destination: MOODLE_99_STABLE
The destination refspec neither matches an existing ref on the remote
nor begins with refs/, and we are unable to guess a prefix based on the source ref.
error: failed to push some refs to 'git@github.com:YOUR_GITHUB_USERNAME/moodle.git'

In the above example, "MOODLE_99_STABLE", is the name of the new branch that does not exist in your public repository. To fix the error, you need to create the new branch on your public repository, using the following commands, replacing "MOODLE_99_STABLE" with the name of the new branch you wish to create:

git checkout MOODLE_99_STABLE
git push origin MOODLE_99_STABLE:MOODLE_99_STABLE

The above code will create a new copy of the "MOODLE_99_STABLE" branch in your local repository. If you do not need to keep a local copy of the new branch - and probably you do not need it, you then can remove it from your local repository as follows:

git checkout master
git branch -D MOODLE_99_STABLE

== Preparing a patch ==

As said earlier at this page, you never work on standard Moodle branches directly. Every time you are going to edit something, switch to a local branch. Fork the local branch off the standard branch you think it should be merged to. So if you are working on a patch for 1.9 or 2.0, fork the branch off MOODLE_19_STABLE or MOODLE_20_STABLE, respectively. Patches for the next [[Moodle versions|major version]] should be based on the master branch.

git checkout -b MDL-xxxxx-master_brief_name origin/master

Note that if you forget to specify the starting point, the branch is based on the currently checked-out branch. It may not be what you want. It is recommended to always specify the starting point.

To check the current branch, run

git branch

The current branch is highlighted.

Now go and fix the issue with your favorite IDE. Check the status of the files, view the change to be committed and finally commit the change:

vim filename.php
git status
git diff
git commit -a

For the commit message, please do respect the guidelines given on in the article [[Coding style#Git_commits|Coding_style]].

Note that this is safe as the commit is recorded just locally, nothing is sent to any server yet (as it would in CVS). To see history of the commits, use

git log

Once your local branch contains the change (note that it may consists of several patches) and you are happy with it, publish the branch at your public repository:

git push origin MDL-xxxxx-master_brief_name

Now as your branch is published, you can ask Moodle core developers to review it and eventually integrate it into the standard Moodle repository.

=== Changing commit message, reordering and squashing commits ===

It often happens that you made a mistake in your patch or in the commit message and helpful CiBot pointed it out for you. You can "rewrite the history" and change the existing commits.

Option 1. Reset all the changes in the branch and commit again.

git reset --mixed origin/master

Now all your changes are still present but all commits on top of "master" branch are gone. You can create a new commit

Option 2. Discover '''git rebase --interactive''' - this is a powerful tool to change the sequence of commit, change the commit messages, squash commits, etc. We will not cover it here, there are many articles in the Internet about it, for example: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History.

Whatever option you chose, you have "rewritten the history" and you can not simply push the changes to github again because they would need to overwrite the commits that were already pushed. If you try "git push MDL-xxxxx-master_brief_name" you will get an error message suggesting you to force push.
To force push the changed commits use:

git push -f origin MDL-xxxxx-master_brief_name

If an error occurs because you are still using the git protocol (read only), use this command :

git remote set-url origin https://github.com/<user.name>/moodle.git

A prompt will ask for your credentials, if you previously setup your SSH public key you can also use this one :

git remote set-url origin git@github.com:<user.name>/moodle.git

=== Checking if a branch has already been merged ===

After some time contributing to Moodle you would have a lot of branches both in your local repository and in your public repository. To prune their list and delete those that were accepted by upstream, use the following

git fetch upstream (1)
git branch --merged upstream/master (2)
git branch --merged upstream/MOODLE_20_STABLE (3)

The command (1) fetches the changes from your upstream repository at git.moodle.org (remember that git-fetch does not modify your working dir so it is safe to run it whenever). Command (2) and (3) print all branches that are merged into the upstream master branch and MOODLE_20_STABLE branch, respectively. To delete these local branches, use

git branch -d MDL-xxxxx-master_accepted_branch

The similar approach can be used to check the branches published at your origin repository at github.com

git fetch origin (1)
git fetch upstream
git branch -r --merged upstream/master (2)
git branch -r --merged upstream/MOODLE_20_STABLE (3)

The command (1) makes sure that you have all your branches from github.com recorded as the remote tracking branch locally. Commands (2) and (3) work the same as in the previous example but they list remote tracking branches only ([http://www.kernel.org/pub/software/scm/git/docs/git-branch.html see -r param]). To delete a branch at github.com, use

git push origin :MDL-xxxxx-master_branch_to_delete

This syntax may look weird to you. However it is pretty logical. The general syntax of the git-push command is

git push <repository> <source ref>:<target ref>

so deleting a remote branch can be understood as pushing an "empty (null) reference" to it.

== Peer-reviewing someone else's code ==

To review a branch that someone else pushed into their public repository, you do not need to register a new remote (unless you work with such repository frequently, of course). Let us imagine your friend Alice pushed a work-in-progress branch called 'wip-feature' into her Github repository and asked you to review it. You need to know the read-only address of the repository and the name of the branch.

git fetch git://github.com/alice/moodle.git wip-feature

This will download all required data and will keep the pointer to the tip of the wip-feature branch in a local symbolic reference FETCH_HEAD. To see what's there on that branch, use

git log -p FETCH_HEAD

To see how a particular file looks at Alice's branch

git show FETCH_HEAD:admin/blocks.php

To create a new local branch called 'alice-wip-feature' containing the work by Alice, use

git checkout -b alice-wip-feature FETCH_HEAD

To merge Alice's work into your current branch:

git merge FETCH_HEAD

To see what would be merged into the current branch without actually modifying anything:

git diff ...FETCH_HEAD

Once you are all set and reviewing code, this [[Peer_reviewing_checklist|checklist]] should prove to be useful.

== Rebasing a branch ==

Rebasing is a process when you cut off the branch from its current start point and transplant it to another point. Let us assume the following history exists:

A---B---C topic
/
D---E---F---G master

From this point, the result of the command:

git rebase master topic

would be:

A'--B'--C' topic
/
D---E---F---G master

and would end with 'topic' being your current branch.

You may be asked to rebase your branch submitted for the integration if the submitted branch was based on an outdated commit. The typical case is if you create a new branch as a fork off the upstream master branch on Tuesday. Then on Wednesday, the upstream master branch grows as all changes from the last integration cycle are merged to it. To make diff easy on Github for next weekly pull request review, you want to rebase your branch against the updated master.

git rebase master MDL-xxxxx-master_topic_branch

Note that rebasing effectively rewrites the history of the branch. '''Do not rebase the branch if there is a chance that somebody has already forked it and based their own branch on it.''' For this reason, many Git tutorials discourage from rebasing any branch that has been published. However in Moodle, all branches submitted for integration are potential subject of rebase (even though we try to not to do it often) and you should not base your own branches on them.

=== Conflicts during rebase ===

During the rebase procedure, conflicts may appear. git-status commands reports the conflicted files. Explore them carefully and fix them in your editor (like you would do with CVS). Then add the files with 'git add' command and continue.

vim conflicted.php
git add conflicted.php
git rebase --continue

== Applying changes from one branch to another ==

Most bugs are fixed at a stable branch (like MOODLE_20_STABLE) and the fix must be prepared for other branches, too (like MOODLE_21_STABLE and the main development branch - master). In Moodle, we do not merge stable branches into the master one. So usually the contributor prepares at least two branches - with the fix for the stable branch(es) and with the fix for the master branch.

If you have a patch prepared on a local branch (let us say MDL-xxxxx-20_brief_name), it is possible to re-apply it to another branch.

=== Cherry-picking a single commit ===

Let us have two local Git repositories ~/public_html/moodle21 containing local installation of Moodle 2.1 and ~/public_html/moodledev with the local installation of most recent development version of Moodle. They both use your public repository at github.com as the origin. You have a branch in moodle21 called MDL-xxxxx-21_topic that was forked off MOODLE_21_STABLE. It contains one commit. Now you want to re-apply this commit to a branch MDL-xxxxx-master_topic in moodledev.

cd ~/public_html/moodledev
git checkout -b MDL-xxxxx-master_topic origin/master (1)
git fetch ../moodle21 MDL-xxxxx-21_topic (2)
git cherry-pick FETCH_HEAD (3)

The command (1) creates new local branch forked off the master. The command (2) fetches all data needed to re-apply the topic branch and stores the pointer to the tip of that branch to FETCH_HEAD symbolic reference. The command (3) picks the tip of the branch (the top-most commit on it) and tries to apply it on the current branch.
There is also a variant of the cherry-pick command that supports multiple commits, shortly (see its man page for details): <code bash>$ git cherry-pick A^..B</code> if you want to include from A - see '''^''' - to B, A should be older than B. We will use another approach for cherry-picking multiple commits.

=== Applying a set of patches ===

If the branch MDL-xxxxx-21_topic from the previous example consists of several commits, it may be easier to use git-format-patch and git-am combo to re-apply the whole set of patches (aka patchset). Firstly you will export all commits from the topic branch to files.

cd ~/public_html/moodle21
mkdir .patches
git format-patch -o .patches MOODLE_21_STABLE..MDL-xxxxx-21_topic (1)

The command (1) takes all commits from the topic branch that are not in MOODLE_21_STABLE and exports them one by one to the output directory .patches. Look at the generated files. They contain the patch itself (in diff format) and additional information about the commit. You could eg send these files by email to a friend of yours for peer-review. We will use them in another repository.

cd ~/public_html/moodledev
git checkout -b MDL-xxxxx-master_topic origin/master
git am -3 ../moodle21/.patches/* (1)

The command (1) applies all the files from the .patches directory. When a patch does not apply cleanly, the command tries fall back on 3-way merge (see the -3 parameter). If conflicts occur during the procedure, you can either deal with them and then use `git am --continue` or abort the whole procedure with `git am --abort`.

== See also ==

* [[Git tips]]

; Moodle forum discussions
* [http://moodle.org/mod/forum/discuss.php?d=168094 GIT help needed]
* [http://moodle.org/mod/forum/discuss.php?d=165236 Best way to manage CONTRIB code with GIT]
* [http://moodle.org/mod/forum/discuss.php?d=167063 Handy Git tip for tracking 3rd-party modules and plugins]
* [http://moodle.org/mod/forum/discuss.php?d=167730 Moodle Git repositories]
* [http://moodle.org/mod/forum/discuss.php?d=183409 Git help!! I don't understand rebase enough...]
* [http://moodle.org/mod/forum/discuss.php?d=217617 add MOODLE_24_STABLE to github.com repository]

; External resources
* [https://mirrors.edge.kernel.org/pub/software/scm/git/docs/giteveryday.html Everyday GIT With 20 Commands Or So]
* [https://git-scm.com/book/en/v2 'Pro Git' complete book]
* [http://vimeo.com/14629850 Getting git by Scott Chacon] - an recording of an excellent 1-hour presentation that introducing git, including a simple introduction to what is going on under the hood.
* [http://tjhunt.blogspot.co.uk/2012/03/fixing-bug-in-moodle-core-mechanics.html Tim Hunt's blog: Fixing a bug in Moodle core: the mechanics]
* [https://github.com/k88hudson/git-flight-rules/blob/master/README.md#flight-rules-for-git Flight rules for Git]

[[Category:Git]]

[[ja:開発者用Git]]

Moodle 3.5 release notes

2018-07-17T08:45:51Z

Howardsmiller: /* Server requirements */

[[Releases]] > {{FULLPAGENAME}}

Release date: 17 May 2018

Here is [https://tracker.moodle.org/secure/IssueNavigator!executeAdvanced.jspa?jqlQuery=project+%3D+mdl+AND+resolution+%3D+fixed+AND+fixVersion+in+%28%223.5%22%29+ORDER+BY+priority+DESC&runQuery=true&clear=true the full list of fixed issues in 3.5].

See our [https://docs.moodle.org/35/en/New_features New Features page] for a more user-friendly introduction to Moodle 3.5 with screenshots.

If you are upgrading from previous version, make sure you read the [https://docs.moodle.org/35/en/Upgrading Upgrading] documentation.

==Server requirements==

These are just the minimum supported versions. We recommend keeping all of your software and operating systems up-to-date.

* Moodle upgrade: Moodle 3.1 or later
* PHP version: minimum PHP 7.0.0 ''Note: minimum PHP version has increased since Moodle 3.3''. PHP 7.1.x and 7.2.x are supported too. PHP 7.x could have some [https://docs.moodle.org/dev/Moodle_and_PHP7#Can_I_use_PHP7_yet.3F engine limitations].
* PHP extension '''intl''' is required since Moodle 3.4 (it was recommended in 2.0 onwards)
* (Recommendation only) If you use MySQL or MariaDB, make sure your database supports full UTF-8 (utf8mb4) if you install a new instance of Moodle. CLI script may be used to convert to utf8mb4 if you're upgrading. You may choose to keep using 'utf8_*', but then a warning will show that the database isn't using full UTF-8 support and suggest moving to 'utf8mb4_unicode_ci'. See [[:en:MySQL full unicode support|MySQL full unicode support]] for details. If you do enable utf8mb4 you *must* use the Barracuda file format.

=== Database requirements ===

Moodle supports the following database servers. Again, version numbers are just the minimum supported version. We recommend running the latest stable version of any software.

{| class="nicetable"
|-
! Database
! Minimum version
! Recommended
|-
| [http://www.postgresql.org/ PostgreSQL]
| 9.3
| Latest
|-
| [http://www.mysql.com/ MySQL]
| 5.5.31
| Latest
|-
| [https://mariadb.org/ MariaDB]
| 5.5.31
| Latest
|-
| [http://www.microsoft.com/en-us/server-cloud/products/sql-server/ Microsoft SQL Server]
| 2008
| Latest
|-
| [http://www.oracle.com/us/products/database/overview/index.html Oracle Database]
| 10.2
| Latest
|}

==Client requirements==

=== Browser support ===
Moodle is compatible with any standards compliant web browser. We regularly test Moodle with the following browsers:

Desktop:
* Chrome
* Firefox
* Safari
* Edge
* Internet Explorer

Mobile:
* MobileSafari
* Google Chrome

For the best experience and optimum security, we recommend that you keep your browser up to date. https://whatbrowser.org

Note: Legacy browsers with known compatibility issues with Moodle 3.5:
* Internet Explorer 10 and below
* Safari 7 and below

==Major features==

===GDPR===
* '''MDL-61275 - GDPR Consenting of Minors and Managing, Versioning and Tracking Privacy Policies and User Consents'''
* MDL-61292 - A new admin tool to manage policy documents
* MDL-61423 - Add age and location verification to identify minors
* MDL-61302 - Workflow to allow users to agree to all policies
* MDL-61301 - Report of user agreed policies and their versions
* MDL-61705 - Bulk accept of policies on behalf of users
* MDL-61864 - Include policy tool in core
* MDL-62286 - Add policy link to the site footer

* '''MDL-61306 - GDPR Data Requests and Data Registry'''
* MDL-59718 - A process to send a request to the data protection officer
* MDL-59720 - Delete personal data when it is no longer required
* MDL-61307 - Create a new privacy subsystem
* MDL-61362 - Ability to create data categories and purposes
* MDL-61486 - Data registry with purpose and retention period
* MDL-61489 - Report of plugin/components implementing the Privacy API
* MDL-61499 - Ability to set default purpose and retention periods for context levels
* MDL-61785 - Ability to review and confirm which expired data can be deleted
* MDL-61899 - Include data privacy tool in core
* MDL-61935 - Ability to specify the lawful bases for the collection of personal data

===Question bank tagging improvements===
* MDL-61066 - Expanded tagging functionality for question bank
* MDL-61133 - New modal to add/edit/remove tags on questions
* MDL-61135 - Filter questions by tag
* MDL-61138 - Show the list of questions in the 'Add a random question' dialog
* MDL-61363 - Ability to add question tags at a course level in the edit question form
* MDL-61364 - Manage tags at a question and course context level
* MDL-61380 - Allow filtering/adding random questions by tag for quizzes
* MDL-61410 - Add import/export support for course level question tags
* MDL-61444 - New capabilities for tagging questions

===UX: Usability improvements===
* MDL-62021 - Boost 4.0 Migration
* MDL-56511 - Update bootstrap 4 to final release
* MDL-61657 - Add images to the course cards on the dashboard

===LTI Advantage support===
* MDL-60416 - Add support for LTI Advantage 1.1

===RecordRTC for Atto===
* MDL-60848 - Implement RecordRTC Atto plugin as core feature
* MDL-61973 - Update RecordRTC Atto plugin buttons

===Messaging database tables===
* MDL-61254 - Merge messaging database tables
* MDL-36941 - Create new tables for messaging
* MDL-61255 - Ad-hoc task to upgrade messages to merged table

==Other Highlights==

===Global search===
* MDL-58885 - Add group support
* MDL-59434 - Content aware searching / alternate results sort orders
* MDL-60981 - Reindex a single area
* MDL-61028 - Allow filtering search by user
* MDL-61256 - Search of section titles, summaries

===Functional changes===
* MDL-2051 - Inform student whether and how their selected choice will display
* MDL-32585 - SCORM: option to force new attempts
* MDL-53226 - Add Moodle DB search engine
* MDL-55491 - Use cohort as badge criteria
* MDL-56246 - Add site wide default for grade export: include feedback
* MDL-59875 - Allow badges as criteria for other badges
* MDL-60119 - Feedback - Multiple choice (rated) - remove weights from answer
* MDL-61203 - Allow uploading of profile picture to be used as badge criteria
* MDL-61601 - Allow cohort themes
* MDL-61651 - LTI: line item definition within link to return gradable LTI links
* MDL-60811 - Bulk delete self-registered enrolments on participants page
* MDL-60682 - Ability to set date/time to nearest minute
* MDL-60441 - Ability to add a link to glossary entries
* MDL-58411 - Ability to apply file type restrictions for essay question type
* MDL-56945 - Add easy return path from PDF grading screen to list of submissions
* MDL-52811 - Add force language capability to course settings
* MDL-41090 - Allow teachers to embed files when manually grading questions

===Security issues===

* [https://moodle.org/mod/forum/discuss.php?d=371199 MSA-18-0007] Calculated question type allows remote code execution by Question authors
* [https://moodle.org/mod/forum/discuss.php?d=371200 MSA-18-0008] Users can download any file via portfolio assignment caller class
* [https://moodle.org/mod/forum/discuss.php?d=371201 MSA-18-0009] Portfolio forum caller class allows a user to download any file
* [https://moodle.org/mod/forum/discuss.php?d=371202 MSA-18-0010] User can shift a block from Dashboard to any page
* [https://moodle.org/mod/forum/discuss.php?d=371203 MSA-18-0011] User who did not agree to the site policies can see the site homepage as if they had full site access
* [https://moodle.org/mod/forum/discuss.php?d=371204 MSA-18-0012] Portfolio script allows instantiation of class chosen by user

===For developers===

* MDL-61307 - All plugins must implement [https://docs.moodle.org/dev/Privacy_API Privacy API] to be compliant with GDPR requirements. They must implement the API to report on, export and delete stored user data
* MDL-56511 - Bootstrap is upgraded to final release of version 4
* MDL-61869 - Infer rendering of templatables with no render method
* MDL-61298 - Boost: use navigation node icon

==== Upgrading plugins ====

'''1. Check for changes in core APIs'''

Read lib/upgrade.txt to check for the deprecations and core API changes, make sure you applied them to your plugin. Note that entries there are not sorted by priority but rather by integration time. Below is the list of upgrade.txt files that contain information about upgrading from Moodle 3.4 to Moodle 3.5 (note that if you upgrade from earlier versions there may be more files):

* [https://raw.githubusercontent.com/moodle/moodle/master/lib/upgrade.txt lib/upgrade.txt] changes to various core APIs, deprecations, functions removal
* [https://raw.githubusercontent.com/moodle/moodle/master/calendar/upgrade.txt calendar/upgrade.txt] changes to Calendar API
* [https://raw.githubusercontent.com/moodle/moodle/master/search/upgrade.txt search/upgrade.txt] changes to Global search API
* [https://raw.githubusercontent.com/moodle/moodle/master/message/upgrade.txt message/upgrade.txt] changes to Messages API
* [https://raw.githubusercontent.com/moodle/moodle/master/course/upgrade.txt course/upgrade.txt] changes to Course API

'''2. Check for changes in the API of your plugin type'''

Below is the list of plugin types that had API changes between Moodle 3.4 and 3.5:

* [https://raw.githubusercontent.com/moodle/moodle/master/enrol/upgrade.txt enrol/upgrade.txt] Enrolment method plugins
* [https://raw.githubusercontent.com/moodle/moodle/master/mod/upgrade.txt mod/upgrade.txt] Activity module plugins
* [https://raw.githubusercontent.com/moodle/moodle/master/auth/upgrade.txt auth/upgrade.txt] Authentication plugins
* [https://raw.githubusercontent.com/moodle/moodle/master/course/format/upgrade.txt course/format/upgrade.txt] Course format plugins
* [https://raw.githubusercontent.com/moodle/moodle/master/question/type/upgrade.txt question/type/upgrade.txt] Question type plugins

'''3. Check for changes in the depended plugins'''

If your plugin depends on another plugin or calls methods from another plugin, read upgrade.txt in this plugin directory (if it exists). Below is the list of standard plugins that had changes between Moodle 3.4 and 3.5:

[https://raw.githubusercontent.com/moodle/moodle/master/admin/tool/mobile/upgrade.txt tool_mobile],
[https://raw.githubusercontent.com/moodle/moodle/master/admin/tool/usertours/upgrade.txt tool_usertours],
[https://raw.githubusercontent.com/moodle/moodle/master/mod/assign/upgrade.txt mod_assign],
[https://raw.githubusercontent.com/moodle/moodle/master/mod/feedback/upgrade.txt mod_feedback],
[https://raw.githubusercontent.com/moodle/moodle/master/mod/quiz/upgrade.txt mod_quiz],
[https://raw.githubusercontent.com/moodle/moodle/master/mod/scorm/upgrade.txt mod_scorm],
[https://raw.githubusercontent.com/moodle/moodle/master/theme/boost/upgrade.txt theme_boost]

'''4. Do a smoke test of your plugin with developer debugging mode'''

Make sure to check on both Boost and Clean themes. Bootstrap was upgraded in Moodle 3.5

'''5. Run all behat and phpunit tests'''

==See also==
*[[Moodle 3.4 release notes]]

[[Category:Release notes]]
[[Category:Moodle 3.5]]

[[fr:Notes de mise à jour de Moodle 3.5]]
[[es:Notas de Moodle 3.5]]

Privacy API

2018-05-14T21:21:20Z

Howardsmiller: /* Retrieving the list of contexts */

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The fetching of data is broken into two separate steps:

# Detecting in which Moodle contexts the user has any data; and
# Exporting all data from each of those contexts.

This has been broken into two steps to later allow administrators to exclude certain contexts from an export - e.g. for courses currently in progress.

A third component will later be added to facilitate the deletion of data within these contexts which will help to satisfy the Right to be Forgotten. This will also use the first step.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0-beta/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name needs to be indicated rather than one copy for each preference.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define four functions (the first two of which are dealt with in this section):

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.
* ''delete_data_for_all_users_in_context'' - to delete all data for all users in the specified context.
* ''delete_data_for_user'' - to delete all user data for the specified user, in the specified contexts.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

=====Retrieving the list of contexts=====

You are required to return the list of contexts for which the plugin stores data about the user. These are the standard Moodle contexts - CONTEXT_COURSE, CONTEXT_USER, CONTEXT_MODULE and so on. In many cases the link between the module type and the context is self-evident (e.g. activity modules). In some cases it may be less so. For example, an enrolment plugin (that stores user data, which most don't) would link to course context (users enrol in courses). Other types of plugins may be less obvious but you need to pick something. One way might to consider what context you would use when checking role capabilities. Consider that this will be used to structure the exported data and define what data is deleted when a context is expired.

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

======More complete example======

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

=====Exporting user data=====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

====Plugins which store user preferences====

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

==== Plugins which can have own subplugins ====

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

===== When a parent plugin should and should not provide the interface for its subplugins =====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

=====Example=====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

====Plugins which are subplugins to another plugin====

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

====Plugins which are typically called by a Moodle subsystem====

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

====Exporting data====

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

Note that this will be called for '''any''' context being expired, not only those the plugin holds data for (as returned by get_contexts_for_userid()), so you must carefully check the contexts given.

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$cm = get_coursemodule_from_id('choice', $context->instanceid);
if (!$cm) {
return;
}

$DB->delete_records('choice_answers', ['choiceid' => $cm->instance]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

==Difference between Moodle 3.3 and more recent versions==
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version.
Consequently the privacy API for this version does not have these features.
==Common Questions==
===What to do if you have one plugin that supports multiple branches===
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

====Example====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

===What to do if your plugin must implement a subplugin or subsystem plugin provider===
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
==== Example ====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Privacy API

2018-05-14T21:20:52Z

Howardsmiller: /* Retrieving the list of contexts */

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The fetching of data is broken into two separate steps:

# Detecting in which Moodle contexts the user has any data; and
# Exporting all data from each of those contexts.

This has been broken into two steps to later allow administrators to exclude certain contexts from an export - e.g. for courses currently in progress.

A third component will later be added to facilitate the deletion of data within these contexts which will help to satisfy the Right to be Forgotten. This will also use the first step.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0-beta/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name needs to be indicated rather than one copy for each preference.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define four functions (the first two of which are dealt with in this section):

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.
* ''delete_data_for_all_users_in_context'' - to delete all data for all users in the specified context.
* ''delete_data_for_user'' - to delete all user data for the specified user, in the specified contexts.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

=====Retrieving the list of contexts=====

You are required to return the list of contexts for which the plugin stores data about the user. These are the standard Moodle contexts - CONTEXT_COURSE, CONTEXT_USER, CONTEXT_MODULE and so on. In may cases the link between the module type and the context is self-evident (e.g. activity modules). In some cases it may be less so. For example, an enrolment plugin (that stores user data, which most don't) would link to course context (users enrol in courses). Other types of plugins may be less obvious but you need to pick something. One way might to consider what context you would use when checking role capabilities. Consider that this will be used to structure the exported data and define what data is deleted when a context is expired.

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

======More complete example======

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

=====Exporting user data=====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

====Plugins which store user preferences====

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

==== Plugins which can have own subplugins ====

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

===== When a parent plugin should and should not provide the interface for its subplugins =====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

=====Example=====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

====Plugins which are subplugins to another plugin====

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

====Plugins which are typically called by a Moodle subsystem====

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

====Exporting data====

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

Note that this will be called for '''any''' context being expired, not only those the plugin holds data for (as returned by get_contexts_for_userid()), so you must carefully check the contexts given.

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$cm = get_coursemodule_from_id('choice', $context->instanceid);
if (!$cm) {
return;
}

$DB->delete_records('choice_answers', ['choiceid' => $cm->instance]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

==Difference between Moodle 3.3 and more recent versions==
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version.
Consequently the privacy API for this version does not have these features.
==Common Questions==
===What to do if you have one plugin that supports multiple branches===
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

====Example====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

===What to do if your plugin must implement a subplugin or subsystem plugin provider===
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
==== Example ====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Privacy API

2018-05-09T12:25:45Z

Howardsmiller: /* Retrieving the list of contexts */

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The fetching of data is broken into two separate steps:

# Detecting in which Moodle contexts the user has any data; and
# Exporting all data from each of those contexts.

This has been broken into two steps to later allow administrators to exclude certain contexts from an export - e.g. for courses currently in progress.

A third component will later be added to facilitate the deletion of data within these contexts which will help to satisfy the Right to be Forgotten. This will also use the first step.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0-beta/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name needs to be indicated rather than one copy for each preference.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define four functions (the first two of which are dealt with in this section):

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.
* ''delete_data_for_all_users_in_context'' - to delete all data for all users in the specified context.
* ''delete_data_for_user'' - to delete all user data for the specified user, in the specified contexts.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

=====Retrieving the list of contexts=====

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

To decide which type of context to return (CONTEXT_COURSE, CONTEXT_USER etc.) consider the most appropriate type for your plugin. Typically, this will be the same type of context as you would create to check capabilities. For example, an enrolment plugin will return the course contexts in which it is used.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

======More complete example======

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

=====Exporting user data=====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

====Plugins which store user preferences====

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

==== Plugins which can have own subplugins ====

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

===== When a parent plugin should and should not provide the interface for its subplugins =====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

=====Example=====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

====Plugins which are subplugins to another plugin====

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

====Plugins which are typically called by a Moodle subsystem====

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

====Exporting data====

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

==Difference between Moodle 3.3 and more recent versions==
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version.
Consequently the privacy API for this version does not have these features.
==Common Questions==
===What to do if you have one plugin that supports multiple branches===
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

====Example====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

===What to do if your must implement a subplugin or subsystem plugin provider===
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
==== Example ====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Privacy API

2018-05-09T12:18:51Z

Howardsmiller: /* Standard plugins which store data */

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The fetching of data is broken into two separate steps:

# Detecting in which Moodle contexts the user has any data; and
# Exporting all data from each of those contexts.

This has been broken into two steps to later allow administrators to exclude certain contexts from an export - e.g. for courses currently in progress.

A third component will later be added to facilitate the deletion of data within these contexts which will help to satisfy the Right to be Forgotten. This will also use the first step.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0-beta/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name needs to be indicated rather than one copy for each preference.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define four functions (the first two of which are dealt with in this section):

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.
* ''delete_data_for_all_users_in_context'' - to delete all data for all users in the specified context.
* ''delete_data_for_user'' - to delete all user data for the specified user, in the specified contexts.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

=====Retrieving the list of contexts=====

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

======More complete example======

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

=====Exporting user data=====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

====Plugins which store user preferences====

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

==== Plugins which can have own subplugins ====

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

===== When a parent plugin should and should not provide the interface for its subplugins =====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

=====Example=====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

====Plugins which are subplugins to another plugin====

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

====Plugins which are typically called by a Moodle subsystem====

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

====Exporting data====

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

==Difference between Moodle 3.3 and more recent versions==
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version.
Consequently the privacy API for this version does not have these features.
==Common Questions==
===What to do if you have one plugin that supports multiple branches===
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

====Example====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

===What to do if your must implement a subplugin or subsystem plugin provider===
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
==== Example ====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Talk:Privacy API

2018-05-08T14:49:44Z

Howardsmiller:

==Privacy API author credits==

Thanks to Andrew Nicols with help from Adrian Greeve, Jake Dallimore, Sander Bangma and Jun Pataleta.

This extensive documentation was drafted first in Google Docs ([https://docs.google.com/document/d/1Y7n4Qkez4Tl83rWArOQPQCpE2NeSA2bUa8gOR2r_JFE/edit# GDPR plugin API]), thus the above authors are not credited in the page history.

==Some issues/questions==

Could the get_contexts_for_userid function be explained in a bit more detail. It's unclear why this data is being collected. My particular case in a custom enrolment plugin that contains user data. Is the context the course context, the user context or something else... and why? There are no actual example to go on. --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:09, 8 May 2018 (UTC)

Pretty sure menu structure is all broken on this page. See menu levels 4.2 and 4.3. Sounds like they are not on the same level at all. It's currently rather confusing --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:49, 8 May 2018 (UTC)

Talk:Privacy API

2018-05-08T14:49:16Z

Howardsmiller: /* Privacy API author credits */

==Privacy API author credits==

Thanks to Andrew Nicols with help from Adrian Greeve, Jake Dallimore, Sander Bangma and Jun Pataleta.

This extensive documentation was drafted first in Google Docs ([https://docs.google.com/document/d/1Y7n4Qkez4Tl83rWArOQPQCpE2NeSA2bUa8gOR2r_JFE/edit# GDPR plugin API]), thus the above authors are not credited in the page history.

Could the get_contexts_for_userid function be explained in a bit more detail. It's unclear why this data is being collected. My particular case in a custom enrolment plugin that contains user data. Is the context the course context, the user context or something else... and why? There are no actual example to go on. --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:09, 8 May 2018 (UTC)

Pretty sure menu structure is all broken on this page. See menu levels 4.2 and 4.3. Sounds like they are not on the same level at all. It's currently rather confusing --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:49, 8 May 2018 (UTC)

Talk:Privacy API

2018-05-08T14:48:05Z

Howardsmiller: /* Privacy API author credits */

==Privacy API author credits==

Thanks to Andrew Nicols with help from Adrian Greeve, Jake Dallimore, Sander Bangma and Jun Pataleta.

This extensive documentation was drafted first in Google Docs ([https://docs.google.com/document/d/1Y7n4Qkez4Tl83rWArOQPQCpE2NeSA2bUa8gOR2r_JFE/edit# GDPR plugin API]), thus the above authors are not credited in the page history.

Could the get_contexts_for_userid function be explained in a bit more detail. It's unclear why this data is being collected. My particular case in a custom enrolment plugin that contains user data. Is the context the course context, the user context or something else... and why? There are no actual example to go on. --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:09, 8 May 2018 (UTC)

Pretty sure menu structure is all broken on this page.

Privacy API

2018-05-08T14:36:26Z

Howardsmiller: /* Basic example */

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The fetching of data is broken into two separate steps:

# Detecting in which Moodle contexts the user has any data; and
# Exporting all data from each of those contexts.

This has been broken into two steps to later allow administrators to exclude certain contexts from an export - e.g. for courses currently in progress.

A third component will later be added to facilitate the deletion of data within these contexts which will help to satisfy the Right to be Forgotten. This will also use the first step.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0-beta/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name needs to be indicated rather than one copy for each preference.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define two functions:

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

====Retrieving the list of contexts====

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

=====Basic example=====

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

=====More complete example=====

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

====Exporting user data====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

===Plugins which store user preferences===

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

=== Plugins which can have own subplugins ===

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

==== When a parent plugin should and should not provide the interface for its subplugins ====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

====Example====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

===Plugins which are subplugins to another plugin===

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

===Plugins which are typically called by a Moodle subsystem===

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

===Exporting data===

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

===Difference between Moodle 3.3 and more recent versions===
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version. Consequently the privacy API for this version does not have these features.
====What to do if you have one plugin that supports multiple branches====
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

=====Example=====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

====What to do if your must implement a subplugin or subsystem plugin provider====
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
===== Example =====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Talk:Privacy API

2018-05-08T14:09:34Z

Howardsmiller: /* Privacy API author credits */

==Privacy API author credits==

Thanks to Andrew Nicols with help from Adrian Greeve, Jake Dallimore, Sander Bangma and Jun Pataleta.

This extensive documentation was drafted first in Google Docs ([https://docs.google.com/document/d/1Y7n4Qkez4Tl83rWArOQPQCpE2NeSA2bUa8gOR2r_JFE/edit# GDPR plugin API]), thus the above authors are not credited in the page history.

Could the get_contexts_for_userid function be explained in a bit more detail. It's unclear why this data is being collected. My particular case in a custom enrolment plugin that contains user data. Is the context the course context, the user context or something else... and why? There are no actual example to go on. --[[User:Howard Miller|Howard Miller]] ([[User talk:Howard Miller|talk]]) 14:09, 8 May 2018 (UTC)

Javascript Modules

2018-04-07T15:01:48Z

Howardsmiller: /* Minimum (getting started) module for plugins */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[[grunt]]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

This means you first have to '''install nodejs''' - and its package manager [https://www.npmjs.com/ npm]. The details of how to install those packages will vary by operating system, but on Linux it's probably similar to "sudo apt-get install nodejs npm". There are downloadable packages for other operating systems here: http://nodejs.org/download/. Moodle currently requires node "8.9.x".

Once this is done, you can '''run the command''':

npm install
npm install -g grunt-cli

from the top of the Moodle directory to install all of the required tools. (You may need extra permissions to use the -g option.)

== Development mode ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

define(['jquery'], function($) {

return {
init: function() {

// Put whatever you like here. $ is available
// to you as normal.
$(".someclass").change(function() {
alert("It changed!!");
});
}
};
});
</code>

This code passes the jquery module into our function (parameter $). There are a number of other useful modules available in Moodle, some of which you'll probably need in a practical application. See [[Useful_core_Javascript_modules]]. Simply list them in both the define() first parameter and the function callback. E.g.,
<code javascript>
define(['jquery', 'core/str', 'core/ajax'], function($, str, ajax) {
</code>

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
return {
init: function(first, last) {
}
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

/**
* @module block_overview/helloworld
*/
define(['jquery'], function($) {

/**
* Give me blue.
* @access private
* @return {string}
*/
var makeItBlue = function() {
// We can use our jquery dependency here.
return $('.blue').show();
};

/**
* @constructor
* @alias module:block_overview/helloworld
*/
var greeting = function() {
/** @access private */
var privateThoughts = 'I like the colour blue';

/** @access public */
this.publicThoughts = 'I like the colour orange';

};

/**
* A formal greeting.
* @access public
* @return {string}
*/
greeting.prototype.formal = function() {
return 'How do you do?';
};

/**
* An informal greeting.
* @access public
* @return {string}
*/
greeting.prototype.informal = function() {
return 'Wassup!';
};
return greeting;
});
</code>

The most interesting line above is:
<code>
define(['jquery'], function($) {
</code>

All AMD modules must call "define()" as the first and only global scoped piece of code. This ensures the javascript code contains no global variables and will not conflict with any other loaded module. The name of the module does not need to be specified because it is determined from the filename and component (but it can be listed in a comment for JSDoc as shown here).

The first argument to "define" is the list of dependencies for the module. This argument must be passed as an array, even if there is only one. In this example "jquery" is a dependency. "jquery" is shipped as a core module is available to all AMD modules.

The second argument to "define" is the function that defines the module. This function will receive as arguments, each of the requested dependencies in the same order they were requested. In this example we receive JQuery as an argument and we name the variable "$" (it's a JQuery thing). We can then access JQuery normally through the $ variable which is in scope for any code in our module.

The rest of the code in this example is a standard way to define a Javascript module with public/private variables and methods. There are many ways to do this, this is only one.

It is important that we are returning 'greeting'. If there is no return then your module will be declared as undefined.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? Stuffing all possible required modules in the define call is one solution, but it's ugly and it only works for code that is in an AMD module (what about inline code in the page?). AMD lets you load a dependency any time you like.
<code javascript>

// Load a new dependency.
require(['mod_wiki/timer'], function(timer) {
// timer is available to do my bidding.
});
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'initialise', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
define(['jquery', 'theme_mytheme/jquery.countdown'], function($, c) {
return {
initialise: function ($params) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
}
};
});
</code>

2. Put the javascript into a mustache template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="clock"></span>
{{#js}}
require(['jquery', 'theme_mytheme/jquery.countdown'], function($) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
{{/js}}
</code>

3. Call the javascript directly from php (although who would want to put javascript into php? ergh):
<code php>
$PAGE->requires->js_amd_inline('
require(['theme_mytheme/jquery.countdown'], function(min) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
');
</code>

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)
* [[Useful_core_Javascript_modules]]

[[Category:AJAX]]
[[Category:Javascript]]

AMD Modal

2018-03-04T21:51:18Z

Howardsmiller: /* How do you create a different type of modal? */

{{Moodle 3.2}}

This is currently (2017) the recommended way to create various sorts of pop-up dialogue boxes in JavaScript.

= Why should you use it? =
The AMD modal modules provide a simple interface for creating a modal within Moodle. The module will ensure all accessibility requirements are met including applying the correct aria roles, focus control, aria hiding background elements and locking keyboard navigation.

The modals will fire events for common actions that occur within the modal, e.g. show / hide, for other code to listen to and react accordingly.

Moodle ships with a couple of standard modal types for you to re-use including a simple cancel modal, a confirm (yes/no) modal and a save/cancel modal. Hopefully with more to come!

= How do you create a basic modal? =
If you'd simply like to display a modal with some simple content then all you'll need is the modal factory. The factory provides a create function that accepts some configuration for your modal that the factory will use to create the modal and optionally a trigger element (the element that will open the modal when activated). The create function will return a promise that is resolved with the created modal.

The configuration is provided as an object with key/value pairs. The options are:
{| class="nicetable"
|-
! '''key'''
! '''description'''
|-
| title
| the title to display in the modal header - note: this will render HTML
|-
| body
| the main content to be rendered in the modal body
|-
| footer
| the content to be rendered in the modal footer
|-
| type
| one of the modal types registered with the factory
|-
| large
| a boolean to indicate if the modal should be wider than the default size
|}

Example 1
<code javascript>
define(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

The modals will also accept a promise for the body and footer content. It is expected that the promise will be resolved with two strings, the html content and any associated javascript. This allows the modal module to work natively with the templates module, such as in the example below.

Example 2
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
// Can include JS which is run when modal is attached to DOM.
body: Templates.render('core/modal_test_3', {}),
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Since the modals aren't actually added to the DOM until they are made visible any javascript resolved with the promise is cached and only run after all of the elements are added to the DOM, so you don't have to worry about any special handling in the javascript that is being loaded from the template.

= How do you create a different type of modal? =
Moodle comes with a few specialised types of modals for common use cases. Look at lib/amd/src/modal_factory.js to see what is available. These can be created using the factory, similar to the examples above, by specifying the type of modal in the configuration provided to the create function. Each of the modal types may have different configuration options, for example the save/cancel modal doesn't allow you to set the footer content.

Example 3
<code javascript>
define(['jquery', 'core/modal_factory', 'core/templates'], function($, ModalFactory, Templates) {
var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
// Do what you want with your modal.
});
});
</code>

Each type of modal may fire additional events to allow your code to handle the new functionality being offered. See the modal_events.js module for a list of events that can be fired. For example, if you wanted to have a save/cancel modal that you did some form validation on before saving you could do something like the example below.

Example 4
<code javascript>
define(['jquery', 'core/modal_factory', 'core/modal_events', 'core/templates'],
function($, ModalFactory, ModalEvents, Templates) {

var trigger = $('#create-modal');
ModalFactory.create({
type: ModalFactory.types.SAVE_CANCEL,
title: 'Modal save cancel',
body: 'This modal is a save/cancel modal',
}, trigger)
.done(function(modal) {
modal.getRoot().on(ModalEvents.save, function(e) {
// Stop the default save button behaviour which is to close the modal.
e.preventDefault();
// Do your form validation here.
});
});
});
</code>

= How do you write a new type of modal? =
If you'd like to write a new type of modal to be re-used throughout your code you can extend the default modal implementation and make any customisations you require. In order to create a new modal type you'll need to create a new AMD module and import the core/modal module to extend. You can also optionally create a new modal template that builds upon the core/modal template. Finally, you can register your new type with the modal registry to allow you to create your new type using the modal factory.

For example, let's create a modal that opens a login form:
First we can create the HTML for out modal by including and extending the core modal template. All we need to do is override the block defined in that template with our new title, body and footer content.

Let's assume the file is <your_module>/templates/modal_login.mustache

<code>
{{< core/modal }}
{{$title}}{{#str}} login {{/str}}{{/title}}
{{$body}}
<div class="container">
<form>
<div class="form-group row">
<label for="inputEmail" class="col-sm-2 col-form-label">{{#str}} email {{/str}}</label>
<div class="col-sm-10">
<input type="email" class="form-control" id="inputEmail" placeholder="{{#str}} email {{/str}}">
</div>
</div>
<div class="form-group row">
<label for="inputPassword" class="col-sm-2 col-form-label">{{#str}} password {{/str}}</label>
<div class="col-sm-10">
<input type="password" class="form-control" id="inputPassword" placeholder="{{#str}} password {{/str}}">
</div>
</div>
</form>
</div>
{{/body}}
{{$footer}}
<button type="button" class="btn btn-primary" data-action="login">{{#str}} login {{/str}}</button>
<button type="button" class="btn btn-secondary" data-action="cancel">{{#str}} cancel {{/str}}</button>
{{/footer}}
{{/ core/modal }}
</code>

Next we can create the new AMD module for the login modal. You can extend the existing modal module which will give you all of the core modal functionality for free allowing you to focus on writing only the specific logic you need. In this example we would only need to write the logic for handling the login.

Let's assume the file is <your_module>/amd/src/modal_login.js

<code javascript>
define(['jquery', 'core/notification', 'core/custom_interaction_events', 'core/modal', 'core/modal_registry'],
function($, Notification, CustomEvents, Modal, ModalRegistry) {

var registered = false;
var SELECTORS = {
LOGIN_BUTTON: '[data-action="login"]',
CANCEL_BUTTON: '[data-action="cancel"]',
};

/**
* Constructor for the Modal.
*
* @param {object} root The root jQuery element for the modal
*/
var ModalLogin = function(root) {
Modal.call(this, root);

if (!this.getFooter().find(SELECTORS.LOGIN_BUTTON).length) {
Notification.exception({message: 'No login button found'});
}

if (!this.getFooter().find(SELECTORS.CANCEL_BUTTON).length) {
Notification.exception({message: 'No cancel button found'});
}
};

ModalLogin.TYPE = 'your_module-login';
ModalLogin.prototype = Object.create(Modal.prototype);
ModalLogin.prototype.constructor = ModalLogin;

/**
* Set up all of the event handling for the modal.
*
* @method registerEventListeners
*/
ModalLogin.prototype.registerEventListeners = function() {
// Apply parent event listeners.
Modal.prototype.registerEventListeners.call(this);

this.getModal().on(CustomEvents.events.activate, SELECTORS.LOGIN_BUTTON, function(e, data) {
// Add your logic for when the login button is clicked. This could include the form validation,
// loading animations, error handling etc.
}.bind(this));

this.getModal().on(CustomEvents.events.activate, SELECTORS.CANCEL_BUTTON, function(e, data) {
// Add your logic for when the cancel button is clicked.
}.bind(this));
};

// Automatically register with the modal registry the first time this module is imported so that you can create modals
// of this type using the modal factory.
if (!registered) {
ModalRegistry.register(ModalLogin.TYPE, ModalLogin, '<your_module>/modal_login');
registered = true;
}

return ModalLogin;
});
</code>

Once you have both your template and javascript ready to go then all you need to do is tie them into where ever you'd like to launch the modal. For the purpose of this example let's assume that we've got a page with a login button with id="login" then you might add this javascript to the page.

<code javascript>
define(['jquery', 'core/templates', 'core/modal_factory', '<your_module>/modal_login'], function($, Templates, ModalFactory, ModalLogin) {
var trigger = $('#login');

ModalFactory.create({type: ModalLogin.TYPE}, trigger);
});
</code>

= Related =
* AMD and [[Javascript Modules]]

[[Category:Javascript]]

AJAX

2018-03-03T20:08:38Z

Howardsmiller: /* Ajax in Moodle */

'''AJAX (Asynchronous Javascript and XML)''' is a modern web design technique that allows for more interactivity by making webpages that fetch data in the background and alter themselves without reloading the entire page. This helps to make a page feel much more like an application than a web page. AJAX is a new way of working with existing technologies (including HTML, [[Javascript]], [[CSS]] and the ''XMLHttpRequest object'' amongst others) rather than a new piece of technology in itself.

Although AJAX indicates that XML is used, the term really relates to the group of technologies and in Moodle we tend to favour use of JSON rather than XML as the syntax is lighter and leads to a smaller output. It is also easier to construct from php data structures.

== Ajax in Moodle ==
{{ Moodle 2.9 }}

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions.

Some benefits of this system are:
# No new ajax scripts need auditing for security vulnerabilities
# Multiple requests can be chained in a single http request
# Strict type checking for all parameters and return types
# New webservice functions benefit Ajax interfaces and web service clients

So the steps required to create an ajax interaction are:

# Write or find an existing web service function to handle the ajax interaction: See [[ Web_services ]]
# White list the web service for ajax. To do this, you can define 'ajax' => true in your function's definition, in db/services.php. Only functions that are whitelisted using this mechanism will be available to the ajax script.
# Call the web service from javascript in response to a user action:

Example calling core_get_string:
<code javascript>
require(['core/ajax'], function(ajax) {
var promises = ajax.call([
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'pluginname' } },
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'changerate' } }
]);

promises[0].done(function(response) {
console.log('mod_wiki/pluginname is' + response);
}).fail(function(ex) {
// do something with the exception
});

promises[1].done(function(response) {
console.log('mod_wiki/changerate is' + response);
}).fail(function(ex) {
// do something with the exception
});
});
</code>

Note: This example chains to separate calls to the core_get_string webservice in one http request

Note: Don't actually fetch strings like this, it is just an example, use the 'core/str' module instead.

If there is only a single action, a simpler form is possible (example from Assignment):

<code>
ajax.call([{
methodname: 'mod_assign_submit_grading_form',
args: {assignmentid: assignmentid, userid: this._lastUserId, jsonformdata: JSON.stringify(data)},
done: this._handleFormSubmissionResponse.bind(this, data, nextUserId),
fail: notification.exception
}]);
</code>

('notifcation' comes from the 'core/notification' javascript module and provides a useful popup error message if the call fails)

To update parts of the UI in response to Ajax changes, consider using [[ Templates ]]

For information on writing AJAX scripts for Moodle before Moodle 2.9 see: [[ AJAX pre 2.9 ]]

Watch a video about using templates with webservices and AJAX in Moodle: https://www.youtube.com/watch?v=UTePjRZqAg8

Tricky things to know about using webservices with ajax calls:
# Any call to $PAGE->get_renderer() requires the correct theme be set. If this is done in a webservice - it is likely that the theme needs to be a parameter to the webservice.
# Text returned from a webservice must be properly filtered. This means it must go through external_format_text or external_format_string (since 3.0 - see MDL-51213) with the correct context.
# The correct context for 2 is the most specific context relating to the thing being output e.g. for a user's profile desciption the context is the user context.
# After adding any dynamic content to a page, Moodle's filters need to be notified via M.core.event.FILTER_CONTENT_UPDATED (MDL-51222 makes this easier)
# After adding or changing any webservice definition in db/services.php - you must bump the version number for either the plugin or Moodle and run the upgrade. This will install the webservice in the DB tables so it can be found by ajax.

In some very rare cases - you can mark webservices as safe to call without a session. These should only be used for webservices that return 100% public information and do not consume many resources. A current example is core_get_string. To mark a webservice as safe to call without a session you need to do 2 things.
# Add 'loginrequired' => false to the service definition in db/services.php
# Pass "false" as the 3rd argument to the ajax "call" method when calling the webservice.
The benefit to marking these safe webservice is that (a) they can be called from the login page before we have a session and (b) they will perform faster because they will bypass moodles session code when responding to the webservice call.

== See also ==

* [[ AJAX pre 2.9 ]]
* [[Templates]]
* [[Javascript Modules]]
* [[Javascript FAQ]]
* [[Javascript]]
* [[Firebug#Debugging_AJAX_with_Firebug|Debugging AJAX with Firebug]]

* [http://www.adaptivepath.com/publications/essays/archives/000385.php ''Ajax: A New Approach to Web Applications'', the original Ajax article by Adaptive Path] (This link is now dead, but the article is preserved on the [https://web.archive.org/web/20070225140912/http://www.adaptivepath.com/publications/essays/archives/000385.php Internet Archive])
* [http://developer.mozilla.org/en/docs/AJAX:Getting_Started ''AJAX: Getting Started'' article on developer.mozilla.org]
* [http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html ''10 places you must use AJAX'' by Adam Bosworth] (Also a dead link... [https://web.archive.org/web/20060127015713/http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html Internet Archive copy])
* [http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype ''Considering Ajax, Part 1: Cut through the hype'' from IBM developerworks] (Also a dead link... [https://web.archive.org/web/20080602101238/http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype Internet Archive copy])
* [http://en.wikipedia.org/wiki/Ajax_%28programming%29 Wikipedia article on ''AJAX'']
* [http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ How to Make Your AJAX Applications Accessible: 40 Tutorials and Articles] (Also a dead link... [https://web.archive.org/web/20090225094656/http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ Internet Archive copy])
*[http://www.ajaxload.info/ AJAX loading icon generator]

[[Category:Javascript]]
[[Category:AJAX]]

AJAX

2018-03-03T20:07:52Z

Howardsmiller: /* Ajax in Moodle */

'''AJAX (Asynchronous Javascript and XML)''' is a modern web design technique that allows for more interactivity by making webpages that fetch data in the background and alter themselves without reloading the entire page. This helps to make a page feel much more like an application than a web page. AJAX is a new way of working with existing technologies (including HTML, [[Javascript]], [[CSS]] and the ''XMLHttpRequest object'' amongst others) rather than a new piece of technology in itself.

Although AJAX indicates that XML is used, the term really relates to the group of technologies and in Moodle we tend to favour use of JSON rather than XML as the syntax is lighter and leads to a smaller output. It is also easier to construct from php data structures.

== Ajax in Moodle ==
{{ Moodle 2.9 }}

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions.

Some benefits of this system are:
# No new ajax scripts need auditing for security vulnerabilities
# Multiple requests can be chained in a single http request
# Strict type checking for all parameters and return types
# New webservice functions benefit Ajax interfaces and web service clients

So the steps required to create an ajax interaction are:

# Write or find an existing web service function to handle the ajax interaction: See [[ Web_services ]]
# White list the web service for ajax. To do this, you can define 'ajax' => true in your function's definition, in db/services.php. Only functions that are whitelisted using this mechanism will be available to the ajax script.
# Call the web service from javascript in response to a user action:

Example calling core_get_string:
<code javascript>
require(['core/ajax'], function(ajax) {
var promises = ajax.call([
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'pluginname' } },
{ methodname: 'core_get_string', args: { component: 'mod_wiki', stringid: 'changerate' } }
]);

promises[0].done(function(response) {
console.log('mod_wiki/pluginname is' + response);
}).fail(function(ex) {
// do something with the exception
});

promises[1].done(function(response) {
console.log('mod_wiki/changerate is' + response);
}).fail(function(ex) {
// do something with the exception
});
});
</code>

Note: This example chains to separate calls to the core_get_string webservice in one http request

Note: Don't actually fetch strings like this, it is just an example, use the 'core/str' module instead.

If there is only a single action, a simpler form is possible:

<code>
ajax.call([{
methodname: 'mod_assign_submit_grading_form',
args: {assignmentid: assignmentid, userid: this._lastUserId, jsonformdata: JSON.stringify(data)},
done: this._handleFormSubmissionResponse.bind(this, data, nextUserId),
fail: notification.exception
}]);
</code>

('notifcation' comes from the 'core/notification' javascript module and provides a useful popup error message if the call fails)

To update parts of the UI in response to Ajax changes, consider using [[ Templates ]]

For information on writing AJAX scripts for Moodle before Moodle 2.9 see: [[ AJAX pre 2.9 ]]

Watch a video about using templates with webservices and AJAX in Moodle: https://www.youtube.com/watch?v=UTePjRZqAg8

Tricky things to know about using webservices with ajax calls:
# Any call to $PAGE->get_renderer() requires the correct theme be set. If this is done in a webservice - it is likely that the theme needs to be a parameter to the webservice.
# Text returned from a webservice must be properly filtered. This means it must go through external_format_text or external_format_string (since 3.0 - see MDL-51213) with the correct context.
# The correct context for 2 is the most specific context relating to the thing being output e.g. for a user's profile desciption the context is the user context.
# After adding any dynamic content to a page, Moodle's filters need to be notified via M.core.event.FILTER_CONTENT_UPDATED (MDL-51222 makes this easier)
# After adding or changing any webservice definition in db/services.php - you must bump the version number for either the plugin or Moodle and run the upgrade. This will install the webservice in the DB tables so it can be found by ajax.

In some very rare cases - you can mark webservices as safe to call without a session. These should only be used for webservices that return 100% public information and do not consume many resources. A current example is core_get_string. To mark a webservice as safe to call without a session you need to do 2 things.
# Add 'loginrequired' => false to the service definition in db/services.php
# Pass "false" as the 3rd argument to the ajax "call" method when calling the webservice.
The benefit to marking these safe webservice is that (a) they can be called from the login page before we have a session and (b) they will perform faster because they will bypass moodles session code when responding to the webservice call.

== See also ==

* [[ AJAX pre 2.9 ]]
* [[Templates]]
* [[Javascript Modules]]
* [[Javascript FAQ]]
* [[Javascript]]
* [[Firebug#Debugging_AJAX_with_Firebug|Debugging AJAX with Firebug]]

* [http://www.adaptivepath.com/publications/essays/archives/000385.php ''Ajax: A New Approach to Web Applications'', the original Ajax article by Adaptive Path] (This link is now dead, but the article is preserved on the [https://web.archive.org/web/20070225140912/http://www.adaptivepath.com/publications/essays/archives/000385.php Internet Archive])
* [http://developer.mozilla.org/en/docs/AJAX:Getting_Started ''AJAX: Getting Started'' article on developer.mozilla.org]
* [http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html ''10 places you must use AJAX'' by Adam Bosworth] (Also a dead link... [https://web.archive.org/web/20060127015713/http://www.sourcelabs.com/blogs/ajb/2005/12/10_places_you_must_use_ajax.html Internet Archive copy])
* [http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype ''Considering Ajax, Part 1: Cut through the hype'' from IBM developerworks] (Also a dead link... [https://web.archive.org/web/20080602101238/http://www-128.ibm.com/developerworks/web/library/wa-ajaxtop1/?ca=dgr-lnxw01AjaxHype Internet Archive copy])
* [http://en.wikipedia.org/wiki/Ajax_%28programming%29 Wikipedia article on ''AJAX'']
* [http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ How to Make Your AJAX Applications Accessible: 40 Tutorials and Articles] (Also a dead link... [https://web.archive.org/web/20090225094656/http://www.maxkiesler.com/index.php/weblog/comments/how_to_make_your_ajax_applications_accessible/ Internet Archive copy])
*[http://www.ajaxload.info/ AJAX loading icon generator]

[[Category:Javascript]]
[[Category:AJAX]]

Useful core Javascript modules

2018-03-03T17:26:28Z

Howardsmiller:

{{Moodle 2.9}}

The source code for all these modules (and a number of others) can be found in lib/amd/src.

== Configuration settings (core/config) ==

Example of using config module:
<code javascript>
require(['core/config’], function(mdlcfg) {
console.log(mdlcfg.wwwroot); // outputs the wwwroot of moodle to console
});
</code>

== Language strings (core/str) ==

Example of using language strings module (retrieved via ajax, if the string is not yet loaded)
<code javascript>
require([‘core/str’], function(str) {
// start retrieving the localized string; store the promise that some time in the future the string will be there.
var editaPresent = str.get_string('edita', 'core', stringargument);
// as soon as the string is retrieved, i.e. the promise has been fulfilled,
// edit the text of a UI element so that it then is the localized string
// Note: $.when can be used with an arbitrary number of promised things
$.when(editaPresent).done(function(localizedEditString) {
$("someUIElementSelector").text = localizedEditString;
});
});
</code>

The string will be retrieved via AJAX request on the first use and cached in browser local storage until purge caches or site upgrade

IT IS NOT RECOMMENDED to call:
<code php>
$PAGE->requires->string_for_js('edita', 'core'); // You do not need this!
</code>

Because:
* The list of strings used now needs to be maintained in 2 places
* The strings are always sent, and bloat the page size even if they are not used
* All strings fetched via the AJAX method above are cached in browser local storage anyway, so these strings will never be used 99% of the time

==Notifications (core/notification)==

Examples:
<code javascript>
require(['core/notification’], function(notification) {
notification.alert('Hello', 'Welcome to my site!', 'Continue');
});
</code>

Strings and notifications:
<code javascript>
require(['core/str', 'core/notification’], function(str, notification) {
str.get_strings([
{'key' : 'delete'},
{'key' : 'confirmdeletetag', component : 'tag'},
{'key' : 'yes'},
{'key' : 'no'},
]).done(function(s) {
notification.confirm(s[0], s[1], s[2], s[3], function() {
window.location.href = href;
});
}
).fail(notification.exception);
});
</code>

== URL module (core/url) ==

Useful Functions:
<code javascript>
// Generate an absolute URL by using the relative path to a file and optionally slash arguments
url.fileUrl(relativeScript, slashArgs)
// Generate an absolute URL from the given relative path, include the URL parameters and possibly the current session key
url.relativeUrl(relativePath, params, includeSessionKey)
// Generates an image url using the filename of the image and the Moodle component (core, plugin, etc.) where it can be found
url.imageUrl(imagename, component)
</code>

Example:

<code javascript>
// Prerequisites:
// - A Javascript file is present under $CFG->wwwroot."/path/to/file.js"
require(['core/url'], function(url) {
url.fileUrl("/path/to/file.js", "");
console.log("Generated URL: " + url);
});
</code>

== Ajax Module (core/ajax) ==

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. This saves having to worry about security in ajax functionality because it's all done for you.

For the full story, see [[AJAX]]

== Tree (core/tree) ==
{{Moodle 3.1}}
If you want to add a hierarchical interface, you should use the tree module. It will handle user interaction and accessibility for you.

<code javascript>
define(['jquery', 'core/tree'], function($, Tree) {
return {
init: function() {
new Tree("css/jquery selector of the tree container");
}
};
});
</code>

Read more about the [[Tree]] module

== Modal (core/modal) ==
{{Moodle 3.2}}

If you'd like to add a modal to your page the modal modules will handle all of the magic for you, all you need to bring is your content!
<code javascript>
require(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

Read more about the [[AMD_Modal]] module

==See also==
* [[Javascript Modules]]
* [[Fragment]]

[[Category:AJAX]]
[[Category:Javascript]]

Useful core Javascript modules

2018-03-03T17:05:42Z

Howardsmiller: /* URL module (core/url) */

{{Moodle 2.9}}

== Configuration settings (core/config) ==

Example of using config module:
<code javascript>
require(['core/config’], function(mdlcfg) {
console.log(mdlcfg.wwwroot); // outputs the wwwroot of moodle to console
});
</code>

== Language strings (core/str) ==

Example of using language strings module (retrieved via ajax, if the string is not yet loaded)
<code javascript>
require([‘core/str’], function(str) {
// start retrieving the localized string; store the promise that some time in the future the string will be there.
var editaPresent = str.get_string('edita', 'core', stringargument);
// as soon as the string is retrieved, i.e. the promise has been fulfilled,
// edit the text of a UI element so that it then is the localized string
// Note: $.when can be used with an arbitrary number of promised things
$.when(editaPresent).done(function(localizedEditString) {
$("someUIElementSelector").text = localizedEditString;
});
});
</code>

The string will be retrieved via AJAX request on the first use and cached in browser local storage until purge caches or site upgrade

IT IS NOT RECOMMENDED to call:
<code php>
$PAGE->requires->string_for_js('edita', 'core'); // You do not need this!
</code>

Because:
* The list of strings used now needs to be maintained in 2 places
* The strings are always sent, and bloat the page size even if they are not used
* All strings fetched via the AJAX method above are cached in browser local storage anyway, so these strings will never be used 99% of the time

==Notifications (core/notification)==

Examples:
<code javascript>
require(['core/notification’], function(notification) {
notification.alert('Hello', 'Welcome to my site!', 'Continue');
});
</code>

Strings and notifications:
<code javascript>
require(['core/str', 'core/notification’], function(str, notification) {
str.get_strings([
{'key' : 'delete'},
{'key' : 'confirmdeletetag', component : 'tag'},
{'key' : 'yes'},
{'key' : 'no'},
]).done(function(s) {
notification.confirm(s[0], s[1], s[2], s[3], function() {
window.location.href = href;
});
}
).fail(notification.exception);
});
</code>

== URL module (core/url) ==

Useful Functions:
<code javascript>
// Generate an absolute URL by using the relative path to a file and optionally slash arguments
url.fileUrl(relativeScript, slashArgs)
// Generate an absolute URL from the given relative path, include the URL parameters and possibly the current session key
url.relativeUrl(relativePath, params, includeSessionKey)
// Generates an image url using the filename of the image and the Moodle component (core, plugin, etc.) where it can be found
url.imageUrl(imagename, component)
</code>

Example:

<code javascript>
// Prerequisites:
// - A Javascript file is present under $CFG->wwwroot."/path/to/file.js"
require(['core/url'], function(url) {
url.fileUrl("/path/to/file.js", "");
console.log("Generated URL: " + url);
});
</code>

== Ajax Module (core/ajax) ==

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. This saves having to worry about security in ajax functionality because it's all done for you.

For the full story, see [[AJAX]]

== Tree (core/tree) ==
{{Moodle 3.1}}
If you want to add a hierarchical interface, you should use the tree module. It will handle user interaction and accessibility for you.

<code javascript>
define(['jquery', 'core/tree'], function($, Tree) {
return {
init: function() {
new Tree("css/jquery selector of the tree container");
}
};
});
</code>

Read more about the [[Tree]] module

== Modal (core/modal) ==
{{Moodle 3.2}}

If you'd like to add a modal to your page the modal modules will handle all of the magic for you, all you need to bring is your content!
<code javascript>
require(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

Read more about the [[AMD_Modal]] module

==See also==
* [[Javascript Modules]]
* [[Fragment]]

[[Category:AJAX]]
[[Category:Javascript]]

Useful core Javascript modules

2018-03-03T17:05:24Z

Howardsmiller: /* URL module (core/url) */

{{Moodle 2.9}}

== Configuration settings (core/config) ==

Example of using config module:
<code javascript>
require(['core/config’], function(mdlcfg) {
console.log(mdlcfg.wwwroot); // outputs the wwwroot of moodle to console
});
</code>

== Language strings (core/str) ==

Example of using language strings module (retrieved via ajax, if the string is not yet loaded)
<code javascript>
require([‘core/str’], function(str) {
// start retrieving the localized string; store the promise that some time in the future the string will be there.
var editaPresent = str.get_string('edita', 'core', stringargument);
// as soon as the string is retrieved, i.e. the promise has been fulfilled,
// edit the text of a UI element so that it then is the localized string
// Note: $.when can be used with an arbitrary number of promised things
$.when(editaPresent).done(function(localizedEditString) {
$("someUIElementSelector").text = localizedEditString;
});
});
</code>

The string will be retrieved via AJAX request on the first use and cached in browser local storage until purge caches or site upgrade

IT IS NOT RECOMMENDED to call:
<code php>
$PAGE->requires->string_for_js('edita', 'core'); // You do not need this!
</code>

Because:
* The list of strings used now needs to be maintained in 2 places
* The strings are always sent, and bloat the page size even if they are not used
* All strings fetched via the AJAX method above are cached in browser local storage anyway, so these strings will never be used 99% of the time

==Notifications (core/notification)==

Examples:
<code javascript>
require(['core/notification’], function(notification) {
notification.alert('Hello', 'Welcome to my site!', 'Continue');
});
</code>

Strings and notifications:
<code javascript>
require(['core/str', 'core/notification’], function(str, notification) {
str.get_strings([
{'key' : 'delete'},
{'key' : 'confirmdeletetag', component : 'tag'},
{'key' : 'yes'},
{'key' : 'no'},
]).done(function(s) {
notification.confirm(s[0], s[1], s[2], s[3], function() {
window.location.href = href;
});
}
).fail(notification.exception);
});
</code>

== URL module (core/url) ==

Useful Functions:
<code javascript>
// Generate an absolute URL by using the relative path to a file and optionally slash arguments
url.fileUrl(relativeScript, slashArgs)
// Generate an absolute URL from the given relative path, include the URL parameters and possibly the current session key
url.relativeUrl(relativePath, params, includeSessionKey)
// Generates an image url using the filename of the image and the Moodle component (core, plugin, etc.) where it can be found
url.imageUrl(imagename, component)
</code>

Example:

<code javascript>
// Prerequisites:
// - A Javascript file is present under $CFG->wwwroot."/path/to/file.js"
require(['core/url'], function(url) {
url.fileUrl("/path/to/file.js", "");
console.log("Generated URL: " + url);
});
</code>

=== Ajax Module (core/ajax) ===

The preferred way to write new ajax interactions in Moodle is to use the javascript module "core/ajax" which can directly call existing web service functions. This saves having to worry about security in ajax functionality because it's all done for you.

For the full story, see [[AJAX]]

== Tree (core/tree) ==
{{Moodle 3.1}}
If you want to add a hierarchical interface, you should use the tree module. It will handle user interaction and accessibility for you.

<code javascript>
define(['jquery', 'core/tree'], function($, Tree) {
return {
init: function() {
new Tree("css/jquery selector of the tree container");
}
};
});
</code>

Read more about the [[Tree]] module

== Modal (core/modal) ==
{{Moodle 3.2}}

If you'd like to add a modal to your page the modal modules will handle all of the magic for you, all you need to bring is your content!
<code javascript>
require(['jquery', 'core/modal_factory'], function($, ModalFactory) {
var trigger = $('#create-modal');
ModalFactory.create({
title: 'test title',
body: '<p>test body content</p>',
footer: 'test footer content',
}, trigger)
.done(function(modal) {
// Do what you want with your new modal.
});
});
</code>

Read more about the [[AMD_Modal]] module

==See also==
* [[Javascript Modules]]
* [[Fragment]]

[[Category:AJAX]]
[[Category:Javascript]]

Javascript Modules

2018-03-03T16:42:04Z

Howardsmiller: /* Minimum (getting started) module for plugins */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[http://gruntjs.com/ grunt]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

This means you first have to '''install nodejs''' - and its package manager [https://www.npmjs.com/ npm]. The details of how to install those packages will vary by operating system, but on Linux it's probably similar to "sudo apt-get install nodejs npm". There are downloadable packages for other operating systems here: http://nodejs.org/download/. Moodle currently requires node "8.9.x".

Once this is done, you can '''run the command''':

npm install
npm install -g grunt-cli

from the top of the Moodle directory to install all of the required tools. (You may need extra permissions to use the -g option.)

== Development mode ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

define(['jquery'], function($) {

return {
init: function() {

// Put whatever you like here. $ is available
// to you as normal.
$(".someclass").change(function() {
alert("It changed!!");
});
}
};
});
</code>

This code passes the jquery module into our function (parameter $). There are a number of other useful modules available in Moodle, some of which you'll probably need in a practical application. See [[Useful_core_Javascript_modules]].

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
return {
init: function(first, last) {
}
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

/**
* @module block_overview/helloworld
*/
define(['jquery'], function($) {

/**
* Give me blue.
* @access private
* @return {string}
*/
var makeItBlue = function() {
// We can use our jquery dependency here.
return $('.blue').show();
};

/**
* @constructor
* @alias module:block_overview/helloworld
*/
var greeting = function() {
/** @access private */
var privateThoughts = 'I like the colour blue';

/** @access public */
this.publicThoughts = 'I like the colour orange';

};

/**
* A formal greeting.
* @access public
* @return {string}
*/
greeting.prototype.formal = function() {
return 'How do you do?';
};

/**
* An informal greeting.
* @access public
* @return {string}
*/
greeting.prototype.informal = function() {
return 'Wassup!';
};
return greeting;
});
</code>

The most interesting line above is:
<code>
define(['jquery'], function($) {
</code>

All AMD modules must call "define()" as the first and only global scoped piece of code. This ensures the javascript code contains no global variables and will not conflict with any other loaded module. The name of the module does not need to be specified because it is determined from the filename and component (but it can be listed in a comment for JSDoc as shown here).

The first argument to "define" is the list of dependencies for the module. This argument must be passed as an array, even if there is only one. In this example "jquery" is a dependency. "jquery" is shipped as a core module is available to all AMD modules.

The second argument to "define" is the function that defines the module. This function will receive as arguments, each of the requested dependencies in the same order they were requested. In this example we receive JQuery as an argument and we name the variable "$" (it's a JQuery thing). We can then access JQuery normally through the $ variable which is in scope for any code in our module.

The rest of the code in this example is a standard way to define a Javascript module with public/private variables and methods. There are many ways to do this, this is only one.

It is important that we are returning 'greeting'. If there is no return then your module will be declared as undefined.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? Stuffing all possible required modules in the define call is one solution, but it's ugly and it only works for code that is in an AMD module (what about inline code in the page?). AMD lets you load a dependency any time you like.
<code javascript>

// Load a new dependency.
require(['mod_wiki/timer'], function(timer) {
// timer is available to do my bidding.
});
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'initialise', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
define(['jquery', 'theme_mytheme/jquery.countdown'], function($, c) {
return {
initialise: function ($params) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
}
};
});
</code>

2. Put the javascript into a mustache template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="clock"></span>
{{#js}}
require(['jquery', 'theme_mytheme/jquery.countdown'], function($) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
{{/js}}
</code>

3. Call the javascript directly from php (although who would want to put javascript into php? ergh):
<code php>
$PAGE->requires->js_amd_inline('
require(['theme_mytheme/jquery.countdown'], function(min) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
');
</code>

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)
* [[Useful_core_Javascript_modules]]

[[Category:AJAX]]
[[Category:Javascript]]

Javascript Modules

2018-03-03T16:41:42Z

Howardsmiller: /* Minimum (getting started) module for plugins */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[http://gruntjs.com/ grunt]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

This means you first have to '''install nodejs''' - and its package manager [https://www.npmjs.com/ npm]. The details of how to install those packages will vary by operating system, but on Linux it's probably similar to "sudo apt-get install nodejs npm". There are downloadable packages for other operating systems here: http://nodejs.org/download/. Moodle currently requires node "8.9.x".

Once this is done, you can '''run the command''':

npm install
npm install -g grunt-cli

from the top of the Moodle directory to install all of the required tools. (You may need extra permissions to use the -g option.)

== Development mode ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

define(['jquery'], function($) {

return {
init: function() {

// Put whatever you like here. $ is available
// to you as normal.
$(".someclass").change(function() {
alert("It changed!!");
});
}
};
});
</code>

This code passes the jquery module into our function (parameter $). There are a number of other useful functions available in Moodle, some of which you'll probably need in a practical application. See [[Useful_core_Javascript_modules]].

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
return {
init: function(first, last) {
}
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

/**
* @module block_overview/helloworld
*/
define(['jquery'], function($) {

/**
* Give me blue.
* @access private
* @return {string}
*/
var makeItBlue = function() {
// We can use our jquery dependency here.
return $('.blue').show();
};

/**
* @constructor
* @alias module:block_overview/helloworld
*/
var greeting = function() {
/** @access private */
var privateThoughts = 'I like the colour blue';

/** @access public */
this.publicThoughts = 'I like the colour orange';

};

/**
* A formal greeting.
* @access public
* @return {string}
*/
greeting.prototype.formal = function() {
return 'How do you do?';
};

/**
* An informal greeting.
* @access public
* @return {string}
*/
greeting.prototype.informal = function() {
return 'Wassup!';
};
return greeting;
});
</code>

The most interesting line above is:
<code>
define(['jquery'], function($) {
</code>

All AMD modules must call "define()" as the first and only global scoped piece of code. This ensures the javascript code contains no global variables and will not conflict with any other loaded module. The name of the module does not need to be specified because it is determined from the filename and component (but it can be listed in a comment for JSDoc as shown here).

The first argument to "define" is the list of dependencies for the module. This argument must be passed as an array, even if there is only one. In this example "jquery" is a dependency. "jquery" is shipped as a core module is available to all AMD modules.

The second argument to "define" is the function that defines the module. This function will receive as arguments, each of the requested dependencies in the same order they were requested. In this example we receive JQuery as an argument and we name the variable "$" (it's a JQuery thing). We can then access JQuery normally through the $ variable which is in scope for any code in our module.

The rest of the code in this example is a standard way to define a Javascript module with public/private variables and methods. There are many ways to do this, this is only one.

It is important that we are returning 'greeting'. If there is no return then your module will be declared as undefined.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? Stuffing all possible required modules in the define call is one solution, but it's ugly and it only works for code that is in an AMD module (what about inline code in the page?). AMD lets you load a dependency any time you like.
<code javascript>

// Load a new dependency.
require(['mod_wiki/timer'], function(timer) {
// timer is available to do my bidding.
});
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'initialise', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
define(['jquery', 'theme_mytheme/jquery.countdown'], function($, c) {
return {
initialise: function ($params) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
}
};
});
</code>

2. Put the javascript into a mustache template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="clock"></span>
{{#js}}
require(['jquery', 'theme_mytheme/jquery.countdown'], function($) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
{{/js}}
</code>

3. Call the javascript directly from php (although who would want to put javascript into php? ergh):
<code php>
$PAGE->requires->js_amd_inline('
require(['theme_mytheme/jquery.countdown'], function(min) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
');
</code>

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)
* [[Useful_core_Javascript_modules]]

[[Category:AJAX]]
[[Category:Javascript]]

Javascript Modules

2018-03-03T16:38:09Z

Howardsmiller: /* Useful links */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[http://gruntjs.com/ grunt]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

This means you first have to '''install nodejs''' - and its package manager [https://www.npmjs.com/ npm]. The details of how to install those packages will vary by operating system, but on Linux it's probably similar to "sudo apt-get install nodejs npm". There are downloadable packages for other operating systems here: http://nodejs.org/download/. Moodle currently requires node "8.9.x".

Once this is done, you can '''run the command''':

npm install
npm install -g grunt-cli

from the top of the Moodle directory to install all of the required tools. (You may need extra permissions to use the -g option.)

== Development mode ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

define(['jquery'], function($) {

return {
init: function() {

// Put whatever you like here. $ is available
// to you as normal.
$(".someclass").change(function() {
alert("It changed!!");
});
}
};
});
</code>

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
return {
init: function(first, last) {
}
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

/**
* @module block_overview/helloworld
*/
define(['jquery'], function($) {

/**
* Give me blue.
* @access private
* @return {string}
*/
var makeItBlue = function() {
// We can use our jquery dependency here.
return $('.blue').show();
};

/**
* @constructor
* @alias module:block_overview/helloworld
*/
var greeting = function() {
/** @access private */
var privateThoughts = 'I like the colour blue';

/** @access public */
this.publicThoughts = 'I like the colour orange';

};

/**
* A formal greeting.
* @access public
* @return {string}
*/
greeting.prototype.formal = function() {
return 'How do you do?';
};

/**
* An informal greeting.
* @access public
* @return {string}
*/
greeting.prototype.informal = function() {
return 'Wassup!';
};
return greeting;
});
</code>

The most interesting line above is:
<code>
define(['jquery'], function($) {
</code>

All AMD modules must call "define()" as the first and only global scoped piece of code. This ensures the javascript code contains no global variables and will not conflict with any other loaded module. The name of the module does not need to be specified because it is determined from the filename and component (but it can be listed in a comment for JSDoc as shown here).

The first argument to "define" is the list of dependencies for the module. This argument must be passed as an array, even if there is only one. In this example "jquery" is a dependency. "jquery" is shipped as a core module is available to all AMD modules.

The second argument to "define" is the function that defines the module. This function will receive as arguments, each of the requested dependencies in the same order they were requested. In this example we receive JQuery as an argument and we name the variable "$" (it's a JQuery thing). We can then access JQuery normally through the $ variable which is in scope for any code in our module.

The rest of the code in this example is a standard way to define a Javascript module with public/private variables and methods. There are many ways to do this, this is only one.

It is important that we are returning 'greeting'. If there is no return then your module will be declared as undefined.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? Stuffing all possible required modules in the define call is one solution, but it's ugly and it only works for code that is in an AMD module (what about inline code in the page?). AMD lets you load a dependency any time you like.
<code javascript>

// Load a new dependency.
require(['mod_wiki/timer'], function(timer) {
// timer is available to do my bidding.
});
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'initialise', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
define(['jquery', 'theme_mytheme/jquery.countdown'], function($, c) {
return {
initialise: function ($params) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
}
};
});
</code>

2. Put the javascript into a mustache template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="clock"></span>
{{#js}}
require(['jquery', 'theme_mytheme/jquery.countdown'], function($) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
{{/js}}
</code>

3. Call the javascript directly from php (although who would want to put javascript into php? ergh):
<code php>
$PAGE->requires->js_amd_inline('
require(['theme_mytheme/jquery.countdown'], function(min) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
');
</code>

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)
* [[Useful_core_Javascript_modules]]

[[Category:AJAX]]
[[Category:Javascript]]

Using the File API in Moodle forms

2018-02-26T17:33:12Z

Howardsmiller: /* Add file manager element */

{{Moodle 2.0}}

{{Moodle 3.X}}

See an example of using a filemanager in an activity module in: https://github.com/CARLOSEDUARDOVIEIRA/moodle-uploadfile.

This document shows you exactly how to use Moodle forms to get files from users in a standard and secure way.

==Overview==

In Moodle 2.0 all files are stored in a central database accessible via the [[File API|File API]], and every file is associated with a component and a "file area" in Moodle, such as a particular module.

A common use case is to provide a form (using Moodle's [[lib/formslib.php|Forms API]]) which allows users to upload or import files as attachments or media embedded into HTML.

Normally this works like this:
# User starts creation or re-edits an existing item in Moodle (eg forum post, resource, glossary entry etc)
# User presses some sort of button to browse for new files to attach or embed
# User sees our "Choose file..." dialog, which contains one or more repository instances.
# User chooses a file, the corresponding [[Repository plugins|Repository plugin]] takes care of copying the file into a "draft file area" within Moodle
# File appears in the text or as an attachment in the form.
# When the user hits save, the [[File API|File API]] is invoked to move the file from the draft file area into a permanent file area associated with that data

This document shows you exactly how to use Moodle forms to interact with users in a standard and secure way.

If you just want to write code to manipulate Moodle files internally (without user input) then see [[File API]].

==Form elements==

In Moodle 2.0 there are three file-related form elements for interacting with users:

# filemanager - the way to attach one or more files as a set
# editor - the way to specify a textarea with a HTML editor, and all the handling of images and movies within that HTML
# filepicker - a way to specify one file for the case when you want to process the file and throw it away

In Moodle 1.9 there were two other types which are now '''deprecated''' (they work, but please do not use these anymore)
# file - used to just allow a normal file upload from the desktop only.
# htmleditor - this old method of embedding a HTML editor in a textarea is not able to support repositories etc.

===filepicker===

File picker (''filepicker'') is a direct replacement of the older ''file'' formslib element.

It is intended for situations when you want the user to upload '''one''' file so you can process it and delete it, such as when you are importing data from a CSV file.
If you want a file that remains part of the Moodle storage and will reappear when you reopen the form, then you should use a filemanager instead (and restrict it to a single file, if necessary).

==== Using the filepicker element ====

<code php>
$mform->addElement('filepicker', 'userfile', get_string('file'), null,
array('maxbytes' => $maxbytes, 'accepted_types' => '*'));
</code>

==== Obtain the chosen file ====

To get the contents of the file:

<code php>
$content = $mform->get_file_content('userfile');
</code>

To get the name of the chosen file:

<code php>
$name = $mform->get_new_filename('userfile');
</code>

To save the chosen file to the server filesystem (such as to moodledata folder):

<code php>
$success = $mform->save_file('userfile', $fullpath, $override);
</code>

To store the chosen file in the Moodle files pool:

<code php>
$storedfile = $mform->save_stored_file('userfile', ...);
</code>

=== filemanager ===

The File Manager element improves on file picker by allowing you to manage more than one file. It is expected that the files will be stored permanently for future use (such as forum and glossary attachments).

==== Add file manager element ====

Example:
<code php>
$mform->addElement('filemanager', 'attachments', get_string('attachment', 'moodle'), null,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'areamaxbytes' => 10485760, 'maxfiles' => 50,
'accepted_types' => array('document'), 'return_types'=> FILE_INTERNAL | FILE_EXTERNAL));
</code>

Here are the fields for filemanager:

;'filemanager':This is a filemanager element :)
;elementname:The unique name of the element in the form
;elementlabel:The label string that users see
;attributes:(leave it as null)
;options: an array of further options for the filepicker (see below)

The options array can contain:

;subdirs:(Default 1) Are subdirectories allowed? (true or false)
;maxbytes:(Default 0) Restricts the size of each individual file.
;areamaxbytes:(Default 0) Restricts the total size of all the files.
;maxfiles:(Default -1) Restricts the total number of files.
;accepted_types:(Default *) You can specify what file types are accepted by filemanager. All current file types are listed in this file: [http://cvs.moodle.org/moodle/lib/filestorage/file_types.mm moodle/lib/filestorage/file_types.mm]. This is a [http://freemind.sourceforge.net/wiki/index.php/Main_Page freemind] file: if it is edited the changes will be immediately reflected in Moodle.

As of 2.3 file_types.mm is no longer used, instead the file types are listed in <code>get_mimetypes_array()</code> in lib/classes/filelib.php (https://github.com/moodle/moodle/blob/master/lib/classes/filetypes.php).

Example usage: '''array('audio', 'video', 'document')''', you can include file extensions as well, for example: '''array('.txt', '.jpg', 'audio')'''.

==== Load existing files into draft area ====

<code php>
if (empty($entry->id)) {
$entry = new stdClass;
$entry->id = null;
}

$draftitemid = file_get_submitted_draft_itemid('attachments');

file_prepare_draft_area($draftitemid, $context->id, 'mod_glossary', 'attachment', $entry->id,
array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));

$entry->attachments = $draftitemid;

$mform->set_data($entry);

</code>

==== Store updated set of files ====

<code php>
if ($data = $mform->get_data()) {
// ... store or update $entry
file_save_draft_area_files($data->attachments, $context->id, 'mod_glossary', 'attachment',
$entry->id, array('subdirs' => 0, 'maxbytes' => $maxbytes, 'maxfiles' => 50));
}
</code>

===editor===
There are two ways of using the editor element in code, the first one is easier but expects some standardized fields. The second method is more low level.

====Simple use====
# name database fields: ''textfield'', ''textfieldformat'' (and ''textfieldtrust'' if required)
# create options array. note that context is the best, most local context you have available.
#:<code php>$textfieldoptions = array('trusttext'=>true, 'subdirs'=>true, 'maxfiles'=>$maxfiles, 'maxbytes'=>$maxbytes, 'context'=>$context);</code>
# add editor ''textfield_editor'' to moodle form, pass options through custom data in form constructor, set $data->id to null if data not exist yet
#:<code php>$mform->addElement('editor', 'textfield_editor', get_string('fieldname', 'somemodule'), null, $textfieldoptions);</code>
# prepare data
#:<code php>$data = file_prepare_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>
# get submitted data and after inserting/updating of data
#:<code php>$data = file_postupdate_standard_editor($data, 'textfield', $textfieldoptions, $context, 'mod_somemodule', 'somearea', $data->id);</code>

Real world examples are in mod/glossary/edit.php and mod/glossary/comment.php

====Low level use====

When using editor element you need to preprocess and postprocess the data:
# detect if form was already submitted (usually means draft is area already exists) - ''file_get_submitted_draft_itemid()''
# prepare draft file area, temporary storage of all files attached to the text - ''file_prepare_draft_area()''
# convert encoded relative links to absolute links - ''file_prepare_draft_area()''
# create form and set current data
# after submission the changed files must be merged back into original area - ''file_save_draft_area_files()''
# absolute links have to be replaced by relative links - ''file_save_draft_area_files()''

=====Replace old htmleditor with editor=====

The file picker has been integrated with with TinyMCE to make the editor element. This new element should support all types on editors and should be able to switch them on-the-fly. Instances of the old htmleditor element in your forms should be replaced by the new editor element, this may need adding of new format and trusttext columns. For example:
<code php>
$mform->addElement('editor', 'entry', get_string('definition', 'glossary'), null,
array('maxfiles' => EDITOR_UNLIMITED_FILES));
</code>
The editor element can take following options: maxfiles, maxbytes, subdirs and changeformat. Please note that the embedded files is optional feature and is not expected be used everywhere.

'''Note''': the editor element now includes text format option. You should no longer use the separate format element type.

=====Prepare current data - text and files=====

<code php>
if (empty($entry->id)) {
$entry = new object();
$entry->id = null;
$entry->definition = '';
$entry->format = FORMAT_HTML;
}

$draftid_editor = file_get_submitted_draft_itemid('entry');
$currenttext = file_prepare_draft_area($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $entry->definition);
$entry->entry = array('text'=>$currenttext, 'format'=>$entry->format, 'itemid'=>$draftid_editor);

$mform->set_data($entry);
</code>

If there are multiple files, they will share the same itemid.

=====Obtain text, format and save draft files=====

To retrieve editor content, you need to use following code:

<code php>
if ($fromform = $mform->get_data()) {
// content of editor
$messagetext = $fromform->entry['text'];
// format of content
$messageformat = $fromform->entry['format'];
}
</code>

When a user selects a file using the file picker, the file is initially stored in a draft file area, and a URL is inserted into the HTML in the editor that lets the person editing the content (but no one else) see the file.

When the user submits the form, we then need to save the draft files to the correct place in permanent storage. (Just like you have to call $DB->update_record('tablename', $data); to have the other parts of the form submission stored correctly.)

The save_files_from_draft_area function and replace absolute links with internal relative links do:
<code php>
$messagetext = file_save_draft_area_files($draftid_editor, $context->id, 'mod_glossary', 'entry',
$entry->id, array('subdirs'=>true), $messagetext);
</code>
; $context->id, 'component', 'proper_file_area' and $entry->id : correspond to the contextid, filearea and itemid columns in the [[File_API#Table:_files|files table]].
; $messagetext : this is the message text. As the files are saved to the real file area, the URLs in this content are rewritten.

All URLs in content that point to files managed to the File API are converted to a form that starts '@@PLUGINFILE@@/' before the content is stored in the database. That is what we mean by rewriting.

== File serving==

=== Convert internal relative links to absolute links ===

Before text content is displayed to the user, any URLs in the '@@PLUGINFILE@@/' form in the content need to be rewritten to the real URL where the user can access the files.
<code php>
$messagetext = file_rewrite_pluginfile_urls($messagetext, 'pluginfile.php',
$context->id, 'mod_mymodule', 'proper_file_area', $itemid);
</code>
; $messagetext : is the content containing the @@PLUGINFILE@@ URLs from the database.
; 'pluginfile.php' : there are a number of different scripts that can serve files with different permissions checks. You need to specify which one to use.
; $context->id, 'mod_mymodule', 'proper_file_area', $itemid : uniquely identifies the file area, as before.

=== Implement file serving access control ===

Attachments and embedded images should have the same access control like the text itself, in majority of cases these files are served using pluginfile.php. Access control is defined in ''module/lib.php'' file in function ''module_pluginfile()''.

== File browsing support ==
Only owner of each file area is allowed to use low level File API function to access files, other parts of Moodle should use file browsing API.

Activities may specify browsing support in own module/lib.php file by implementing functions module_get_file_areas() and module_get_file_info().

== Upgrading your code ==
Here I will attempt to describe some simple steps you can take to upgrade your file-handling form elements from pre-2.0 code to 2.0. We will use the example of glossary, since it has been used above.

=== Preparing your options ===
Unless you are happy with the defaults, you will need to define an array of options for each file-handling form element. You could define it at different places, but it's best to put it in one place and make the array(s) available to other files if they need it. In the majority of cases, this will be in a file like edit.php

Previous code in mod/glossary/edit.php:
<code php>
$mform =& new mod_glossary_entry_form(null, compact('cm', 'glossary', 'hook', 'mode', 'e', 'context'));
</code>
New code:
<code php>
$maxbytes = $course->maxbytes;
// Could also use $CFG->maxbytes if you are not coding within a course context

$definitionoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes, 'trusttext'=>true,
'context'=>$context);
$attachmentoptions = array('subdirs'=>false, 'maxfiles'=>99, 'maxbytes'=>$maxbytes);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Note that the data being passed to the form constructor have changed also, but this is not part of the file API changes, I just include them to avoid confusion.

These options are for the htmleditor (definition field) and the filemanager (attachment field). They are used by a file called edit_form.php.

=== Element preparation ===
Before we look at this, however, we need to "prepare" the elements so that they can correctly display existing embedded images and attached files when you are editing a record instead of just creating one. So, let's take the code we've got so far in edit.php and add to it:

Currently upgraded code in edit.php:
<code php>
$mform = new mod_glossary_entry_form(null, array(
'current'=>$entry,
'cm'=>$cm,
'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
New code with element preparation:
<code php>
$entry = file_prepare_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_prepare_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
$mform = new mod_glossary_entry_form(null, array('current'=>$entry, 'cm'=>$cm, 'glossary'=>$glossary,
'definitionoptions'=>$definitionoptions,
'attachmentoptions'=>$attachmentoptions));
</code>
Things to note:
* $entry in this case is simply a stdClass object which may either represent a new glossary entry or an existing one.
* $entry->id must be the unique identifier for the current object. If we are creating a new entry, it will be null, but in all cases it must be defined.
* These two functions (file_prepare_standard_editor and file_prepare_standard_filemanager) are shortcuts functions that take care of some of the tedious setting up for you, but they make a couple of assumptions:
*# You '''must''' name the form element as {element}_editor or {element}_filemanager (see next section)
*# You '''must''' have at least the following fields in the database: {element} and {element}summary, as described earlier in this documentation

We can now look at the upgrades needed in the form definition file.

=== Form definition ===
Previous code in mod/glossary/edit_form.php:
<code php>
$mform->addElement('htmleditor', 'definition', get_string('definition', 'glossary'),
array('rows'=>20));
$mform->setType('definition', PARAM_RAW);
$mform->addRule('definition', null, 'required', null, 'client');
$mform->setHelpButton('definition', array('writing', 'richtext'), false, 'editorhelpbutton');
$mform->addElement('format');
// a bit further...
$this->set_upload_manager(new upload_manager('attachment', true, false, $COURSE, false, 0,
true, true, false));
$mform->addElement('file', 'attachment', get_string('attachment', 'forum'));
$mform->setHelpButton('attachment', array('attachment', get_string('attachment', 'glossary'),
'glossary'));
</code>
New code:
<code php>
$definitionoptions = $this->_customdata['definitionoptions'];
$attachmentoptions = $this->_customdata['attachmentoptions'];
// a bit further...
$mform->addElement('editor', 'definition_editor', get_string('definition', 'glossary'), null,
$definitionoptions);
$mform->setType('definition_editor', PARAM_RAW);
$mform->addRule('definition_editor', get_string('required'), 'required', null, 'client');
// a bit further...
$mform->addElement('filemanager', 'attachment_filemanager', get_string('attachment', 'glossary'),
null, $attachmentoptions);
$mform->setHelpButton('attachment_filemanager', array('attachment2',
get_string('attachment', 'glossary'), 'glossary'));
</code>

Note the following:
* The format element and the help button are no longer required for the HTML editor element
* The name of the form element needs to be changed by adding '_editor' or '_manager' to the original name. This is a naming convention that is used by a couple of functions we will look at shortly
* Make sure $definitionoptions has context parameter, else system context is used and editor will not respect filter settings.

=== Handling submitted data ===
The final step is to handle the submitted data properly, i.e. retrieve the files and save them to disk, associating them with the record we have just created (a glossary entry in our example). This happens in edit.php:

Previous code in edit.php:
<code php>
// Section that updates an entry:
$todb->id = $e;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
$todb->attachment = $newfilename;
}

// Section that adds an entry:
if ($todb->id = insert_record("glossary_entries", $todb)) {
$e = $todb->id;
$dir = glossary_file_area_name($todb);
if ($mform->save_files($dir) and $newfilename = $mform->get_new_filename()) {
set_field("glossary_entries", "attachment", $newfilename, "id", $todb->id);
}
}
</code>
New code:
<code php>
// $todb was renamed to $entry, and the code was refactored
// so that the file-handling code is only used once for either an add or an update action.
// If an entry is being added, $DB->insert() has already been called, so we have a valid $entry->id
$entry = file_postupdate_standard_editor($entry, 'definition', $definitionoptions, $context,
'mod_glossary', 'entry', $entry->id);
$entry = file_postupdate_standard_filemanager($entry, 'attachment', $attachmentoptions, $context,
'mod_glossary', 'attachment', $entry->id);
// store the updated value values
$DB->update_record('glossary_entries', $entry);
</code>
Things to note:
* If you are adding a new record, you will still need to call update_record after calling the file_postupdate* functions

=== Gotchas ===
A few things to keep in mind:
* Make sure that you instantiate the moodle form before any call to $OUTPUT->header()

== See also ==

* [[File API]]
* [[Using the file API]]
* [[Repository plugins]]
* [[Portfolio API]]
* MDL-14589 - File API Meta issue
* [https://moodle.org/mod/forum/discuss.php?d=207748 Adding a text editor to a Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=157953#p692822 Button actions in Moodle form]
* [https://moodle.org/mod/forum/discuss.php?d=351013#p1416554 File Picker]

[[Category:Files]]
[[Category:Repositories]]

Running acceptance test

2017-11-24T13:32:16Z

Howardsmiller: /* 2. Set up Selenium */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode was not tested.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running" when running the behat tests). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>' (if you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files)
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Running acceptance test

2017-11-24T13:31:05Z

Howardsmiller: /* 1. Background */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode was not tested.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>' (if you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files)
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Running acceptance test

2017-11-24T13:30:15Z

Howardsmiller: /* 5. Run Behat tests */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode is not considered in this section.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>' (if you don't want to run all the tests add '<tt>--tags="@something"</tt>' where the @something refers to the tags at the top of most feature files)
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Running acceptance test

2017-11-24T13:29:31Z

Howardsmiller: /* 5. Run Behat tests */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode is not considered in this section.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>' (if you don't want to run all the tests add '<tt>--tag="@something"</tt>' where the @something refers to the tags at the top of most feature files)
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Running acceptance test

2017-11-24T12:53:52Z

Howardsmiller: /* 2. Set up Selenium */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode is not considered in this section.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running). Try the following
# Run '<tt>Xvfb -ac :99 -screen 0 1280x1024x16 &</tt>'
# Then immediately, '<tt>export DISPLAY=:99</tt>'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>'
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Running acceptance test

2017-11-24T12:53:23Z

Howardsmiller: /* 2. Set up Selenium */

== Short version ==

...or how I got it to work on Ubuntu and some of the problems encountered.

You need a bunch of browsers and terminal windows open to do this.

==== 1. Background ====

# I am using the desktop version of Ubuntu 17.04 so there are no issues about running this software in headless mode. Running in headless mode is not considered in this section.
# Moodle is version 3.3 and is a fully installed and working version using the 'standard' Ubuntu LAMP stack.

==== 2. Set up Selenium ====

# Download the Selenium Standalone Server from [http://www.seleniumhq.org/download/ http://www.seleniumhq.org/download/]. It's a single JAR file, put it anywhere handy.
# Download the Chrome driver from [https://sites.google.com/a/chromium.org/chromedriver/ https://sites.google.com/a/chromium.org/chromedriver/] (Forget trying to use Firefox, the latest version is not compatible)
# Unzip the driver (it's a single file) and copy to /usr/local/bin (should work anywhere on the path)
# If not installed already, '<tt>sudo apt install default-jre</tt>'
# Start Selenium - '<tt>java -jar /path/to/your/selenium/server/selenium-server-standalone-N.NN.N.jar -port 4444</tt>'
# check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

If running headless or the above doesn't work ("Selenium server is not running). Try the following
# Run 'Xvfb -ac :99 -screen 0 1280x1024x16 &'
# Then immediately, 'export DISPLAY=:99'
# The run the Selenium command as above

==== 3. Set up Moodle ====

# Create a new 'dataroot' area for files especially for behat adjusting permissions accordingly.
# If not there already, add Section 11 from config-dist.php to your config.php file and review the settings.
# $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.
# $CFG->behat_dataroot should point to the directory you created above
# $CFG->behat_prefix should be fine.
# Set up $CFG->behat_profiles to select Chrome as the browser...

$CFG->behat_profiles = [
'default' => [
'browser' => 'chrome',
'extensions' => [
'Behat\MinkExtension' => [
'selenium2' => [
'browser' => 'chrome',
]
]
]
]
];

==== 4. Configure Behat for Moodle ====

# Run '<tt>php admin/tool/behat/cli/init.php</tt>' (from the root of your Moodle install). This installs all the required software and creates the test version of Moodle.

==== 5. Run Behat tests ====

# Run '<tt>php admin/tool/behat/cli/run.php</tt>'
# After some initial setup dots should start to go by. It's a while before Selenium is first accessed. On the Linux desktop a new Chrome window appears and the testing process 'remote control' starts (hopefully!)

== Prerequisite ==
Before initializing acceptance test environment for running behat, you should ensure:
# [[Acceptance_testing#Requirements Meet min. system requirements for running tests]]
# [[Acceptance_testing#Installation Have set min. config variable in config.php for behat]]
# [[Acceptance_testing#Installation Downloaded composer dependencies]]

== Overview ==
Acceptance tests (also known as behat), use [http://www.seleniumhq.org/download/ Selenium server] and can be run as:
# '''Single run:''' In single run, only one behat run is executed. So all features are executed in this single run.
# '''Parallel runs:''' (Since Moodle 3.0) Parallel runs allow dev's to execute multiple behat runs together. This was introduced to get acceptance tests results faster. To achieve this:
#* Features are divided between multiple behat runs
#* Symlinks behatrun{x} (x being the run process number), are created pointing to moodle directory, so site for run 1 is accessible via https://localhost/moodle/behatrun1
#* Process number is included as suffix to $CFG->behat_prefix.
#* Process number is suffixed to $CFG->behat_dataroot.

== Step 1: Initialise acceptance test environment ==
Before running acceptance tests, environment needs to be initialised for acceptance testing.

=== Single run ===
For initialising acceptance tests for single run, above command is sufficient.
<code>
php admin/tool/behat/cli/init.php
</code>

=== Parallel runs ===
For initialising acceptance tests for parallel runs, you can use one of the following options
# '''-j or --parallel''' (required) Number of parallel behat run to initialise
# '''-m or --maxruns''' (optional) Max parallel site which should be initialised at one time. If your system is slow, then you can initialise sites in chucks.
# '''--fromrun''' (optional) Initialise site to run specified run from. Used for running acceptance tests on different vms
# '''--torun''' (optional) Initialise site to run specified run till. Used for running acceptance tests on different vms
# '''-o or --optimize-runs''' (optional) This option will split features with specified tags in all parallel runs, so they are executed first when parallel run gets executed.
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified themes (comma separated list of themes)
<code>
// Below command will initialise moodle to run 2 parallel tests.
php admin/tool/behat/cli/init.php --parallel=2
</code>

== Step 2: Running acceptance test environment ==
=== Single run ===
Run either of the following commands. For more options '''vendor/bin/behat --help''' or http://docs.behat.org/guides/6.cli.html
<code>
vendor/bin/behat --config /path/to/your/CFG_behat_dataroot/behatrun/behat/behat.yml
</code>
<code>
php admin/tool/behat/cli/run.php
</code>

=== Parallel runs ===
For running parallel runs, use following command
<code>
php admin/tool/behat/cli/run.php
</code>
Following optional options are available for custom run:
# '''--feature''' Only execute specified feature file (Absolute path of feature file).
# '''--suite''' Features for specified theme will be executed.
# '''--replace''' Replace args string with run process number, useful for output and reruns.
# '''--fromrun''' Execute run starting from (Used for parallel runs on different vms)
# '''--torun''' Execute run till (Used for parallel runs on different vms)
# '''-a or --add-core-features-to-theme''' (optional) Since Moodle 3.2. Use this option to add all core features to specified theme's (comma separated list)
# Behat options can be passed for filtering features/scenarios:
#* In case you don't want to run Javascript tests, use the Behat tags option to skip them, '''--tags="~@javascript"'''
#* In case you want to run specific scenario, use the Behat name option to run it, '''--name="Filter user accounts by role and cohort"'''
#* In case you want to run specific feature file, use the Behat feature option to run it, '''--feature="/PATH/TO/MOODLE/admin/tests/behat/filter_users.feature"'''

=== Common options for running tests ===
==== Tests filters ====
With the '''--tags''' or the '''-name''' Behat options you can filter which tests are going to run or which ones are going to be skipped. There are a few tags that you might be interested in:
* '''@javascript''': All the tests that runs in a browser using Javascript; they require Selenium to be running, otherwise an exception will be thrown.
* '''@_file_upload''': All the tests that involves file uploading or any OS feature that is not 100% part of the browser. They should only be executed when Selenium is running in the same machine where the tests are running.
* '''@_alert''': All the tests that involves Javascript dialogs (alerts, confirms...) are using a feature that is OS-dependant and out of the browser scope, so they should be tag appropriately as not all browsers manage them properly.
* '''@_switch_window''': All the tests that are using the '''I switch to "NAME" window''' step should be tagged as not all browsers manage them properly.
* '''@_switch_iframe''': All the tests that are using the '''I switch to "NAME" window''' steps should be tagged as it is an advanced feature and some browsers may have problems dealing with them
* '''@_cross_browser''': All the tests that should run against multiple combinations of browsers + OS in a regular basis. The features that are sensitive to different combinations of OS and browsers should be tagges as @_cross_browser.
* '''@componentname''': Moodle features uses the [https://docs.moodle.org/dev/Frankenstyle Frankenstyle] component name to tag the features according to the Moodle subsystem they belong to.

==== Output formats ====
Since Moodle 3.1 option for output is:
<code>
--format=pretty --out=/path/to/pretty.txt --format=moodle_progress --out=std
</code>
Before Moodle 3.1 option for output was:
<code>
--format='moodle_progress,pretty' --out=',/path/to/pretty.txt'
</code>
Following output formats are supported:
# '''progress''': Prints one character per step.
# '''pretty''': Prints the feature as is.
# '''junit''': Outputs the failures in JUnit compatible files.
# '''moodle_progress''': Prints Moodle branch information and dots for each step.
# '''moodle_list''': List all scenarios.
# '''moodle_stepcount''': List all features with total steps in each feature file. Used for parallel run.
# '''moodle_screenshot''': (since Moodle 3.1) Take screenshot and core dump of each step. With following options you can dump either or both.
## --format-settings '{"formats": "image"}'**: will dump image only
## --format-settings '{"formats": "html"}'**: will dump html only.
## --format-settings '{"formats": "html,image"}'**: will dump both.
## --format-settings '{"formats": "html", "dir_permissions": "0777"}'**
If you want to see the failures immediately (rather than waiting ~3 hours for all the tests to finish) then either use the -v option to output a bit more information, or change the output format using --format. Format 'pretty' ('''-f pretty''') is sufficient for most cases, as it outputs each step outcomes in the command line making easier to see the progress.

== Advance usage ==
=== Rerun failed scenarios ===
With slow systems or parallel run you might see some random failures, to rerun only failed scenarios (to eliminate random failures), use --rerun option
# '''Single run:''' --run="absolute_path_to_empty_file" (Behat will record failed scenarios in this file, and when run again only failed scenarios will be run)
# '''Parallel run:''' --rerun="absolute_path_to_empty_file_{runprocess}.txt --replace="{runprocess}" ({runprocess} will be replaced with the process number for recording fails in the specific run process).
'''Since Moodle 3.1 --rerun option don't accept any value, as it is handled internally by behat'''

=== Running behat with specified theme (Since Moodle 3.2) ===
You can run behat with any theme installed. To execute behat with specified theme use '''--suite={THEME_NAME}''' option, while running behat. By default the features in theme behat folder will be executed for the suite. But if you want to run all core features with the specific theme then initialise acceptance test with -a option. For example, '''-a {THEME_NAME}'''
Also if you want a specific theme for your behat tests, use the $CFG->theme variable to set the theme.

==== Override behat core context for theme suite ====
To override behat step definitions so as to run behat with specified theme, you should create a contexts within '''/theme/{MYTHEME}/tests/behat/''' with prefix behat_theme_{MYTHEME}_ and suffixed with the context being overridden. For example, if you want to override behat_mod_forum context, then you should create a class /theme/{MYTHEME}/tests/behat/mod_forum/behat_theme_{MYTHEME}_behat_mod_forum.php

==== Blacklist behat context or features to run in theme suite ====
To blacklist contexts/ features to be executed by theme suite you should create a /theme/{MYTHEME}/tests/behat/blacklist.json file with following format. Following will not use step_definitions from behat_grade and behat_navigation while running theme suite. Also, scenarios in auth/tests/behat/login.feature and grade/tests/behat/grade_hidden_items.feature won't be executed with theme suite.
<code>
{
"contexts": [
"behat_grade",
"behat_navigation",
],
"features": [
"auth/tests/behat/login.feature",
"grade/tests/behat/grade_hidden_items.feature",
]
}
</code>
==== Override core behat selectors ====
To override behat selectors in specific theme, you should create a class behat_theme_{MYTHEME}_behat_selectors in /theme/{MYTHEME}/tests/behat/behat_theme_{MYTHEME}_behat_selectors.php extending behat_selectors.

=== Use php built in web server ===
You can use php built-in-web server for executing behat runs. To do so:
# Open a command line interface and '''cd /to/your/moodle/dirroot'''
# '''php -S localhost:8000''' (This is the test site URL that moodle uses by default, if you want to use another one you can override it in config.php with $CFG->behat_wwwroot attribute; more info in https://docs.moodle.org/dev/Acceptance_testing#Advanced_usage or config-dist.php)
# Update $CFG->behat_wwwroot = localhost:8000; in config.php

=== Define custom options for parallel runs ===
You can set following custom config options for parallel runs via $CFG->behat_parallel_run. It's an array of options where 1st array is for 1st run and so on.
<code>
array (
'dbtype' => 'mysqli',
'dblibrary' => 'native',
'dbhost' => 'localhost',
'dbname' => 'moodletest',
'dbuser' => 'moodle',
'dbpass' => 'moodle',
'behat_prefix' => 'mdl_',
'wd_host' => 'http://127.0.0.1:4444/wd/hub',
'behat_wwwroot' => 'http://127.0.0.1/moodle',
'behat_dataroot' => '/home/example/bht_moodledata'
)
</code>

To set different selenium servers for parallel runs, you can use following. NOTE: Running parallel (headless) runs on different selenium servers avoid random focus failures.
<code>
$CFG->behat_parallel_run = array (
array ('wd_host' => 'http://127.0.0.1:4444/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4445/wd/hub'),
array ('wd_host' => 'http://127.0.0.1:4446/wd/hub'),
);
</code>

=== Running acceptance tests with different browser ===
By default behat will run with Firefox browser through Selenium. By adding the following code to your config.php you can change the selected browser that is run when behat is invoked. You will need to run php admin/tool/behat/cli/init.php for changes to take effect. Then use --profile='chrome'.

<code>
$CFG->behat_profiles = array(
'chrome' => array(
'browser' => 'chrome',
'tags' => '@javascript',
)
);
</code>
[[Acceptance_testing/Browsers|More info about alternative browsers]]

=== Start multiple selenium servers ===
From command line Start selenium servers at different ports (say 4444, 4445, 4446 for 3 parallel runs)
<code>
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4444 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4445 &
java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar -port 4446
</code>

=== Using Docker to start selenium server ===
==== What is Docker ====
Docker is a app container, it's a kind of virtual machine, but only for one app, service, so you can download
a docker image and run a selenium server without worry in how to configure selenium in your machine, one for chrome, others for firefox, you either don't need to install the browsers in your machine
To install docker follow this link; https://docs.docker.com/engine/installation/

==== Selenium docker images ====
There is many docker images available, for many browser, the complete list is in https://hub.docker.com/u/selenium/
for moodle you can use standalone version.
You can download specific selenium version too, for example, for firefox, moodle recommend selenium 2.53.1, see: [https://docs.moodle.org/dev/Acceptance_testing/Browsers/Working_combinations_of_OS%2BBrowser%2Bselenium What version do I need?]

so the command will be:
<code>
docker run -d -p4444:4444 selenium/standalone-firefox:2.53.1-beryllium
</code>
to see all available version click in tags. For firefox you can find at: https://hub.docker.com/r/selenium/standalone-firefox/tags/

==== Change config.php file ====
In config.php file you must change the $CFG->behat_wwwroot= to your network card (NIC) ip address, you can't use
localhost , 127.0.0.1, ... or selenium docker server will fail

== NOTE ==
# Start the Selenium server (in case you want to run tests that involves Javascript)
#* (See http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/ for running 'headless' Firefox and xvfm in a server environment)
#* Open another command line interface and '''java -jar /path/to/your/selenium/server/selenium-server-standalone-2.NN.N.jar'''

=== Trouble shooting ===
=== New step definitions or features are not executed ===
If you are adding new tests or steps definitions update the tests list
<code>
php admin/tool/behat/cli/util.php --enable
</code>
''' For parallel runs, all options for initialising parallel runs are valid '''

=== Tests are failing ===
If you followed all the steps and you receive an unknown weird error probably your system's Firefox version is not compatible with the Selenium version you are running. Please refer Working combinations to ensure you have correct [[Acceptance_testing/Browsers#Working_combinations_of_OS.2BBrowser.2Bselenium]] of them to run acceptance test.

=== Disable acceptance test environment ===
if you want to prevent access to test environment
<code>
php admin/tool/behat/cli/util.php --disable
</code>
'''Note that if you have the HTTP_PROXY environment variable set, which you may have had to do to run composer, then you also need to set NO_PROXY=localhost.'''

== External links ==
* Vagrant profile with Moodle and Behat preconfigured: https://github.com/mackensen/moodle-hat
* Docker containers for Moodle Developers and Behat: https://github.com/moodlehq/moodle-docker
* Docker environment with Behat preconfigured : https://github.com/tobiga/docker_moodle_environment

[[Category:Quality Assurance]]

Templates

2017-11-13T13:48:56Z

Howardsmiller: /* How do I call a template from php? */

{{Moodle 2.9}}

= Templates =

== What is a template? ==
A template is an alternative to writing blocks of HTML directly in javascript / php by concatenating strings. The end result is the same, but templates have a number of advantages:
* It is easier to see the final result of the template because the code for a template is very close to what the final HTML will look like
* Because the templating language is intentionally limited, it is hard to introduce complex logic into a template. This makes it far easier for a theme designer to override a template, without breaking the logic
* Templates can be rendered from javascript. This allows ajax operations to re-render a portion of the page.

== How do I write a template? ==
Templates are written in a language called "[http://mustache.github.io/mustache.5.html Mustache]". Mustache is written as HTML with additional tags used to format the display of the data. Mustache tags are made of 2 opening and closing curly braces <code xml>{{tag}}</code>. There are a few variations of these tags that behave differently.
* <code xml>{{raiden}}</code> This is a simple variable substitution. The variable named "raiden" will be searched for in the current context (and any parent contexts) and when a value is found, the entire tag will be replaced by the variable (HTML escaped).
* <code xml>{{{galaga}}}</code> This is an unescaped variable substitution. Instead of escaping the variable before replacing it in the template, the variable is included raw. This is useful when the variable contains a block of HTML (for example).
* <code xml>{{#lemmings}} jump off cliff {{/lemmings}}</code> These are opening and closing section tags. If the lemmings variable exists and evaluates to "not false" value, the variable is pushed on the stack, the contents of the section are parsed and included in the result. If the variable does not exist, or evaluates to false - the section will be skipped. If the variable lemmings evaluates to an array, the section will be repeated for each item in the array with the items of the array on the context. This is how to output a list.
* <code xml>{{^lemmings}} enjoy view {{/lemmings}}</code> Equivalent of "if-not" block, there is no "else" in mustache.
* <code xml>{{> franken_style/name_of_template }}</code> This is a partial. Think of it like an include. Templates can include other templates using this tag. In the called template, the data it sees (for including values is the same as the data available where the partial is included.
* <code xml>{{$blockvar}} ... {{/blockvar}}</code> This is a block variable. It defines a section of the template that can be overridden when it's included in another template.
* <code xml>{{< template_name}} ... {{/template_name}}</code> This is similar to including a partial but specifically indicates that you'd like to override one or more block variables defined within the template you're including. You can override the block variables by defining a block variable within these tags that matches the name of the block variable you'd like to override in the included template.

So - putting this all together:

recipe.mustache
<code xml>
<h3>{{recipename}}</h3>
<p>{{description}}</p>
<h4>Ingredients</h4>
<ol>
{{#ingredients}}
<li>{{.}}</li>
{{/ingredients}}
</ol>
<h4>Steps</h4>
<ol>
{{#steps}}
<li>{{{.}}}</li>
{{/steps}}
</ol>
{{ > ratethisrecipe }}
</code>

When given this data:
<code javascript>
{
recipename: "Cheese sandwich",
description: "Who doesn't like a good cheese sandwich?",
ingredients: ["bread", "cheese", "butter"],
steps: ["<p>Step 1 is to spread the butter on the bread</p>", "<p>Step 2 is to put the cheese "in" the bread (not on top, or underneath)</p>"]
}
</code>

Gives this: <span style="font-size:4em">😋</span>

More info - there are much clearer explanations of templates on the [http://mustache.github.io/mustache.5.html Mustache] website. Try reading those pages "before" posting on stack overflow :) .

=== Blocks (Moodle 3.0 onwards) ===
Blocks are a feature of Mustache that deserves a special mention. The are used as a form of inheritance - and are crucial to building a library of re-usable templates. To make use of "blocks" you define a parent template with replaceable sections. Each of those sections is marked with a "blocks" tag like this (A blocks tag looks like a regular tag, but the variable name is preceded with $):

<code xml>
<section>
<h1>{{$sectionheading}}Default heading{{/sectionheading}}</h1>
<div>
{{$content}}Content for section{{/content}}
</div>
</section>
</code>

Now - wherever I need to re-use this template I can include it and replace the content of those sections at the same time.

<code xml>
{{< section}}
{{$sectionheading}}Latest News{{/sectionheading}}
{{$content}}Nothing happened today - sorry!{{/content}}
{{/section}}
</code>

Notice that when I include a template and I want to make use of blocks - the include tag points the other way:
<code xml>
{{< section}}
</code>
Not
<code xml>
{{> section}}
</code>

Blocks support looping and many other cool things - for more info see https://github.com/bobthecow/mustache.php/wiki/BLOCKS-pragma

== Where do I put my templates? ==

Templates go in the <componentdir>/templates folder and must have a .mustache file extension. When loading templates the template name is <componentname>/<filename> (no file extension).

So "mod_lesson/timer" would load the template at mod/lesson/templates/timer.mustache.

Note: Do not try and put your templates in sub folders under the "/templates" directory. This is not supported and will not work.

== How do I call a template from javascript? ==

Rendering a template from javascript is fairly easy. There is a new AMD module that can load/cache and render a template for you.

<code javascript>
// This is AMD code for loading the "core/templates" module. see [Javascript Modules].
require(['core/templates'], function(templates) {

// This will be the context for our template. So {{name}} in the template will resolve to "Tweety bird".
var context = { name: 'Tweety bird', intelligence: 2 };

// This will call the function to load and render our template.
templates.render('block_looneytunes/profile', context);

// It returns a promise that needs to be resoved.
.then(function(html, js) {
// Here eventually I have my compiled template, and any javascript that it generated.
// The templates object has append, prepend and replace functions.
templates.appendNodeContents('.block_looneytunes .content', source, javascript);
}).fail(function(ex) {
// Deal with this exception (I recommend core/notify exception function for this).
});
});
</code>

Under the hood, this did many clever things for us. It loaded the template via an ajax call if it was not cached. It found any missing lang strings in the template and loaded them in a single ajax request, it split the JS from the HTML and returned us both in easy to use way. Read on for how to nicely deal with the javascript parameter.

Note: with some nice chaining and sugar, we can shorten the above example quite a bit:
<code javascript>
require(['core/templates', 'core/notification'], function(templates, notification) {
var context = { name: 'Tweety bird', intelligence: 2 };
templates.render('block_looneytunes/profile', context)
.then(doneCallback)
.fail(notification.exception);
});
</code>

== What if a template contains javascript? ==

Sometimes a template requires that some JS be run when it is added to the page in order to give it more features. In the template we can include blocks of javascript, but we should use a special section tag that has a "helper" method registered to handle javascript carefully.

Example
profile.mustache
<code xml>
<div id="profile">
<p>Name: {{name}}</p>
<p>Intelligence: {{intelligence}}</p>
</div>
{{#js}}
require(['jquery'], function($) {
// Effects! Can we have "blink"?
$('#profile').slideDown();
});
{{/js}}
</code>

If this template is rendered by PHP, the javascript is separated from the HTML, and is appended to a special section in the footer of the page "after" requirejs has loaded. This provides the optimal page loading speed. If the template is rendered by javascript, the javascript source will be passed to the "done" handler from the promise. Then, when the "done" handler has added the template to the DOM, it can call
<code javascript>templates.runTemplateJS(javascript);</code>
which will run the javascript (by creating a new script tag and appending it to the page head).

== What other helpers can I use? ==

The implementation of these helpers is in classes like [https://github.com/moodle/moodle/blob/master/lib/classes/output/mustache_string_helper.php#L31 \core\output\mustache_string_helper]. This is set up in the [https://github.com/moodle/moodle/blob/master/lib/outputrenderers.php#L100 get_mustache method in renderer_base] (There should be no need to look at the implementation, unless you are interested.) If you are considering adding a new helper, it should be limited to display logic - and you need to be able to create identical helpers for the javascript and php implementations (javascript ones go in lib/amd/src/templates.js).

=== {{# str }} ===

There is a string helper for loading language strings.

<code xml>
{{# str }} helloworld, mod_greeting {{/ str }}
</code>

The first 2 parameters are the string id and the component name. So this is effectively Mustache variant of <code>get_string('helloworld', 'mod_greeting')</code>.

The optional third parameter defines the value for the string's <code>$a</code> placeholder:

<code xml>
{{# str }} iscool, mod_cool, David Beckham {{/ str }}
</code>

This example would effectively do what <code>get_string('iscool', 'mod_cool', 'David Beckham')</code> does in Moodle PHP code.

Variable tags are allowed to define the value of the <code>$a</code> placeholder:

<code xml>
{{# str }} iscool, mod_cool, {{ name }} {{/ str }}
</code>

For strings that accept complex placeholder, see the following section.

=== {{# quote }} ===

As shown in the previous section, the <code>{{# str }}</code> helper may need complex data structures passed as the value of the <code>$a</code> placeholder. You can use a JSON object syntax in that case:

<code xml>
{{# str }} iscool, mod_cool, { "firstname": "David", "lastname": "Beckham" } {{/ str }}
</code>

If you wanted to use the context values instead of literal strings, you might intuitively use something like this:

<code xml>
{{! DO NOT DO THIS !}}
{{# str }} iscool, mod_cool, { "firstname": "{{ firstname }}", "lastname": "{{ lastname }}" } {{/ str }}
</code>

There is a potential problem though. If the variable tag <code>{{ firstname }}</code> or <code>{{ lastname }}</code> evaluates to a string containing the double quote character, that will break the JSON syntax. We need to escape the double quotes potentially appearing in the variable tags. For this, use the <code>{{# quote }}</code> helper:

<code xml>
{{! This is OK !}}
{{# str }} iscool, mod_cool, { "firstname": {{# quote }}{{ firstname }}{{/ quote }}, "lastname": {{# quote }}{{ lastname }}{{/ quote }} } {{/ str }}
</code>

See MDL-52136 for details.
=== {{# pix }} ===

There is a pix icon helper for generating pix icon tags.

<code xml>
{{# pix }} t/edit, core, Edit David Beckham {{/ pix }}
</code>

The first 2 parameters are the string id and the component name, the rest is the alt text for the image.

=== {{# userdate }} (Moodle 3.3 onwards) ===
{{Moodle 3.3}}
This mustache template helper will format unix timestamps into a given human readable date format while using the user's timezone settings configured (if any) in Moodle. The helper will accept hardcoded values, context variables, or other helpers.

The recommended way to use this helper is to use the string helper to get one of the core Moodle formats because they have been translated into other languages so you'll get multi-lang support for free (that's a pretty good deal!).

==== Using the string helper (recommended) ====
<code>
// Assuming we had a context variable named "time" set to 1293876000.
{{#userdate}} {{time}}, {{#str}} strftimedate {{/str}} {{/userdate}}
</code>
This will ask the Moodle server for the string "strftimedate" and will use the value (which in this case is "%d %B %Y") to format the user date. So the resulting formatted timestamp from the userdate helper will be "01 January 2011".

==== Using context variables ====
<code>
// Assuming we had a context variable named "time" set to 1293876000.
{{#userdate}} {{time}}, %A, %d %B %Y, %I:%M %p {{/userdate}}
</code>
Will produce "Saturday, 01 January 2011, 10:00 AM"

==== Using hardcoded values ====
<code xml>
{{#userdate}} 1293876000, %A, %d %B %Y, %I:%M %p {{/userdate}}
</code>
Will produce "Saturday, 01 January 2011, 10:00 AM"

=== {{# shortentext }} (Moodle 3.3 onwards) ===
{{Moodle 3.3}}
This helper can be used to shorten a large amount of text to a specified length and will append a trailing ellipsis to signify that the text has been shortened.

The algorithm will attempt to preserve words while shortening to text. Words, for the purposes of the helper, are considered to be groups of consecutive characters broken by a space or, in the case of a multi-byte character, after the completion of the multi-byte character (rather than in the middle of the character).

It will also attempt to preserve HTML in the text by keeping the opening and closing tags. Only text within the tags will be considered when calculating how much should be truncated to reach the desired length.

The helper takes two comma separated arguments. The first is the desired length and the second is the text to be shortened. Both can be provided as context variables.

==== Plain text ====
<code>
{{#shortentext}} 30, long text without any tags blah de blah blah blah what {{/shortentext}}
</code>
Will produce:
<code>
long text without any tags ...
</code>

==== HTML text ====
<code>
{{#shortentext}} 30, <div class='frog'><p><blockquote>Long text with tags that will be chopped off but <b>should be added back again</b></blockquote></p></div></p> {{/shortentext}}
</code>
Will produce:
<code>
<div class='frog'><p><blockquote>Long text with tags that ...</blockquote></p></div>
</code>

==== Multi-byte text ====
<code>
{{#shortentext}} 6, 𠮟𠮟𠮟𠮟𠮟𠮟𠮟𠮟𠮟𠮟𠮟𠮟 {{/shortentext}}
</code>
Will produce:
<code>
𠮟𠮟𠮟...
</code>

== How do I call a template from php? ==

The templates in php are attached to the renderers. There is a renderer method "render_from_template($templatename, $context)" that does the trick.

Note $context is nothing to do with normal Moodle 'contexts'. It is the data passed to the template (the 'context' mentioned in mustache documentation).

== How do templates work with renderers? ==

Extra care must be taken to ensure that the data passed to the context parameter is useful to the templating language. The template language cannot:
* Call functions
* Perform any boolean logic
* Render renderables
* Do capability checks
* Make DB queries

So - I have "some" data in my renderable and some logic and HTML generation in my render method for that renderable - how do I refactor this to use a template?

The first thing to note, is that you don't have to use a template if you don't want to. It just means that themers will still have to override your render method, instead of just overriding the template. But if you DO want to use a template, you will earn "cred" with themers, and you will be able to re-render parts of your interface from javascript in response to ajax requests without reloading the whole page (that's cool).

There is a simple pattern to use to hook a template into a render method. If you make your renderable implement templatable as well as renderable - it will have to implement a new method "export_for_template(renderer_base $output)". This method takes the data stored in the renderable and "flattens it" so it can be used in a template. If there is some nested data in the renderable (like other renderables) and they do not support templates, they can be "rendered" into the flat data structure using the renderer parameter. It should return an stdClass with properties that are only made of simple types: int, string, bool, float, stdClass or arrays of these types. Then the render method can updated to export the data and render it with the template.

In the renderable:
<code php>
/**
* Export this data so it can be used as the context for a mustache template.
*
* @return stdClass
*/
public function export_for_template(renderer_base $output) {
$data = new stdClass();
$data->canmanage = $this->canmanage;
$data->things = array();
foreach ($this->things as $thing) {
$data->things[] = $thing->to_record();
}
$data->navigation = array();
foreach ($this->navigation as $button) {
$data->navigation[] = $output->render($button);
}

return $data;
}
</code>

In the renderer class:
<code php>
/**
* Defer to template.
*
* @param mywidget $widget
*
* @return string HTML for the page
*/
render(mywidget $widget) {
$data = $widget->export_for_template($this);
return $this->render_from_template('mywidget', $data);
}
</code>

== How to I override a template in my theme? ==

Templates can be overridden a bit easier than overriding a renderer. First - find the template that you want to change. E.g. "mod/wiki/templates/ratingui.mustache". Now, create a sub-folder under your themes "templates" directory with the component name of the plugin you are overriding. E.g "theme/timtam/templates/mod_wiki". Finally, copy the ratingui.mustache file into the newly created "theme/timtam/templates/mod_wiki" and edit it. You should see your changes immediately if theme designer mode is on. Note: templates are cached just like CSS, so if you are not using theme designer mode you will need to purge all caches to see the latest version of an edited template. If the template you are overriding contains a documentation comment (see next section) it is recommended to remove it, it will still show the documentation in the template library.

== Should I document my templates? ==

Yes!!!! Theme designers need to know the limits of what they can expect to change without breaking anything. As a further benefit - your beautiful new template can be displayed in the "Template Library" tool shipped with Moodle. In order to provide nice documentation and examples for the Template Library, you should follow these conventions when documenting your template.

=== Add a documentation comment to your template ===
Mustache comments look like this:
<pre>
{{!
I am a comment.
I can span multiple lines.
}}
</pre>

The template library will look for a mustache comment that contains this special marker as the documentation to display, and the source of an example context.

<code>
@template component/templatename
</code>

==== Useful things to include in the documentation for a template ====
===== Classes required for JS =====
This is a list of classes that are used by the javascript for this template. If removing a class from an element in the template will break the javascript, list it here.

===== Data attributes required for JS =====
This is a list of data attributes (e.g. data-enhance="true") that are used by the javascript for this template. If removing a data attribute from an element in the template will break the javascript, list it here.

===== Context variables required for this template =====
This is a description of the data that may be contained in the context that is passed to the template. Be explicit and document every attribute.

===== Example context (JSON) =====
The Template Library will look for this data in your documentation comment as it allows it to render a "preview" of the template right in the Template Library. This is useful for theme designers to test all the available templates in their new theme to make sure they look nice in a new theme. It is also useful to make sure the template responds to different screen sizes, languages and devices. The format is a JSON-encoded object that is passed directly into the render method for this template.

==== A full example ====
lib/templates/pix_icon.mustache
<pre>
{{!
This file is part of Moodle - http://moodle.org/

Moodle is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

Moodle is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with Moodle. If not, see <http://www.gnu.org/licenses/>.
}}
{{!
@template core/pix_icon

Moodle pix_icon template.

The purpose of this template is to render a pix_icon.

Classes required for JS:
* none

Data attributes required for JS:
* none

Context variables required for this template:
* attributes Array of name / value pairs.

Example context (json):
{
"attributes": [
{ "name": "src", "value": "http://moodle.com/wp-content/themes/moodle/images/logo-hat2.png" },
{ "name": "class", "value": "iconsmall" }
]
}

}}
<img {{#attributes}}{{name}}="{{value}}" {{/attributes}}/>
</pre>

== Coding style for templates ==
This section documents some coding style guidelines to follow when writing templates. The reason for these guidelines is to promote consistency, and interoperability of the templates.

=== Include GPL at the top of each template ===

Templates are a form of code and it is appropriate to license them like any other code.

===Include a documentation comment for each template===

The exception is when you are overriding a template, if the documentation from the parent still applies, you do not need to copy it to the overridden template.

===Use data-attributes for JS hooks===

Data attributes are ideal for adding javascript hooks to templates because:
* Classes are meant for styling - theme designers should be able to change the classes at will without breaking any functionality.
* IDs must be unique in the page, but it is not possible to control how many times the same template might be included in the page.
* Data attributes can have meaningful names and can be efficiently queried with a selector

===Avoid custom CSS for templates===

This is not a hard rule, but a preference. We already have too much CSS in Moodle - where ever possible we should try and re-use the existing CSS instead of adding new CSS to support every new template.

===Re-use core templates as much as possible===

First we need to build the core set of reusable templates - but once that is in place we should always try to re-use those core templates to build interfaces. This will make Moodle more consistent, attractive and customisable.

===Do use the CSS framework classes directly in the templates===

We have bootstrap in core - so lets make the most of it. There is no problem using bootstrap classes in core templates, as long as the "base" theme is also tested, and an overridden template is added there if required.

===Avoid IDs for styling or javascript===

IDs should never evet be used for styling as they have a high CSS specificity, and so are hard to override. In addition, IDs should be unique in the page, which implies that a template could only be used once in a page. IDs are also not ideal for javascript, for the same reason (must be unique in a page).

The only acceptable case to use an ID is you need to create a one to one connection between the JS and template. In this case use the uniqid helper to generate an ID that will not conflict with any other template on the page, and use it as part of the ID.

<code php>
<div id="{{uniqid}}-somethingspecific">
</div>
{{#js}}
callFunction('{{uniqid}}-somethingspecific');
{{/js}}
</code>

===Follow CSS coding style===

https://docs.moodle.org/dev/CSS_coding_style

Use hyphens as word-separators for class names.
Use lower case class names.

===Wrap each template in one node with a classname that matches the template name===

Generate a class name by combining the component and template names and separating words with underscore.

e.g.

<code php>
<div class="core_user_header">
...
</div>
</code>

[[Category:AJAX]]
[[Category:Javascript]]
[[Category:Output]]

===Iterating over php arrays in a mustache template===

Mustache treats hashes and arrays differently because of cross language compatibility
In php arrays and hashes are the same, but mustache treats them differently
It decides a php array is a hash and will not iterate over it if it is non 0 indexed and/or has a gap in the key numbers
so in short
you need to
<code php>
$datafortemplate->mylist = array_values($myarraywithnonnumerickeys)
</code>
you could also use
<code php>
$datafortemplate->mylist = new ArrayIterator($myarraywithnonnumerickeys);
</code>
BUT this fails when $myarraywithnonnumerickeys is empty and you try to use
<code html5>
{{#mylist}}
with an array iterator if mylist is empty this block will not run
{{/mylist}}
{{^mylist}}
with an array iterator mylist will not run this block either because it is not quite empty
{{/mylist}}
</code>

===Test for not empty array in a mustache template===

Short answer you can't. Include a specific property in the context like "hasusers".

In php
<pre>
{{#users.0}} blah {{/users.0}}
</pre>

will work but please don't use this syntax, it will break if the template is used in javascript AND it mucks up the context.

In javascript

<pre>
{{#users.length}} blah {{/users.length}}
</pre>

will work but please don't use this syntax, it will break if the template is used in php.

===Squash whitespace in a template===
Sometimes whitespace is significant, for example inside a link it will show with an underline. If you want 2 mustache tags from separate lines to be rendered with no whitespace between them you can use mustache comments to squash the whitespace.

<pre>
<a href="blah">{{!
}}{{icon}}{{!
}}{{name}}</a>
</pre>

== How do I write core templates? ==
Core templates should ideally be simple generic components that can be used within other templates to create more complex page layouts. They should be flexible enough for developers and themers to easily use without having to replace the template. The templates should attempt to encapsulate some core structure for the element as well as key classes while allowing the content to be easily overridden. Ultimately we want to avoid having duplicate HTML copied from template to template where possible, particularly if the HTML element has some classes associated with it.

Mustache relies on variables to substitute context data into the template but unfortunately it's very unlikely that the the names of the context data will match what the template is expecting for all the places that the template might be used. So in order to allow easy extensibility and avoid having to duplicate templates just to rename the variables we can wrap them in block variables which would allow the template that is including our template to replace that variable with one from it's own context inline.

There are a few key points to keep in mind when writing a core template:
* Consider how your template will actually be used. Try writing a test page that uses your template to help discover some of the assumptions you might have in the template.
* The example context you provide in the template is mostly just for showing the template in the template library and is likely not how your template will actually be used. Most uses of the template will have a different context all together.
* Try to enforce a core structure but avoid enforcing a specific context. Content should be overridable.
* Use block variables to indicate sections of your template that people are likely to want to change. Typically where they will be wanting to substitute in their own content.
* Try to keep any javascript that accompanies the template as decoupled from the HTML / CSS structure of the template as possible. Instead of relying on the existence of certain HTML elements or CSS classes it is generally better to leverage data-attributes which can be added to any element.

=== An example: tabs ===
Let's go through an example to illustrate how you might build a core template. For the example we'll be building a tabs template, since it's a fairly complex component that requires the use of block variables and javascript.

First we can create a basic template to get the general structure down, let's call it tabs. Here's what it might look like:
<pre>
<div id="{{ uniqid }}-tab-container">
<ul role="tablist" class="nav nav-tabs">
{{# tabs }}
<li role="tab"
data-target="{{ uniqid }}-{{ id }}"
data-selected-class="active"
aria-controls="{{ uniqid }}-{{ id }}"
aria-selected="false" tabindex="-1">

<a href="#">{{{ name }}}</a>
</li>
{{/ tabs }}
</ul>
<div class="tab-content">
{{# tabs }}
<div role="tabpanel"
class="tab-pane"
id="{{ uniqid }}-{{ id }}">

{{{ content }}}
</div>
{{/ tabs }}
</div>
</div>
{{#js}}
require(['jquery','core/tabs'], function($, tabs) {

var container = $("#{{ uniqid }}-tab-container");
tabs.create(container);
});
{{/js}}
</pre>

The template requires a context that looks something like:
<pre>
{
"tabs": [
{"id":"tab1","name":"Tab 1","content":"This is tab 1 content <a href=\"#\">test</a>"},
{"id":"tab2","name":"Tab 2","content":"This is tab 2 content <a href=\"#\">test</a>"},
{"id":"tab3","name":"Tab 3","content":"This is tab 3 content <a href=\"#\">test</a>"}
]
}
</pre>

The javascript required to power the tabs element (keyboard navigation, show / hide panels etc) is written as an AMD module and is included by the template. The javascript is a little too large to go through here, but some key points to consider when writing it are:
It should ideally be independent of the HTML structure, so if someone wants to completely rewrite the tabs to be different elements (e.g. buttons or a set of divs) then the same javascript can be used without needing to change it. In order to achieve this it is important to identify the key components of the template.

In this case it is a tab list, a tab and it's content. One way to identify these components would be to inspect the structure of the DOM, for example you might say "find me the ul element" when looking for the tab list and then "find my the child li elements" to find the tabs. While this would work, it couples your javascript to the HTML structure and makes it difficult to change later. A different approach would be to use the element attributes, for example you might say "find my the element with the role 'tablist'" to get the tab list and then "find me the elements with the role 'tab'" to get the tabs. This allows the HTML structure to change without breaking the javascript (as long as the correct attributes are set, of course).

Another point of consideration for this example is what class to apply to a tab when it is selected. It makes sense to just apply something like "active" in the javascript, but that once again couples it to a particular CSS framework which makes it more difficult to change without modifying the javascript. In this case I chose to add a data attribute to the element to indicate which class will be set when the tab is selected. This means the javascript doesn't have to guess what the appropriate class is, it can just get it from the template.

Ok, so we've got our basic template. It's time to use it! Let's say we want to create a simple user profile page that might show 2 tabs, the first tab will be the user's name and the second tab will be the user's email address (please excuse the contrived example).

Here's what the page might look like:
<pre>
<html>
<header><title>User Profile</title></header>
<body>
{{< core/tabs }}
</body>
</html>
</pre>

That looks pretty simple! The only problem is, how do I get my content there? I would have to supply a context like this in order to display the tabs I want:
<pre>
{
"tabs": [
{"id":"tab1","name":"Name","content":"Your name is Mr. Test User."},
{"id":"tab2","name":"Email","content":"Your email is testuser@example.com"},
]
}
</pre>

Let's assume that the context for this page doesn't match what the tabs template is expecting though (as will be the case most of the time). Let's assume the tabs template is being rendered with this context:
<pre>
{
"name":"Mr. Test User",
"email":"testuser@example.com"
}
</pre>

Unfortunately, we'll almost certainly never have complete control over all of the contexts that our template will be rendered in which means we'll be expecting people to write new webservices to supply the same data in different formats every time they want to use a template. It becomes an unmanageable problem.

Enter blocks! We can make the template more flexible by defining sections of the template that can be overriden when they are included. Pretty neat! This will allow us to enforce a certain core structure but not enforce a context on the template that is including the tabs.

Let's have another go at that template, this time leverging blocks:
<pre>
<div id="{{ uniqid }}-tab-container">
{{$ tabheader }}
<ul role="tablist" class="nav nav-tabs">
{{$ tablist }}
{{# tabs }}
<li role="tab"
data-target="{{ uniqid }}-{{ id }}"
data-selected-class="active"
aria-controls="{{ uniqid }}-{{ id }}"
aria-selected="false" tabindex="-1">

<a href="#">{{{ name }}}</a>
</li>
{{/ tabs }}
{{/ tablist }}
</ul>
{{/ tabheader }}
{{$ tabbody }}
<div class="tab-content">
{{$ tabcontent }}
{{# tabs }}
<div role="tabpanel"
class="tab-pane"
id="{{ uniqid }}-{{ id }}">

{{{ content }}}
</div>
{{/ tabs }}
{{/ tabcontent }}
</div>
{{/ tabbody }}
</div>
{{#js}}
require(['jquery','core/tabs'], function($, tabs) {

var container = $("#{{ uniqid }}-tab-container");
tabs.create(container);
});
{{/js}}
</pre>

A summary of what we've changed:
* Added a $tabheader block around the tab list, in case someone wants to change the ul element to something else.
* Added a $tablist block around the group of tabs to allow them to be overriden on incldue.
* Added a $tabbody block around the content, in case someone wants to change the content elements from divs.
* Added a $tabcontent block around the tab variable for the content to allow the content to be overriden on inlcude.

Now let's see what using this template looks like for your User Profile page:
<pre>
<html>
<header><title>User Profile</title></header>
<body>
{{> core/tabs }}
{{$ tablist }}
<li role="tab"
data-target="{{ uniqid }}-tab1"
data-selected-class="active"
aria-controls="{{ uniqid }}-tab1"
aria-selected="false" tabindex="-1">

<a href="#">Name</a>
</li>
<li role="tab"
data-target="{{ uniqid }}-tab2"
data-selected-class="active"
aria-controls="{{ uniqid }}-tab2"
aria-selected="false" tabindex="-1">

<a href="#">Email</a>
</li>
{{/ tablist }}
{{$ tabcontent }}
<div role="tabpanel"
class="tab-pane"
id="{{ uniqid }}-tab1">

Your name is {{ name }}.
</div>
<div role="tabpanel"
class="tab-pane"
id="{{ uniqid }}-tab2">

Your email address is {{ email }}.
</div>
{{/ tabcontent }}
{{/ core/tabs }}
</body>
</html>
</pre>

That looks a bit better! Now we've been able to use the blocks to successfully change the template to use the context available to this page, we no longer need a "tabs" array with "name" and "content". Even the javascript will continue to work because we've kept the correct element attributes.

We've still got a slight problem though... In order to change the data for the template we've had to copy & paste the HTML from the original template into our blocks as we do the override. While this works fine in this example, it means we don't quite get the encapsulation we want within the templates since we're leaking internal implementation details. If we ever wanted to change the CSS framework we use for Moodle (say from bootstrap 2 to boostrap 3 or 4) we'd have to find all the places in the code where this tabs template is used and make sure that the HTML is correct in their block overrides.

With that in mind, let's take one more pass at this template and see if we can improve it slightly again. This time we're doing to split the template out into 3 templates.

tabs.mustache:
<pre>
<div id="{{ uniqid }}-tab-container">
{{$ tabheader }}
<ul role="tablist" class="nav nav-tabs">
{{$ tablist }}
{{# tabs }}
{{> core/tab_header_item }}
{{/ tabs }}
{{/ tablist }}
</ul>
{{/ tabheader }}
{{$ tabbody }}
<div class="tab-content">
{{$ tabcontent }}
{{# tabs }}
{{> core/tab_content_item }}
{{/ tabs }}
{{/ tabcontent }}
</div>
{{/ tabbody }}
</div>
{{#js}}
require(['jquery','core/tabs'], function($, tabs) {

var container = $("#{{ uniqid }}-tab-container");
tabs.create(container);
});
{{/js}}
</pre>

tab_header_item.mustache
<pre>
<li role="tab"
data-selected-class="active"
aria-selected="false" tabindex="-1">

<a href="#">{{$ tabname }}{{{ name }}}{{/ tabname }}</a>
</li>
</pre>

tab_content_item.mustache
<pre>
<div role="tabpanel"
class="tab-pane"

{{$ tabpanelcontent }}{{{ content }}}{{/ tabpanelcontent }}
</div>
</pre>

A summary of the changes:
* Split the template into 3, moving the tab into it's own template and the content into it's own and then including them in the tabs template.
* Removed the ids from the tabs and content. The javascript would be updating to assign these ids at runtime so that they don't need to be provided as part of the template context.
* Added a $tabname block for in the tab_header_item template to make the name flexible on import.
* Added a $tabpanelcontant block in the tab_content_item template to make the content flexible on import.

Cool, so let's see what that looks like in our example now:
<pre>
<html>
<header><title>User Profile</title></header>
<body>
{{> core/tabs }}
{{$ tablist }}
{{< core/tab_header_item }}
{{$ tabname }}Name{{/ tabname }}
{{/ core/tab_header_item }}
{{< core/tab_header_item }}
{{$ tabname }}Email{{/ tabname }}
{{/ core/tab_header_item }}
{{/ tablist }}
{{$ tabcontent }}
{{< core/tab_content_item }}
{{$ tabpanelcontent }}Your name is {{ name }}.{{/ tabpanelcontent }}
{{/ core/tab_content_item }}
{{< core/tab_content_item }}
{{$ tabpanelcontent }}Your email address is {{ email }}.{{/ tabpanelcontent }}
{{/ core/tab_content_item }}
{{/ tabcontent }}
{{/ core/tabs }}
</body>
</html>
</pre>

And we're done! After making the changes above we've been able to keep the benefits of the previous change to allow the context changes but we've also removed the need to copy & paste the HTML everywhere. Instead we're able to use the child templates with a few additional blocks defined to get the content in there.

Now if we want to change tabs HTML or CSS frameworks we can just change the core tabs templates and this page will receive the updates for free.

Javascript Modules

2017-10-20T14:14:49Z

Howardsmiller: /* Minimum (getting started) module for plugins */

{{Moodle 2.9}}

= Javascript Modules =

== What is a Javascript module and why do I care? ==

A Javascript module is nothing more than a collection of Javascript code that can be used (reliably) from other pieces of Javascript.

== Why should I package my code as a module? ==

By packaging your code as a module you break your code up into smaller reusable pieces. This is good because:

a) Each smaller piece is simpler to understand / debug

b) Each smaller piece is simpler to test

c) You can re-use common code instead of duplicating it

= How do I write a Javascript module in Moodle? =

Since version 2.9, Moodle supports Javascript modules written using the Asynchronous Module Definition ([https://github.com/amdjs/amdjs-api/wiki/AMD AMD]) API. This is a standard API for creating Javascript modules and you will find many useful third party libraries that are already using this format.

To edit or create an AMD module in Moodle you need to do a couple of things.

== Install grunt ==

The AMD modules in Moodle must be processed by some build tools before they will be visible to your web browser. We use "[http://gruntjs.com/ grunt]" as a build tool to wrap our different processes. Grunt is a build tool written in Javascript that runs in the "[http://nodejs.org/ nodejs]" environment.

This means you first have to '''install nodejs''' - and its package manager [https://www.npmjs.com/ npm]. The details of how to install those packages will vary by operating system, but on Linux it's probably similar to "sudo apt-get install nodejs npm". There are downloadable packages for other operating systems here: http://nodejs.org/download/. Moodle currently requires node "v4" and does not work with the latest node "v6".

Once this is done, you can '''run the command''':

npm install
npm install -g grunt-cli

from the top of the Moodle directory to install all of the required tools. (You may need extra permissions to use the -g option.)

== Development mode ==

To avoid having to constantly run grunt, make sure you set the following in your config.php

<code php>
// Prevent JS caching
$CFG->cachejs = false;
</code>

Moodle will now run your module from the amd/src module. Don't forget to switch this off and run 'grunt' before deploying the new version!

In this mode - if you get a strange message in your javascript console like "No define call for core/first" it means you have a syntax error in the javascript you are developing.
Or, "No define call for theme_XXX/loader" as you are probably missing the 'src' folder with relevant JS files. which might happen when you turn debugging ON on a theme that was bought, without 'src' folder :-(

== Running grunt ==

You can run grunt in your plugin's 'amd' directory and it will only operate on your modules. If you're having problems or just want to check your work it is worth running for the 'lint' feature. This can find basic problems. This sub-directory support wont work on Windows unfortunately but there is an alternative: Run grunt from the top directory with the --root=path/to/dir to limit execution to a sub-directory.

See [[Grunt#Running_grunt]] for more details of specific grunt commands which can be used.

If you get the error message

/usr/bin/env: node: No such file or directory

Then see the thread https://github.com/nodejs/node-v0.x-archive/issues/3911

On Ubuntu 14.04 this fixed it for me:

sudo ln -fs /usr/bin/nodejs /usr/local/bin/node

Note: Once you have run grunt and built your code, you will then need to purge Moodle caches otherwise the changes made to your minified files may not be picked up by Moodle.

== Minimum (getting started) module for plugins ==

This shows the absolute minimum module you need to get started adding modules to your plugins. It's actually quite simple...

<code javascript>
// Put this file in path/to/plugin/amd/src
// You can call it anything you like

define(['jquery'], function($) {

return {
init: function() {

// Put whatever you like here. $ is available
// to you as normal.
$(".someclass").change(function() {
alert("It changed!!");
});
}
};
});
</code>

The idea here is that we will run the 'init' function from our (PHP) code to set things up. This is called from PHP like this...

<code php>
$PAGE->requires->js_call_amd('frankenstyle_path/your_js_filename', 'init');
</code>

Don't forget to supply the complete '[[Frankenstyle]]' path. The .js is not needed.

js_call_amd takes a third parameter which is an ''array'' of parameters. These will translate to individual parameters in the 'init' function call. For example...

<code php>
$PAGE->requires->js_call_amd('block_iomad_company_admin/department_select', 'init', array($first, $last));
</code>

...calls

<code javascript>
return {
init: function(first, last) {
}
</code>

A more comprehensive explanation follows...

== "Hello World" I am a Javascript Module ==
Lets now create a simple Javascript module so we can see how to lay things out.

Each Javascript module is contained in a single source file in the <componentdir>/amd/src folder. The final name of the module is taken from the file name and the component name. E.g. block_overview/amd/src/helloworld.js would be a module named "block_overview/helloworld". the name of the module is important when you want to call it from somewhere else in the code.

After running grunt - the minified Javascript files are stored in the <componentdir>/amd/build folder. The javascript files are renamed to show that they are minified (helloworld.js becomes helloworld.min.js).

Don't forget to add the built files (the ones in amd/build) to your git commits, or in production no-one will see your changes.

Lets create a simple module now:

blocks/overview/amd/src/helloworld.js
<code javascript>
// Standard license block omitted.
/*
* @package block_overview
* @copyright 2015 Someone cool
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

/**
* @module block_overview/helloworld
*/
define(['jquery'], function($) {

/**
* Give me blue.
* @access private
* @return {string}
*/
var makeItBlue = function() {
// We can use our jquery dependency here.
return $('.blue').show();
};

/**
* @constructor
* @alias module:block_overview/helloworld
*/
var greeting = function() {
/** @access private */
var privateThoughts = 'I like the colour blue';

/** @access public */
this.publicThoughts = 'I like the colour orange';

};

/**
* A formal greeting.
* @access public
* @return {string}
*/
greeting.prototype.formal = function() {
return 'How do you do?';
};

/**
* An informal greeting.
* @access public
* @return {string}
*/
greeting.prototype.informal = function() {
return 'Wassup!';
};
return greeting;
});
</code>

The most interesting line above is:
<code>
define(['jquery'], function($) {
</code>

All AMD modules must call "define()" as the first and only global scoped piece of code. This ensures the javascript code contains no global variables and will not conflict with any other loaded module. The name of the module does not need to be specified because it is determined from the filename and component (but it can be listed in a comment for JSDoc as shown here).

The first argument to "define" is the list of dependencies for the module. This argument must be passed as an array, even if there is only one. In this example "jquery" is a dependency. "jquery" is shipped as a core module is available to all AMD modules.

The second argument to "define" is the function that defines the module. This function will receive as arguments, each of the requested dependencies in the same order they were requested. In this example we receive JQuery as an argument and we name the variable "$" (it's a JQuery thing). We can then access JQuery normally through the $ variable which is in scope for any code in our module.

The rest of the code in this example is a standard way to define a Javascript module with public/private variables and methods. There are many ways to do this, this is only one.

It is important that we are returning 'greeting'. If there is no return then your module will be declared as undefined.

== Loading modules dynamically ==
What do you do if you don't know in advance which modules will be required? Stuffing all possible required modules in the define call is one solution, but it's ugly and it only works for code that is in an AMD module (what about inline code in the page?). AMD lets you load a dependency any time you like.
<code javascript>

// Load a new dependency.
require(['mod_wiki/timer'], function(timer) {
// timer is available to do my bidding.
});
</code>

== Including an external javascript/jquery library ==
If you want to include a javascript / jquery library downloaded from the internet you can do so as follows:

'''Warning: if the library you download, supports AMD but is already "named" you will not be able to include it directly'''
e.g.
<code javascript>
define("typeahead.js", *[ "jquery" ], function(a0) {
return factory(a0);
});
</code>
will not work, as moodle injects it's own define name when loading the library.

If the library is in AMD format and has a define:
e.g. i want to include the jquery final countdown timer on my page ( hilios.github.io/jQuery.countdown/ )
* download the module in both normal and minified versions
* place the modules in your moodle install e.g. your custom theme dir, or plugin dir
* /theme/mytheme/amd/src/jquery.countdown.js

you can now include the module and initialise it (there are multiple ways to do this)
php:

1. Create your own AMD module and initialise it:

In your PHP file:
<code php>
$this->page->requires->js_call_amd('theme_mytheme/countdowntimer', 'initialise', $params);
</code>

Javascript module:
<code javascript>
// /theme/mytheme/amd/src/countdowntimer.js
define(['jquery', 'theme_mytheme/jquery.countdown'], function($, c) {
return {
initialise: function ($params) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
}
};
});
</code>

2. Put the javascript into a mustache template:
<code javascript>
// /theme/mytheme/templates/countdowntimer.mustache
<span id="clock"></span>
{{#js}}
require(['jquery', 'theme_mytheme/jquery.countdown'], function($) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
{{/js}}
</code>

3. Call the javascript directly from php (although who would want to put javascript into php? ergh):
<code php>
$PAGE->requires->js_amd_inline('
require(['theme_mytheme/jquery.countdown'], function(min) {
$('#clock').countdown('2020/10/10', function(event) {
$(this).html(event.strftime('%D days %H:%M:%S'));
});
});
');
</code>

== Embedding AMD code in a page ==
So you have created lots of cool Javascript modules. Great. How do we actually call them? Any javascript code that calls an AMD module must execute AFTER the requirejs module loader has finished loading. We have provided a function "js_call_amd" that will call a single function from an AMD module with parameters.

<code php>
$PAGE->requires->js_call_amd($modulename, $functionname, $params);
</code>

that will "do the right thing" with your block of AMD code and execute it at the end of the page, after our AMD module loader has loaded.
Notes:
* the $modulename is the 'componentname/modulename' discussed above
* the $functionname is the name of a public function exposed by the amd module.
* the $params is an array of params passed as arguments to the function. These should be simple types that can be handled by json_encode (no recursive arrays, or complex classes please).
* if the size of the params array is too large (> 1Kb), this will produce a developer warning. Do not attempt to pass large amounts of data through this function, it will pollute the page size. A preferred approach is to pass css selectors for DOM elements that contain data-attributes for any required data, or fetch data via ajax in the background.

AMD / JS code can also be embedded on a page via mustache templates
see here: https://docs.moodle.org/dev/Templates#What_if_a_template_contains_javascript.3F

== But I have a mega JS file I don't want loaded on every page? ==
Loading all JS files at once and stuffing them in the browser cache is the right choice for MOST js files, there are probably some exceptions. For these files, you can rename the javascript file to end with the suffix "-lazy.js" which indicates that the module will not be loaded by default, it will be requested the first time it is used. There is no difference in usage for lazy loaded modules, the require() call looks exactly the same, it's just that the module name will also have the "-lazy" suffix.

== Useful links ==
* [https://assets.moodlemoot.org/sites/15/20171004085436/JavaScript-AMD-with-RequireJS-presented-by-Daniel-Roperto-Catalyst.pdf JavaScript AMD with RequireJS] presented by Daniel Roperto, Catalyst. (MoodleMOOT AU 2017)

[[Category:AJAX]]
[[Category:Javascript]]