MoodleDocs - User contributions [en]

Output callbacks

2020-03-04T08:20:55Z

Jan.dagefoerde: /* before_http_headers */ Add reference to after_config

There are cases where we want any plugin to contribute to a chunk of the output of any given page. We want this to be loosely coupled so you don't have say an admin tool's code leaking into your theme. In the past many plugins had an installation step like "Copy this into your theme header" which is what we want to completely avoid.

There are a variety of [[Callbacks]] which enable any plugin to add or modify certain parts of the output and at certain stages in the rendering process. These callbacks are probably most useful for local plugins or admin tools which bring some functionality to the whole site instead of just certain pages. But they can also be used by any plugin to conditionally augment the output too.

= add_htmlattributes =

{{Moodle 3.3}}

This is used to append extra attributes into the html element of the page which are required elsewhere. An example might be a facebook block plugin which uses the opengraph js libraries which need the xml namespace to be setup. It should return an array of attribute key and values like this:

<code>
function tool_facebook_add_htmlattributes() {
return array(
'xmlns:og' => 'http://ogp.me/ns#',
);
}
</code>

Because multiple plugins can add extra attributes there is potential namespace clash issues. ie in the Facebook opengraph example stick with "xmlns:og" as specified by Facebook so that if multiple plugins declare the same key / value then they will match. Not that duplicates attribute keys are merged.

= before_footer =

{{Moodle 3.3}}

This enables you to easily inject a chunk of JS or CSS into every page, for instance an analytics tool like Google Analytics or Facebook pixel. It only has side effects and it's return value is ignored:

<code>
function tool_mytool_before_footer() {
global $PAGE;
$PAGE->requires->js_init_code("alert('before_footer');");
}
</code>

= before_http_headers =

{{Moodle 3.3}}

This enables you to easily inject a HTTP header into every page. It only has side effects and it's return value is ignored:

<code>
function tool_headertest_before_http_headers() {
header("X-CustomHeader: SomeValue");
}
</code>

Note that this is called after the page is generated and just prior to the headers being sent. So internal system state may have already been changed (eg adding events to the log) and the page generation may have been expensive. You should not use this callback for things like redirecting away under some conditions (we need a new callback earlier in the process for this -- maybe the new (M3.8+) [[Login_callbacks#after_config|after_config]] callback is an alternative).

= before_standard_html_head =

{{Moodle 3.3}}

This is an API alternative to appending to $CFG->additionalhtmlhead and could be used for adding meta tags or similar to the page. It MUST return a string containing well a formed html chunk, or at minimum an empty string.

<code>
function tool_headtag_before_standard_html_head() {
return "<meta name='foo' value='before_top_of_body_html' />\n";
}
</code>

= before_standard_top_of_body_html =

{{Moodle 3.3}}

This enables a plugin to insert a chunk of html at the start of the html document. Typical use cases include some sort of alert notification, but in many cases the [[Notifications]] may be a better fit. It MUST return a string containing a well formed chunk of html, or at minimum an empty string.

<code>
function tool_callbacktest_before_standard_top_of_body_html() {
return "<div style='background: red'>Before standard top of body html</div>";
}
</code>

= render_navbar_output =

{{Moodle 3.2}}

TBA

Plugin types

2020-03-02T09:44:45Z

Jan.dagefoerde: /* List of Moodle plugin types */ Add Custom fields type

{{Plugins development}}

The M in Moodle stands for modular. The easiest and most maintainable way to add new functionality to Moodle is by writing one of these types of plugin.

== List of Moodle plugin types ==

{| class="nicetable"
|-
! Plugin type
! Component name ([[Frankenstyle]])
! Moodle path
! Description
! Moodle versions
|-
| [[Activity modules]]
| mod
| /mod
| Activity modules are essential types of plugins in Moodle as they provide activities in courses. For example: Forum, Quiz and Assignment.
| 1.0+
|-
| [[Antivirus plugins]]
| antivirus
| /lib/antivirus
| Antivirus scanner plugins provide functionality for virus scanning user uploaded files using third-party virus scanning tools in Moodle. For example: ClamAV.
| 3.1+
|-
| [[Assign_submission_plugins|Assignment submission plugins]]
| assignsubmission
| /mod/assign/submission
| Different forms of assignment submissions
| 2.3+
|-
| [[Assign_feedback_plugins|Assignment feedback plugins]]
| assignfeedback
| /mod/assign/feedback
| Different forms of assignment feedbacks
| 2.3+
|-
| [[Book tools]]
| booktool
| /mod/book/tool
| Small information-displays or tools that can be moved around pages
| 2.1+
|-
| [[Custom fields]]
| customfield
| /customfield/field
| Custom field types, used e. g. in Custom course fields
| 3.7+
|-
| [[Database fields]]
| datafield
| /mod/data/field
| Different types of data that may be added to the Database activity module
| 1.6+
|-
| [[Database presets]]
| datapreset
| /mod/data/preset
| Pre-defined templates for the Database activity module
| 1.6+
|-
| [[External tool source|LTI sources]]
| ltisource
| /mod/lti/source
| LTI providers can be added to external tools easily through the external tools interface see [https://docs.moodle.org/en/External_tool Documentation on External Tools]. This type of plugin is specific to LTI providers that need a plugin that can register custom handlers to process LTI messages
| 2.7+
|-
| [[File Converters]]
| fileconverter
| /files/converter
| Allow conversion between different types of user-submitted file. For example from .doc to PDF.
| 3.2+
|-
| [[LTI services]]
| ltiservice
| /mod/lti/service
| Allows the implementation of LTI services as described by the IMS LTI specification
| 2.8+
|-
| [[Machine learning backends]]
| mlbackend
| /lib/mlbackend
| Prediction processors for analytics API
| 3.4+
|-
| [[Quiz reports]]
| quiz
| /mod/quiz/report
| Display and analyse the results of quizzes, or just plug miscellaneous behaviour into the quiz module
| 1.1+
|-
| [[Quiz access rules]]
| quizaccess
| /mod/quiz/accessrule
| Add conditions to when or where quizzes can be attempted, for example only from some IP addresses, or student must enter a password first
| 2.2+
|-
| [[SCORM reports]]
| scormreport
| /mod/scorm/report
| Analysis of SCORM attempts
| 2.2+
|-
| [[Workshop grading strategies]]
| workshopform
| /mod/workshop/form
| Define the type of the grading form and implement the calculation of the grade for submission in the [[Workshop]] module
| 2.0+
|-
| [[Workshop allocation methods]]
| workshopallocation
| /mod/workshop/allocation
| Define ways how submissions are assigned for assessment in the [[Workshop]] module
| 2.0+
|-
| [[Workshop evaluation methods]]
| workshopeval
| /mod/workshop/eval
| Implement the calculation of the grade for assessment (grading grade) in the [[Workshop]] module
| 2.0+
|-
| [[Blocks]]
| block
| /blocks
| Small information-displays or tools that can be moved around pages
| 2.0+
|-
| [[Question types]]
| qtype
| /question/type
| Different types of question (e.g. multiple-choice, drag-and-drop) that can be used in quizzes and other activities
| 1.6+
|-
| [[Question behaviours]]
| qbehaviour
| /question/behaviour
| Control how student interact with questions during an attempt
| 2.1+
|-
| [[Question formats|Question import/export formats]]
| qformat
| /question/format
| Import and export question definitions to/from the question bank
| 1.6+
|-
| [[Filters|Text filters]]
| filter
| /filter
| Automatically convert, highlight, and transmogrify text posted into Moodle.
| 1.4+
|-
| [[Editors]]
| editor
| /lib/editor
| Alternative text editors for editing content
| 2.0+
|-
| [[Atto|Atto editor plugins]]
| atto
| /lib/editor/atto/plugins
| Extra functionality for the Atto text editor
| 2.7+
|-
| [[TinyMCE editor plugins]]
| tinymce
| /lib/editor/tinymce/plugins
| Extra functionality for the TinyMCE text editor.
| 2.4+
|-
| [[Enrolment plugins]]
| enrol
| /enrol
| Ways to control who is enrolled in courses
| 2.0+
|-
| [[Authentication plugins]]
| auth
| /auth
| Allows connection to external sources of authentication
| 2.0+
|-
| [[Admin tools]]
| tool
| /admin/tool
| Provides utility scripts useful for various site administration and maintenance tasks
| 2.2+
|-
| [[Log stores]]
| logstore
| /admin/tool/log/store
| Event logs storage back-ends
| 2.7+
|-
| [[Availability conditions]]
| availability
| /availability/condition
| Conditions to restrict user access to activities and sections.
| 2.7+
|-
| [[Calendar types]]
| calendartype
| /calendar/type
| Defines how dates are displayed throughout Moodle
| 2.6+
|-
| [[Messaging consumers]]
| message
| /message/output
| Represent various targets where messages and notifications can be sent to (email, sms, jabber, ...)
| 2.0+
|-
| [[Course formats]]
| format
| /course/format
| Different ways of laying out the activities and blocks in a course
| 1.3+
|-
| [[Data formats]]
| dataformat
| /dataformat
| Formats for data exporting and downloading
| 3.1+
|-
| [[User profile fields]]
| profilefield
| /user/profile/field
| Add new types of data to user profiles
| 1.9+
|-
| [[Reports]]
| report
| /report
| Provides useful views of data in a Moodle site for admins and teachers
| 2.2+
|-
| [[Course reports]]
| coursereport
| /course/report
| Reports of activity within the course
| Up to 2.1 (for 2.2+ see [[Reports]])
|-
| [[Gradebook export]]
| gradeexport
| /grade/export
| Export grades in various formats
| 1.9+
|-
| [[Gradebook import]]
| gradeimport
| /grade/import
| Import grades in various formats
| 1.9+
|-
| [[Gradebook reports]]
| gradereport
| /grade/report
| Display/edit grades in various layouts and reports
| 1.9+
|-
| [[Grading methods|Advanced grading methods]]
| gradingform
| /grade/grading/form
| Interfaces for actually performing grading in activity modules (eg Rubrics)
| 2.2+
|-
| [[MNet services]]
| mnetservice
| /mnet/service
| Allows to implement remote services for the [[MNet]] environment (deprecated, use web services instead)
| 2.0+
|-
| [[Webservice protocols]]
| webservice
| /webservice
| Define new protocols for web service communication (such as SOAP, XML-RPC, JSON, REST ...)
| 2.0+
|-
| [[Repository plugins]]
| repository
| /repository
| Connect to external sources of files to use in Moodle
| 2.0+
|-
| [[Portfolio plugins]]
| portfolio
| /portfolio
| Connect external portfolio services as destinations for users to store Moodle content
| 1.9+
|-
| [[Search engines]]
| search
| /search/engine
| Search engine backends to index Moodle's contents.
| 3.1+
|-
| [[Media players]]
| media
| /media/player
| Pluggable media players
| 3.2+
|-
| [[Plagiarism plugins]]
| plagiarism
| /plagiarism
| Define external services to process submitted files and content
| 2.0+
|-
| [[Cache store]]
| cachestore
| /cache/stores
| Cache storage back-ends.
| 2.4+
|-
| [[Cache locks]]
| cachelock
| /cache/locks
| Cache lock implementations.
| 2.4+
|-
| [[Themes]]
| theme
| /theme
| Change the look of Moodle by changing the the HTML and the CSS.
| 2.0+
|-
| [[Local plugins]]
| local
| /local
| Generic plugins for local customisations
| 2.0+
|-
| [[Assignment types|Legacy assignment types]]
| assignment
| /mod/assignment/type
| Different forms of assignments to be graded by teachers
| 1.x - 2.2
|-
| [[Admin reports|Legacy admin reports]]
| report
| /admin/report
| Provides useful views of data in a Moodle site, for admins only.
| Up to 2.1 (for 2.2+ see [[Reports]])
|}

== Obtaining the list of plugin types known to your Moodle ==

To get the most exact list of types in your version of Moodle, use the following script. Put it to a file in the root directory of your Moodle installation and execute it via command line.

<code php>
<?php
define('CLI_SCRIPT', true);
require('config.php');

$pluginman = core_plugin_manager::instance();

foreach ($pluginman->get_plugin_types() as $type => $dir) {
$dir = substr($dir, strlen($CFG->dirroot));
printf("%-20s %-50s %s".PHP_EOL, $type, $pluginman->plugintype_name_plural($type), $dir);
}
</code>

==Things you can find in all plugins==

Although there are many different types of plugin, there are some things that work the same way in all plugin types, and we have [[Things that work the same in all plugin types|a page that describes them]].

Additionally you probably want to look at the page [[Plugin files]].

== Naming conventions ==

Warning if you have to choose a plugin (directory) name. The name is validated by the method <tt>lib/classes/component.php::is_valid_plugin_name()</tt> with a regexp: <tt>/^[a-z](?:[a-z0-9_](?!__))*[a-z0-9]+$/</tt>. In particular, the minus (-) character is not considered as valid, and the plugin will be silently ignored if the name is not valid.

There is an exception for [[Activity modules|activity modules]] that can not have the underscore in their name for legacy reasons.

== See also ==

* [[Guidelines_for_contributed_code|Guidelines for contributing code]]
* [[Core APIs]]
* [[Frankenstyle]]
* [http://moodle.org/plugins Moodle Plugins directory]
* [[Tutorial]] to help you learn how to write plugins for Moodle from start to finish, while showing you how to navigate the most important developer documentation along the way.

[[Category:Coding guidelines|Plugins]]
[[Category:Plugins]]

Privacy API

2018-11-26T08:25:16Z

Jan.dagefoerde: /* See also */ Link to CLI script collection

The [https://en.wikipedia.org/wiki/General_Data_Protection_Regulation General Data Protection Regulation] (GDPR) is an EU directive that looks at providing users with more control over their data and how it is processed. This regulation will come into effect on 25th of May 2018 and covers any citizen or permanent resident of the European Union. The directive will be respected by a number of other countries outside of the European Union.

To help institutions become compliant with this new regulation we are adding functionality to Moodle. This includes a number of components, amongst others these include a user’s right to:

* request information on the types of personal data held, the instances of that data, and the deletion policy for each;
* access all of their data; and
* be forgotten.

The compliance requirements also extend to installed plugins (including third party plugins). These need to also be able to report what information they store or process regarding users, and have the ability to provide and delete data for a user request.

This document describes the proposed API changes required for plugins which will allow a Moodle installation to become GDPR compliant.

Target Audience: The intended audience for this document is Moodle plugin developers, who are aiming to ensure their plugins are updated to comply with GDPR requirements coming into effect in the EU in May, 2018.

==Personal data in Moodle==

From the GDPR Spec, Article 4:

''‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;''

In Moodle, we need to consider two main types of personal data; information entered by the user and information stored about the user. The key difference being that information stored about the user will have come from a source other than the user themselves. Both types of data can be used to form a profile of the individual.

The most obvious clue to finding personal data entered by the user is the presence of a userid on a database field. Any data on the record (or linked records) pertaining to that user may be deemed personal data for that user, including things like timestamps and record identification numbers. Additionally, any free text field which allows the user to enter information must also be considered to be the personal data of that user.

Data stored about the user includes things like ratings and comments made on a student submission. These may have been made by an assessor or teacher, but are considered the personal data of the student, as they are considered a reflection of the user’s competency in the subject matter and can be used to form a profile of that individual.

The sections that follow outline what you need to do as a plugin developer to ensure any personal data is advertised and can be accessed and deleted according to the GDPR requirements.

==Background==

===Architecture overview===

[[File:MoodlePrivacyMetadataUML.png|thumb|UML diagram of the metadata part of the privacy subsystem]]
[[File:MoodlePrivacyRequestUML.png|thumb|UML diagram of the request providers part of the privacy subsystem]]

A new system for Privacy has been created within Moodle. This is broken down into several main parts and forms the ''core_privacy'' subsystem:

* Some metadata providers - a set of PHP interfaces to be implemented by components for that component to describe the kind of data that it stores, and the purpose for its storage;
* Some request providers - a set of PHP interfaces to be implemented by components to allow that component to act upon user requests such as the Right to be Forgotten, and a Subject Access Request; and
* A manager - a concrete class used to bridge components which implement the providers with tools which request their data.

All plugins will implement one metadata provider, and zero, one or two request providers.

The 'request' providers are responsible for several separate areas:

# Detecting in which Moodle contexts a specific user has any personal data;
# Exporting all personal data from each of those contexts for that user;
# Deleting all personal data from each of those contexts for that user;
# Detecting which users have personal data in a specific Moodle context;
# Deleting all personal data for each of those users in that context; and
# Deleting all personal data for all users in a specific context.

The export and delete phases use data from the detection phase, which allows for the possibility to exclude all data from certain contexts, as required.

Please refer to the inline phpdocs of the [https://github.com/moodle/moodle/blob/v3.5.0/privacy/classes/manager.php#L31 core_privacy::manager class] for detailed description of the interfaces, their hierarchy and meaning.

Please note that support for locating and removing multiple users in a single context was added in MDL-62560 for Moodle 3.6, 3.5.3, and 3.4.6.
This functionality has been added to allow support for removal of user data for users subject to speific role criterion, and to support expiry of the same data by role too.

===Implementing a provider===

All plugins will need to create a concrete class which implements the relevant metadata and request providers. The exact providers you need to implement will depend on what data you store, and the type of plugin. This is covered in more detail in the following sections of the document.

In order to do so:

# You must create a class called ''provider'' within the namespace ''\your_pluginname\privacy''.
# This class must be created at ''path/to/your/plugin/classes/privacy/provider.php''.
# You must have your class implement the relevant metadata and request interfaces.

==Plugins which do not store personal data==

Many Moodle plugins do not store any personal data. This is usually the case for plugins which just add functionality, or which display the data already stored elsewhere in Moodle.

Some examples of plugin types which might fit this criteria include themes, blocks, filters, editor plugins, etc.

Plugins which cause data to be stored elsewhere in Moodle (e.g. via a subsystem call) are considered to store data.

One examples of a plugin which does not store any data would be the Calendar month block which just displays a view of the user’s calendar. It does not store any data itself.

An example of a plugin which must not use the null provider is the Comments block. The comments block is responsible for data subsequently being stored within Moodle. Although the block doesn’t store anything itself, it interacts with the comments subsystem and is the only component which knows how that data maps to a user.

===Implementation requirements===

In order to let Moodle know that you have audited your plugin, and that you do not store any personal user data, you must implement the ''\core_privacy\local\metadata\null_provider'' interface in your plugin’s provider.

These null providers can only be implemented where a plugin has:

* no external links (e.g. sends data to an external service like an LTI provider, repository plugin which you can search on)
* no database tables which store user data (including IP addresses)
* no user preferences

The ''null_provider'' requires you to define one function ''get_reason()'' which returns the language string identifier within your component.

====Example====

''blocks/calendar_month/classes/privacy/provider.php''

<code php>
<?php
// …

namespace block_calendar_month\privacy;

class provider implements
// This plugin does not store any personal user data.
\core_privacy\local\metadata\null_provider {

/**
* Get the language string identifier with the component's language
* file to explain why this plugin stores no data.
*
* @return string
*/
public static function get_reason() : string {
return 'privacy:metadata';
}
}
</code>

''blocks/calendar_month/lang/en/block_calendar_month.php''

<code php>
<?php
// …

...
$string['privacy:metadata'] = 'The Calendar block only displays existing calendar data.';
...
</code>

That’s it. Congratulations, your plugin now implements the Privacy API.

==Plugins which store personal data==

Many Moodle plugins do store some form of personal data.

In some cases this will be stored within database tables in your plugin, and in other cases this will be in one of Moodle’s core subsystems - for example your plugin may store files, ratings, comments, or tags.

Plugins which do store data will need to:

* Describe the type of data that they store;
* Provide a way to export that data; and
* Provide a way to delete that data.

Data is described via a ''metadata'' provider, and it is both exported and deleted via an implementation of a ''request'' provider.

These are both explained in the sections below.

===Describing the type of data you store===

In order to describe the type of data that you store, you must implement the ''\core_privacy\local\metadata\provider'' interface.

This interfaces requires that you define one function: ''get_metadata''.

There are several types of item to describe the data that you store. These are for:

* Items in the Moodle database;
* Items stored by you in a Moodle subsystem - for example files, and ratings; and
* User preferences stored site-wide within Moodle for your plugin

Note: All fields should include a description from a language string within your plugin.

====Example====

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …

namespace mod_forum\privacy;
use core_privacy\local\metadata\collection;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider {

public static function get_metadata(collection $collection) : collection {

// Here you will add more items into the collection.

return $collection;
}
}
</code>

====Indicating that you store content in a Moodle subsystem====

Many plugins will use one of the core Moodle subsystems to store data.

As a plugin developer we do not expect you to describe those subsystems in detail, but we do need to know that you use them and to know what you use them for.

You can indicate this by calling the ''add_subsystem_link()'' method on the ''collection''.

=====Relevant subsystems=====

You are likely to need to indicate use of the following subsystems that store user data:

* Ratings (if users are allowed to rate items within your plugin)
* Tags (if users can tag items within your plugin)
* Comments (if users can make comments on items in your plugin)
* Questions (if the plugin uses core question types)
* Filesystem (if users can attach files to items within your plugin)
* ...?

Some subsystems which store user data do not need to be listed:

* (TBC - how about global search? Nothing lists it that I could see.)

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_subsystem_link(
'core_files',
[],
'privacy:metadata:core_files'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:core_files'] = 'The forum stores files which have been uploaded by the user to form part of a forum post.';
</code>

====Describing data stored in database tables====

Most Moodle plugins will store some form of user data in their own database tables.

As a plugin developer you will need to describe each database table, and each field which includes user data.

It is a matter of judgement which fields contain user data and which don't. Anything entered by, or directly about, the user probably counts as user data but it may be useful to include additional fields that explain the context of the data.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_database_table(
'forum_discussion_subs',
[
'userid' => 'privacy:metadata:forum_discussion_subs:userid',
'discussionid' => 'privacy:metadata:forum_discussion_subs:discussionid',
'preference' => 'privacy:metadata:forum_discussion_subs:preference',

],
'privacy:metadata:forum_discussion_subs'
);

return $collection;
}
</code>

''mod/forum/lang/en/forum.php''

<code php>
<?php

$string['privacy:metadata:forum_discussion_subs'] = 'Information about the subscriptions to individual forum discussions. This includes when a user has chosen to subscribe to a discussion, or to unsubscribe from one where they would otherwise be subscribed.';
$string['privacy:metadata:forum_discussion_subs:userid'] = 'The ID of the user with this subscription preference.';
$string['privacy:metadata:forum_discussion_subs:discussionid'] = 'The ID of the discussion that was subscribed to.';
$string['privacy:metadata:forum_discussion_subs:preference'] = 'The start time of the subscription.';
</code>

====Indicating that you store site-wide user preferences====

Many plugins will include one or more user preferences. Unfortunately this is one of Moodle’s older components and many of the values stored are not pure user preferences. Each plugin should be aware of how it handles its own preferences and is best placed to determine whether they are site-wide preferences, or per-instance preferences.

Whilst most of these will have a fixed name (e.g. ''filepicker_recentrepository''), some will include a variable of some kind (e.g. ''tool_usertours_tour_completion_time_2''). Only the general name (in this case "tool_usertours_tour_completion_time_") needs to be indicated, rather than one copy for each possible value of the variable.

Also, these should only be ''site-wide'' user preferences which do not belong to a specific Moodle context.

In the above examples:

* Preference ''filepicker_recentrepository'' belongs to the file subsystem, and is a site-wide preference affecting the user anywhere that they view the filepicker.
* Preference ''tool_usertours_tour_completion_time_2'' belongs to user tours. User tours are a site-wide feature which can affect many parts of Moodle and cross multiple contexts.

In some cases a value may be stored in the preferences table but is known to belong to a specific context within Moodle. In these cases they should be stored as metadata against that context rather than as a site-wide user preference.

You can indicate this by calling the ''add_user_preference()'' method on the ''collection''.

Any plugin providing user preferences must also implement the ''\core_privacy\local\request\preference_provider''.

=====Example=====

''admin/tool/usertours/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_user_preference('tool_usertours_tour_completion_time',
'privacy:metadata:preference:tool_usertours_tour_completion_time');

return $collection;
}
</code>

''admin/tool/usertours/lang/en/tool_usertours.php''

<code php>
<?php

$string['privacy:metadata:tool_usertours_tour_completion_time'] = 'The time that a specific user tour was last completed by a user.';
</code>

====Indicating that you export data to an external location====

Many plugins will interact with external systems - for example cloud-based services. Often this external location is configurable within the plugin either at the site or the instance level.

As a plugin developer you will need to describe each ''type'' of target destination, alongside a list of each exported field which includes user data.
The ''actual'' destination does not need to be described as this can change based on configuration.

You can indicate this by calling the ''add_external_location_link()'' method on the collection.

=====Example=====

''mod/lti/classes/privacy/provider.php''

<code php>
public static function get_metadata(collection $collection) : collection {

$collection->add_external_location_link('lti_client', [
'userid' => 'privacy:metadata:lti_client:userid',
'fullname' => 'privacy:metadata:lti_client:fullname',
], 'privacy:metadata:lti_client');

return $collection;
}
</code>

''mod/lti/lang/en/lti.php''

<code php>
<?php

$string['privacy:metadata:lti_client'] = 'In order to integrate with a remote LTI service, user data needs to be exchanged with that service.';
$string['privacy:metadata:lti_client:userid'] = 'The userid is sent from Moodle to allow you to access your data on the remote system.';
$string['privacy:metadata:lti_client:fullname'] = 'Your full name is sent to the remote system to allow a better user experience.';
</code>

===Providing a way to export user data===

In order to export the user data that you store, you must implement the relevant request provider.

We have named these request providers because they are called in response to a specific request from a user to access their information, however they also deal with the removal of user data following it's expiry.

There are several different types of request provider, and you may need to implement several of these, depending on the type and nature of your plugin.

Broadly speaking plugins will fit into one of the following categories:

* Plugins which are a subplugin of another plugin. Examples include ''assignsubmission'', ''atto'', and ''datafield'';
* Plugins which are typically called by a Moodle subsystem. Examples include ''qtype'', and ''profilefield'';
* All other plugins which store data.

Most plugins will fit into this final category, whilst other plugins may fall into several categories.
Plugins which ''define'' a subplugin will also be responsible for collecting this data from their subplugins.

A final category exists - plugins which store user preferences. In some cases this may be the ''only'' provider implemented.

====Standard plugins which store data====

A majority of Moodle plugins will fit into this category and will be required to implement the ''\core_privacy\local\request\plugin\provider'' interface. This interface requires that you define four functions (the first two of which are dealt with in this section):

* ''get_contexts_for_userid'' - to explain where data is held within Moodle for your plugin; and
* ''export_user_data'' - to export a user’s personal data from your plugin.
* ''delete_data_for_all_users_in_context'' - to delete all data for all users in the specified context.
* ''delete_data_for_user'' - to delete all user data for the specified user, in the specified contexts.

These APIs make use of the Moodle ''context'' system to hierarchically store this data.

In addition to these requirements, since Moodle 3.4.6, 3.5.3, any plugin which implements the plugin provider interface must also implement the ''\core_privacy\local\request\core_userlist_provider'' provider and implement functions:

* ''get_users_in_context'' - to locate the users who hold any personal data in a specific context; and
* ''delete_data_for_users'' - to delete data for multiple users in the specified context.

=====Retrieving the list of contexts=====

You are required to return the list of contexts for which the plugin stores data about the user. These are the standard Moodle contexts - CONTEXT_COURSE, CONTEXT_USER, CONTEXT_MODULE and so on. In many cases the link between the module type and the context is self-evident (e.g. activity modules). In some cases it may be less so. For example, an enrolment plugin (that stores user data, which most don't) would link to course context (users enrol in courses). Other types of plugins may be less obvious but you need to pick something. One way might to consider what context you would use when checking role capabilities. Consider that this will be used to structure the exported data and define what data is deleted when a context is expired.

Contexts are retrieved using the ''get_contexts_for_userid'' function which takes the ID of the user being fetched, and returns a list of contexts in which the user has any data.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {}
</code>

The function returns a ''\core_privacy\local\request\contextlist'' which is used to keep a set of contexts together in a fixed fashion.

Because a Subject Access Request covers ''every'' piece of data that is held for a user within Moodle, efficiency and performance is highly important. As a result, contexts are added to the ''contextlist'' by defining one or more SQL queries which return just the contextid. Multiple SQL queries can be added as required.

Many plugins will interact with specific subsystems and store data within them.
These subsystems will also provide a way in which to link the data that you have stored with your own database tables.
At present these are still a work in progress and only the ''core_ratings'' subsystem includes this.

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$contextlist = new \core_privacy\local\request\contextlist();

$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
WHERE (
d.userid = :discussionuserid
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'discussionuserid' => $userid,
];

$contextlist->add_from_sql($sql, $params);

return $contextlist;
}
</code>

======More complete example======

The following example includes a link to core_rating.
It will find any forum, forum discussion, or forum post where the user has any data, including:

* Per-forum digest preferences;
* Per-forum subscription preferences;
* Per-forum read tracking preferences;
* Per-discussion subscription preferences;
* Per-post read data (if a user has read a post or not); and
* Per-post rating data.

In the case of the rating data, this will include any post where the user has rated the post of another user.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$ratingsql = \core_rating\privacy\provider::get_sql_join('rat', 'mod_forum', 'post', 'p.id', $userid);
// Fetch all forum discussions, and forum posts.
$sql = "SELECT c.id
FROM {context} c
INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
INNER JOIN {forum} f ON f.id = cm.instance
LEFT JOIN {forum_discussions} d ON d.forum = f.id
LEFT JOIN {forum_posts} p ON p.discussion = d.id
LEFT JOIN {forum_digests} dig ON dig.forum = f.id
LEFT JOIN {forum_subscriptions} sub ON sub.forum = f.id
LEFT JOIN {forum_track_prefs} pref ON pref.forumid = f.id
LEFT JOIN {forum_read} hasread ON hasread.forumid = f.id
LEFT JOIN {forum_discussion_subs} dsub ON dsub.forum = f.id
{$ratingsql->join}
WHERE (
p.userid = :postuserid OR
d.userid = :discussionuserid OR
dig.userid = :digestuserid OR
sub.userid = :subuserid OR
pref.userid = :prefuserid OR
hasread.userid = :hasreaduserid OR
dsub.userid = :dsubuserid OR
{$ratingsql->userwhere}
)
";

$params = [
'modname' => 'forum',
'contextlevel' => CONTEXT_MODULE,
'postuserid' => $userid,
'discussionuserid' => $userid,
'digestuserid' => $userid,
'subuserid' => $userid,
'prefuserid' => $userid,
'hasreaduserid' => $userid,
'dsubuserid' => $userid,
];
$params += $ratingsql->params;

$contextlist = new \core_privacy\local\request\contextlist();
$contextlist->add_from_sql($sql, $params);

return $contextlist;

}
</code>

=====Retrieving the users in a context=====

You are required to return the list of users holding personal data in a context for which the plugin stores data about the user.
This method is very similar to the ''get_contexts_for_userid'' function but has some important distinctions:

* It takes a ''userlist'' as an argument and does not require that you, as a developer, create it yourself;
* There is no need to return any value; and
* The argument for the ''userlist::add_from_sql'' are different to those for ''contextlist::add_from_sql'' (this is because we learnt some important lessons after the initial implementation).

<code php>
/**
* Get the list of users who have data within a context.
*
* @param userlist $userlist The userlist containing the list of users who have data in this context/plugin combination.
*/
public static function get_users_in_context(userlist $userlist) {}
</code>

======Basic example======

The following example simply fetches the contextid for all forums where a user has a single discussion (note: this is an incomplete example):

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Get the list of contexts that contain user information for the specified user.
*
* @param int $userid The user to search.
* @return contextlist $contextlist The list of contexts used in this plugin.
*/
public static function get_contexts_for_userid(int $userid) : contextlist {
$context = $userlist->get_context();

if (!is_a($context, \context_module::class)) {
return;
}

$params = [
'instanceid' => $context->instanceid,
'modulename' => 'forum',
];

// Discussion authors.
$sql = "SELECT d.userid
FROM {course_modules} cm
JOIN {modules} m ON m.id = cm.module AND m.name = :modulename
JOIN {forum} f ON f.id = cm.instance
JOIN {forum_discussions} d ON d.forum = f.id
WHERE cm.id = :instanceid";
$userlist->add_from_sql('userid', $sql, $params);

// Forum authors.
$sql = "SELECT p.userid
FROM {course_modules} cm
JOIN {modules} m ON m.id = cm.module AND m.name = :modulename
JOIN {forum} f ON f.id = cm.instance
JOIN {forum_discussions} d ON d.forum = f.id
JOIN {forum_posts} p ON d.id = p.discussion
WHERE cm.id = :instanceid";
$userlist->add_from_sql('userid', $sql, $params);

// ...
}
</code>

=====Exporting user data=====

After determining where in Moodle your plugin holds data about a user, the ''\core_privacy\manager'' will then ask your plugin to export all user data for a subset of those locations.

This is achieved through use of the ''export_user_data'' function which takes the list of approved contexts in a ''\core_privacy\local\request\approved_contextlist'' object.

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user data for the specified user, in the specified contexts, using the supplied exporter instance.
*
* @param approved_contextlist $contextlist The approved contexts to export information for.
*/
public static function export_user_data(approved_contextlist $contextlist) {}
</code>

The ''approved_contextlist'' includes both the user record, and a list of contexts, which can be retrieved by either processing it as an Iterator, or by calling ''get_contextids()'' or ''get_contexts()'' as required.

Data is exported using a ''\core_privacy\local\request\content_writer'', which is described in further detail below.

====Plugins which store user preferences====

Many plugins store a variety of user preferences, and must therefore export them.

Since user preferences are a site-wide preference, these are exported separately to other user data.
In some cases the only data present is user preference data, whilst in others there is a combination of user-provided data, and user preferences.

Storing of user preferences is achieved through implementation of the ''\core_privacy\local\request\user_preference_provider'' interface which defines one required function -- ''export_user_preferences''.

You need to provide a description of the value of the user preference. (This description is particularly useful in cases where the value might be, say, 3, but it actually means 'Alphabetical order'.) Most likely you can use an existing language string from your plugin.

=====Example=====

''mod/forum/classes/privacy/provider.php''

<code php>
/**
* Export all user preferences for the plugin.
*
* @param int $userid The userid of the user whose data is to be exported.
*/
public static function export_user_preferences(int $userid) {
$markasreadonnotification = get_user_preference('markasreadonnotification', null, $userid);
if (null !== $markasreadonnotification) {
switch ($markasreadonnotification) {
case 0:
$markasreadonnotificationdescription = get_string('markasreadonnotificationno', 'mod_forum');
break;
case 1:
default:
$markasreadonnotificationdescription = get_string('markasreadonnotificationyes', 'mod_forum');
break;
}
writer::export_user_preference('mod_forum', 'markasreadonnotification', $markasreadonnotification, $markasreadonnotificationdescription);
}
}
</code>

==== Plugins which can have own subplugins ====

Many plugin types are also able to define their own subplugins and will need to define a contract between themselves and their subplugins in order to fetch their data.

This is required as the parent plugin and the child subplugin should be separate entities and the parent plugin must be able to function if one or more of its subplugins are uninstalled.

The parent plugin is responsible for defining the contract, and for interacting with its subplugins, though we intend to create helpers to make this easier.

The parent plugin should define a new interface for each type of subplugin that it defines. This interface should extend the ''\core_privacy\local\request\plugin\subplugin_provider'' interface.

===== When a parent plugin should and should not provide the interface for its subplugins =====

There can be cases when there is no point for a plugin to provide the "subplugin_provider" based interface, even if it has own subplugins. See the Atto or TinyMCE editors as real examples.

If the parent plugin has no data passed through to the subplugins, there is no benefit in defining a subplugin provider. For example, Atto subplugins are just used to enhance the functionality and they never receive anything like a context. Most of the time we need to define a subplugin provider, but in cases where there is no data passed from the plugin to its subplugins, there is no need to define the subplugin provider. If the subplugins still do store personal data that are not related to the parent plugin in any way, then subplugins should define their own standard provider.

Compare with something like mod_assign where the subplugins store data for the parent and that data is contextually relevant to the parent plugin. In those cases the subplugin stores data for the plugin and it only makes sense to do so in the context of its parent plugin.

=====Example=====

The following example defines the contract that assign submission subplugins may be required to implement.

The assignment module is responsible for returning the contexts of all assignments where a user has data, but in some cases it is unaware of all of those cases - for example if a Teacher comments on a student submission it may not be aware of these as the information about this interaction may not be stored within its own tables.

''mod/assign/privacy/assignsubmission_provider.php''

<code php>
<?php
// …

namespace mod_assign\privacy;
use \core_privacy\local\metadata\collection;

interface assignsubmission_provider extends
// This Interface defines a subplugin.
\core_privacy\local\request\plugin\subplugin_provider {

/**
* Get the SQL required to find all submission items where this user has had any involvements.
*
* @param int $userid The user to search.
* @return \stdClass Object containing the join, params, and where used to select a these records from the database.
*/
public static function get_items_with_user_interaction(int $userid) : \stdClass ;

/**
* Export all relevant user submissions information which match the combination of userid and attemptid.
*
* @param int $userid The user to search.
* @param \context $context The context to export this submission against.
* @param array $subcontext The subcontext within the context to export this information
* @param int $attid The id of the submission to export.
*/
public static function export_user_submissions(int $userid, \context $context, array $subcontext, int $attid) ;

}
</code>

====Plugins which are subplugins to another plugin====

If you are developing a sub-plugin of another plugin, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subplugin type should define a new interface which extends the ''\core_privacy\local\request\plugin\subplugin_provider'' interface and it is up to the parent plugin to define how they will interact with their children.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''mod/pluginname/classes/privacy/provider.php''

<code php>
<?php
// …
namespace assignsubmission\onlinetext;

class provider implements
// This plugin does store personal user data.
\core_privacy\local\metadata\provider,

// This plugin is a subplugin of assign and must meet that contract.
\mod_assign\privacy\assignsubmission_provider {
}
</code>

====Plugins which are typically called by a Moodle subsystem====

There are a number of plugintypes in Moodle which are typically called by a specific Moodle subsystem.

Some of these are ''only'' called by that subsystem, for example plugins which are of the ''plagiarism'' plugintype should never be called directly, but are always invoked via the ''core_plagiarism'' subsystem.

Conversely, there maybe other plugintypes which can be called both via a subsystem, and in some other fashion. We are still determining whether any plugintypes currently fit this pattern.

If you are developing a plugin which belongs to a specific subsystem, then you will have to look at the relevant plugin in order to determine the exact contract.

Each subsystem will define a new interface which extends the ''\core_privacy\local\request\plugin\subsystem_provider'' interface and it is up to that subsystem to define how they will interact with those plugins.

The principles remain the same, but the exact implementation will differ depending upon requirements.

''plagiarism/detectorator/classes/privacy/provider.php''

<code php>
<?php
// …
namespace plagiarism_detectorator\privacy;

class provider implements
// This plugin does export personal user data.
\core_privacy\local\metadata\provider,

// This plugin is always linked against another activity module via the Plagiarism API.
\core_plagiarism\privacy\plugin_provider {
}
</code>

====Exporting data====

Any plugin which stores data must also export it.

To cater for this the privacy API includes a ''\core_privacy\local\request\content_writer'', which defines a set of functions to store different types of data.

Broadly speaking data is broken into the following types:

* Data - this is the object being described. For example the post content in a forum post;
* Related data - this is data related to the object being stored. For example, ratings of a forum post;
* Metadata - This is metadata about the main object. For example whether you are subscribed to a forum discussion;
* User preferences - this is data about a site-wide preference;
* Files - Any files that you are stored within Moodle on behalf of this plugin; and
* Custom files - For custom file formats - e.g. a calendar feed for calendar data. These should be used sparingly.

Each piece of data is stored against a specific Moodle ''context'', which will define how the data is structured within the exporter.
Data, and Related data only accept the ''stdClass'' object, whilst metadata should be stored as a set of key/value pairs which include a description.

In some cases the data being stored belongs within an implicit structure. For example, one forum has many forum discussions, which each have a number of forum posts. This structure is represented by an ''array'' referred to as a ''subcontext''.

The ''content_writer'' must ''always'' be called with a specific context, and can be called as follows:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

writer::with_context($context)
->export_data($subcontext, $post)
->export_area_files($subcontext, 'mod_forum', 'post', $post->id)
->export_metadata($subcontext, 'postread', (object) ['firstread' => $firstread], new \lang_string('privacy:export:post:postread'));
</code>

Any text field which supports Moodle files must also be rewritten:

''mod/forum/classes/privacy/provider.php''

<code php>
<?php
// …
use \core_privacy\local\request\writer;

$post->message = writer::with_context($context)
->rewrite_pluginfile_urls($subcontext, 'mod_forum', 'post', $post->id, $post->message);

</code>

===Providing a way to delete user data===

Deleting user data is also implemented in the request interface. There are two methods that need to be created. The first one to remove all user data from a context, the other to remove user data for a specific user in a list of contexts.

====Delete for a context====

A context is given and all user data (for all users) is to be deleted from the plugin. This will be called when the retention period for the context has expired to adhere to the privacy by design requirement. Retention periods are set in the Data registry.

Note that this will be called for '''any''' context being expired, not only those the plugin holds data for (as returned by get_contexts_for_userid()), so you must carefully check the contexts given.

When expiring content for a high-level context such as a course context, the function will be called not only with the course context but also with each context within the course: each module context, each block context, etc. At each level you should only expire data specifically related to that exact context. For a module plugin, this generally means you should only take action for CONTEXT_MODULE contexts and only if they relate to an instance of your module, as shown in the following example. (In the rare case where your module also stores user data related to the course, and not just for a module instance, then you do need to implement CONTEXT_COURSE level contexts to expire that course-level data.)

<code php>
/**
* Delete all personal data for all users in the specified context.
*
* @param context $context Context to delete data from.
*/
public static function delete_data_for_all_users_in_context(\context $context) {
global $DB;

if ($context->contextlevel != CONTEXT_MODULE) {
return;
}

$cm = get_coursemodule_from_id('choice', $context->instanceid);
if (!$cm) {
return;
}

$DB->delete_records('choice_answers', ['choiceid' => $cm->instance]);
}
</code>

====Delete personal information for a specific user and context(s)====

An ''approved_contextlist'' is given and user data related to that user should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/choice/classes/privacy/provider.php''

<code php>
public static function delete_data_for_user(approved_contextlist $contextlist) {
global $DB;

if (empty($contextlist->count())) {
return;
}
$userid = $contextlist->get_user()->id;
foreach ($contextlist->get_contexts() as $context) {
$instanceid = $DB->get_field('course_modules', 'instance', ['id' => $context->instanceid], MUST_EXIST);
$DB->delete_records('choice_answers', ['choiceid' => $instanceid, 'userid' => $userid]);
}
}
</code>

====Delete personal information for several users in a specific context====

An ''approved_userlist'' is given and user data related to all users in the specified context should either be completely deleted, or overwritten if a structure needs to be maintained. This will be called when a user has requested the right to be forgotten when per-role overrides exist, or when performing a per-role expiry of a context. All attempts should be made to delete this data where practical while still allowing the plugin to be used by other users.

''mod/chat/classes/privacy/provider.php''

<code php>
/**
* Delete multiple users within a single context.
*
* @param approved_userlist $userlist The approved context and user information to delete information for.
*/
public static function delete_data_for_users(approved_userlist $userlist) {
global $DB;

$context = $userlist->get_context();
$cm = $DB->get_record('course_modules', ['id' => $context->instanceid]);
$chat = $DB->get_record('chat', ['id' => $cm->instance]);

list($userinsql, $userinparams) = $DB->get_in_or_equal($userlist->get_userids(), SQL_PARAMS_NAMED);
$params = array_merge(['chatid' => $chat->id], $userinparams);
$sql = "chatid = :chatid AND userid {$userinsql}";

$DB->delete_records_select('chat_messages', $sql, $params);
$DB->delete_records_select('chat_messages_current', $sql, $params);
$DB->delete_records_select('chat_users', $sql, $params);
}
</code>

==Difference between Moodle 3.3 and more recent versions==
Moodle 3.3 has a minimum requirement of php 5.6 and so type hinting and return type declarations are not supported in this version.
Consequently the privacy API for this version does not have these features.
==Common Questions==
===What to do if you have one plugin that supports multiple branches===
This is something that we have considered and we have put in place a polyfill. This gets around the restrictions of one version having type hinting and return type declarations while another does not.

====Example====
To use the polyfill include the legacy polyfill trait and create the necessary static methods but with an underscore (shown below).
<code php>
class provider implements
\core_privacy\local\metadata\provider,
\core_privacy\local\request\plugin\provider {

// This trait must be included.
use \core_privacy\local\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}
</code>

===What to do if your plugin must implement a subplugin or subsystem plugin provider===
For subplugins (e.g. assignsubmission, assignfeedback, quiz report, quiz access rules), or subsystems which have a plugintype relationship (portfolio, plagiarism, and others), they will also define their own legacy polyfill.

In this instance you will need to include the trait for both the core polyfill, and the provider polyfill as appropriate.
==== Example ====
<code php>
class provider implements
// This plugin has data and must therefore define the metadata provider in order to describe it.
\core_privacy\local\metadata\provider,

// This is a plagiarism plugin. It interacts with the plagiarism subsystem rather than with core.
\core_plagiarism\privacy\plagiarism_provider {

// This trait must be included to provide the relevant polyfill for the metadata provider.
use \core_privacy\local\legacy_polyfill;

// This trait must be included to provide the relevant polyfill for the plagirism provider.
use \core_plagiarism\privacy\plagiarism_provider\legacy_polyfill;

// The required methods must be in this format starting with an underscore.
public static function _get_metadata(collection $collection) {
// Code for returning metadata goes here.
}

// This is one of the polyfilled methods from the plagiarism provider.
public static function _export_plagiarism_user_data($userid, \context $context, array $subcontext, array $linkarray) {
// ...
}
</code>

== Tips for development ==

* While implementing the privacy API into your plugin, there are CLI scripts that can help you to test things on the fly. Just don't forget these are not supposed to replace proper unit tests. See [[Privacy API/Utilities]] for details.
* Inherit Unit tests from the <code php>core_privacy\tests\provider_testcase</code>, not <code php>advanced_testcase</code>. Advanced test case doesn't reset the Privacy content_writer between tests!

==See also==

* [[Subject Access Request FAQ]]
* [[:en:GDPR|GDPR]] in the user documentation
* [[Privacy API/Utilities]] provides CLI scripts that are helpful during development

[[Category:Privacy]]
[[Category:GDPR]]
[[Category:API]]

Repository plugins

2018-08-10T14:35:10Z

Jan.dagefoerde: /* supported_returntypes() */

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[:en:OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an empty array.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public function type_config_form($mform) {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

Repository plugins

2018-08-10T14:31:48Z

Jan.dagefoerde: supported_returntypes(): Add FILE_CONTROLLED_LINK and clarify different kinds of return types using an example

{{Repository plugins}}
== Introduction ==

Repository plugin allow Moodle to bring contents into Moodle from external repositories.

===Prerequisites===
Before starting coding, it is necessary to know how to use repository administration pages and how to use the file picker.

===Overview===

The 3 different parts to write
# Administration - You can customise the way administrators and users can configure their repositories.
# File picker integration - The core of your plugin, it will manage communication between Moodle and the repository service, and also the file picker display.
# I18n - Internationalization should be done at the same time as you're writing the other parts.

== History ==

Repository plugins exists from 2.0

== Example ==
*[[Box.net Repository Plugin|Box.net Repository Plugin]]
*[[Flickr Repository Plugin|Flickr Repository Plugin]]
*[[Moodle Repository Plugin|Remote Moodle Repository Plugin]]

==Creating new repository plugin==
# Create a folder for your plugin in ''/repository/'' e.g. ''/repository/myplugin''
# Create the following files in your plugin folder:
#* ''/repository/myplugin/lib.php''
#* ''/repository/myplugin/pix/icon.png'' - the icon displayed in the file picker (16x16)
#* ''/repository/myplugin/[[version.php]]''
#* ''/repository/myplugin/lang/en/repository_myplugin.php'' - language file
#* ''/repository/myplugin/db/access.php''
# Declare class '''repository_myplugin extends repository''' in your lib.php
# In your repository_myplugin class overwrite function get_listing() to '''return array('list' => array());'''
# Add at least strings '''$string['pluginname']''' and '''$string['configplugin']''' to your language file
# Add capability 'repository/myplugin:view' to your access.php file
# Create install and upgrade scripts (optional or you can do it later)
# Login as admin on your website and run upgrade
# Open Site Administration->Plugins->Repositories->Manage Repositories and make your repository 'Enabled and visible'

For a more detailed explanation of each of each of the files that have been created here, along with code examples, see [[Repository plugin files]].

==Administration APIs==

===Fixed settings===

These are settings that are hard-coded into your repository plugin and can only be updated by changing the plugin code.

====supported_returntypes()====
Return any combination of the following values:
* FILE_INTERNAL - the file is uploaded/downloaded and stored directly within the Moodle file system.
* FILE_EXTERNAL - the file stays in the external repository and is accessed from there directly.
* FILE_REFERENCE - the file stays in the external repository but may be cached locally. In that case it should be synchronised automatically, as required, with any changes to the external original.
* FILE_CONTROLLED_LINK - the file remains in the external repository. By "uploading" it, ownership of the file (in the remote system) is changed so that the [[OAuth_2_services#Connecting_a_system_account|system account in the external repository]] becomes the new owner of the file. Later, if the file is accessed, the system account is responsible for granting access to users.

<code php>
function supported_returntypes() {
return FILE_INTERNAL | FILE_EXTERNAL | FILE_REFERENCE | FILE_CONTROLLED_LINK;
}
</code>

[[File:File options.png|thumb|Choices offered resulting from the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.]]
The return values influence the choices offered to a user when selecting a file in file picker. Consider the screenshot excerpt on the right, which is from the dialogue that appears directly after choosing (but before uploading) a file in mod_resource. The three options result from FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED_LINK being present. FILE_REFERENCE corresponds to the "alias/shortcut" option.
The option FILE_EXTERNAL is never reflected in the file picker for mod_resource, so its absence or presence in supported_returntypes() is never reflected here. However, FILE_EXTERNAL is the only return type supported by mod_url: For mod_url, file picker will only(!) list repositories that support FILE_EXTERNAL.

This implies that a plugin that uses a file picker is able to narrow the set of supported return types. For example, assignsubmission_file disallows FILE_EXTERNAL and FILE_REFERENCE.

In the end, which type is
used by Moodle depends on the choices made by the end user (e.g. inserting a link, will result in 'FILE_EXTERNAL'-related functions being used, using a 'shortcut/alias' will result in the 'FILE_REFERENCE'-related functions being used).

====supported_filetypes()====
Optional. Returns '*' for all file types (default implementation), or an array of types or groups (e.g. array('text/plain', 'image/gif', 'web_image') )
<code php>
function supported_filetypes() {
//return '*';
//return array('image/gif', 'image/jpeg', 'image/png');
return array('web_image');
}
</code>
For a full list of possible types and groups, look in lib/filelib.php, function get_mimetypes_array().

===Global settings===

These are settings that are configured for the whole Moodle site and not per instance of your plugin. All of these are optional, without them there will be no configuration options in the Site administration > Plugins > Repositories > Myplugin page.

====get_type_option_names()====
''This function must be declared static''<br>
Optional. Return an array of string. These strings are setting names. These settings are shared by all instances.
Parent function returns an empty array.

For example:
<code php>
public static function get_type_option_names() {
return array_merge(parent::get_type_option_names(), array('rootpath'));
}
</code>

====type_config_form($mform, $classname='repository')====
Optional. This is for modifying the Moodle form displaying the plugin settings. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to display the standard repository plugin settings along with the custom ones use:
<code php>
public function type_config_form($mform) {
parent::type_config_form($mform);

$rootpath = get_config('repository_someplugin', 'rootpath');
$mform->addElement('text', 'rootpath', get_string('rootpath', 'repository_someplugin'), array('size' => '40'));
$mform->setDefault('rootpath', $rootpath);
}
</code>

====type_form_validation($mform, $data, $errors)====
Optional. Use this function if you need to validate some variables submitted by plugin settings form. To use it, check through the associative array of data provided ('settingname' => value) for any errors. Then push the items to $error array in the format ("fieldname" => "human readable error message") to have them highlighted in the form.

With the example above, this function may look like:
<code php>
public static function type_form_validation($mform, $data, $errors) {
if (!is_dir($data['rootpath'])) {
$errors['rootpath'] = get_string('invalidrootpath', 'repository_someplugin');
}
return $errors;
}
</code>

===Instance settings===
These functions relate to a specific instance of your plugin (e.g. the URL and login details to access a specific webdav repository). All of these are optional, without them, the instance settings form will only contain a single 'name' field.

==== get_instance_option_names()====
''This function must be declared static''<br>
Optional. Return an array of strings. These strings are setting names. These settings are specific to an instance.
If the function returns an empty array, the API will consider that the plugin displays only one repository in the file picker.
Parent function returns an empty array. This is equivalent to ''get_type_option_names()'', but for a specific instance.
<code php>
public static function get_instance_option_names() {
return array('fs_path'); // From repository_filesystem
}
</code>

====instance_config_form($mform)====
Optional. This is for modifying the Moodle form displaying the settings specific to an instance. This is equivalent to ''type_config_form($mform, $classname)'' but for instances. [[lib/formslib.php Form Definition]] has details of all the types of elements you can add to the settings form.

For example, to add a required text box called email_address:
<code php>
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', $strrequired, 'required', null, 'client');
</code>

''Note: ''mform'' has by default a name text box (cannot be removed).''

Parent function does nothing.

====instance_form_validation($mform, $data, $errors)====
Optional. This allows us to validate what has been submitted in the instance configuration form. This is equivalent to ''type_form_validation($mform, $data, $errors), but for instances. For example:
<code php>
public static function instance_form_validation($mform, $data, $errors) {
if (empty($data['email_address'])) {
$errors['email_address'] = get_string('invalidemailsettingname', 'repository_flickr_public');
}
return $errors;
}
</code>

====Getting / updating settings====

Both global and instance settings can be retrieved, from within the plugin, via $this->get_option('settingname') and updated via $this->set_option(array('settingname' => 'value')).

====plugin_init()====
''This function must be declared static''<br>
Optional. This function is called when the administrator adds the plugin. So unless the administrator deletes the plugin and re-adds it, it should be called only once.
Parent function does nothing.

===Example of using the settings===

As an example, let's create a Flickr plugin for accessing a public flickr account. The plugin will be called "Flickr Public".

Firstly the skeleton:

<code php>
<?php
/**
* repository_flickr_public class
* Moodle user can access public flickr account
*
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*/
class repository_flickr_public extends repository {
}
</code>

Then consider the question "What does my plugin do?"

In the Moodle file picker, we want to display some flickr public repositories directly linked to a flickr public account. For example ''My Public Flickr Pictures'', and also ''My Friend's Flickr Pictures''. When the user clicks on one of these repositories, the public pictures are displayed in the file picker.

In order to access to a flickr public account, the plugin needs to know the email address of the Flickr public account owner. So the administrator will need to set an email address for every repository. Let's add an "email address" setting to every repository.

<code php>
//We tell the API that the repositories have specific settings: "email address"
public static function get_instance_option_names() {
return array('email_address');
}

//We add an "email address" text box to the create/edit repository instance Moodle form
public function instance_config_form($mform) {
$mform->addElement('text', 'email_address', get_string('emailaddress', 'repository_flickr_public'));
$mform->addRule('email_address', get_string('required'), 'required', null, 'client');
}
</code>

So at this moment all our Flickr Public Repositories will have a specific email address. However this is not enough. In order to communicate with Flickr, Moodle needs to know a Flickr API key (http://www.flickr.com/services/api/). This API key is the same for any repository. We could add it with the email address setting but the administrator would have to enter the same API key for every repository. Hopefully the administrator can add settings to the plugin level, impacting all repositories. The code is similar the repository instance settings:
<code php>
//We tell the API that the repositories have general settings: "api_key"
public static function get_type_option_names() {
return array('api_key');
}

//We add an "api key" text box to the create/edit repository plugin Moodle form (also called a Repository type Moodle form)
public function type_config_form($mform) {
//the following line is needed in order to retrieve the API key value from the database when Moodle displays the edit form
$api_key = get_config('flickr_public', 'api_key');
$mform->addElement('text', 'api_key', get_string('apikey', 'repository_flickr_public'),
array('value'=>$api_key,'size' => '40'));
$mform->addRule('api_key', get_string('required'), 'required', null, 'client');
}
</code>

Have we finished yet?

Yes! We have created everything necessary for the administration pages. But let's go further. It would be good if the user can enter any "Flickr public account email address" in the file picker. In fact we want to display in the file picker a Flickr Public repository that the Moodle administrator can never delete. Let's add:

<code php>
//this function is only called one time, when the Moodle administrator add the Flickr Public Plugin into the Moodle site.
public static function plugin_init() {
//here we create a default repository instance. The last parameter is 1 in order to set the instance as readonly.
repository::static_function('flickr_public','create', 'flickr_public', 0, get_system_context(),
array('name' => 'default instance','email_address' => null),1);
}
</code>

That's all - the administration part of our Flickr Public plugin is done. For your information, Box.net, Flickr, and Flickr Public all have similar administration APIs.

==Repository APIs==
=== Quick Start ===
First of all, the File Picker using intensively Ajax you will need a easy way to debug. Install [[FirePHP]] (MDL-16371) and make it works. It will save you a lot of time. (You might give the [http://moodle.org/mod/forum/discuss.php?d=119961 FirePHP plugin for Moodle] a try, it's still work in progress, though.)

* Your first question when you write your plugin specification is 'Does the user need to log-in'? If they do, in your plugin you have to detect user session in constructor() function, and use print_login() if required, see more details below.
* For most of plugins, you need to establish a connection with the remote repository. This connection can be done into the get_listing(), constructor() function, see more details below.
* You wanna retrieve the file that the user selected, rewrite get_file() if required, see more details below.
* Optional question that you should ask yourself is 'Does the user can execute a search', if they do, you will have to rewrite search() method, see more details below.

===Functions you *MUST* override===

These functions cover the basics of initialising your plugin each time the repository is accessed and listing the files available to the user from within the plugin.

====__construct($respoitoryid, $context=SYSCONTEXTID, $options=array(), $readonly=0)====
Should be overridden to do any initialisation required by the repository, including:
* logging in via optional_param, if required - see 'print_login', below
* getting any options from the database

The possible items in the $options array are:
* 'ajax' - bool, true if the user is using the AJAX filepicker
* 'mimetypes' - array of accepted mime types, or '*' for all types

Calling parent::__construct($repositoryid, $context, $options, $readonly); is essential and will set up various required member variables:
* $this->id - the repository instance id (the ID of the entry in mdl_repository_instances)
* $this->context - the context in which the repository instance can be found
* $this->instance - the repository instance record (from mdl_repository_instances)
* $this->readonly - whether or not the settings can be changed
* $this->options - the above options, combined with the settings saved in the database
* $this->name - as specified by $this->get_name()
* $this->returntypes - as specified by $this->supported_returntypes()

====get_listing($path="", $page="")====
This function will return a list of files to be displayed to the user, the list must be a array like this:
<code php>
$list = array(
//this will be used to build navigation bar
'path'=>array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder')),
'manage'=>'http://webmgr.moodle.com',
'list'=> array(
array('title'=>'filename1', 'date'=>'1340002147', 'size'=>'10451213', 'source'=>'http://www.moodle.com/dl.rar'),
array('title'=>'folder', 'date'=>'1340002147', 'size'=>'0', 'children'=>array())
)
);
</code>
Amongst other details, this returns a '''title''' for each file (to be displayed in the filepicker) and the '''source''' for the file (which will be included in the request to 'download' the file into Moodle or to generate a link to the file). Directories return a '''children''' value, which is either an empty array (if 'dynload' is specified) or an array of the files and directories contained within it.

'''The full specification of list element:'''
<code php>
array(
// 'path' is used to build navigation bar to show the current folder, so you need to include all parents folders
// array(array('name'=>'root','path'=>'/'), array('name'=>'subfolder', 'path'=>'/subfolder'))
// This will result in: /root/subfolder as current directory
'path' => (array) this will be used to build navigation bar
// 'dynload' tells file picker to fetch list dynamically.
// When user clicks the folder, it will send a ajax request to server side.
// Default value is false but note that non-Javascript file picker always acts as if dynload was set to true
'dynload' => (bool) use dynamic loading,
// if you are using pagination, 'page' and 'pages' parameters should be set.
// It is not recommended to use pagination and subfolders at the same time, the tree view mode can not handle it correctly
'page' => (int) which page is this list
'pages' => (int) how many pages. If number of pages is unknown but we know that the next page exists repository may return -1
'manage' => (string) url to file manager for the external repository, if specified will display link in file picker
'help' => (string) url to the help window, if specified will display link in file picker
'nologin' => (bool) requires login, default false, if set to true the login link will be removed from file picker
'norefresh' => (bool) no refresh button, default false
'logouttext' => (string) in case of nologin=false can substitute the text 'Logout' for logout link in file picker
'nosearch' => (bool) no search link, default false, if set to true the search link will be removed from file picker
'issearchresult' => (bool) tells that this listing is the result of search
// for repositories that actually upload a file: set 'upload' option to display an upload form in file picker
'upload' => array( // upload manager
'label' => (string) label of the form element,
'id' => (string) id of the form element
),
// 'list' is used by file picker to build a file/folder tree
'list' => array(
array( // file
'title' => (string) file name,
'shorttitle' => (string) optional, if you prefer to display a short title
'date' => (int) UNIX timestamp, default value for datemodified and datecreated,
'datemodified' => (int) UNIX timestamp when the file was last modified [2.3+],
'datecreated' => (int) UNIX timestamp when the file was last created [2.3+],
'size' => (int) file size in bytes,
'thumbnail' => (string) url to thumbnail for the file,
'thumbnail_width' => (int) the width of the thumbnail image,
'thumbnail_height' => (int) the height of the thumbnail image,
'source' => plugin-dependent unique path to the file (id, url, path, etc.),
'url' => the accessible url of file,
'icon' => (string) url to icon of the image (24x24px), if omitted the moodle filetype icon will be used [2.3+],
'realthumbnail' => (string) url to image preview to be lazy-loaded when scrolled to it (if it requires to be generated and can not be returned as 'thumbnail') [2.3+],
'realicon' => (string) url to image preview in icon size (24x24) [2.3+],
'author' => (string) default value for file author,
'license' => (string) default value for license (short name, see class license_manager),
'image_height' => (int) if the file is an image, image height in pixels, null otherwise [2.3+],
'image_width' => (int) if the file is an image, image width in pixels, null otherwise [2.3+]
),
array( // folder - similar to file, has also 'path' and 'children' but no 'source' or 'url'
'title' => (string) folder name,
'shorttitle' => (string) optional, if you prefer to display a short title
'path' => (string) path to this folder. In case of dynload=true (and for non-JS filepicker) the value will be passed to repository_xxx::get_listing() in order to retrieve children
'date', 'datemodified', 'datecreated', 'thumbnail', 'icon' => see above,
'children' => array(
// presence of this attribute actually tells file picker that this is a folder. In case of dynload=true, it should be empty array
// otherwise it is a nested list of contained files and folders
)
),
)
// The 'object' tag can be used to embed an external web page or application within the filepicker
'object' => array(
'type' => (string) e.g. 'text/html', 'application/x-shockwave-flash'
'src' => (string) the website address to embed in the object
)
)
</code>
Dynamically loading
Some repositories contain many files which cannot load in one time, in this case, we need dynamically loading to fetch them step by step, files in subfolder won't be listed until user click the folder in file picker treeview.

As a plug-in developer, if you set dynload flag as '''true''', you should return files and folders (set children as a null array) in current path only instead of building the whole file tree.

Example of dynamically loading
See [http://cvs.moodle.org/moodle/repository/alfresco/lib.php?view=log Alfresco] plug-in

The use of the '''object''' tag, instead of returning a ''list'' of files, allows you to embed an external file chooser within the repository panel. See [[Repository plugins embedding external file chooser]] for details about how to do this.

===User login (optional)===
If the plugin requires login from the user at the time when they use it, then these functions can be used.

====print_login====
Returns an array of the elements required in the login form. If no login form is required, then the default implementation of this will redirect to the files list. If $this->options['ajax'] is not set, then an HTML-snippet with the login fields (but not the form tags) should be output, instead of returning the form details.
<code php>
public function print_login() { // From repository_alfresco
if ($this->options['ajax']) {
$user_field = new stdClass();
$user_field->label = get_string('username', 'repository_alfresco').': ';
$user_field->id = 'alfresco_username';
$user_field->type = 'text';
$user_field->name = 'al_username';

$passwd_field = new stdClass();
$passwd_field->label = get_string('password', 'repository_alfresco').': ';
$passwd_field->id = 'alfresco_password';
$passwd_field->type = 'password';
$passwd_field->name = 'al_password';

$ret = array();
$ret['login'] = array($user_field, $passwd_field);
return $ret;
} else { // Non-AJAX login form - directly output the form elements
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_alfresco').'</label></td>';
echo '<td><input type="text" name="al_username" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_alfresco').'</label></td>';
echo '<td><input type="password" name="al_password" /></td></tr>';
echo '</table>';
echo '<input type="submit" value="Enter" />';
}
}
</code>
This will help to generate a form by file picker which contains user name and password input elements.

If your login form is static and never changes, you can add ''$ret['allowcaching'] = true;'' and filepicker will not send the request to the server every time user opens the login/search form.

For plugins that do not fully process the login via a popup window, the submitted details can be retrieved, from within the '__construct' function, via $submitted = optional_param('fieldname', [defaultvalue], PARAM_INT/PARAM_TEXT).
<code php>
public function __construct($repositoryid, $context = SYSCONTEXTID, $options = array()) {
// Taken from repository_alfresco

/* Skipping code that is not relevant to user login */

$this->alfresco = new Alfresco_Repository($this->options['alfresco_url']);
$this->username = optional_param('al_username', '', PARAM_RAW);
$this->password = optional_param('al_password', '', PARAM_RAW);
try{
// deal with user logging in
if (empty($SESSION->{$this->sessname}) && !empty($this->username) && !empty($this->password)) {
$this->ticket = $this->alfresco->authenticate($this->username, $this->password);
$SESSION->{$this->sessname} = $this->ticket;
} else {
if (!empty($SESSION->{$this->sessname})) {
$this->ticket = $SESSION->{$this->sessname};
}
}
$this->user_session = $this->alfresco->createSession($this->ticket);
$this->store = new SpacesStore($this->user_session);
} catch (Exception $e) {
$this->logout();
}
$this->current_node = null;

/* Skipping code that is not relevant to user login */

}
</code>

Many types include a single element of type 'popup' with the param 'url' pointing at the URL used to authenticate the repo instance.
<code php>
public function print_login(){ // Code taken from repository_boxnet
$t = $this->boxclient->getTicket();
if ($this->options['ajax']) {
$popup_btn = new stdClass();
$popup_btn->type = 'popup';
$popup_btn->url = ' https://www.box.com/api/1.0/auth/' . $t['ticket'];

$ret = array();
$ret['login'] = array($popup_btn);
return $ret;
} else {
echo '<table>';
echo '<tr><td><label>'.get_string('username', 'repository_boxnet').'</label></td>';
echo '<td><input type="text" name="boxusername" /></td></tr>';
echo '<tr><td><label>'.get_string('password', 'repository_boxnet').'</label></td>';
echo '<td><input type="password" name="boxpassword" /></td></tr>';
echo '<input type="hidden" name="ticket" value="'.$t['ticket'].'" />';
echo '</table>';
echo '<input type="submit" value="'.get_string('enter', 'repository').'" />';
}
}
</code>

====check_login====
This function will return a boolean value to tell Moodle whether the user has logged in.
By default, this function will return true.
<code php>
public function check_login() { // Taken from repository_alfresco
global $SESSION;
return !empty($SESSION->{$this->sessname});
}
</code>
====logout====
When a user clicks the logout button in file picker, this function will be called. You may clean up the session or disconnect the connection with remote server here. After this the code should return something suitable to display to the user (usually the results of calling $this->print_login() ):
<code php>
public function logout() { // Taken from repository_alfresco
global $SESSION;
unset($SESSION->{$this->sessname});
return $this->print_login();
}
</code>

===Transferring files to Moodle (optional)===
These functions all relate to transferring the files into Moodle, once they have been chosen in the filepicker. All of them are optional and have default implementations which are often suitable to use as they are.

====get_file_reference($source)====
This function takes $source as in user input, parses and cleans it (recommended to call clean_param()). It prepares the reference to the file in repository-specific format that would be passed on to methods get_file(), get_link(), get_moodle_file(), get_file_by_reference() and/or stored in DB in case of creating a shortcut to file. For the most of repositories it is just clean $source value. For has_moodle_files-repositories this function also changes encoding.

====get_file($url, $filename = "")====
For FILE_INTERNAL or FILE_REFERENCE this function is called at the point when the user has clicked on the file and then on 'select this file' to add it to the filemanager / editor element. It does the actual transfer of the file from the repository and onto the Moodle server. The default implementation is to download the $url via CURL. The $url parameter is the $reference returned by get_file_reference (above, but usually the same as the 'source' returned by 'get_listing'). The $filename should usually be processed by $path = $this->prepare_file($filename), giving the full 'path' where the file should be saved locally. This function then returns an array, containing:
* path - the local path where the file was saved
* url - the $url param passed into the function

<code php>
public function get_file($url, $filename = '') {
// Default implementation from the base 'repository' class
$path = $this->prepare_file($filename); // Generate a unique temporary filename
$c = new curl;
$result = $c->download_one($url, null, array('filepath' => $path, 'timeout' => self::GETFILE_TIMEOUT));
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>
<code php>
public function get_file($reference, $filename = '') {
// Slightly extended version taken from repository_equella
global $USER;
// Extract the details saved in the 'source' param by
// repository/equella/callback.php (now in the $reference paramater)
$ref = @unserialize(base64_decode($reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}
$path = $this->prepare_file($filename);
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie'=>$cookiepathname));
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::GETFILE_TIMEOUT));
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}
if ($result !== true) {
throw new moodle_exception('errorwhiledownload', 'repository', '', $result);
}
return array('path'=>$path, 'url'=>$url);
}
</code>

====get_link($url)====
Used with FILE_EXTERNAL to convert a reference (from 'get_file_reference', but ultimately from the output of 'get_listing') into a URL that can be used directly by the end-user's browser. Usually just returns the original $url, but may need further transformation based on the internal implementation of the repository plugin.

{{Moodle 2.3}}====get_file_source_info($source)====
Takes the 'source' field from 'get_listing' (as returned by the user's browser) and returns the value to be stored in files.source field in DB (regardless whether file is picked as a copy or by reference). It indicates where the file came from. It is advised to include either full URL here or indication of the repository.
Examples: 'Dropbox: /filename.jpg', 'http://fullurl.com/path/file', etc.
This value will be used to display warning message if reference can not be restored from backup. Also it can (although not has to) be used in get_reference_details() to produce the human-readable reference source in the fileinfo dialogue in the file manager.

===Search functions (optional)===

These functions allow you to implement search functionality within your repository.

====print_search====
When a user clicks the search button on file picker, this function will be called to return a search form. By default, it will create a form with single search bar - you can override it to create a advanced search form.

A custom search form must include the following:
* A text field element named '''s''', this is where users will type in their search criteria

The following fields are automatically inserted in Moodle 2.3+ (but may need to be manually included in earlier versions):
* A hidden element named '''repo_id''' and the value must be the id of the repository instance
* A hidden element named '''ctx_id''' and the value must be the context id of the repository instance
* A hidden element named '''sesskey''' and the value must be the session key

<code php>
public function print_search() {
// The default implementation in class 'repository'
global $PAGE;
$renderer = $PAGE->get_renderer('core', 'files');
return $renderer->repository_default_searchform();
}

// From core_files_renderer (repository/renderer.php)
public function repository_default_searchform() {
$str = '<div class="fp-def-search"><input name="s" value='.get_string('search', 'repository').' /></div>';
return $str;
}
</code>

<code php>
public function print_search() {

// label search name
$param = array('for' => 'label_search_name');
$title = get_string('search_name', 'myrepo_search_name');
$html .= html_writer::tag('label', $title, $param);
$html .= html_writer::empty_tag('br');

// text field search name
$attributes['type'] = 'text';
$attributes['name'] = 's';
$attributes['value'] = '';
$attributes['title'] = $title;
$html .= html_writer::empty_tag('input', $attributes);
$html .= html_writer::empty_tag('br');

return $html;
}
</code>

====search($search_text, $page = 0)====
Return the results of doing the search. Any additional parameters from the search form can be retrieved by $param = optional_param('paramname', [defaultvalue], PARAM_INT / PARAM_TEXT);. The return should return an array containing:
* list - with the same layout as the 'list' element in 'get_listing'
<code php>
public function search($search_text, $page = 0) {
// Example from repoistory_googledocs
$gdocs = new google_docs($this->googleoauth);

$ret = array();
$ret['dynload'] = true;
$ret['list'] = $gdocs->get_file_list($search_text);
return $ret;
}
</code>

====global_search()====
Return true if should be included in a search throughout all repositories (currently not available via the UI)

{{Moodle 2.3}}===Repository support for returning file as alias/shortcut===

From Moodle 2.3 it became possible to link to the file from external (or internal) repository by reference. In UI it is called “create alias/shortcut”. This creates a row in {files} table but the contents of the file is not stored. Although it may be cached by repository if developer wants to.

Make sure that function supported_returntypes() returns FILE_REFERENCE among other types.

Note that external file is synchronised by moodle when UI wants to show the file size.

====get_reference_file_lifetime()====
Return minimum number of seconds before checking for changes to the file (default implementation = 1 day)
<code php>
public function get_reference_file_lifetime($ref) {
return 60 * 60 * 24; // One day
}
</code>

====sync_individual_file(stored_file $storedfile)====
Called after the file has reached the 'lifetime' specified above to see if it should now be synchronised (default implementation is to return true)
<code php>
public function sync_individual_file(stored_file $storedfile) {
return true;
}
</code>

====get_reference_details($reference, $filestatus = 0)====
Returns human-readable information about where the original file is stored (to be displayed in the filepicker properties box). It is usually prefixed with repository name and semicolon (e.g. 'Myrepository: http://url.to.file'). $reference is the 'source' output by 'get_listing'. $filestatus can be either 0 (OK - default) or 666 (source file missing).
<code php>
public function get_reference_details($reference, $filestatus = 0) {
// Example taken from repository_equella
if (!$filestatus) {
$ref = unserialize(base64_decode($reference));
return $this->get_name(). ': '. $ref->filename;
} else {
return get_string('lostsource', 'repository', '');
}
}
</code>

====get_file_by_reference($reference)====
Returns up-to-date information about the original file, only called when the 'lifetime' is reached and 'sync_individual_file' returns true.
* for image files - download the file and return either $ret->filepath (full path on the server), $ret->handle (open handle to the file) or $ret->content (raw data from the file) to allow the file to be saved into the Moodle filesystem and the thumbnail to be updated
* for non-image files - avoid downloading the file (if possible) and just return $ret->filesize to update that information
* for missing / inaccessible files - return null
Remember this function may be called quite a lot, as the filemanager often wants to know the filesize.
<code php>
public function get_file_by_reference($reference) {
// Example taken from repository_equella
global $USER;
// Extract the remote file identifier
$ref = @unserialize(base64_decode($reference->reference));
if (!isset($ref->url) || !($url = $this->appendtoken($ref->url))) {
// Occurs when the user isn't known..
return null;
}

// Download the file details
$return = null;
$cookiepathname = $this->prepare_file($USER->id. '_'. uniqid('', true). '.cookie');
$c = new curl(array('cookie' => $cookiepathname));
if (file_extension_in_typegroup($ref->filename, 'web_image')) {
// The file is an image - download and return the file path
$path = $this->prepare_file('');
$result = $c->download_one($url, null, array('filepath' => $path, 'followlocation' => true, 'timeout' => self::SYNCIMAGE_TIMEOUT));
if ($result === true) {
$return = (object)array('filepath' => $path);
}
} else {
// The file is not an image - just get the file details
$result = $c->head($url, array('followlocation' => true, 'timeout' => self::SYNCFILE_TIMEOUT));
}
// Delete cookie jar.
if (file_exists($cookiepathname)) {
unlink($cookiepathname);
}

$this->connection_result($c->get_errno());
$curlinfo = $c->get_info();
if ($return === null && isset($curlinfo['http_code']) && $curlinfo['http_code'] == 200
&& array_key_exists('download_content_length', $curlinfo)
&& $curlinfo['download_content_length'] >= 0) {
// we received a correct header and at least can tell the file size
$return = (object)array('filesize' => $curlinfo['download_content_length']);
}
return $return;
}
</code>

====send_file($storedfile, $lifetime=86400, $filter=0, $forcedownload=false, array $options = null)====
Send the requested file back to the user's browser. The 'reference' for the file can be found via $storedfile->get_reference(). If the file is not found / no longer exists, the function 'send_file_not_found()' should be used. Otherwise the file should be output directly, via the most appropriate method - e.g. use a 'Location: ' header to redirect to the external URL; or download the file and cache within the Moodle filesystem (possibly using '$this->import_external_file_contents()'), then call 'send_stored_file'. Note, it is up to the repository developer to decide whether to actually download the file or to return a locally cached copy instead.
<code php>
public function send_file($stored_file, $lifetime=86400 , $filter=0, $forcedownload=false, array $options = null) {
// Example taken from repository_equella
$reference = unserialize(base64_decode($stored_file->get_reference()));
$url = $this->appendtoken($reference->url);
if ($url) {
header('Location: ' . $url);
} else {
send_file_not_found();
}
}
</code>
An example of caching files within the Moodle filesystem can be found in repository_dropbox.

===Misc functions===

A couple of other useful functions to be aware of.

====get_name()====
Returns the human-readable name for this instance of the plugin (the default implementation should usually be fine and this function can be useful when doing any output to the user).

====cron()====
For any background tasks that need to be scheduled (rarely needed). The minimum time between calls is specified in the version.php file (but the maximum time depends on the server settings for the Moodle install).

== I18n - Internationalization ==
These following strings are required in ''moodle/repository/myplugin/lang/en/repository_myplugin.php'' or ''moodle/lang/en/repository_myplugin.php'':

<code php>
$string['pluginname'] = 'Flickr Public';
$string['configplugin'] = 'Flickr Public configuration';
$string['pluginname_help'] = 'A Flickr public repository';
</code>

==See also==

*[[Plugins]]
*[[Repository_Interface_for_Moodle/Course/User| Repository Interface for Moodle/Course/User]]
*[[QA:Use Case Number Attribution| Use Case Number Attribution]]
* MDL-16543 - A list of officially supported repository plugins
* MDL-16543 - Template plugin for developers

[[Category:Repositories]]
[[Category:Plugins]]

File:File options.png

2018-08-10T14:17:02Z

Jan.dagefoerde: Choices offered as a result of the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.

Choices offered as a result of the values FILE_INTERNAL, FILE_REFERENCE, and FILE_CONTROLLED LINK, respectively. Whether FILE_EXTERNAL is present is never reflected in this list.

XMLDB defining an XML structure

2018-01-30T12:32:53Z

Jan.dagefoerde: add hint that fields are a comma-separated string

[[XMLDB Documentation|XMLDB Documentation]] > [[XMLDB roadmap|Roadmap]] > Defining one XML structure
----

== Justification ==

Before Moodle 1.7, all the DB install and upgrade was developed twice (once to handle MySQL installations and another to handle PostgreSQL installations). This approach, although working, has caused some headaches in the past, mainly because it was really difficult to keep both lines of development 100% on sync. Some developers do they work against one RDBMS and it was complex to develop to the other one (two test environments, skills on both databases, slower development cycle...). And all this was happening with ''only'' two supported RDBMS!

One of the main objectives of Moodle 1.7 is to extend the the number of supported RDBMS to other flavours (more exactly, to Oracle and MSSQL). And the old approach (one line of development for each DB) could become an absolute nightmare.

Because of this we have planned to build one structure to define all the DB objects used by Moodle. This structure will provide the necessary level of abstraction to be shared by all the RDBMS systems, so the "multiple lines of development" explained in the previous paragraph will be out forever, giving us one robust and well defined way to handle DB objects independently of the underlying RDBMS being used.

== Implementation ==

Initially all our best wishes were to use the [http://phplens.com/lens/adodb/docs-datadict.htm#xmlschema AdoDB XML Schema]. As Moodle is using ADOdb libraries to communicate with databases it sounded like the natural approach to solve the problem. But, finally, two reasons prevented us to use it:

# Although working, it seems to be one feature in progress, with important changes/evolutions arriving at the time of write this document.
# Its lack of support for "prefixes" (one Moodle key feature, to allow multiple instances to run in the same server), would force us to create some awful tricks to generate the objects.

So, finally, we decided to build our own XML files, with everything we need to define every object present in the DB.

== The XMLDB editor ==
[[XMLDB_editor | Main article]]

Although the XML is pretty simple to read (and to write), one of the major drawbacks was its easy and error-prone adoption by the developers. Also some problems with versioning systems getting crazy with XML files (thanks ML!) pointed us to the requirement to use one high-density format (it means, physically '''long lines''') in our XML files.

After some intense thoughts we decided to build one specialised editor for our XML format. This editor should be easy to use and provide support for all the objects present one Moodle DB. And it's done (and will support future enhancements easily, we hope).

The XMLDB Editor makes the addition of tables/fields/keys/indexes practically a trivial task, allowing the developer to spend the time coding and improving things instead of fighting against XML files and the errors caused by manual editing (of course, the developer is free to use such extra-time as desired, beers, dance, books, music...) ;-)

All the new '''install.xml''' files, present under each '''db''' directory in Moodle can be edited (and we recommend it) with just some clicks and keystrokes. Those '''install.xml''' will contain all the info needed to generate the specific objects needed for each RDBMS supported. Obviously, such files, are the neutral replacement for all the *.sql files used until now.

=== Launching ===

Just login to your server as an administrator and, under the Miscellaneous tab of the Administration Block, you'll see a new link pointing to the "XMLDB Editor".

One '''important note''' is that, to be able to handle files properly, the web server needs write access to all those "db" directories where the "install.xml" files reside (and to the files themselves, of course). ;-)

That's all!

=== Use===

We really think the XMLDB Editor is pretty easy to use, so here you won't see a complete guide to use it. We highly recommend you to play with it for a while, viewing how it works and how it modifies the '''install.xml''' files.

It's organised in a top-botton structure, where you start '''loading''' (or '''creating''') a new XMLDB file. Then, you can '''edit''' such file and its '''general structure''' will be showed. This structure have two type of elements, '''tables''' and '''statements''' and the XMLDB Editor allows you to '''add''', '''edit''', '''delete''', and '''move''' them easily. Also, for initial creation of tables, one small but effective '''reverse-enginery''' tool has been developed (only under MySQL) allowing you to retrofit any table from the DB to the XMLDB Editor.

'''Note: If you can't click on the create links....''' you must first create the /db folder (as shown in the list, but it may not really exist) and then make sure it is writeable by the webserver

While editing tables you will see their '''fields''', '''keys''' and '''indexes''' and you'll be able to handle all them easily. Note that some fields can be no-editable. It uses to be because they are being used in some way (part of one key or index) and the idea is to warn you about that.

Fields can be edited and you can specify their '''name''', '''type''', '''length''', '''decimals''', '''null-ability''', '''defaults''' and so one. Exactly the same for both '''keys''' and '''indexes'''.

While editing statements, you must think about them like "collections of sentences". Once you select the '''type''' (only inserts are allowed for now) and '''table''' you are interested you'll be able to introduce the exact values easily, being able to '''duplicate''' them easily to gain some speed if you have a lot of sentences in your development. Sentences can be '''edited''' and '''deleted''' easily too.

One interesting feature is that all the XMLDB Editor pages allow you to enter one '''comment''' about the item being modified (table, index, key, field, statement...). Use it at your entire needs, sure it helps other developers to understand a bit more the DB model.

Please, don't forget to read and understand the next section, where we talk about '''some important guidelines''' to create and handle XMLDB files.

== Conventions ==

Apart of the [[Database| Database Structures guidelines]], some more conventions should be followed:

# About names:
## All lowercase names (tables, indexes, keys and fields).
## Table names and field names must use only a-z, 0-9 and _ chars. Table names can be at most 28 characters long; column names at most 30 characters.
## Key and index names under the XMLDB Files must be formed by concatenating the name of the fields present in the key/index with the '"-" (minus) character.
## Primary key always must be named "primary" (this is one exception to the previous convention).
## It's highly recommended to avoid [[XMLDB_reserved_words|reserved words]] completely. We know we have some of them now but they should be completely out for next releases.
# About NULLS
## Avoid to create all the fields as NOT NULL with the ''silly'' default value <nowiki>''</nowiki> (empty string). The underlying code used to create tables will handle it properly but the XMLDB structure must be REAL. Read more in the [[XMLDB_Problems#NOT_NULL_fields_using_a_DEFAULT_.27.27_clause|Problems Page]].
# About FOREIGN KEYS
## Under the tables of every XMLDB file, you must define the existing '''Foreign Keys''' (FK) properly. This will allow everybody to know a bit better the structure, allow to evolve to a better constrained system in the future and will provide the underlying code with the needed info to create the proper indexes.
## Note that, if you define any field combination as FK you won't have to create any index on that fields, the code will do it automatically!
## Respect Convention 1.3
# About UNIQUE KEYS
## Declare any fields as UNIQUE KEY (UK) only if they are going to be used as target for one FK. Create unique indexes instead.
## Respect Convention 1.3

== One example: the assignment module ==

Here we are going to examine the [http://cvs.moodle.org/moodle/mod/assignment/db/install.xml?view=markup current implementation of the XMLDB Schema for the assignment module] (a simple one). It has been completely generated with the XMLDB Editor but it's nice to know a bit more about the XML internals.

As you can see the structure is pretty simple:

* XMLDB
** TABLES, one or more, each one with
*** FIELDS
*** KEYS
*** INDEXES
** STATEMENTS, none or more, each one with
*** SENTENCES

First of all you should note that all the elements contain the '''PREVIOUS''' and '''NEXT''' attributes. They allow us to keep everything ordered although it isn't meaningful at all from the RDBMS perspective. Also the '''COMMENT''' field is present everywhere to be used as desired.

=== The TABLE element ===

We can ignore the TABLE element, as it's simply one container for the internals (FIELDS, KEYS and INDEXES). Let's go to examine them a bit more:

==== The FIELD element ====

It maps with one field in the DB (obviously). For each field you can define its '''name''', '''type''' (from a list of [[XMLDB column types|neutral types]]), '''length''', '''decimals''' (for some types), '''notnull''' (true/false), '''unsigned''' (true/false), '''sequence''' (if it's autonumeric or serial, true/false), '''enum''' (true/false), '''enumvalues''' (the list of values if the field has been declared as enum, for example <tt>'frog','toad','newt'</tt>) and '''default''' (to assign a default value).

So, in our example, we have two tables, assignment and assignment_submissions, each one with its own fields, defining all the information related above. Please note that naming conventions are followed.

==== The KEY element ====

Here is where all the PRIMARY KEYS (PK), UNIQUE KEYS (UK) and FOREIGN KEYS (FK) will be defined. For each key we define its '''name''', '''type''', '''fields''' (that belongs to it) and optionally (if the key is one FK) the target '''reftable''' and '''reffields'''.

In our example, the assignment table has one (mandatory!) PK (called, "primary", rules are rules) built with the "id" field.

The other table, the "assignment_submissions" one, also has its PK (called "primary" once more) and one FK, with the field "assignment" pointing to the field "id" of the table "assignment". Note that the FK follows the name conventions and its name is, simply, the name of the fields being part of it ("assignment"). Also, the FK has as target to one PK of the same module.

Finally, note that there isn't any index created for all these keys. Moodle will generate them automatically when the table is created. All the keys will have their corresponding index. Point. ;-)

==== The INDEX element ====

Where all the indexes will be defined. For each index you can define its '''name''', '''unique''' (true/false) and the '''fields''' (as a comma-separated string) that it comprises. Please note that naming conventions are followed.

Also, some "obvious index", like the one based in the "assignment" field of the "assignment_submissions" table doesn't exist. Yes, you know why: Because such column has been defined as a FK and the index will be automatically created (see previous section).

=== The STATEMENT element ===

This is the other '''big container''' in the XMLDB Schema (at the same level as the '''TABLES''' one) and we can define its '''name''', '''type''' (only insert allowed for now) and '''table''' (against the sentences will be executed).

Every statement is a collection of '''sentences'''

==== The SENTENCE element ====

Each sentence implies one simple action to be performed against the DB and it can be defined as the "missing part of the SQL statement". In our example, we have one statement, of type "insert" on table "log_display". With this Moodle knows the initial part of the sentence, i.e:

INSERT INTO log_display

and then the text will be added to create this:

INSERT INTO log_display
(module, action, mtable, field)
VALUES
('assignment', 'view', 'assignment', 'name')

There is one important trick when handling sentences, although they aren't in the assignment example. Take a look to the [http://cvs.moodle.org/moodle/lib/db/install.xml?view=co Core Tables XML Schema] (it's a huge one!). If you go near the end, to the statements section, you will see some sentences like this:

<SENTENCE TEXT="....VALUES ('user', 'view', 'user', 'CONCAT(firstname," ",lastname)')"/>

Such "CONCAT" function isn't standard at all (only MySQL supports it), but don't worry, we'll transform it to the correct concatenation operators for other RDBMS. Just be sure to use the syntax showed above.

== DTD and XML schema ==

Not sure if this will be usable for somebody but here you can find one [http://cvs.moodle.org/moodle/lib/xmldb/xmldb.dtd?view=co automatically generated DTD] for the XMLDB files. Also one [http://cvs.moodle.org/moodle/lib/xmldb/xmldb.xsd?view=co automatically generated XML Schema] is available. Any improvement/fix to them will be welcome!

== See also ==

* [[XMLB List of files to create|List of files to create]]: The list of files to be created from scratch. Used to follow the progress.
* http://www.hitsw.com/xml_utilites/: One online XML-DTD-Schema converter.

[[Category:XMLDB]]
[[Category:DB]]

Lock API

2017-08-17T14:33:10Z

Jan.dagefoerde: /* Usage */

{{Moodle 2.7}}

{{Infobox Project
|name = Lock
|state = Done
|tracker = MDL-25500
|discussion = https://moodle.org/mod/forum/discuss.php?d=229139
|assignee = Damyon
}}

= Locks =

Locking is required whenever you need to prevent 2 processing accessing the same resource at the same time. The prime candidate for locking in Moodle is cron. This will allow multiple cron processes to work on different parts of cron at the same time with no risk that they will conflict (work on the same job at the same time).

This is a new feature for Moodle 2.7 and will not be available in older versions.

= When to use locking =

When you want to prevent multiple requests from accessing the same resource at the same time. Accessing a resource is a vague description, but it could be e.g. running a slow running task in the background, running different parts of cron etc.

= Performance =
Locking is not meant to be fast. Do not use it in code that will be triggered many times in a single request (e.g. MUC). It is meant to be always correct - even for multiple nodes in a cluster. This implies that the locks are communicated among all the nodes in the cluster, and hence it will never be super quick.

= Usage =

The locking API is used by getting an instance of a lock_factory, and then using it to retrieve locks, and eventually releasing them. You are required to release all your locks, even on the event of failures.

<code php>

// 5 seconds.
$timeout = 5;
// A namespace for the locks. Must be prefixed with the component name to prevent conflicts.
$locktype = 'mod_assign_download_submissions';
// Resource key - needs to uniquely identify the resource that is to be locked. E.g. If you
// want to prevent a user from running multiple course backups - include the userid in the key.
$resource = 'user:' . $USER->id;

// Get an instance of the currently configured lock_factory.
$lockfactory = \core\lock\lock_config::get_lock_factory($locktype);

// Get a new lock for the resource, wait for it if needed.
if ($lock = $lockfactory->get_lock($resource, $timeout)) {
// We have exclusive access to the resource, do the slow zip file generation...

if ($someerror) {
// Always release locks on failure.
$lock->release();
print_error('blah');
}

// Release the lock once finished.
$lock->release();

} else {
// We did not get access to the resource in time, give up.
throw new moodle_exception('locktimeout');
}
</code>

= Use a different lock type from the default =
Change the $CFG->lock_factory setting to one of the other lock types included with core. These are all documented in config-dist.php.

= Implementing new lock types =
If you really want to do this you can. I probably wouldn't recommend it - because the core lock types should be very reliable - and the performance is not really a concern.

Add a new local_XXX plugin with an autoloaded class that implements \core\lock\lock_factory.
Set $config->lock_factory to the full namespaced path to your class e.g.
<code php>
$config->lock_factory = '\\local_redis\\lock\\redis_lock_factory';
</code>
Note: the extra backslashes are just required because it's a string and php insists on moronic syntax for namespaces.

Note: lib/tests/lock_test.php has unit tests that can be run on a custom lock instance to verify it for correctness (run_on_lock_factory).

Composer

2017-07-06T14:03:41Z

Jan.dagefoerde: Use stable Moodle repository in order to avoid that people download Moodle from the integration server

= Use composer to download moodle code =
Composer.json now includes meta information and hence composer can be used to download the moodle code base.
You can do it by creating composer.json file with following information
<code>
{
"repositories": [
{
"type": "vcs",
"url": "https://github.com/moodle/moodle.git"
}
],
"require": {
"moodle/moodle": "v3.2.0"
}
}
</code>
And then execute
<code>php composer.phar install</code>

Creating a theme settings page

2016-03-10T13:46:05Z

Jan.dagefoerde: More accurate description how a $settings object is created.

{{Template:Themes}}This document looks at how to create a settings page for your Moodle 2.x.x theme and how to make use of those settings within the CSS and layout files for your theme.

This is a pretty advanced topic and will require that you have at least an intermediate knowledge of PHP, CSS, and development in general.

==Before we begin==
[[Image:Theme.settings.page.03.png|350px|thumb|Our end goal. The settings page.]]
[[Image:Theme.settings.page.10.png|350px|thumb|And what it can do.]]
There is a huge body of knowledge that we must cover in following through this document and as such I think the best way to write this is as a tutorial.

My intentions for this tutorial are to replicate the standard theme but with a settings page that allows the administrator to set a background colour, set a logo to use with the page, and probably several other minor settings to change the way in which the theme is displayed.

I will start this tutorial by creating a new theme which will be largely a copy/paste of the current standard theme. I expect that anyone working through this tutorial has previously read the tutorial I wrote on [[Themes 2.0 creating your first theme|creating your first theme]]. If you haven't go read it now because I'm not going to go into much detail until we get to the actual process of customising the theme and introducing the settings page.

So before we finally get this started please ensure you can check off everything on the following requirements list.
* Have a Moodle installation that has already been installed and configured and is ready to use.
* Have full read/write access to that installation.
* Be prepared to delete that installation at the end of this... we will destroy it!
* Have a development environment prepared and ready to use. This includes:
** Your favourite editor installed, running, and pointed at the themes directory of your installation.
** Your browser open and your site visible.
** A bottomless coffee pot... decaf won't help you with this one.
* Have set the following settings:
** '''themedesignermode''' if you don't know what this is please read the [[Themes 2.0 creating your first theme|creating your first theme]] tutorial.
** '''allowthemechangeonurl''' turn this on, it allows you to change themes on the URL and is very handy when developing themes. ''Site Administration > Appearance > Themes > Theme settings''
** '''langstringcache''' if you don't turn this off you won't see your strings when they are added. ''Site Administration > Language > Language settings''
* And finally an insane ambition to create a customisable theme.

For those interested the theme that I create throughout this tutorial can be downloaded from the forum post in which I announce this document: http://moodle.org/mod/forum/discuss.php?d=152053
<br style="clear:right;" />

==Our goals for this tutorial==
The following is just a list goals that I hope to achieve during this tutorial. They are laid out here so that I can easily refer back to them and so that you can easily find them.
# Create a new theme called '''demystified''' based upon the standard theme within Moodle 2.0.
# Make some minor changes to that theme to allow us to more easily see what is going on.
# Create a settings page for the demystified theme.
# Add several settings to our settings page.
# Use some of those settings to alter our CSS.
# Use the rest of those settings within our layout file..
# Discuss the good, the bad, and limits of what we have just created.

So I can see you are all very excited about this point and that you would love to know what settings we are going to create; So here they are:

A setting to ...
* change the background colour (CSS).
* set the path to an image that we will use as a logo on all pages (Layout files).
* override the width of the block regions (CSS).
* allow a note to be added to the footer of all pages (Layout files).
* allow custom CSS to be written to do anything the user wants. (CSS)

==Creating the demystified theme==
Before we start here I want to remind you that I am going to look at this only briefly as I am making the assumption that you have read the [[Themes 2.0 creating your first theme|creating your first theme]] tutorial.

Well lets get into it....

The first thing we need to do is create a directory for our theme which we will call demystified.

So within your Moodle directory create the following folder '''moodle/theme/demystified'''. At the same time you can also create the following files and folders which we will get to soon.
* The file '''moodle/theme/demystified/config.php''' for our config information.
* The directory '''moodle/theme/demystified/layout''' for our layout files.
* The directory '''moodle/theme/demystified/style''' for our css files.
* The file '''moodle/theme/demystified/style/core.css''' which will contain our special CSS.

Next we will copy the layout files from the base theme to our new theme demystified. We are basing the demystified theme on the standard theme however that doesn't use it's own layout files it uses the base theme's layout files so those are the ones we want.

The reason that we are coping these layout files is that later on in this tutorial we will be modifying them to make use of our new settings... so copy all of the layout files from '''moodle/theme/base/layout''' to '''moodle/theme/demystified/layout'''.

There should be three files that you just copied:
# embedded.php
# frontpage.php
# general.php

Now we need to populate demystified/config.php with the settings for our new theme. They are as follows:
<code php>
$THEME->name = 'demystified';
</code>
Simply sets the name of our theme.

<code php>
$THEME->parents = array('standard','base');
</code>
This theme is extending both the standard theme and the base theme. Remember when extending a theme you also need to extend its parents or things might not work correctly.

<code php>
$THEME->sheets = array('core');
</code>
This tells our theme that we want to use the file '''demystified/style/core.css''' with this theme.

<div style="height:300px;overflow-y:scroll;">
<code php>
$THEME->layouts = array(
// Most backwards compatible layout without the blocks - this is the layout used by default
'base' => array(
'file' => 'general.php',
'regions' => array(),
),
// Standard layout with blocks, this is recommended for most pages with general information
'standard' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// Main course page
'course' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
'options' => array('langmenu'=>true),
),
'coursecategory' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// part of course, typical for modules - default page layout if $cm specified in require_login()
'incourse' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// The site home page.
'frontpage' => array(
'file' => 'frontpage.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// Server administration scripts.
'admin' => array(
'file' => 'general.php',
'regions' => array('side-pre'),
'defaultregion' => 'side-pre',
),
// My dashboard page
'mydashboard' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
'options' => array('langmenu'=>true),
),
// My public page
'mypublic' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
'login' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('langmenu'=>true),
),

// Pages that appear in pop-up windows - no navigation, no blocks, no header.
'popup' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
// No blocks and minimal footer - used for legacy frame layouts only!
'frametop' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('nofooter'=>true),
),
// Embeded pages, like iframe/object embeded in moodleform - it needs as much space as possible
'embedded' => array(
'file' => 'embedded.php',
'regions' => array(),
'options' => array('nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
// Used during upgrade and install, and for the 'This site is undergoing maintenance' message.
// This must not have any blocks, and it is good idea if it does not have links to
// other places - for example there should not be a home link in the footer...
'maintenance' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('noblocks'=>true, 'nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
);
</code>
</div>
Now that all looks very complicated however its really not too bad as it is just copied from the base theme's config.php file. We can do this because we copied the layout files from the base theme to begin with and for the time being there are no changes that we wish to make. Simply open up '''theme/base/config.php''' and copy the layouts from there.

And that is it. The config.php file for our demystified theme is complete. The full source is shown below:
<div style="height:300px;overflow-y:scroll;">
<code php>
<?php

// This file is part of Moodle - http://moodle.org/
//
// Moodle is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Moodle is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Moodle. If not, see <http://www.gnu.org/licenses/>.

/**
* The demystified theme config file
*
* This theme was created to document the process of adding a settings page to a theme
*
* @copyright 2010 Sam Hemelryk
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/

// The name of our theme
$THEME->name = 'demystified';

// The other themes this theme extends
$THEME->parents = array('standard','base');

// The CSS files this theme uses (located in the style directory)
$THEME->sheets = array('core');

// The layout definitions for this theme
$THEME->layouts = array(
// Most backwards compatible layout without the blocks - this is the layout used by default
'base' => array(
'file' => 'general.php',
'regions' => array(),
),
// Standard layout with blocks, this is recommended for most pages with general information
'standard' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// Main course page
'course' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
'options' => array('langmenu'=>true),
),
'coursecategory' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// part of course, typical for modules - default page layout if $cm specified in require_login()
'incourse' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// The site home page.
'frontpage' => array(
'file' => 'frontpage.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
// Server administration scripts.
'admin' => array(
'file' => 'general.php',
'regions' => array('side-pre'),
'defaultregion' => 'side-pre',
),
// My dashboard page
'mydashboard' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
'options' => array('langmenu'=>true),
),
// My public page
'mypublic' => array(
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
),
'login' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('langmenu'=>true),
),

// Pages that appear in pop-up windows - no navigation, no blocks, no header.
'popup' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
// No blocks and minimal footer - used for legacy frame layouts only!
'frametop' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('nofooter'=>true),
),
// Embeded pages, like iframe/object embeded in moodleform - it needs as much space as possible
'embedded' => array(
'file' => 'embedded.php',
'regions' => array(),
'options' => array('nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
// Used during upgrade and install, and for the 'This site is undergoing maintenance' message.
// This must not have any blocks, and it is good idea if it does not have links to
// other places - for example there should not be a home link in the footer...
'maintenance' => array(
'file' => 'general.php',
'regions' => array(),
'options' => array('noblocks'=>true, 'nofooter'=>true, 'nonavbar'=>true, 'nocustommenu'=>true),
),
);
</code>
</div>

The screenshot below shows both the directory structure we have now created and the theme presently.

[[Image:Theme.settings.page.01.png]]

To view the theme so far open you browser and enter the URL of your site followed by '''?theme=demystified'''. You should see the theme that we just created which will look exactly like the base standard theme.

The final thing that we want to do is add a little bit of CSS to the demystified theme that will both visually set this theme apart from the standard theme and second build a the base which our settings can later extend.

I added the following snippet of CSS to the file '''demystified/style/core.css'''.
<code css>
html {background-color:#DDD;}
body {margin:30px;padding:0;border:1px solid #333;border-width:0 10px 0 10px;background-color:#333;}
body #page {background-color:#FFF;position:relative;top:-10px;}
.block .header {background-image:none;background-color:#0C5CAC;border:1px solid #0C5CAC;color:#FFF;}
.block {border-color:#4BA7FF;background-color:#DDEEFF;}
.block .content {background-color:#F1F8FF;}
a:link,
a:visited {color:#0C5CAC;}
a:hover {color:#C77500;}
#page #page-header {background-color:#0C5CAC;margin:0;padding:0;width:100%;color:#fff;}
#page #page-header a:link, #page #page-header a:visited {color:#FFAC02}
#page #page-header .navbar, #page #page-header .navbar a:link, #page #page-header .navbar a:visited {color:#0C5CAC;}
</code>

The CSS that we have just added to our theme sets a couple of colours on the front page. Presently this is the only CSS I will add, I know it isn't complete by any means but it achieves it's purpose as the screenshot below illustrates.

[[Image:Theme.settings.page.02.png|715px|thumb|left|The newly styles demystified theme]]<br style="clear:both;" />

And with that I will move on to the real purpose of this tutorial, creating the settings page

==Setting up the settings page==
With the demystified theme set up it is time to create the settings page. This is where the real PHP fun begins.

For those of you who happen to be familiar with development of modules, blocks or other plugin types you have probably encountered settings pages before and this is not going to be any different.

However for those who haven't which I imagine is most of you this is going to be quite a challenge. I will try to walk through this step by step however if at any point you get stuck don't hesitate to ask in the forums as I imagine you will get a speedy response.

===How settings pages work in Moodle===
Settings pages can be used by nearly every plugin type, of which themes is of course one. The way in which it all works isn't too tricky to understand.

All of the settings for Moodle can be configured through the administrator interfaces when logged in. I am sure that everyone here has seen those pages and has changed a setting or two before so you will all know what I am talking about. Well the settings page for a theme is no different. It will be shown in the administration pages tree under '''Appearance > Themes''' and all we have to do is tell Moodle what settings there are.

This is done by creating a settings.php file within our theme into which we will add code that tells Moodle about the settings we want to add/use.

When telling Moodle about each setting we are simply creating a new ''admin_setting'' instance of the type we want and the properties we want and then adding it to our settings page.

There is really not much more too it at this level. Things can get very complex very fast so the best thing we can do now is start creating our settings.php file for the demystified theme and see where it leads us.

===Creating the settings page===
So as mentioned before we need a settings.php file which we will create now. To begin with create the file '''theme/demystified/settings.php''' and open it in your favourite editor so its ready to go.

Before we start adding code however lets just remember the settings that we want to create:
* change the background colour (CSS).
* set the path to an image that we will use as a logo on all pages (Layout files).
* override the width of the block regions (CSS).
* allow a note to be added to the footer of all pages (Layout files).
* allow custom CSS to be written to do anything the user wants. (CSS)

Alright.

Now thinking about this the first setting is as basic as it gets, all we need is a text box that the user can type a colour into.

The second is to allow a logo to be used in the header of each page. What we want here is a path but should it be a physical path e.g. C:/path/to/image.png or should it be a web path e.g. <nowiki>http://mysite.com/path/to/image.png</nowiki>?
For the purpose of this tutorial I am going to go with a web path because it is going to be easier to code and will hopefully be a little easier to understand to begin with.

The third setting is a little more complex. For this I want a drop down box with some specific widths that the administrator can select.

The forth and the fifth settings are both pretty straight forward, there we want a textarea into which the user can enter what ever they want and we will do something useful with it.

Now that we have an understanding about the settings we wish to define pull up your editor and lets start coding....

<code php>
<?php

/**
* Settings for the demystified theme
*/
defined('MOODLE_INTERNAL') || die;

if ($ADMIN->fulltree) {
</code>
This is the first bit of code we must enter, the first line is of course just the opening php tag, secondly we have a comment that describes this file. The surrounding part of Moodle has already created an '''admin_settingspage''' object which can be referenced as '''$settings'''. Therefore, it does not need to be created by us. It already represents an entry in the Moodle administration entry. We proceed with testing for '''$ADMIN->fulltree''' in order to find out whether actual settings should be initialised.

For more background information: Look at the class '''\admin_root''' in lib/adminlib.php to find out about '''$ADMIN->fulltree''' and have a look at admin/settings/appearance.php to find out how '''"$themedir/settings.php"''' is included.

====Background colour====

With the page now created lets add our first setting: Background colour.

<code php>
// Background colour setting
$name = 'theme_demystified/backgroundcolor';
$title = get_string('backgroundcolor','theme_demystified');
$description = get_string('backgroundcolordesc', 'theme_demystified');
$default = '#DDD';
$setting = new admin_setting_configtext($name, $title, $description, $default, PARAM_CLEAN, 12);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
</code>
Thankfully this isn't as difficult as it initially looks.

The first line of code is creating a variable for the name of the background colour setting. In this case it is '''theme_demystified/backgroundcolor'''.

The name is very important, for the setting to be usable we have to follow a strict naming convention. '''theme_''themename''/''settingname''''' where '''''themename''''' is the name of the theme the setting belongs to and '''''settingname''''' is the name for the setting by which we will use it.

The second line of code creates a variable that contains the title of the setting. This is what the user sees to the right of the setting on the settings page and should be a short description of the setting. Here we are again using the ''get_string'' method so we will need to remember to add that string later on.

The third line of code sets the description. This should describe what the setting does or how it works and again we will use the get_string method.

The fourth line creates a variable that will be used as the default value for the setting. Because this setting is a colour we want an HTML colour to be the default value.

The fifth line is where we put it all together. Here we create a new '''admin_setting_configtext''' object. This object will represent the background colour setting.

When we create it we need to give it 6 different things.
# The name of the setting. In this case we have a variable '''$name'''.
# The title for this setting. We used the variable '''$title'''.
# The description of the setting '''$description'''.
# The default value for the setting. '''$default''' is the variable this.
# The type of value we want the user to enter. For this we have used PARAM_CLEAN which tells Moodle to get rid of any nasties from what the user enters.
# The size of the field. In our case 12 characters will be plenty.

The sixth and final line of code adds our newly created setting to the administration page we created earlier.

That is it we have successfully created and added our first setting, however there are several more to settings to do, and there are a couple of important things that you need to be aware of before we move on.

First: There are several different types of settings that you can create and add to a page, and each one may differ in what they need you to give them. In this case it was name, title, description, default, type, and size. However other settings will likely require different things. Smart editors like Netbeans or Eclipse can tell you what is required, otherwise you will need to research it.

Second: Normally settings are declared on one line as follows:
<code php>
$setting->add(new admin_setting_configtext('theme_demystified/backgroundcolor', get_string('backgroundcolor','theme_demystified'), get_string('backgroundcolordesc', 'theme_demystified'), '#DDD', PARAM_CLEAN, 12));
</code>
While this is structurally identical as all we have done is move everything onto one line and do away with the variables it is a little harder to read when you are learning all of this.

====The logo file====
Time to create the second setting that will allow the user to enter a URL to an image they wish to use as the logo on their site.
<code php>
// Logo file setting
$name = 'theme_demystified/logo';
$title = get_string('logo','theme_demystified');
$description = get_string('logodesc', 'theme_demystified');
$setting = new admin_setting_configtext($name, $title, $description, '', PARAM_URL);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
</code>
The first thing that you will notice about this setting that it is very similar to the first setting, in fact all we have changed is the name, title, description, and default value. We have however changed the value type from PARAM_CLEAN to PARAM_URL, this makes sure the user enters a URL. You will also notice that for this one we don't set a size for the field as we have no idea how long the URL will be.

====Block region width====
The third setting should allow the user to set a width for the block regions that will be used as columns.

For this setting I want to do something a little different from the previous two, here I want to use a select box so that the user selects a width for the column from a list I provide.
<code php>
// Block region width
$name = 'theme_demystified/regionwidth';
$title = get_string('regionwidth','theme_demystified');
$description = get_string('regionwidthdesc', 'theme_demystified');
$default = 200;
$choices = array(150=>'150px', 170=>'170px', 200=>'200px', 240=>'240px', 290=>'290px', 350=>'350px', 420=>'420px');
$setting = new admin_setting_configselect($name, $title, $description, $default, $choices);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
</code>
So looking at the code: The first four lines you will recognise. $name, $title, $description, and $default are all being set.

The fifth line of code however introduces something new. Of course in order to have a select box we have to have options, in this case we have an array of options stored in the variable $choices.

The array of options is constructed of a collection of '''''value''' => '''label''''' pairs. Notice how we don't add '''px''' to the value. This is is very intentional as later on I need to do a little bit of math with that value so we need it to be a number.

The lines after should look familiar again, the only difference being that instead of a ''admin_setting_configtext'' setting we have created a ''admin_setting_configselect'' for which we must give the choices for the select box as the fifth argument.

Woohoo, we've just created our first select box setting.

====Foot note====
Now to create the foot note setting. Here we want the user to be able to enter some arbitrary text that will be used in the footer of the page. For this I want the user to be able to enter some HTML so I will create an editor setting.
<code php>
// Foot note setting
$name = 'theme_demystified/footnote';
$title = get_string('footnote','theme_demystified');
$description = get_string('footnotedesc', 'theme_demystified');
$setting = new admin_setting_confightmleditor($name, $title, $description, '');
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
</code>
How simple is that!

It is just about identical to the first two settings except that for this we have created a ''admin_setting_confightmleditor'' setting rather than a text setting.

Note: You can also set the columns and rows for the editor setting using the fifth and sixth arguments.

====Custom CSS====
The final setting is to allow the user to add some custom CSS to the theme that will be used on every page. I want this to be a plain textarea into which the user can enter CSS.
<code php>
// Custom CSS file
$name = 'theme_demystified/customcss';
$title = get_string('customcss','theme_demystified');
$description = get_string('customcssdesc', 'theme_demystified');
$setting = new admin_setting_configtextarea($name, $title, $description, '');
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
</code>
Just like the editor or text settings. It's getting very easy now!

====Finishing settings.php====
With all of our settings defined and added to our page that we created right at the beginning it is time to finish it all off.
<code php>
} // This is the closing brace that encloses all the above settings.
</code>
The above line is the final line for the page. It is adding the page that we have created '''$setting''' to the admin tree structure. In this case it is adding it to the themes branch.

The following is the completed source for our settings.php ..... for your copy/paste pleasure.
<div style='height:300px;overflow:auto;'>
<code php>
<?php
/**
* Settings for the demystified theme
*/
defined('MOODLE_INTERNAL') || die;

if ($ADMIN->fulltree) {

// Background colour setting
name = 'theme_demystified/backgroundcolor';
$title = get_string('backgroundcolor','theme_demystified');
$description = get_string('backgroundcolordesc', 'theme_demystified');
$default = '#DDD';
$setting = new admin_setting_configtext($name, $title, $description, $default, PARAM_CLEAN, 12);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);

// Logo file setting.
$name = 'theme_demystified/logo';
$title = get_string('logo','theme_demystified');
$description = get_string('logodesc', 'theme_demystified');
$setting = new admin_setting_configtext($name, $title, $description, '', PARAM_URL);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);

// Block region width.
$name = 'theme_demystified/regionwidth';
$title = get_string('regionwidth','theme_demystified');
$description = get_string('regionwidthdesc', 'theme_demystified');
$default = 200;
$choices = array(150=>'150px', 170=>'170px', 200=>'200px', 240=>'240px', 290=>'290px', 350=>'350px', 420=>'420px');
$setting = new admin_setting_configselect($name, $title, $description, $default, $choices);
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);

// Foot note setting.
$name = 'theme_demystified/footnote';
$title = get_string('footnote','theme_demystified');
$description = get_string('footnotedesc', 'theme_demystified');
$setting = new admin_setting_confightmleditor($name, $title, $description, '');
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);

// Custom CSS file.
$name = 'theme_demystified/customcss';
$title = get_string('customcss','theme_demystified');
$description = get_string('customcssdesc', 'theme_demystified');
$setting = new admin_setting_configtextarea($name, $title, $description, '');
$setting->set_updatedcallback('theme_reset_all_caches');
$settings->add($setting);
}
</code>
</div>

===Creating a language file and adding our strings===
As I'm sure none of you have forgotten, throughout the creation of the our settings.php page, we used a lot of strings that I mentioned we would set later on. Well now is the time to set those strings.

First up create the following directories and file for our language strings:
* Directory '''theme/demystified/lang'''
* Directory '''theme/demystified/lang/en'''
* File '''theme/demystified/lang/theme_demystified.php'''

What we have created here is the required structure for Moodle to start looking for language strings.

First Moodle locates the lang directory, once found it looks within that directory for another directory that uses the character code for the language the user has selected, by default this is '''en''' for English. Once that is found it looks for the appropriate language file, in this case '''theme_demystified.php''' from which it will load all language strings for our theme.

If English isn't your chosen language simply replace the ''en'' directory with one that uses your chosen languages character code (two letters).

We can now add our language strings to '''theme/demystified/lang/theme_demystified.php'''. Copy and paste the following lines of PHP into this file.
<code php>
<?php

/**
* This file contains the strings used by the demystified theme
*/

$string['backgroundcolor'] = 'Background colour';
$string['backgroundcolordesc'] = 'This sets the background colour for the theme.';
$string['configtitle'] = 'Demystified theme';
$string['customcss'] = 'Custom CSS';
$string['customcssdesc'] = 'Any CSS you enter here will be added to every page allowing your to easily customise this theme.';
$string['footnote'] = 'Footnote';
$string['footnotedesc'] = 'The content from this textarea will be displayed in the footer of every page.';
$string['logo'] = 'Logo';
$string['logodesc'] = 'Enter the URL to an image to use as the logo for this site. Should be http://www.yoursite.com/path/to/logo.png';
$string['pluginname'] = 'Demystified';
$string['regionwidth'] = 'Column width';
$string['regionwidthdesc'] = 'This sets the width of the two block regions that form the left and right columns.';
</code>

In the above lines of code I have added an entry for each language string we used within ''settings.php''. When adding language strings like this make sure you use single quotes and try to keep things alphabetical - it helps greatly when managing strings.

Now when we view the settings page there will not be any errors or strings missing.

===Having a look at what we have created===
Now that we have created our settings page (settings.php) and added all of the language strings it is time to have a look at things in your browser.

Open your browser and enter the URL to your site. When you arrive at your site login as an administrator.

If you are not redirected to view the new settings change your URL to <nowiki>http://www.yoursite.com/admin/</nowiki> and your will see a screen to set the new theme settings we have just created. This lets us know that everything has worked correctly.

At any point now you are able to log in as administrator and within the settings block browse to '''Site administration > Appearance > Themes > Demystified theme''' to change those settings.

The screenshot below shows you what you should see at this point:

[[Image:Theme.settings.page.03.png|715px|thumb|left|The settings page we just created]]
<br style="clear:both;" />

==Using the settings in CSS==
With the settings page now created and operational it is time to make use of our new settings. The settings that we want to use within our CSS is as follows:

; backgroundcolor : Will be used to set the background colour in CSS.
; regionwidth : Will be the width of the column for CSS.
; customcss : Will be some custom CSS to add to our stylesheet.

At this point those names are the names we used for our setting with the slash and everything before it having been removed.

Before we start tearing into some code it is important that we have a look at what we are going to do and how we are going to go about it.

===How it all works within Moodle===
The first thing to understand is that while Moodle allows you to create a settings page and automates it inclusion and management right into the administration interfaces there is no smart equivalent for using the settings. This is simply because there is no way to predict how people will want to use the settings.

However don't think of this as a disadvantage, in fact it is quite the contrary. Although we can't just ''use'' our settings we can take full control over how and where we use them. It means it will take a little more code but in the end that will work to our advantage as we can do anything we want.

Moodle does help us out a little but not in an obvious way. The first thing that Moodle does is look for a config variable '''csspostprocess''' that should be the name of a function which we want called to make any changes to the CSS after it has been prepared.

The second thing Moodle does is include a lib.php from the theme's directory if one exists (also for the themes the current theme extends.) which ensures that as long as we write our code within '''theme/demystified/lib.php''' it will be included and ready to be used.

The third and final thing Moodle does that will help us out here is ensure that by the time any of code is ready to execute the settings have been prepared and are ready to be used within a theme config object which is passed into our ''csspostprocess'' function.

===Our plan===
As you have already probably guessed we will need to create a function to make the changes to the CSS that we want. We will then set the theme config option '''$THEME->csspostprocess''' to the name of our function.

By doing this when Moodle builds the CSS file it will call our function afterwards with the CSS and the theme object that contains our setting.

Now we know that we will use the ''csspostprocess'' function but how are we going to change the CSS, we could get the function to add CSS, or we could get the function to replace something within the CSS. My personal preference is to replace something within the CSS, just like what is happening with images. If you want to use an image within CSS you would write <nowiki>[[pix:theme|imagename]]</nowiki>.

For settings I am going to use <nowiki>[[setting:settingname]]</nowiki>, this way it looks a bit like something you are already familiar with.

What we need to decide upon next is the best way in which to replace our settings tag with the settings that the user has set.

There are two immediate options available to us:
# Make the ''csspostprocess'' function do all the work.
# Make the ''csspostprocess'' function call a separate function for each setting.
Solution 1 might sound like the simplest however it is going to result in a '''VERY''' complex function. Remember the user might have left settings blank or entered something that wouldn't be valid so we would need to make the sure there is some validation and a good default.
Because of this I think that solution 2 is the better solution.

So we are going to need a ''csspostprocess'' function and then a function for each of the three settings we have that will do the replacements.
<code php>
// This is our css post process function
function demystified_process_css($css, $theme) {};
// This replaces [[setting:backgroundcolor]] with the background colour
function demystified_set_backgroundcolor($css, $backgroundcolor) {};
// This replaces [[setting:regionwidth]] with the correct region width
function demystified_set_regionwidth() {$css, $regionwidth};
// This replaces [[setting:customcss]] with the custom css
function demystified_set_customcss() {$css, $customcss};
</code>

What you should note about the above functions is that they all start with the theme's name. This is required to ensure that the functions are named uniquely as it is VERY unlikely that someone has already created these functions.

So with our plan set out lets start writing the code.

===Writing the code===
The very first thing that we need to do is create a lib.php for our theme into which our css processing functions are going to go. So please at this point create '''theme/demystified/lib.php''' and open it in your editor ready to go.

The first bit of code we have to write is the function that will be called by Moodle to do the processing '''demystified_process_css'''.

Before we start out please remember that the wonderful thing about coding is that there is any number of solutions to a problem. The solutions that you are seeing here in this tutorial are solutions that I have come up with to meet fulfil the needs of the tutorial without being so complex that they are hard to understand. This probably isn't how I would go about it normally but this is a little easier to understand for those who aren't overly familiar with PHP and object orientation.

====The function: demystified_process_css====

<code php>
function demystified_process_css($css, $theme) {

if (!empty($theme->settings->backgroundcolor)) {
$backgroundcolor = $theme->settings->backgroundcolor;
} else {
$backgroundcolor = null;
}
$css = demystified_set_backgroundcolor($css, $backgroundcolor);

if (!empty($theme->settings->regionwidth)) {
$regionwidth = $theme->settings->regionwidth;
} else {
$regionwidth = null;
}
$css = demystified_set_regionwidth($css, $regionwidth);

if (!empty($theme->settings->customcss)) {
$customcss = $theme->settings->customcss;
} else {
$customcss = null;
}
$css = demystified_set_customcss($css, $customcss);

return $css;
}
</code>

So lets look at the things that make up this function:

<code php>
function demystified_process_css($css, $theme) {
//.....
return $css
}
</code>

This of course is the function declaration.

The function gets given two variables, the first '''$css''' is a pile of CSS as one big string, and the second is the theme object '''$theme''' that contains all of the configuration, options, and settings for our theme.

It then returns the ''$css'' variable, essentially returning the modified CSS.

<code php>
//...
if (!empty($theme->settings->backgroundcolor)) {
$backgroundcolor = $theme->settings->backgroundcolor;
} else {
$backgroundcolor = null;
}
$css = demystified_set_backgroundcolor($css, $backgroundcolor);
//...
</code>

Here we are processing our first setting ''backgroundcolor''.

The first thing that we need to do is check whether it has been set and whether it has a value. If it has then we store that value in '''$backgroundcolor'''. It is doesn't have a value then we set ''$backgroundcolor'' to null. This ensures that ''$backgroundcolor'' is set because if it isn't then you are going to get a notice (if you have debugging on).

The final line of this block calls the function '''demystified_set_backgroundcolor'''. We haven't written this function yet but we will shortly. When we call it we give it the ''$css'' variable that contains all of the CSS and we give it the background colour variable ''$backgroundcolor''. Once this function is finished it returns the ''$css'' variable with all of the changes made much like how our css processing function works.

What you should also note about this code is this:
<code php>
$theme->settings->backgroundcolor
</code>
As mentioned earlier ''$theme'' is an object that contains all of the configuration and settings for our theme. The ''$theme'' object has a $settings property which contains all of the settings for our theme, and finally the settings property contains a variable backgroundcolor that is the value the user entered for that setting. That is how we get a settings value.

<code php>
if (!empty($theme->settings->regionwidth)) {
$regionwidth = $theme->settings->regionwidth;
} else {
$regionwidth = null;
}
$css = demystified_set_regionwidth($css, $regionwidth);

if (!empty($theme->settings->customcss)) {
$customcss = $theme->settings->customcss;
} else {
$customcss = null;
}
$css = demystified_set_customcss($css, $customcss);
</code>

These two routines are nearly identical to the routine above. For both the regionwidth and the customcss we make sure it has a value and then store it in a variable. We then call the relevant function to make the changes for that setting.

Now that we have the general processing function it is time to write the three functions we have used but not written, ''demystified_set_backgroundcolor'', ''demystified_set_regionwidth'', ''demystified_set_customcss''.

====The function: demystified_set_backgroundcolor====

First up demystified_set_backgroundcolor.

<code php>
/**
* Sets the background colour variable in CSS
*
* @param string $css
* @param mixed $backgroundcolor
* @return string
*/
function demystified_set_backgroundcolor($css, $backgroundcolor) {
$tag = '[[setting:backgroundcolor]]';
$replacement = $backgroundcolor;
if (is_null($replacement)) {
$replacement = '#DDDDDD';
}
$css = str_replace($tag, $replacement, $css);
return $css;
}
</code>

Ok so what is happening here?

First we need a variable '''$tag''' that contains the tag we are going to replace. As mentioned earlier we are going to use tags that look like the image tags you are already familiar with ''<nowiki>[[setting:settingname]]</nowiki>'', in this case ''<nowiki>[[setting:backgroundcolor]]</nowiki>''.

Next I am going to create a variable called '''$replacement''' into which I put '''$backgroundcolor'''.

The '''IF''' statement that comes next checks ''$replacement'' to make sure it is not null. If it is then we need to set it to a default value. In this case I have used ''#DDD'' as that was the default for the settings.

The line after the IF statement puts it all together. The ''str_replace'' function that we are calling takes three arguments in this order:
# The text to search for.
# The text to replace it with.
# The text to do the replacement in.
It then returns the text with all of the replacements made. So in this case we are replacing the tag with the background colour and it is returning the changed CSS.

The final thing is to return the ''$css'' variable which now contains the correct background colour.

====The function: demystified_set_regionwidth====

Next we have the demystified_set_regionwidth function.
<code php>
/**
* Sets the region width variable in CSS
*
* @param string $css
* @param mixed $regionwidth
* @return string
*/
function demystified_set_regionwidth($css, $regionwidth) {
$tag = '[[setting:regionwidth]]';
$doubletag = '[[setting:regionwidthdouble]]';
$replacement = $regionwidth;
if (is_null($replacement)) {
$replacement = 200;
}
$css = str_replace($tag, $replacement.'px', $css);
$css = str_replace($doubletag, ($replacement*2).'px', $css);
return $css;
}
</code>

This function is very similar to the above function however there is one key thing we are doing different. We are doing two replacements.

# The first replacement is for the width that the user selected. In this case I am replacing the tag ''<nowiki>[[setting:regionwidth]]</nowiki>'' with the width.
# The second replacement is for the width x 2. This is because the page layout requires that the width be doubled for some of the CSS. Here I will replace ''<nowiki>[[setting:regionwidthdouble]]</nowiki>'' with the doubled width.

Remember because it is still just a number we need to add '''px''' to the end of each before we do the replacement.

So the overall process of this function is:
# Define the two tags as '''$tag''' and '''$doubletag'''.
# Make '''$replacement''' the region width ''$regionwidth''.
# Set ''$replacement'' to a default value of 200 is it is null.
# Replace '''<nowiki>[[setting:regionwidth]]</nowiki>''' with the width.
# Replace '''<nowiki>[[setting:regionwidthdouble]]</nowiki>''' with the width x 2.
# Return the changed CSS.

====The function: demystified_set_customcss====

The final function that we need to write is the demystified_set_customcss function.
<code php>
/**
* Sets the custom css variable in CSS
*
* @param string $css
* @param mixed $customcss
* @return string
*/
function demystified_set_customcss($css, $customcss) {
$tag = '[[setting:customcss]]';
$replacement = $customcss;
if (is_null($replacement)) {
$replacement = '';
}
$css = str_replace($tag, $replacement, $css);
return $css;
}
</code>

This function is just like the first function. I'm going to let you work it out on your own.

And that is it no more PHP... Hallelujah I can hear you yelling. The final thing we need to do is tell our theme about the functions we have written and put the settings tags into the CSS.

===Finishing it all off===

So there are two things we have to do in order to complete this section and have our the settings page implemented and our settings being used.

First we need to tell our theme that we want to use the function ''demystified_process_css'' as the ''csspostprocess'' function. This is done very simply by adding the following line of PHP to the bottom of our theme's config.php file '''theme/demystified/config.php'''.

<code php>
$THEME->csspostprocess = 'demystified_process_css';
</code>

With that done the only thing left is to add the settings tag into the CSS. Remember those settings tags are:

; <nowiki>[[setting:backgroundcolor]]</nowiki> : We need to add this where ever we want the background colour setting to be used.
; <nowiki>[[setting:regionwidth]]</nowiki> : We need to add this where ever we want to set the width of the block regions.
; <nowiki>[[setting:regionwidthdouble]]</nowiki> : We need to add this where ever we want to set the doubled width of the block regions.
; <nowiki>[[setting:customcss]]</nowiki> : We need to add this to the bottom of the CSS file that we want the custom CSS added to.

So lets make those changes in CSS now, open up your ''core.css'' file and replace the CSS with the CSS below:
<code css>
/** Background color is a setting **/
html {background-color:[[setting:backgroundcolor]];}
body {margin:30px;padding:0;border:1px solid #333;border-width:0 10px 0 10px;background-color:#333;}
body #page {background-color:#FFF;position:relative;top:-10px;}
.block .header {background-image:none;background-color:#0C5CAC;border:1px solid #0C5CAC;color:#FFF;}
.block {border-color:#4BA7FF;background-color:#DDEEFF;}
.block .content {background-color:#F1F8FF;}
a:link,
a:visited {color:#0C5CAC;}
a:hover {color:#C77500;}
#page #page-header {background-color:#0C5CAC;margin:0;padding:0;width:100%;color:#fff;}
#page #page-header a:link, #page #page-header a:visited {color:#FFAC02}
#page #page-header .navbar, #page #page-header .navbar a:link, #page #page-header .navbar a:visited {color:#0C5CAC;}
/** Override the region width **/
#page-content #region-main-box {left:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidthdouble]];}
#page-content #region-main-box #region-post-box #region-pre {width:[[setting:regionwidth]];left:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box #region-post {width:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidthdouble]];}
.side-pre-only #page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidth]];}
.side-pre-only #page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidth]];}
.side-post-only #page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidth]];}
.side-post-only #page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidth]];}
/** Custom CSS **/
[[setting:customcss]]
</code>

You will notice that <nowiki>[[setting:backgroundcolor]]</nowiki> has been used for the html tags background colour:
<code css>
html {background-color:[[setting:backgroundcolor]];}
</code>

We have also set the width of the block regions by adding <nowiki>[[setting:regionwidth]]</nowiki> as the width for region-pre and region-post as shown below:
<code css>
#page-content #region-main-box {left:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidthdouble]];}
#page-content #region-main-box #region-post-box #region-pre {width:[[setting:regionwidth]];left:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box #region-post {width:[[setting:regionwidth]];}
#page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidthdouble]];}
.side-pre-only #page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidth]];}
.side-pre-only #page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidth]];}
.side-post-only #page-content #region-main-box #region-post-box {margin-left:-[[setting:regionwidth]];}
.side-post-only #page-content #region-main-box #region-post-box #region-main-wrap #region-main {margin-left:[[setting:regionwidth]];}
</code>
You'll notice here that we have to set several different widths and margins using the regionwidth setting and make use of the special regionwidthdouble setting that we added.

The final thing that we did was add the <nowiki>[[setting:customcss]]</nowiki> to the bottom of the file to ensure that the custom CSS comes last (and therefore can override all other CSS).

And with that we are finished. The screenshot below shows how this now looks in the browser if I set the background colour setting to '''<span style='color:#FFA800;'>#FFA800</span>''' and made the column width 240px;

[[Image:Theme.settings.page.04.png|715px|thumb|left|Our settings in action]]<br style="clear:both;" />

==Using the settings within our layout files==
Now that we have utilised the first three settings within our theme's CSS file it is time to implement the other two settings within the layout files so that they are written directly into the page.

You'll be glad to know this is no where near as difficult as utilising settings within a CSS file although it still does require a little bit of PHP.

First up is the logo setting. Into this setting the user is able to enter the URL to an image to use as the logo for the site. In my case I want this to be just a background logo on top of which I want to position the page header.

Before I start there is one thing I need to do however and that is create a default logo background that gets shown if the user hasn't set a specific logo file. To do this I simply created an image '''logo.jpg''' and put it into a pix directory within the demystified theme. You should end up with '''theme/demystified/pix/logo.jpg'''.

Next open up the front-page layout file ''theme/demystified/layout/frontpage.php''. At the top of the file is the PHP that checks what blocks regions the page has and a bit of other stuff. Well right below the existing bit of PHP we want to add the following code:
<code php>
if (!empty($PAGE->theme->settings->logo)) {
//$logourl = $PAGE->theme->settings->logo; // not working give no link to the picture. give the name of the picture. /logo.jpg
$logourl = $PAGE->theme->setting_file_url('logo', 'logo'); // tested and works fine
} else {
$logourl = $OUTPUT->pix_url('logo', 'theme');
}
</code>
What we are doing here is creating a variable called $logourl that will contain the URL to a logo file that was either entered by the user or if they left it blank is that of our default logo file.

There are two things that you should notice about this. First the logo setting can be retrieved through '''$PAGE->theme->settings->logo''' and second we get the default logo url by calling '''$OUTPUT->pix_url('logo', 'theme')'''.

Now that we have the logo URL we are going to use we need to add an image to the header section of the page as shown below:
<code php>
<div id="page-header" class="clearfix">
<img class="sitelogo" src="<?php echo $logourl;?>" alt="Custom logo here" />
<h1 class="headermain"><?php echo $PAGE->heading ?></h1>
....
</code>

If you save that and browse to your sites front page you will notice that the logo file is now being shown. Hooray. However it is probably not styled too nicely so lets quickly fix that. Open up the core.css file and add the following lines of CSS to the bottom of the file.
<code css>
#page-header {position:relative;min-height:100px;}
#page-header .sitelogo {float:left;}
#page-header .headermain {position:absolute;left:0.5em;top:50px;margin:0;float:none;font-size:40px;}
</code>
These three lines position the image correctly and if you now refresh everything should appear perfectly.

With the logo done the last setting we need to deal with is the footnote setting. The idea with this setting was that the administrator could enter some text into the editor and it would be displayed in the footer of the page.

This is probably the easiest setting to implement.

Within the front page layout file that we edited above add the following lines below those we added previously.

<code php>
if (!empty($PAGE->theme->settings->footnote)) {
$footnote = $PAGE->theme->settings->footnote;
} else {
$footnote = '';
}
</code>

Here we are just collecting the footnote into a variable '''$footnote''' and setting a default footnote comment if the user hasn't entered one.

We can now echo the $footnote variable within the page footer. This can be done as shown below.

<code php>

<div id="page-footer">
<div class="footnote"><?php echo $footnote; ?></div>
<p class="helplink">
<?php echo page_doc_link(get_string('moodledocslink')) ?>
</p>
</code>

And with that done we are finished! Congratulations if you got this far.

The screenshot below shows the demystified theme we have just created that is styled by the settings page.

[[Image:Theme.settings.page.05.png|715px|thumb|left|My finished demystified theme]]<br style="clear:both" />

==Testing our creation==
Congratulations to you! If you've made it this far you have done very well. Now it is time to have a look at what we have created and give it a quick test run.

So in this tutorial we have achieved the following:

* We created a theme called demystified that is based on the standard theme.
* We added a settings page to our new theme.
* We added the following settings to our settings page:
** We can set a background colour through the admin interface.
** We can change the logo of the site.
** We can change the block region width.
** We can add a footnote to the page footer
** We can add some custom CSS to seal the deal.
* Those settings were then used in the CSS files for our theme.
* They were also used in the layout files for our theme.
* And here we are testing it all.

The screenshots below show my testing process and I change each setting and view the outcome. The great thing about this is that with theme designer mode on you see the changes as soon as the form refreshes.

[[Image:Theme.settings.page.06.png|300px|thumb|left|Default settings]]
[[Image:Theme.settings.page.07.png|300px|thumb|left|Changed the background colour setting]]
[[Image:Theme.settings.page.08.png|300px|thumb|left|Changed the logo and region width]]
[[Image:Theme.settings.page.09.png|300px|thumb|left|Added a footnote]]
[[Image:Theme.settings.page.10.png|300px|thumb|left|Added some custom CSS]]
<br style="clear:both" />

==The different settings you can use==
During this tutorial we use four different kinds of settings, a text box, text area, a select (dropdown) and the editor however as I'm sure you have all guessed there is many more some of which will certainly be useful for theme settings.

The following are examples of some of the different settings. I should add that these build upon what we have already done within the tutorial but are not included in the download.

===Colour picker===
[[Image:Theme.settings.page.11.png|400px|thumb|The colour picker]]
I can hear you all asking now 'Why didn't you mention this one earlier?' well the answer is simple it didn't exist when I first wrote the tutorial. It is an admin setting that I wrote several days after because of the fantastic effort people were putting into trying out settings pages.

A bit about the colour picker. First up it is a text box that when the page loads turns into a colour picker that can be used to select a colour and can even preview the selected colour in the page (by clicking the preview button). It is designed to be very easy to use and the code is just about as simple as creating a normal text box setting.

In the image to the left I have replaced the background colour setting we created during the tutorial with the colour picker. Lets have a look at the code involved for that.
<code php>
// Background colour setting
$name = 'theme_demystified/backgroundcolor';
$title = get_string('backgroundcolor','theme_demystified');
$description = get_string('backgroundcolordesc', 'theme_demystified');
$default = '#DDD';
$previewconfig = array('selector'=>'html', 'style'=>'backgroundColor');
$setting = new admin_setting_configcolourpicker($name, $title, $description, $default, $previewconfig);
$temp->add($setting);
</code>
So first thing you should notice is that the first four lines of code have not changed at all!

The first change we have is to create a variable '''$previewconfig'''.
This variable is used to tell the colour picker what to do when the user clicks the preview button. It should be an array that contains two things, first a selector, and second a style.
# The selector is a CSS selector like ''.page .header h2''.
# The style is a what should change, there are two immediate values it can be, either '''backgroundColor''' or '''color'''.
In my case the selector is '''html''' to target the html tag and the style is backgroundColor to change the background colour.

The second change is to use '''admin_setting_configcolourpicker''' instead of ''admin_setting_configtext''. The arguments are nearly identical as well except that there is an extra argument to which we give the '''$previewconfig''' variable.

And that is it! it is all you need to get the colour picker up and running.

'''Extra note:''' The $previewconfig variable is optional. If you don't want a preview button the simply set ''$previewconfig = null''

==Common pitfalls and useful notes==

This section is really just a collection of pointers, tips, notes, and other short useful stuff that may be of use to those attempting this tutorial.

# First up and most importantly there are very few limitations to what you achieve in this fashion.
# If you get stuck or need a hand ask in the forums, there's always someone round who can help.
# If you do something really cool let us know, I know everyone in the community loves finding our what others are achieving.
# You don't have to use ''admin_settingpage'' you can also use '''admin_externalpage''' if you prefer. It should be noted however that it is not the preferred way to do it as admin_externalpage's were only left in Moodle 2.0 for backwards compatibility (although they are more flexible one day they will be deprecated or removed). Thank you to Darryl for raising this.
# If you find that your language strings aren't being used (you are getting language string notices) and you have double checked that you have turned '''langstringcache''' off then you may need to delete the language cache directory. This is located in the moodledata directory for you installation: moodledata/cache/lang/*. You should delete the directory for your chosen language by default ''en''.

==See also==

* [[Adding theme upgrade code]]
* [[Styling and customising the dock]]
* Using Moodle [http://moodle.org/mod/forum/discuss.php?d=152053 Theme 2.0: Adding a settings page to your theme] forum discussion

Jenkins Integration Testing for Plugin Development

2015-05-06T12:41:18Z

Jan.dagefoerde: Suddenly (as of M2.9), a Moodle dependency seems to require the intl extension as well.

<p>
This article describes how to use a Jenkins continuous integration server for automating PHPUnit and Behat tests for Moodle and Moodle plugins. The configuration uses Moodle's way of running PHPUnit and Behat tests using the command line.<br>
</p>

<p>
Integration scenario: We are developing a Moodle plugin. We want to run automated tests, ensuring that it runs correctly on multiple Moodle platform versions, e.g., Moodle 2.7 and Moodle 2.8. This is motivated by the fact that we announce compatibility with these versions in the Moodle plugin directory, and we want to - at least - know, when a plugin becomes incompatible with an older version or needs to be changed. Since we are only running one recent Moodle version for development, automated tests with other versions are useful for us.<br>
</p>

<p>
Prerequisites for this manual: Ubuntu-based integration server, headless (i.e. without a graphical environment), here: Ubuntu 14.04. <br>
</p>

== Installation of required software ==
<p>
<ul>
<li>Install Jenkins as a Ubuntu native package, as described on their website (instructions currently at http://pkg.jenkins-ci.org/debian/).</li>
<li>Install a Moodle runtime environment. A web server is not needed, since PHP 5's standalone server will be used. The following commands were used for our system:</li>
</ul>
</p>
<div style='padding-left: 30pt'>
<code>sudo apt-get install postgresql<br />
sudo apt-get install php5-cli php-pear php5-curl php5-pgsql php5-gd php5-intl</code>
</div>

<p>
<ul>
<li>Install git, which will be used to obtain the sources of Moodle and of your plugin.</li>
</ul>
<div style='padding-left: 30pt'>
<code>sudo apt-get install git</code><br>
</div>
<ul>
<li>For acceptance tests, you need to use a browser. Therefore you need to be able to run graphical applications. On headless servers (i.e. those without a display), you can install Xvfb, emulating a display which will be used by your browser.</li>
</ul>
<div style='padding-left: 30pt'>
<code>sudo apt-get install xvfb firefox</code><br>
</div>
<ul>
<li>Furthermore, acceptance tests require the Selenium standalone server to be installed. Download from http://www.seleniumhq.org/download/ (here: Version 2.45.0) and unpack it at a jenkins-accessible location, e.g. $JENKINS_HOME/selenium (here: /var/lib/jenkins/selenium).</li>
</ul>
</p>

== Basic configuration of Jenkins using the shell ==
<p>
<ul>
<li>Configuration of the init script can be found at <code>/etc/default/jenkins</code>. It is not strictly necessary to change anything here, but we set <code>HTTP_PORT=-1</code> and added <code>HTTPS_PORT=8998</code> to make the server only via HTTPS. In that case, you also need to add <code>--httpsPort=$HTTPS_PORT</code> to the <code>JENKINS_ARGS</code> variable and restart Jenkins.</li>
<li>Furthermore, Moodle requires a <code>config.php</code> which is not contained in a repository. Therefore, configuration files need to be stored at a separate location, from where they can be pulled in. For our scenario, we stored the configuration file at <code>$JENKINS_HOME/moodle-config/config_$target.php</code>, where $target indicates a Moodle branch to build against (e.g., /var/lib/jenkins/moodle-config/config_MOODLE_28_STABLE.php).</li>
</ul>
</p>

== Jenkins server configuration ==
<p>
For all following steps you need to open Jenkins in your browser.<br>
</p>

<p>
Install or update the following plugins:<br>
<ul>
<li>Xvfb plugin</li>
<li>Multiple SCMs plugin</li>
<li>Matrix project plugin</li>
<li>JUnit plugin</li>
<li>GitHub plugin</li>
<li>Job Configuration History Plugin (optional, but very useful)</li>
</ul>
</p>

<p>
Configure Jenkins in the following way (Manage Jenkins → Configure System):<br>
<ul>
<li>Git</li>
<ul>
<li>Add a git installation. Choose an arbitrary name and set the path to <code>git</code> </li>
</ul>
<li>Git plugin</li>
<ul>
<li>Set the fields <code>Global Config user.name Value</code> and <code>Global Config user.email Value</code>. In our case, we set those values to account and e-mail of a technical GitHub account that is only used by the Jenkins server</li>
</ul>
</ul>
</p>

<p>
Create a job for your integration scenario:<br>
<ul>
<li>On the dashboard, select New Item. </li>
<li>Choose Multi-configuration project and define an Item name without any spaces.</li>
<li>On the next page, define the following settings (Change to your requirements. Remember our integration scenario described above!):</li>
<ul>
<li>GitHub project: URL to the web page of your repository</li>
<li>Source Code management:</li>
<ul>
<li>Choose <code>Multiple SCMs</code>.</li>
<li>Use <code>Add SCM</code> to add two git repositories.</li>
<li>For the first repository,</li>
<ul>
<li>set the URL to <code>git://git.moodle.org/moodle.git</code>.</li>
<li>Since this is a public repository, credentials are not required.</li>
<li>Under Additional Behaviours, add <code>Advanced clone behaviours</code> and check the option <code>Shallow clone</code>.</li>
</ul>
<li>For the second repository,</li>
<ul>
<li>set the URL to the URL of your git repository.</li>
<li>If this is a private repository, add credentials for accessing it (consider SSH authentication!)</li>
<li>Under Additional Behaviours, add <code>Check out to a sub-directory</code> and, for <code>Local subdirectory for repo</code>, enter the path where you would deploy your module in a usual Moodle installation (e.g., <code>mod/yourmodule</code>), without surrounding slashes.</li>
<li>Under Additional Behaviours, add <code>Advanced clone behaviours</code> and check the option <code>Shallow clone</code>.</li>
</ul>
</ul>
<li>Build Triggers:</li>
<ul>
<li>Check Build when a change is pushed to GitHub.</li>
<li>Check Poll SCM. As a schedule, define e.g. <code>H/5 * * * *</code> for polling every five minutes.</li>
</ul>
<li>Configuration Matrix:</li>
<ul>
<li>Add two user-defined axes.</li>
<ul>
<li>For the first axis, set name to <code>target</code> and values to <code>MOODLE_28_STABLE MOODLE_27_STABLE</code>. These values correspond to the branch names used in the Moodle repository.</li>
<li>For the second axis, set name to <code>testtype</code> and values to <code>UnitTest BehatTest</code>. These are used later on to perform different test types - be patient :).</li>
</ul>
<li>Check Run each configuration sequentially.</li>
</ul>
<li>Build Environment:</li>
<ul>
<li>Check Start Xvfb before the build, and shut it down after.</li>
</ul>
<li>Build:</li>
<ul>
<li>Add Execute shell</li>
<ul>
<li>Enter the following commands (Note that we use <code>$target</code> here, which always holds the current value of <code>target</code> in a particular configuration. Furthermore, this copies an appropriate config.php file into the current workspace):</li>
</ul>
</ul>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>git checkout $target;<br />
curl -s https://getcomposer.org/installer | php;<br />
php composer.phar install --prefer-source;<br />
cp $JENKINS_HOME/moodle-config/config_$target.php config.php;</code>
</div>

<p>
<ul style='padding-left: 60pt'>
<li>Add Conditional step (single)</li>
<ul>
<li>Run?: <code>Strings match</code>, String 1: <code>${testtype}</code>, String 2: <code>UnitTest</code> (the motivation is that this command is only run for unit tests. In contrast, the other step will be for behat acceptance tests).</li>
<li>Builder: <code>Execute shell</code></li>
<li>Enter the following commands:</li>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>mkdir -p ../moodledata/phpunit;<br />
php admin/tool/phpunit/cli/init.php;<br />
vendor/bin/phpunit --log-junit results/phpunit/phpunit.xml --group mod_ratingallocate || true;</code>
</div>

<p>
<ul style='padding-left: 60pt'>
<li>Add Conditional step (single)</li>
<ul>
<li>Run?: <code>Strings match</code>, String 1: <code>${testtype}</code>, String 2: <code>BehatTest</code>.</li>
<li>Builder: <code>Execute shell</code></li>
<li>Enter the following commands (replace <code>[SeleniumVersion]</code>, <code>[ModuleTag]</code>, and <code>[JobName]</code> accordingly):</li>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>mkdir -p ../moodledata/behat;<br />
php admin/tool/behat/cli/init.php;<br />
java -jar /var/lib/jenkins/selenium/selenium-server-standalone-[SeleniumVersion].jar > /dev/null 2>&1 &<br />
SELENIUM_PID=$!<br />
php -S localhost:8000 > /dev/null 2>&1 &<br />
PHP_PID=$!<br />
vendor/bin/behat --config /var/lib/jenkins/jobs/[JobName]/workspace/target/moodledata/behat/behat/behat.yml --tags '@[ModuleTag]' --format moodle_progress,junit --out ,behatlog || true;<br />
kill $SELENIUM_PID<br />
kill $PHP_PID</code>
</div>

<p>
<ul style='padding-left: 30pt'>
<li>Post-build Actions:</li>
<ul>
<li>Add Publish JUnit test result report</li>
<ul>
<li>Change the pattern for <code>Test report XMLs</code> to the value <code>**/phpunit/phpunit.xml,**/behatlog/*.xml</code></li>
</ul>
</ul>
<li>Save the configuration and start the first build to test it.</li>
</ul>
</p>

[[Category:Developer tools]]

Jenkins Integration Testing for Plugin Development

2015-05-05T10:19:15Z

Jan.dagefoerde: Created page with "<p> This article describes how to use a Jenkins continuous integration server for automating PHPUnit and Behat tests for Moodle and Moodle plugins. The configuration uses Mood..."

<p>
This article describes how to use a Jenkins continuous integration server for automating PHPUnit and Behat tests for Moodle and Moodle plugins. The configuration uses Moodle's way of running PHPUnit and Behat tests using the command line.<br>
</p>

<p>
Integration scenario: We are developing a Moodle plugin. We want to run automated tests, ensuring that it runs correctly on multiple Moodle platform versions, e.g., Moodle 2.7 and Moodle 2.8. This is motivated by the fact that we announce compatibility with these versions in the Moodle plugin directory, and we want to - at least - know, when a plugin becomes incompatible with an older version or needs to be changed. Since we are only running one recent Moodle version for development, automated tests with other versions are useful for us.<br>
</p>

<p>
Prerequisites for this manual: Ubuntu-based integration server, headless (i.e. without a graphical environment), here: Ubuntu 14.04. <br>
</p>

== Installation of required software ==
<p>
<ul>
<li>Install Jenkins as a Ubuntu native package, as described on their website (instructions currently at http://pkg.jenkins-ci.org/debian/).</li>
<li>Install a Moodle runtime environment. A web server is not needed, since PHP 5's standalone server will be used. The following commands were used for our system:</li>
</ul>
</p>
<div style='padding-left: 30pt'>
<code>sudo apt-get install postgresql<br />
sudo apt-get install php5-cli php-pear php5-curl php5-pgsql php5-gd</code>
</div>

<p>
<ul>
<li>Install git, which will be used to obtain the sources of Moodle and of your plugin.</li>
</ul>
<div style='padding-left: 30pt'>
<code>sudo apt-get install git</code><br>
</div>
<ul>
<li>For acceptance tests, you need to use a browser. Therefore you need to be able to run graphical applications. On headless servers (i.e. those without a display), you can install Xvfb, emulating a display which will be used by your browser.</li>
</ul>
<div style='padding-left: 30pt'>
<code>sudo apt-get install xvfb firefox</code><br>
</div>
<ul>
<li>Furthermore, acceptance tests require the Selenium standalone server to be installed. Download from http://www.seleniumhq.org/download/ (here: Version 2.45.0) and unpack it at a jenkins-accessible location, e.g. $JENKINS_HOME/selenium (here: /var/lib/jenkins/selenium).</li>
</ul>
</p>

== Basic configuration of Jenkins using the shell ==
<p>
<ul>
<li>Configuration of the init script can be found at <code>/etc/default/jenkins</code>. It is not strictly necessary to change anything here, but we set <code>HTTP_PORT=-1</code> and added <code>HTTPS_PORT=8998</code> to make the server only via HTTPS. In that case, you also need to add <code>--httpsPort=$HTTPS_PORT</code> to the <code>JENKINS_ARGS</code> variable and restart Jenkins.</li>
<li>Furthermore, Moodle requires a <code>config.php</code> which is not contained in a repository. Therefore, configuration files need to be stored at a separate location, from where they can be pulled in. For our scenario, we stored the configuration file at <code>$JENKINS_HOME/moodle-config/config_$target.php</code>, where $target indicates a Moodle branch to build against (e.g., /var/lib/jenkins/moodle-config/config_MOODLE_28_STABLE.php).</li>
</ul>
</p>

== Jenkins server configuration ==
<p>
For all following steps you need to open Jenkins in your browser.<br>
</p>

<p>
Install or update the following plugins:<br>
<ul>
<li>Xvfb plugin</li>
<li>Multiple SCMs plugin</li>
<li>Matrix project plugin</li>
<li>JUnit plugin</li>
<li>GitHub plugin</li>
<li>Job Configuration History Plugin (optional, but very useful)</li>
</ul>
</p>

<p>
Configure Jenkins in the following way (Manage Jenkins → Configure System):<br>
<ul>
<li>Git</li>
<ul>
<li>Add a git installation. Choose an arbitrary name and set the path to <code>git</code> </li>
</ul>
<li>Git plugin</li>
<ul>
<li>Set the fields <code>Global Config user.name Value</code> and <code>Global Config user.email Value</code>. In our case, we set those values to account and e-mail of a technical GitHub account that is only used by the Jenkins server</li>
</ul>
</ul>
</p>

<p>
Create a job for your integration scenario:<br>
<ul>
<li>On the dashboard, select New Item. </li>
<li>Choose Multi-configuration project and define an Item name without any spaces.</li>
<li>On the next page, define the following settings (Change to your requirements. Remember our integration scenario described above!):</li>
<ul>
<li>GitHub project: URL to the web page of your repository</li>
<li>Source Code management:</li>
<ul>
<li>Choose <code>Multiple SCMs</code>.</li>
<li>Use <code>Add SCM</code> to add two git repositories.</li>
<li>For the first repository,</li>
<ul>
<li>set the URL to <code>git://git.moodle.org/moodle.git</code>.</li>
<li>Since this is a public repository, credentials are not required.</li>
<li>Under Additional Behaviours, add <code>Advanced clone behaviours</code> and check the option <code>Shallow clone</code>.</li>
</ul>
<li>For the second repository,</li>
<ul>
<li>set the URL to the URL of your git repository.</li>
<li>If this is a private repository, add credentials for accessing it (consider SSH authentication!)</li>
<li>Under Additional Behaviours, add <code>Check out to a sub-directory</code> and, for <code>Local subdirectory for repo</code>, enter the path where you would deploy your module in a usual Moodle installation (e.g., <code>mod/yourmodule</code>), without surrounding slashes.</li>
<li>Under Additional Behaviours, add <code>Advanced clone behaviours</code> and check the option <code>Shallow clone</code>.</li>
</ul>
</ul>
<li>Build Triggers:</li>
<ul>
<li>Check Build when a change is pushed to GitHub.</li>
<li>Check Poll SCM. As a schedule, define e.g. <code>H/5 * * * *</code> for polling every five minutes.</li>
</ul>
<li>Configuration Matrix:</li>
<ul>
<li>Add two user-defined axes.</li>
<ul>
<li>For the first axis, set name to <code>target</code> and values to <code>MOODLE_28_STABLE MOODLE_27_STABLE</code>. These values correspond to the branch names used in the Moodle repository.</li>
<li>For the second axis, set name to <code>testtype</code> and values to <code>UnitTest BehatTest</code>. These are used later on to perform different test types - be patient :).</li>
</ul>
<li>Check Run each configuration sequentially.</li>
</ul>
<li>Build Environment:</li>
<ul>
<li>Check Start Xvfb before the build, and shut it down after.</li>
</ul>
<li>Build:</li>
<ul>
<li>Add Execute shell</li>
<ul>
<li>Enter the following commands (Note that we use <code>$target</code> here, which always holds the current value of <code>target</code> in a particular configuration. Furthermore, this copies an appropriate config.php file into the current workspace):</li>
</ul>
</ul>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>git checkout $target;<br />
curl -s https://getcomposer.org/installer | php;<br />
php composer.phar install --prefer-source;<br />
cp $JENKINS_HOME/moodle-config/config_$target.php config.php;</code>
</div>

<p>
<ul style='padding-left: 60pt'>
<li>Add Conditional step (single)</li>
<ul>
<li>Run?: <code>Strings match</code>, String 1: <code>${testtype}</code>, String 2: <code>UnitTest</code> (the motivation is that this command is only run for unit tests. In contrast, the other step will be for behat acceptance tests).</li>
<li>Builder: <code>Execute shell</code></li>
<li>Enter the following commands:</li>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>mkdir -p ../moodledata/phpunit;<br />
php admin/tool/phpunit/cli/init.php;<br />
vendor/bin/phpunit --log-junit results/phpunit/phpunit.xml --group mod_ratingallocate || true;</code>
</div>

<p>
<ul style='padding-left: 60pt'>
<li>Add Conditional step (single)</li>
<ul>
<li>Run?: <code>Strings match</code>, String 1: <code>${testtype}</code>, String 2: <code>BehatTest</code>.</li>
<li>Builder: <code>Execute shell</code></li>
<li>Enter the following commands (replace <code>[SeleniumVersion]</code>, <code>[ModuleTag]</code>, and <code>[JobName]</code> accordingly):</li>
</ul>
</ul>
</p>
<div style='padding-left: 120pt'>
<code>mkdir -p ../moodledata/behat;<br />
php admin/tool/behat/cli/init.php;<br />
java -jar /var/lib/jenkins/selenium/selenium-server-standalone-[SeleniumVersion].jar > /dev/null 2>&1 &<br />
SELENIUM_PID=$!<br />
php -S localhost:8000 > /dev/null 2>&1 &<br />
PHP_PID=$!<br />
vendor/bin/behat --config /var/lib/jenkins/jobs/[JobName]/workspace/target/moodledata/behat/behat/behat.yml --tags '@[ModuleTag]' --format moodle_progress,junit --out ,behatlog || true;<br />
kill $SELENIUM_PID<br />
kill $PHP_PID</code>
</div>

<p>
<ul style='padding-left: 30pt'>
<li>Post-build Actions:</li>
<ul>
<li>Add Publish JUnit test result report</li>
<ul>
<li>Change the pattern for <code>Test report XMLs</code> to the value <code>**/phpunit/phpunit.xml,**/behatlog/*.xml</code></li>
</ul>
</ul>
<li>Save the configuration and start the first build to test it.</li>
</ul>
</p>

[[Category:Developer tools]]

sandbox

2015-05-05T08:03:07Z

Jan.dagefoerde:

For playing around in...
a