MoodleDocs - User contributions [en]

Hooks spec

2025-02-09T22:14:57Z

Brendanheywood:

{{Template:WillNotMigrate}}

= Moodle hooks/features (Abandoned) =

{{Note|This proposal was worked on for several years but no consensus was reached. A decision was made by the integrators to abandon this project as it has consumed too much of the development communities time and resources. The text on this page is kept for historical reasons only. See the comments on MDL-44078 for the full discussion.|gotcha}}

Then a later effort to introduce hooks then DID land in core, and now most callbacks on progressively being converted to hooks:

https://moodledev.io/docs/4.5/apis/core/hooks

If you are looking for the existing callback based API please see:

https://docs.moodle.org/dev/Callbacks

The full discussion is available on MDL-44078 and in forum: https://moodle.org/mod/forum/discuss.php?d=254508 and https://moodle.org/mod/forum/discuss.php?d=327349 but this issue is dead, please lets not keep discussing it.

==Terms used==

Terminology in the discussion around this feature has often been confusing because of the ways that other systems have used them (eg Drupal).

As a guide - the following terms are equivalent and any other interpretation is not allowed:

* '''hook''' === hook point === trigger === caller
* '''callback''' === listener === executor

== Goals ==

# Create a consistent/simple/performant/secure way to create extension points where all plugins can run code (never specific plugin types) and modify data (no restrictions - not tied specifically to events)
# Create a consistent/simple/performant/secure way to call code from another specific plugin
# Allow implementations in an autoloaded class
# Allow implementations in lib.php (only for legacy)
# Allow testing hook implementations even if no plugin in core currently implements a callback
# Enforce organisation and categorisation of all "callers" and "executors" in a plugin/subsystem/core

=== Disregarded Goals ===
This section is only kept to show the history of this issue - these goals were initially listed, but since then nobody requested and they seem like a bad idea (overly complex - no use case).
# Provide an admin UI to control the executed plugins for each hook (order and enable/disable)
# Dropping in a hook implementation from some other large framework/web app that does not suit Moodle
# Blowing this feature up into some full controller for business logic / dependency injection / bloat

== Current solutions ==

* Defining functions in /plugindir/lib.php with the name <pluginname>_<hookname> and querying all plugins if somebody implements function and/or list of plugins implementing it. *This is the current implementation of hooks.* Disadvantage: functions must be defined in lib.php therefore they are always included even when function is not there (performance loss); no UI to configure if only one plugin can implement feature; difficult for developers because they don't know if they can or how they can implement a hook (no separate place for documentation).

* Events system. The core notifies whoever wants to listen to it about something that happens. Disadvantage: one-way communication, no feedback. Advantage: plugins cannot break important core functionality.

* $CFG variables. These are fine if it makes sense to have a setting to choose different implementations of a feature (e.g. lock factories).

* Creating a new plugin type. Disadvantage: very long process, involves lots of documentation, creating lots of strings and UI, may be available in major releases only. Also not applicable for minor features. Forcing a "feature" to be split amongst many plugins.

* Renderers. These can be used to override a lot of standard Moodle behaviour, particularly on the view side. Only can be overridden by themes

== Implementation ==

* Hook definition is a class stored in /classes/hook/ folder (of core component or plugin). They have a function execute() that calls all registered callbacks and passes the actual instance of the hook definition class to them. Some hooks may be called on one component/plugin at a time only.

* Hooks to be executed must be registered in /db/hooks.php so it is possible to determine all the hooks called by a plugin/subsystem.

* Plugins that want to register hook executions (callbacks) do so in /db/hooks.php

* Hook manager, is responsible for caching of the list of existing hooks and their callbacks. It also has an interface with list of all hooks and callbacks.

=== Example ===

Typical example is a hook that allows plugins to add fields to the existing form. Note that actual hook classes may be much more complicated, with lots of methods that can be called from callbacks and also with properties/methods that can be called by the component who executed the hook to access the collected data from plugins.

0. Base class does almost nothing, it's just a suggestion for implementation, methods are not final (unlike events) '''/lib/classes/hook/base.php'''
<pre>
namespace core\hook;
abstract class base {
private $isexecuting = false;
public function execute($component = null) {
if ($this->isexecuting) {
throw new \moodle_exception('Already executing');
}
$this->isexecuting = true;
\core\hook\manager::dispatch($this, $component);
$this->isexecuting = false;
return $this;
}
}
</pre>

1. '''/lib/classes/hook/courseresetform.php'''

<pre>
namespace core\hook;
public class courseresetform extends \core\hook\base {
private $mform;
public function get_mform() {
return $this->mform;
}
public static function create($mform) {
$hook = new self();
$hook->mform = $mform;
return $hook;
}
}
</pre>

2. '''/course/reset_form.php''', function course_reset_form::definition()
at the moment this function looks for function xxx_reset_course_form_definition() in all mod plugins and call this function passing $mform. We can have a legacy hook execution that does this for backward compatibility but it will just call:
<pre>
\core\hook\courseresetform::create($mform)->execute();
</pre>
The hook also must be listed in db/hooks.php
<pre>
// Hooks that exist in Moodle core.
$hooks = array(
'\core\hook\courseresetform', // Allows components/plugins to extend the course reset form.
);
</pre>

3. Now any plugin type and not only module but also blocks (existing issue MDL-24359) can register their callbacks in '''/blocks/myblock/db/hooks.php'''.
<pre>
$callbacks = array(
array(
'hookname' => '\core\hook\courseresetform',
'callback' => array('block_myblock_hook_callbacks', 'courseresetform'),
//'includefile' => '/optional/file/path.php',
),
);
</pre>

4. Actual callback implementation from /blocks/myblock/classes/hook_callbacks.php:
<pre>
class block_myblock_hook_callbacks {
public static function courseresetform(\core\hook\courseresetform $courseresetformhook) {
$courseresetformhook->get_mform()->addElement('checkbox', 'Reset contents of block Myblock');
}
}
</pre>

== Use cases ==

# Navigation: we want to allow any plugin to put own items anywhere
# Recent activity in course
# Global search: similar to above but result will be list of title+url+abstract
# Front page: We have $CFG->customfrontpageinclude for including a file displaying frontpage contents. It could be also hook/feature
# Extending mimetypes list
# Extending licenses list
# Adding fields to the standard forms. This may be a comprehensive feature that includes several functions - adding fields, validation, processing.

== Comparison with Moodle Events ==

Differences with [[Events API]]:
* events are one way communication (from one to many) - hooks are two way communication (usually many to one)
* event data must not be modified - hook callbacks may alter the hook instance data
* events are triggered after actions - hooks can be executed at any useful stage (pre, post, validation, etc.)
* event structure is fixed - hooks can be arbitrary classes that may extend the base
* database transaction may cause event buffering - hook manager does does not consider transactions
* all events are logged (this is contentious) - hooks are not logged

Session locks

2023-06-05T03:15:18Z

Brendanheywood:

== What is a session lock? ==
When you create a normal moodle page and include config.php then by default a large amount of moodle bootstrapping runs and you will have the $SESSION global setup for you. While the php script is executing it holds a session lock and at the end it writes back to the session an unlocks it. This means that only a single script can run at a time for a given user while holding the session lock. In practice most scripts run quite fast and most of the time is spent sending the result back over the network and so you often don't notice this. But if you have a long running page like a report or backup this starts to be noticed by the end user as a pause while the affected page is blocked waiting for another page to finish execution on the server.

When writing higher performance code it is better to reduce or eliminate the session locks where possible.
== Debugging session lock issues ==
If you have 'pages that are slow' and you profile them and see that you are waiting on a lock to free up then this is potentially an easy thing to fix to improve your overall performance.
$CFG->debugsessionlock = 5; // Time in seconds
When a session is locked for more N seconds a debugging call will be made with details of what the other page was which is holding onto the lock. A good strategy is to set this to a value like 30 seconds and then gradually reduce this down to a very small number of seconds as you identify and fix various issues. As a rule of thumb in an ideal world no page should ever lock the session for more than a second.
== Session unlocking ==
By default core assumes that you might need to mutate the $SESSION object so it will hold a lock on the session until the page finished and a shutdown handler will release the session lock.

If you are working on any page which is potentially long running, then you should cleanly separate logic which runs early which could mutate the session from the long running processing code and unlock the session.
\core\session\manager::write_close();
== Read only session in pages ==
For this to work, READONLY sessions must be enabled as well needing your code to support it.

If you know ahead of time that you will never mutate the session, but you still need to be able to read it, then you can declare your page to be read only. This will mean your page will never block the session in another http request.
define('READ_ONLY_SESSION', true);
== Read only sessions in web services ==
The same is possible in web services. When you declare your web service you can specify it will not need a session lock:
'core_message_get_unread_conversations_count' => array(
'classname' => 'core_message_external',
'methodname' => 'get_unread_conversations_count',
'classpath' => 'message/externallib.php',
'description' => 'Retrieve the count of unread conversations for a given user',
'type' => 'read',
'ajax' => true,
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE),
'''<nowiki>'readonlysession' => true, // We don't modify the session.</nowiki>'''
),
== No session at all ==
If your script doesn't actually need $SESSION in the first place then save even more processing and locks by declaring:
define('NO_MOODLE_COOKIES', true);
== No config is needed ==
Going to the absolute extreme, if you do not even need the full moodle bootstrap to run then you can skip it via:
define('ABORT_AFTER_CONFIG', true);
[[ja:セッションロック_(Dev_docs)]]

Session locks

2023-06-05T03:10:57Z

Brendanheywood:

== What is a session lock? ==
When you create a normal moodle page and include config.php then by default a large amount of moodle bootstrapping runs and you will have the $SESSION global setup for you. While the php script is executing it holds a session lock and at the end it writes back to the session an unlocks it. This means that only a single script can run at a time for a given user while holding the session lock. In practice most scripts run quite fast and most of the time is spent sending the result back over the network and so you often don't notice this. But if you have a long running page like a report or backup this starts to be noticed by the end user as a pause while the victim page is blocked waiting for another page to finish execution on the server.

This is a safe starting assumption but when writing higher performance code it is better to reduce or eliminate the session locks where possible.
== Debugging session lock issues ==
If you have 'pages that are slow' and you profile them and see that you are waiting on a lock to free up then this is potentially an easy thing to fix to improve your overall performance.
$CFG->debugsessionlock = 5; // Time in seconds
When a session is locked for more N seconds a debugging call will be made with details of what the other page was which is holding onto the lock.
== Session unlocking ==
By default core assumes that you might need to mutate the $SESSION object so it will hold a lock on the session until the page finished and a shutdown handler will release the session lock.
If you are working on any page which is potentially long running, then you should cleanly separate logic which runs early which could mutate the session from the long running processin code and unlock the session.
\core\session\manager::write_close();
== Read only session in pages ==
For this to work, READONLY sessions must be enabled as well needing your code to support it. Not all session

If you know ahead of time that you will never mutate the session, but you still need to be able to read it, then you can declare your page to be read only. This will mean your page will never block the session in another http request.
define('READ_ONLY_SESSION', true);
== Read only sessions in web services ==
The same is possible in web services. When you declare your web service you can specify it will not need a session lock:
'core_message_get_unread_conversations_count' => array(
'classname' => 'core_message_external',
'methodname' => 'get_unread_conversations_count',
'classpath' => 'message/externallib.php',
'description' => 'Retrieve the count of unread conversations for a given user',
'type' => 'read',
'ajax' => true,
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE),
'''<nowiki>'readonlysession' => true, // We don't modify the session.</nowiki>'''
),
== No session at all ==
If your script doesn't actually need $SESSION in the first place then save even more processing and locks by declaring:
define('NO_MOODLE_COOKIES', true);
== No config is needed ==
Going to the absolute extreme, if you do not even need the full moodle bootstrap to run then you can skip it via:
define('ABORT_AFTER_CONFIG', true);
[[ja:セッションロック_(Dev_docs)]]

Error pages

2023-05-23T04:44:11Z

Brendanheywood: /* 403 in Nginx */

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== 403 in Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex
=== 403 in Nginx ===
<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php?code=404
</syntaxhighlight>If you want to transform all 403 errors into 404's then:<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php?code=404
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.
== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== 404 in Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== 404 in nginx ===
<syntaxhighlight lang="php">
error_page 404 = /error/index.php
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.
== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Cache API

2023-05-10T13:29:45Z

Brendanheywood: /* Setting requirements */

This document details the Cache API aka MUC aka Moodle's Universal Cache.
I've chosen to use a tutorial/example flow for this document always working with a theoretical module plugin called myplugin.
There is also a [[Cache API - Quick reference]] if you would rather read that.
==Basic usage==
It's very easy to get started with the Cache API. It is designed to be as easy and as quick to use as possible and is predominantely self contained.
All you need to do is add a definition for your cache and you are ready to start working with the Cache API.
===Creating a definition===
Cache definitions exist within the db/caches.php file for a component/plugin. 
In the case of core that is the moodle/lib/db/caches.php file, in the case of a module that would be moodle/mod/myplugin/db/caches.php.

The definition is used API in order to understand a little about the cache and what it is being used for, it also allows the administrator to set things up especially for the definition if they want.
From a development point of view the definition allows you to tell the API about your cache, what it requires, and any (if any) advanced features you want it to have. 
The following shows a basic definition containing just the bare minimum:
<syntaxhighlight lang="php">
// moodle/mod/myplugin/db/caches.php
$definitions = [
'somedata' => [
'mode' => cache_store::MODE_APPLICATION,
]
];
</syntaxhighlight>
This informs the API that the myplugin module has a cache called ''somedata'' and that it is an application (globally shared) cache.

When creating a definition thats the bare minimum, to provide an area ''(somedata)'' and declare the type of the cache application, session, or request.
* An application cache is a shared cache, all users can access it.
* Session caches are scoped to a single users session, but may not actually be stored in the session.
* Request caches you can think of as static caches, only available to the user owning the request, and only alive until the end of the request.
There are of course many more options available that allow you to really take the cache by the reigns, you can read about some of the important ones further on, or skip ahead to [[#The definition]] section which details the available options in full.

Please note that for each definition a language string with the name ''cachedef_'' followed by the name of the definition is expected. Using the example above you would have to define:
<syntaxhighlight lang="php">
// moodle/mod/myplugin/lang/en/mod_myplugin.php
$string['cachedef_somedata'] = 'This is the description of the cache somedata';
</syntaxhighlight>
===Getting a cache object===
Once your definition has been created you should bump the version number so that Moodle upgrades and processes the definitions file at which point your definition will be useable.

Now within code you can get a cache object corresponding to the definition created earlier.
<syntaxhighlight lang="php">
$cache = cache::make('mod_myplugin', 'somedata');
</syntaxhighlight>
The cache::make method is a factory method, it will create a cache object to allow you to work with your cache. The cache object will be one of several classes chosen by the API based upon what your definition contains. All of these classes will extend the base cache class, and in nearly all cases you will get one of cache_application, cache_session, or cache_request depending upon the mode you selected.
===Using your cache object===
Once you have a cache object (will extend the cache class and implements cache_loader) you are ready to start interacting with the cache.

Of course there are three basic basic operations. get, set, and delete.

The first is to send something to the cache.
<syntaxhighlight lang="php">
$result = $cache->set('key', 'value');
</syntaxhighlight>
Easy enough. The key must be an int or a string. The value can be absolutely anything your want that is [https://www.php.net/manual/en/function.serialize.php serializable].
The result is true if the operation was a success, false otherwise.

The second is to fetch something from the cache.
<syntaxhighlight lang="php">
$data = $cache->get('key');
</syntaxhighlight>
$data will either be whatever was being stored in the cache, or false if the cache could not find the key.

The third and final operation is delete.
<syntaxhighlight lang="php">
$result = $cache->delete('key');
</syntaxhighlight>
Again just like set the result will either be true if the operation was a success, or false otherwise.

You can also set, get, and delete multiple key=>value pairs in a single transaction.
<syntaxhighlight lang="php">
$result = $cache->set_many([
'key1' => 'data1',
'key3' => 'data3'
]);
// $result will be the number of pairs sucessfully set.

$result = $cache->get_many(['key1', 'key2', 'key3']);
print_r($result);
// Will print the following:
// array(
// 'key1' => 'data1',
// 'key2' => false,
// 'key3' => 'data3'
// )

$result = $cache->delete_many(['key1', 'key3']);
// $result will be the number of records sucessfully deleted.
</syntaxhighlight>
That covers the basic operation of the Cache API. 
In many situations there is not going to be any more to it than that.
==Ad-hoc Caches==
This is the alternative method of using the cache API. 
It involves creating a cache using just the required params at the time that it is required. It doesn't require that a definition exists making it quicker and easier to use, however it can only use the default settings and is only recommended for insignificant caches (rarely used during operation, never to be mapped or customised, only existsing in a single place in code).

Once a cache object has been retrieved it operates exactly as the same as a cache that has been created for a definition.

To create an ad-hoc cache you would use the following:
<syntaxhighlight lang="php">
$cache = cache::make_from_params(cache_store::MODE_APPLICATION, 'mod_myplugin', 'mycache');
</syntaxhighlight>
Really don't be lazy, if you don't have a good reason to use an ad-hoc cache you should be spending a extra 5 minutes creating a definition.
==The definition==
The above section illustrated how to create a basic definition, specifying just the area name (the key) and the mode for the definition. Those being the two required properties for a definition. 
There are many other options that will let you make the most of the Cache API and will undoubtedly be required when implementing and converting cache solutions to the Cache API.

The following details the options available to a definition and their defaults if not applied:
<syntaxhighlight lang="php">
$definitions = [
// The name of the cache area is the key. The component/plugin will be picked up from the file location.
'area' => [
'mode' => cache_store::MODE_*,
'simplekeys' => false,
'simpledata' => false,
'requireidentifiers' => ['ident1', 'ident2'],
'requiredataguarantee' => false,
'requiremultipleidentifiers' => false,
'requirelockingread' => false,
'requirelockingwrite' => false,
'requirelockingbeforewrite' => false,
'maxsize' => null,
'overrideclass' => null,
'overrideclassfile' => null,
'datasource' => null,
'datasourcefile' => null,
'staticacceleration' => false,
'staticaccelerationsize' => null,
'ttl' => 0,
'mappingsonly' => false,
'invalidationevents' => ['event1', 'event2'],
'canuselocalstore' => false
'sharingoptions' => cache_definition::SHARING_DEFAULT,
'defaultsharing' => cache_definition::SHARING_DEFAULT,
],
];
</syntaxhighlight>
===Setting requirements===
The definition can specify several requirements for the cache. 
This includes identifiers that must be provided when creating the cache object, that the store guarantees data stored in it will remain there until removed, a store that supports multiple identifiers, and finally read/write locking.
The options for these are as follows:
; simplekeys : [bool] Set to true if your cache will only use simple keys for its items. Simple keys consist of digits, underscores and the 26 chars of the english language. a-zA-Z0-9_ If true the keys won't be hashed before being passed to the cache store for gets/sets/deletes. It will be better for performance and possible only because we know the keys are safe.
; simpledata : [bool] If set to true we know that the data is scalar or array of scalar. If true, the data values will be stored as they are. Otherwise they will be serialised first.
; requireidentifiers : [array] An array of identifiers that must be provided to the cache when it is created.
; requiredataguarantee : [bool] If set to true then only stores that can guarantee data will remain available once set will be used.
; requiremultipleidentifiers : [bool] If set to true then only stores that support multiple identifiers will be used.
; requirelockingread : [bool] If set to true then a lock will be gained before reading from the cache store. It is recommended not to use this setting unless 100% absolutely positively required.

; requirelockingbeforewrite :[bool] If set to true then a lock must be gained and held during expensive computation such as the generation of modinfo before writing to the cache store by the calling code. This is to prevent cache stampedes. After gaining the lock code must check to ensure the cache hasn't already been updated by another process. This is so far only used by course modinfo application caches presently.
===Cache modifiers===
You are also to modify the way in which the cache is going to operate when working for your definition. 
By enabling the static option the Cache API will only ever generate a single cache object for your definition on the first request for it, further requests will be returned the original instance. This greatly speeds up the collecting of a cache object. 
Enabling persistence also enables a static store within the cache object, anything set to the cache, or retrieved from it will be stored in that static array for the life of the request.
This makes the persistence options some of the most powerful. If you know you are going to be using you cache over and over again or if you know you will be making lots of requests for the same items then this will provide a great performance boost.
Of course the static storage of cache objects and of data is costly in terms of memory and should only be used when actually required, as such it is turned off by default.
As well as persistence you can also set a maximum number of items that the cache should store (not a hard limit, its up to each store) and a time to live (ttl) although both are discouraged as efficient design negates the need for both in most situations.
; staticacceleration : [bool] This setting does two important things. First it tells the cache API to only instantiate the cache structure for this definition once, further requests will be given the original instance. Second the cache loader will keep an array of the items set and retrieved to the cache during the request. This has several advantages including better performance without needing to start passing the cache instance between function calls, the downside is that the cache instance + the items used stay within memory. Consider using this setting when you know that there are going to be many calls to the cache for the same information or when you are converting existing code to the cache and need to access the cache within functions but don't want to add it as an argument to the function.
; staticaccelerationsize : [int] This supplements the above setting by limiting the number of items in the caches persistent array of items. Tweaking this setting lower will allow you to minimise the memory implications above while hopefully still managing to offset calls to the cache store.
; ttl : [int] A time to live for the data (in seconds). It is strongly recommended that you don't make use of this and instead try to create an event driven invalidation system (even if the event is just time expiring, better not to rely on ttl). Not all cache stores will support this natively and there are undesired performance impacts if the cache store does not.
; maxsize : [int] If set this will be used as the maximum number of entries within the cache store for this definition. Its important to note that cache stores don't actually have to acknowledge this setting or maintain it as a hard limit.
; canuselocalstore : [bool] This setting specifies whether the cache can safely be local to each frontend in a cluster which can avoid latency costs to a shared central cache server. The cache needs to be carefully written for this to be safe. It is conceptually similar to using $CFG->localcachedir (can be local) vs $CFG->cachedir (must be shared). Look at purify_html() in lib/weblib.php for an example.

===Overriding a cache loader===
This is a super advanced feature and should not be done. ever. Unless you have an very good reason to do so.
It allows you to create your own cache loader and have it be used instead of the default cache loader class. The cache object you get back from the make operations will be an instance of this class.
; overrideclass : [string] A class to use as the loader for this cache. This is an advanced setting and will allow the developer of the definition to take 100% control of the caching solution. Any class used here must inherit the cache_loader interface and must extend default cache loader for the mode they are using.
; overrideclassfile : [string] Suplements the above setting indicated the file containing the class to be used. This file is included when required.
===Specifying a data source===
This is a great wee feature, especially if your code is object orientated.

It allows you to specify a class that must inherit the cache_data_source object and will be used to load any information requested from the cache that is not already being stored.

When the requested key cannot be found in the cache the data source will be asked to load it. The data source will then return the information to the cache, the cache will store it, and it will then return it to the user as a request of their get request. Essentially no get request should ever fail if you have a data source specified.
; datasource : [string] A class to use as the data loader for this definition. Any class used here must inherit the cache_data_source interface.
; datasourcefile : [string] Suplements the above setting indicated the file containing the class to be used. This file is included when required.
GOTCHA: Note that, in Moodle versions prior to 3.8.6 and 3.9.3, if caching is disabled then *nothing* will be loaded through the data source which is probably not what you expect (rather than the data source being loaded every time but never cached). See also:

https://tracker.moodle.org/browse/MDL-42012
===Misc settings===
The following are stand along settings that don't fall into any of the above categories.
; invalidationevents :[array] An array of events that should cause this cache to invalidate some or all of the items within it. Note that these are NOT normal moodle events and predates the [[Events API]] . Instead these are arbitrary strings which can be used by cache_helper::purge_by_event('changesincoursecat'); to mark multiple caches as invalid at once without the calling code knowing which caches are affected.
; mappingsonly : [bool] If set to true only the mapped cache store(s) will be used and the default mode store will not. This is a super advanced setting and should not be used unless absolutely required. It allows you to avoid the default stores for one reason or another.
; sharingoptions : [int] The sharing options that are appropriate for this definition. Should be the sum of the possible options.
; defaultsharing : [int] The default sharing option to use. It's highly recommended that you don't set this unless there is a very specific reason not to use the system default.
==The loader==

== Localized stores for distributed high performance caching ==
Most cache definitions are simple in that the code expects to be able to purge the cache, or update it, and for this to be universally available from then on to all code which loads from this cache again. But as you scale up having a single mega shared cache store doesn't work well for a variety of reasons, including extra latency between multiple front ends and the shared cache service, the number of connections the cache server can handle, the cost of IO between services, and depending on the cache definition issues with locking while writing.

So if you want very high performance caching then you need to write you code so that it can support being distributed, or localized, which means that each front end can have it's own independent cache store. But this architecture means that you have no direct way to communicate from code running in one place to invalidate the caches on all the other front ends. In order to achieve this you need to carefully construct cache keys so that if the content changes then it uses a new cache key, which will of course be a cache miss and then it will regenerate using fresh data. There are multiple ways to achieve this, a couple common example strategies are below.
=== When should you localize? ===
Not all caches are good candidates to localize and some can have a detrimental effect if localized for the wrong reasons.
[[File:When to localize cache.png|alt=When to localize a cache|none|thumb|683x683px|When to localize a cache]]
=== Revision numbers as key suffix ===
A simple method is storing a version number somewhere and appending that to your key. This is the most common method used in Moodle, simply store a number somewhere which is globally shared such as in a custom db field, or using set_config / get_config.

One small edge case with this approach is you now need to make sure that the incrementing code is atomic, which means you should use a DB transaction, or gain a lock, before bumping the version so you don't get two conflicting changes ending up on the same version with a race condition. However if you are anticipating high turnover rate of the cache you probably have a deeper issue, see 'Fast churning keys' below.

One potential benefit of a simple version number strategy if your cache misses are very expensive, is that you can check for the presence of a version, and if it doesn't exist it is easy to simply retrieve the previous version and use it in the mean time, while you could generate the new version asynchronously. This is an advanced concept and depends on the use case and has the obvious disadvantage of showing stale data.

An example of this strategy in Moodle is the theme versions cache (NOTE: this is not actually in MUC but the caching concepts are the same):

https://github.com/moodle/moodle/blob/master/lib/outputlib.php#L88-L100

It works best with a cache store that supports Least Recently Used garbage collection.
=== Revision numbers as value suffix instead of key suffix ===
This is conceptually the same as above but has two important differences. Firstly because the key itself is always the same then only a single version of some value will be stored on disk or in memory for any given cache store which makes it much more efficient in terms of storage. But secondly this comes with a higher coding complexity cost because it will no longer be guaranteed to be correct because a local cache could return a hit with a stale version. So if you need it to be correct you will need to parse out the revision from the value and then confirm the revision is correct before using the value. If it is invalid then you need to treat it as a miss and handle that. One way is to rebuild and set it, but this loses the advantage of primary and final caches (see below). A better way is to delete the local cache but not the final cache by passing a second param of false to delete, and then getting the value again which will repopulate the local cache from the shared cache:
$cache->delete('foo', false); // Delete the primary / local stale version
$cache->get('foo'); // Get the final / shared version (which may also be stale!)
But this also is imperfect if there are 3 layers of caching, see [https://tracker.moodle.org/browse/MDL-72837 MDL-72837] for a full discussion and a possible new api to handle this.

This method is how the course modinfo cache is localized.
=== Using time as a key suffix ===
Another common approach is to use a timestamp, this is how some of the core cache numbers work, see increment_revision_number() for some examples. This has the benefit of not needing any transaction or locking, but you do run the risk of two processes clashing if they happen to run in the same second.

It may look like Moodle theme-related caching uses this strategy, but actually if you look at [https://github.com/moodle/moodle/blob/df0e58adb140f90712bcd3229ad936d3b4bc15d9/lib/outputlib.php#L88 the code for theme_get_next_revision], you will see that it is actually guaranteeing to generate a unique new integer which mitigates the clashing mentioned above. It is just making that integer close to current time-stamp, to make it more self-documenting.

https://github.com/moodle/moodle/blob/master/lib/datalib.php#L1131-L1145

It works best with a cache store that supports Least Recently Used garbage collection.
=== Content hashes as keys ===
A great strategy is to use a digest or hash such as sha1 / sha256 of the content stored inside the cache, or in even more advanced scenarios a hash of the dependencies of the content in the cache. This guarantees that the key will be unique, and can be truly distributed without any synchronous communication between the front ends.

This strategy can work very well when building up large cache items from many smaller cache fragments in a nested tree structure, eg when caching a structure which is built of other cache items, eg 10 chunks of html to get combined into a large chunk to be cached, you would append the 10 hashes of the 10 smaller chunks and then hash that to use as the key for the combined cache item. This means you can mutate a small part of a tree structure and quickly re-generate the whole tree without expensively regenerating all of the other branches and leaves in the tree.

This is known as 'Russian Doll' caching:

https://blog.appsignal.com/2018/04/03/russian-doll-caching-in-rails.html

This is a very common strategy is many distributed systems, outside of the context of caching, and is conceptually how git works internally which is why commits are a sha1 hash.

This strategy works well if you may periodically change back and forth to previous states which will still be present and so be immediate hits without re-warming. It works best with a cache store that supports Least Recently Used garbage collection.
=== Primary and Final caches ===
Because http requests are generally assigned randomly or round-robin to front ends, when a cache item version changes you will now effectively have an empty cache on every front end. As you get the same cache item again and again on each front end it will continue to be a cache miss on each local box until they are all warm, which can ironically mean that on average for a fast changing, but not often requested cache item, your cache hit rate will be very low and much worse than if you had a shared cache. The solution here is to have multiple levels of caches setup, a local cache backed by a shared cache. You do not need to do anything special in the code to support this if your cache is already localizable, the MUC manages this for you, ie if you request a key which is missing from the local cache it will then request it from the shared cache. If present it will copy it back to the local cache and then return the value. If it is not present in either then your fall back code will generate the item, and it will be written to both cache stores at the same time.

This is especially important if you are scaling up and down the number of frontends quickly. If you suddenly need more horizontal scale and create a bunch of new front ends with empty caches and no shared cache they will all consume even more resources warming up and loading the shared services such as the database and filesystem.

A good rule of thumb is to pair similar types of local and shared caches together. For instance it is very common to store the Moodle string cache in APCu because it is very heavily used, so an in-memory cache is the fastest and is well paired this a shared cache like Redis. coursemodinfo on the other hand is often very large so isn't as practical to store in Redis so it is usually cached on disk, so you could have a local file cache (which could often be very fast SSD) and pair it with a shared disk cache in dataroot (often much slower over NFS or Gluster etc).

As you scale even bigger, a new bottleneck can appear when purging a shared disk cache ie when you deploy a new version. A full purge needs to iterate over and remove and sync a very large number of files, which can take some time. See MDL-69088 for a proposed fix.
=== Beware fast churning keys ===
A big concern when designing a cache is how fast you anticipate it changing. If it contains very fast moving data but sparsely requested data (see above) then you can end up in a situation where you are effectively just using the shared final cache, and wasting latency and space and IO cloning data to the local cache where is may not be hit again very much. As always caching is a balancing act trading off between CPU, time and disk, and ultimately money.

Even if your cache is able to use a local store that doesn't mean it actually will be configured to be local (and your code can't tell either way). So a wasteful cache item will consume much more space storing all the previous versions of its items even if it isn't localized, and it will be much worse if it is.
=== Time-To-Live for distributed caches ===
Another consideration is the total size of your cache stores across all the front ends. As cache keys change they are never be invalidated or purged. So you should have in place some process to garbage collect stale items. This is more a concern for the cache store implementations and the configuration but worth considering. Some stores are deleted on upgrade, or have a Time-To-Live or a Least Recently Used strategy for deleting stale items.
==Using a data source==

==Developing cache plugins==
==Miscellaneous==
* Checkout important [https://moodle.org/local/chatlogs/index.php?q=cache discussion about the Cache API at the Moodle developer (Telegram) chat]

Cache API

2023-05-10T13:28:14Z

Brendanheywood: Add docs for requirelockingbeforewrite

This document details the Cache API aka MUC aka Moodle's Universal Cache.
I've chosen to use a tutorial/example flow for this document always working with a theoretical module plugin called myplugin.
There is also a [[Cache API - Quick reference]] if you would rather read that.
==Basic usage==
It's very easy to get started with the Cache API. It is designed to be as easy and as quick to use as possible and is predominantely self contained.
All you need to do is add a definition for your cache and you are ready to start working with the Cache API.
===Creating a definition===
Cache definitions exist within the db/caches.php file for a component/plugin. 
In the case of core that is the moodle/lib/db/caches.php file, in the case of a module that would be moodle/mod/myplugin/db/caches.php.

The definition is used API in order to understand a little about the cache and what it is being used for, it also allows the administrator to set things up especially for the definition if they want.
From a development point of view the definition allows you to tell the API about your cache, what it requires, and any (if any) advanced features you want it to have. 
The following shows a basic definition containing just the bare minimum:
<syntaxhighlight lang="php">
// moodle/mod/myplugin/db/caches.php
$definitions = [
'somedata' => [
'mode' => cache_store::MODE_APPLICATION,
]
];
</syntaxhighlight>
This informs the API that the myplugin module has a cache called ''somedata'' and that it is an application (globally shared) cache.

When creating a definition thats the bare minimum, to provide an area ''(somedata)'' and declare the type of the cache application, session, or request.
* An application cache is a shared cache, all users can access it.
* Session caches are scoped to a single users session, but may not actually be stored in the session.
* Request caches you can think of as static caches, only available to the user owning the request, and only alive until the end of the request.
There are of course many more options available that allow you to really take the cache by the reigns, you can read about some of the important ones further on, or skip ahead to [[#The definition]] section which details the available options in full.

Please note that for each definition a language string with the name ''cachedef_'' followed by the name of the definition is expected. Using the example above you would have to define:
<syntaxhighlight lang="php">
// moodle/mod/myplugin/lang/en/mod_myplugin.php
$string['cachedef_somedata'] = 'This is the description of the cache somedata';
</syntaxhighlight>
===Getting a cache object===
Once your definition has been created you should bump the version number so that Moodle upgrades and processes the definitions file at which point your definition will be useable.

Now within code you can get a cache object corresponding to the definition created earlier.
<syntaxhighlight lang="php">
$cache = cache::make('mod_myplugin', 'somedata');
</syntaxhighlight>
The cache::make method is a factory method, it will create a cache object to allow you to work with your cache. The cache object will be one of several classes chosen by the API based upon what your definition contains. All of these classes will extend the base cache class, and in nearly all cases you will get one of cache_application, cache_session, or cache_request depending upon the mode you selected.
===Using your cache object===
Once you have a cache object (will extend the cache class and implements cache_loader) you are ready to start interacting with the cache.

Of course there are three basic basic operations. get, set, and delete.

The first is to send something to the cache.
<syntaxhighlight lang="php">
$result = $cache->set('key', 'value');
</syntaxhighlight>
Easy enough. The key must be an int or a string. The value can be absolutely anything your want that is [https://www.php.net/manual/en/function.serialize.php serializable].
The result is true if the operation was a success, false otherwise.

The second is to fetch something from the cache.
<syntaxhighlight lang="php">
$data = $cache->get('key');
</syntaxhighlight>
$data will either be whatever was being stored in the cache, or false if the cache could not find the key.

The third and final operation is delete.
<syntaxhighlight lang="php">
$result = $cache->delete('key');
</syntaxhighlight>
Again just like set the result will either be true if the operation was a success, or false otherwise.

You can also set, get, and delete multiple key=>value pairs in a single transaction.
<syntaxhighlight lang="php">
$result = $cache->set_many([
'key1' => 'data1',
'key3' => 'data3'
]);
// $result will be the number of pairs sucessfully set.

$result = $cache->get_many(['key1', 'key2', 'key3']);
print_r($result);
// Will print the following:
// array(
// 'key1' => 'data1',
// 'key2' => false,
// 'key3' => 'data3'
// )

$result = $cache->delete_many(['key1', 'key3']);
// $result will be the number of records sucessfully deleted.
</syntaxhighlight>
That covers the basic operation of the Cache API. 
In many situations there is not going to be any more to it than that.
==Ad-hoc Caches==
This is the alternative method of using the cache API. 
It involves creating a cache using just the required params at the time that it is required. It doesn't require that a definition exists making it quicker and easier to use, however it can only use the default settings and is only recommended for insignificant caches (rarely used during operation, never to be mapped or customised, only existsing in a single place in code).

Once a cache object has been retrieved it operates exactly as the same as a cache that has been created for a definition.

To create an ad-hoc cache you would use the following:
<syntaxhighlight lang="php">
$cache = cache::make_from_params(cache_store::MODE_APPLICATION, 'mod_myplugin', 'mycache');
</syntaxhighlight>
Really don't be lazy, if you don't have a good reason to use an ad-hoc cache you should be spending a extra 5 minutes creating a definition.
==The definition==
The above section illustrated how to create a basic definition, specifying just the area name (the key) and the mode for the definition. Those being the two required properties for a definition. 
There are many other options that will let you make the most of the Cache API and will undoubtedly be required when implementing and converting cache solutions to the Cache API.

The following details the options available to a definition and their defaults if not applied:
<syntaxhighlight lang="php">
$definitions = [
// The name of the cache area is the key. The component/plugin will be picked up from the file location.
'area' => [
'mode' => cache_store::MODE_*,
'simplekeys' => false,
'simpledata' => false,
'requireidentifiers' => ['ident1', 'ident2'],
'requiredataguarantee' => false,
'requiremultipleidentifiers' => false,
'requirelockingread' => false,
'requirelockingwrite' => false,
'requirelockingbeforewrite' => false,
'maxsize' => null,
'overrideclass' => null,
'overrideclassfile' => null,
'datasource' => null,
'datasourcefile' => null,
'staticacceleration' => false,
'staticaccelerationsize' => null,
'ttl' => 0,
'mappingsonly' => false,
'invalidationevents' => ['event1', 'event2'],
'canuselocalstore' => false
'sharingoptions' => cache_definition::SHARING_DEFAULT,
'defaultsharing' => cache_definition::SHARING_DEFAULT,
],
];
</syntaxhighlight>
===Setting requirements===
The definition can specify several requirements for the cache. 
This includes identifiers that must be provided when creating the cache object, that the store guarantees data stored in it will remain there until removed, a store that supports multiple identifiers, and finally read/write locking.
The options for these are as follows:
; simplekeys : [bool] Set to true if your cache will only use simple keys for its items. Simple keys consist of digits, underscores and the 26 chars of the english language. a-zA-Z0-9_ If true the keys won't be hashed before being passed to the cache store for gets/sets/deletes. It will be better for performance and possible only because we know the keys are safe.
; simpledata : [bool] If set to true we know that the data is scalar or array of scalar. If true, the data values will be stored as they are. Otherwise they will be serialised first.
; requireidentifiers : [array] An array of identifiers that must be provided to the cache when it is created.
; requiredataguarantee : [bool] If set to true then only stores that can guarantee data will remain available once set will be used.
; requiremultipleidentifiers : [bool] If set to true then only stores that support multiple identifiers will be used.
; requirelockingread : [bool] If set to true then a lock will be gained before reading from the cache store. It is recommended not to use this setting unless 100% absolutely positively required.
; requirelockingbeforewrite : [bool] If set to true then it expects the calling code to have gained a lock before attempting to write. The lock is be held during expensive computation such as the generation of modinfo
 Remember 99.9% of caches will NOT need this setting. This setting will only be used for application caches presently.
; requirelockingbeforewrite : [bool] If set to true then a lock must be gained before writing to the cache store by the calling code. This is to prevent cache stampedes. After gaining the lock code must check to ensure the cache hasn't already been updated by another process. This is so far only used by course modinfo application caches presently.
===Cache modifiers===
You are also to modify the way in which the cache is going to operate when working for your definition. 
By enabling the static option the Cache API will only ever generate a single cache object for your definition on the first request for it, further requests will be returned the original instance. This greatly speeds up the collecting of a cache object. 
Enabling persistence also enables a static store within the cache object, anything set to the cache, or retrieved from it will be stored in that static array for the life of the request.
This makes the persistence options some of the most powerful. If you know you are going to be using you cache over and over again or if you know you will be making lots of requests for the same items then this will provide a great performance boost.
Of course the static storage of cache objects and of data is costly in terms of memory and should only be used when actually required, as such it is turned off by default.
As well as persistence you can also set a maximum number of items that the cache should store (not a hard limit, its up to each store) and a time to live (ttl) although both are discouraged as efficient design negates the need for both in most situations.
; staticacceleration : [bool] This setting does two important things. First it tells the cache API to only instantiate the cache structure for this definition once, further requests will be given the original instance. Second the cache loader will keep an array of the items set and retrieved to the cache during the request. This has several advantages including better performance without needing to start passing the cache instance between function calls, the downside is that the cache instance + the items used stay within memory. Consider using this setting when you know that there are going to be many calls to the cache for the same information or when you are converting existing code to the cache and need to access the cache within functions but don't want to add it as an argument to the function.
; staticaccelerationsize : [int] This supplements the above setting by limiting the number of items in the caches persistent array of items. Tweaking this setting lower will allow you to minimise the memory implications above while hopefully still managing to offset calls to the cache store.
; ttl : [int] A time to live for the data (in seconds). It is strongly recommended that you don't make use of this and instead try to create an event driven invalidation system (even if the event is just time expiring, better not to rely on ttl). Not all cache stores will support this natively and there are undesired performance impacts if the cache store does not.
; maxsize : [int] If set this will be used as the maximum number of entries within the cache store for this definition. Its important to note that cache stores don't actually have to acknowledge this setting or maintain it as a hard limit.
; canuselocalstore : [bool] This setting specifies whether the cache can safely be local to each frontend in a cluster which can avoid latency costs to a shared central cache server. The cache needs to be carefully written for this to be safe. It is conceptually similar to using $CFG->localcachedir (can be local) vs $CFG->cachedir (must be shared). Look at purify_html() in lib/weblib.php for an example.

===Overriding a cache loader===
This is a super advanced feature and should not be done. ever. Unless you have an very good reason to do so.
It allows you to create your own cache loader and have it be used instead of the default cache loader class. The cache object you get back from the make operations will be an instance of this class.
; overrideclass : [string] A class to use as the loader for this cache. This is an advanced setting and will allow the developer of the definition to take 100% control of the caching solution. Any class used here must inherit the cache_loader interface and must extend default cache loader for the mode they are using.
; overrideclassfile : [string] Suplements the above setting indicated the file containing the class to be used. This file is included when required.
===Specifying a data source===
This is a great wee feature, especially if your code is object orientated.

It allows you to specify a class that must inherit the cache_data_source object and will be used to load any information requested from the cache that is not already being stored.

When the requested key cannot be found in the cache the data source will be asked to load it. The data source will then return the information to the cache, the cache will store it, and it will then return it to the user as a request of their get request. Essentially no get request should ever fail if you have a data source specified.
; datasource : [string] A class to use as the data loader for this definition. Any class used here must inherit the cache_data_source interface.
; datasourcefile : [string] Suplements the above setting indicated the file containing the class to be used. This file is included when required.
GOTCHA: Note that, in Moodle versions prior to 3.8.6 and 3.9.3, if caching is disabled then *nothing* will be loaded through the data source which is probably not what you expect (rather than the data source being loaded every time but never cached). See also:

https://tracker.moodle.org/browse/MDL-42012
===Misc settings===
The following are stand along settings that don't fall into any of the above categories.
; invalidationevents :[array] An array of events that should cause this cache to invalidate some or all of the items within it. Note that these are NOT normal moodle events and predates the [[Events API]] . Instead these are arbitrary strings which can be used by cache_helper::purge_by_event('changesincoursecat'); to mark multiple caches as invalid at once without the calling code knowing which caches are affected.
; mappingsonly : [bool] If set to true only the mapped cache store(s) will be used and the default mode store will not. This is a super advanced setting and should not be used unless absolutely required. It allows you to avoid the default stores for one reason or another.
; sharingoptions : [int] The sharing options that are appropriate for this definition. Should be the sum of the possible options.
; defaultsharing : [int] The default sharing option to use. It's highly recommended that you don't set this unless there is a very specific reason not to use the system default.
==The loader==

== Localized stores for distributed high performance caching ==
Most cache definitions are simple in that the code expects to be able to purge the cache, or update it, and for this to be universally available from then on to all code which loads from this cache again. But as you scale up having a single mega shared cache store doesn't work well for a variety of reasons, including extra latency between multiple front ends and the shared cache service, the number of connections the cache server can handle, the cost of IO between services, and depending on the cache definition issues with locking while writing.

So if you want very high performance caching then you need to write you code so that it can support being distributed, or localized, which means that each front end can have it's own independent cache store. But this architecture means that you have no direct way to communicate from code running in one place to invalidate the caches on all the other front ends. In order to achieve this you need to carefully construct cache keys so that if the content changes then it uses a new cache key, which will of course be a cache miss and then it will regenerate using fresh data. There are multiple ways to achieve this, a couple common example strategies are below.
=== When should you localize? ===
Not all caches are good candidates to localize and some can have a detrimental effect if localized for the wrong reasons.
[[File:When to localize cache.png|alt=When to localize a cache|none|thumb|683x683px|When to localize a cache]]
=== Revision numbers as key suffix ===
A simple method is storing a version number somewhere and appending that to your key. This is the most common method used in Moodle, simply store a number somewhere which is globally shared such as in a custom db field, or using set_config / get_config.

One small edge case with this approach is you now need to make sure that the incrementing code is atomic, which means you should use a DB transaction, or gain a lock, before bumping the version so you don't get two conflicting changes ending up on the same version with a race condition. However if you are anticipating high turnover rate of the cache you probably have a deeper issue, see 'Fast churning keys' below.

One potential benefit of a simple version number strategy if your cache misses are very expensive, is that you can check for the presence of a version, and if it doesn't exist it is easy to simply retrieve the previous version and use it in the mean time, while you could generate the new version asynchronously. This is an advanced concept and depends on the use case and has the obvious disadvantage of showing stale data.

An example of this strategy in Moodle is the theme versions cache (NOTE: this is not actually in MUC but the caching concepts are the same):

https://github.com/moodle/moodle/blob/master/lib/outputlib.php#L88-L100

It works best with a cache store that supports Least Recently Used garbage collection.
=== Revision numbers as value suffix instead of key suffix ===
This is conceptually the same as above but has two important differences. Firstly because the key itself is always the same then only a single version of some value will be stored on disk or in memory for any given cache store which makes it much more efficient in terms of storage. But secondly this comes with a higher coding complexity cost because it will no longer be guaranteed to be correct because a local cache could return a hit with a stale version. So if you need it to be correct you will need to parse out the revision from the value and then confirm the revision is correct before using the value. If it is invalid then you need to treat it as a miss and handle that. One way is to rebuild and set it, but this loses the advantage of primary and final caches (see below). A better way is to delete the local cache but not the final cache by passing a second param of false to delete, and then getting the value again which will repopulate the local cache from the shared cache:
$cache->delete('foo', false); // Delete the primary / local stale version
$cache->get('foo'); // Get the final / shared version (which may also be stale!)
But this also is imperfect if there are 3 layers of caching, see [https://tracker.moodle.org/browse/MDL-72837 MDL-72837] for a full discussion and a possible new api to handle this.

This method is how the course modinfo cache is localized.
=== Using time as a key suffix ===
Another common approach is to use a timestamp, this is how some of the core cache numbers work, see increment_revision_number() for some examples. This has the benefit of not needing any transaction or locking, but you do run the risk of two processes clashing if they happen to run in the same second.

It may look like Moodle theme-related caching uses this strategy, but actually if you look at [https://github.com/moodle/moodle/blob/df0e58adb140f90712bcd3229ad936d3b4bc15d9/lib/outputlib.php#L88 the code for theme_get_next_revision], you will see that it is actually guaranteeing to generate a unique new integer which mitigates the clashing mentioned above. It is just making that integer close to current time-stamp, to make it more self-documenting.

https://github.com/moodle/moodle/blob/master/lib/datalib.php#L1131-L1145

It works best with a cache store that supports Least Recently Used garbage collection.
=== Content hashes as keys ===
A great strategy is to use a digest or hash such as sha1 / sha256 of the content stored inside the cache, or in even more advanced scenarios a hash of the dependencies of the content in the cache. This guarantees that the key will be unique, and can be truly distributed without any synchronous communication between the front ends.

This strategy can work very well when building up large cache items from many smaller cache fragments in a nested tree structure, eg when caching a structure which is built of other cache items, eg 10 chunks of html to get combined into a large chunk to be cached, you would append the 10 hashes of the 10 smaller chunks and then hash that to use as the key for the combined cache item. This means you can mutate a small part of a tree structure and quickly re-generate the whole tree without expensively regenerating all of the other branches and leaves in the tree.

This is known as 'Russian Doll' caching:

https://blog.appsignal.com/2018/04/03/russian-doll-caching-in-rails.html

This is a very common strategy is many distributed systems, outside of the context of caching, and is conceptually how git works internally which is why commits are a sha1 hash.

This strategy works well if you may periodically change back and forth to previous states which will still be present and so be immediate hits without re-warming. It works best with a cache store that supports Least Recently Used garbage collection.
=== Primary and Final caches ===
Because http requests are generally assigned randomly or round-robin to front ends, when a cache item version changes you will now effectively have an empty cache on every front end. As you get the same cache item again and again on each front end it will continue to be a cache miss on each local box until they are all warm, which can ironically mean that on average for a fast changing, but not often requested cache item, your cache hit rate will be very low and much worse than if you had a shared cache. The solution here is to have multiple levels of caches setup, a local cache backed by a shared cache. You do not need to do anything special in the code to support this if your cache is already localizable, the MUC manages this for you, ie if you request a key which is missing from the local cache it will then request it from the shared cache. If present it will copy it back to the local cache and then return the value. If it is not present in either then your fall back code will generate the item, and it will be written to both cache stores at the same time.

This is especially important if you are scaling up and down the number of frontends quickly. If you suddenly need more horizontal scale and create a bunch of new front ends with empty caches and no shared cache they will all consume even more resources warming up and loading the shared services such as the database and filesystem.

A good rule of thumb is to pair similar types of local and shared caches together. For instance it is very common to store the Moodle string cache in APCu because it is very heavily used, so an in-memory cache is the fastest and is well paired this a shared cache like Redis. coursemodinfo on the other hand is often very large so isn't as practical to store in Redis so it is usually cached on disk, so you could have a local file cache (which could often be very fast SSD) and pair it with a shared disk cache in dataroot (often much slower over NFS or Gluster etc).

As you scale even bigger, a new bottleneck can appear when purging a shared disk cache ie when you deploy a new version. A full purge needs to iterate over and remove and sync a very large number of files, which can take some time. See MDL-69088 for a proposed fix.
=== Beware fast churning keys ===
A big concern when designing a cache is how fast you anticipate it changing. If it contains very fast moving data but sparsely requested data (see above) then you can end up in a situation where you are effectively just using the shared final cache, and wasting latency and space and IO cloning data to the local cache where is may not be hit again very much. As always caching is a balancing act trading off between CPU, time and disk, and ultimately money.

Even if your cache is able to use a local store that doesn't mean it actually will be configured to be local (and your code can't tell either way). So a wasteful cache item will consume much more space storing all the previous versions of its items even if it isn't localized, and it will be much worse if it is.
=== Time-To-Live for distributed caches ===
Another consideration is the total size of your cache stores across all the front ends. As cache keys change they are never be invalidated or purged. So you should have in place some process to garbage collect stale items. This is more a concern for the cache store implementations and the configuration but worth considering. Some stores are deleted on upgrade, or have a Time-To-Live or a Least Recently Used strategy for deleting stale items.
==Using a data source==

==Developing cache plugins==
==Miscellaneous==
* Checkout important [https://moodle.org/local/chatlogs/index.php?q=cache discussion about the Cache API at the Moodle developer (Telegram) chat]

Error pages

2023-05-10T03:38:27Z

Brendanheywood:

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== 403 in Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex
=== 403 in Nginx ===
<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php
</syntaxhighlight>If you want to transform all 403 errors into 404's then:<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php?code=404
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.
== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== 404 in Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== 404 in nginx ===
<syntaxhighlight lang="php">
error_page 404 = /error/index.php
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.
== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Error pages

2023-05-09T23:29:44Z

Brendanheywood: /* Nginx */

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex
=== Nginx ===
<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php
</syntaxhighlight>If you want to transform all 403 errors into 404's then:<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php?code=404
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.

== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== nginx ===
<syntaxhighlight lang="php">
error_page 404 = /error/index.php
</syntaxhighlight>Note the '=' char above is important as it allows Moodle to reset the http headers to correctly serve the page in all cases.

== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Error pages

2023-05-09T23:28:16Z

Brendanheywood: /* 403 Forbidden - Web server errors */

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex

=== Nginx ===
<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php
</syntaxhighlight>If you want to transform all 403 errors into 404's then:<syntaxhighlight lang="nginx">
error_page 403 = /error/index.php?code=404
</syntaxhighlight>

== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== nginx ===
<syntaxhighlight lang="php">
error_page 404 = /error/index.php
</syntaxhighlight>
== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Error pages

2023-05-09T23:24:27Z

Brendanheywood: /* 404 File not found - Web server errors */

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex
== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== nginx ===
<syntaxhighlight lang="php">
error_page 404 = /error/index.php;
</syntaxhighlight>
== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Improve Annotation

2022-11-08T22:45:11Z

Brendanheywood:

{{Infobox Project
|state = Proposal
|name = Improve Annotation
|tracker = https://tracker.moodle.org/browse/MDL-76243
|discussion = https://moodle.org/mod/forum/discuss.php?d=422700
|assignee = Catalyst IT
}}
== Introduction ==
This is a proposal for improving Annotation in Moodle by [https://www.catalyst.net.nz Catalyst IT] - thanks to [https://www.xjtlu.edu.cn Xi’an Jiaotong-Liverpool University] for providing support with researching this project.

The “uploadpdf” assignment feedback plugin allows teachers to annotate assignments during the grading process. We would like to improve the user experience for annotation by allowing the teacher to scroll through the pages like a normal document and implement an “Undo” feature, allowing a teacher to undo previous changes like the deletion of an object.

The existing annotation feature in Moodle is built using the YUI Javascript library which has not been maintained by the original developers since 2014 and has been generally deprecated in Moodle (no new YUI code has been generally allowed in Moodle since 2.9).

Moodle has also been improving the core grade API and recently in Moodle 3.8 added a new user interface within the forum activity for grading with the intention of using this same interface for the assignment grading interface but has not yet implemented this in assignment yet.

Investing significant development effort on the existing annotation plugin would be unwise due to the existing use of YUI and we believe it would be more useful to develop a brand new annotation tool in Moodle.

There is also potential for annotation of content to be useful outside the assignment activity – for example in essay questions, forum, workshop activities and 3rd party plugins like mod_annotatepdf so we propose that annotation functionality is added to the core Grade API so that it can be re-used anywhere in Moodle where grades are also used.
== External libraries ==
=== Rendering the PDF ===
Moodle uses a combination of libraries to convert a document to a pdf, then converts the pdf into an image per page to allow annotation and then stiches it all back into a pdf again so that it can be downloaded - this is complex, results in accessibility issues and bugs like MDL-64431.

We propose that we that we render the PDF directly in the browser using Mozilla's [https://github.com/mozilla/pdf.js pdf.js], and then add an annotation UI on top which stores the annotations directly in the PDF.
=== Annotating the PDF ===
Ideally we wouldn't implement our own Annotation library like we have done in the existing plugin, however there doesn’t appear to be many well maintained open source annotation libraries that we can re-use in Moodle.
* https://github.com/highkite/pdfAnnotate – This seems to be the most promising, however it is in early development stage, stores annotations within the pdf itself, Supports the main components that uploadpdf tool currently supports except the “stamp” and doesn’t seem to have built-in undo/redo support. Doesn’t implement a UI, but Moodle would be providing this anyway. Other advantage of using a JS library with annotations stored within the document itself is that there is potential for this to be re-used within an offline mobile app. Using this library would also mean a decision to only support annotations on pdf documents which I think is ok. The developer behind this project has expressed an interest in helping us use this.

*Apache Annotator https://annotator.apache.org/ - Incubating project - allows annotation on any html content which would be a nice idea to allow annotations directly on forum posts etc, however at this state the project only allows you to select text or css selectors – annoation in Moodle requires more flexibility – drawing/circles etc.

* http://annotatorjs.org/ - Unmaintained since 2017hypothes.is – uses a python-based back-end to store annotations.

* https://github.com/bhargavacc/pdf-annotation – unmaintained since 2016

* https://github.com/instructure/pdf-annotate.js – unmaintained since 2018, requires a storage layer for annotations.

* https://moodle.org/plugins/mod_pdfannotator - (3rd party Moodle plugin) Uses Mozilla's pdf.js already, uses https://github.com/instructure/pdf-annotate.js for performing annotations - doesn't contain undo/redo features.
== Prototype / mock-ups ==
See below for a mockup of a possible pdf annotation interface using the new grading UI from the forum activity.

Note: Moodle’s existing interface provides the ability to rotate the image clockwise or anti-clockwise, and the addition of a new “undo” button may require some changes to help make it clear.
[[File:improveannotation-assign-mockup.png|Mockup of new grading interface]]
== Development tasks ==
# Implement Behat tests for the forum grading panel (https://tracker.moodle.org/browse/MDL-66666) to help prevent any regressions while reworking the grading UI.
# Re-work the Forum grading UI so that it is a core function that other activities can re-use.
# Create a new centralised annotation API initally using https://github.com/highkite/pdfAnnotate with a nice Moodle UI (images/buttons etc) but leaving room for the annotation API to be extended further in future with other interfaces like Apache annotator for html content. Initially this would only support in-browser editing with the full pdf being sent back to the server for storing.
# Create a new assignment feedback plugin that re-uses the new core grading UI (based on forum) and integrates the core annotation API
# Add support for “stamps” (custom images) within the pdfAnnotate library.
# Add support for undo process within the pdfAnnotate library and Moodle’s UI.
== Ideas for future development (outside initial scope) ==
* Some PDF files are quite large and sending the full edited pdf back to the server is in-efficient, the highkite library supports being run server side using nodejs - investigate using an optional server-side process to save annotations.
* Add support for annotations to other areas like essay quiz questions, Workshop activity etc.
* Add support for devices like the iPad Pencil.
* Add support to allow grading to occur offline (embedding the JS library within our Mobile app).
* Add support for Apache annotator for annotating html content.

String API

2022-05-23T01:50:31Z

Brendanheywood: /* Should the colon sign be hard-coded or included in a language string? */

==Overview==
The String API is how you get language text strings to use in the user interface. It handles internationalisation issues, and will use a number of settings and environment variables to present the best text to every user. Moodle has a mechanism that allows a number of places to be searched (in order) to find language strings. This enables language strings to be packaged with plugins and avoids the step of having to copy the language files over to the language directory when a plugin is installed.

Moodle also provide general string functions like substr, strlen etc. for multibyte safe, string operations. It uses mbstring or iconv for UTF-8 strings and falls back to typo3.
==Basic concepts==
When it is required to lookup a string, two basic items of information are required.
# The component providing the string.
# The identifier of the string.
Example: The function call <tt>get_string('editingquiz', 'mod_quiz')</tt> will return "Editing quiz" if the current language is English, or relevant translation of the text. Here the string identifier is "editingquiz" and the string is provided by the "mod_quiz" component (that is the Quiz activity module).
===Adding language file to plugin===
Language support for plugin(s) is added by creating a '''lang/en/''' subdirectory in the plugin directory and putting the plugin's English string file there. The name of the string file is supposed to follow the plugin's [[Frankenstyle|component name]], so that it is something like <tt>plugintype_pluginname.php</tt>.

The default "en" language is the [[:en:Language FAQ#Which_is_the_official_language_for_Moodle.3F|Australian English]] which is same as the British English when it comes to spelling and grammar. American English is a separate language pack "en_us".

There is no need to include languages other than the default English, as almost all approved [https://lang.moodle.org/mod/forum/discuss.php?d=2485 plugin strings will be automatically imported into AMOS] for translation by the language packs translators.
===Adding a string to the language file===
Strings are defined via the associative array <tt>$string</tt> provided by the string file. The array key is the string identifier, the value is the string text in the given language.
<syntaxhighlight lang="php">
$string['editingquiz'] = 'Editing quiz';
$string['editingquiz_help'] = 'Help for editing quiz';
</syntaxhighlight>
===Run-time parameters===
Strings can define placeholders like <tt>{$a}</tt> or <tt>{$a->foobar}</tt>. These placeholders are replaced with a value passed to the <tt>get_string()</tt> function call.
<syntaxhighlight lang="php">
// Strings defined in the language file.
$string['greeting'] = 'Dear {$a}';
$string['info'] = 'There are {$a->count} new messages from {$a->from}.';

// Passing values for the placeholders.
echo get_string('greeting', 'tool_example', 'Mr. Anderson');
echo get_string('info', 'tool_example', ['count' => 42, 'from' => 'Mr. Smith']);
</syntaxhighlight>
== Help strings ==
Help strings are texts that are displayed in the help tooltips. There are certain conventions on how to define them.
[[Image:Roles help popup.png|thumb|Help popup containing "More help" link]] Help strings names start with the string name of the setting to which they apply, so that they are kept together in the language file.
* ''stringname'' - identifier of the string we provide the help for, may be used as a title of the help popup
* ''stringname_help'' - string with the help text
* ''stringname_link'' - an optional string naming a path in Moodle Docs, only if required, calculated just like the link in the footer to go to the right language etc, and shown after the help text in the popup as a link with an icon to "More help..."
* ''stringname_desc'' - an admin setting description replacing the legacy ''configstringname''
Example:
<syntaxhighlight lang="php">
$string['submissionsize'] = 'Submission size';
$string['submissionsize_help'] = 'Blah blah blah some useful text, optionally using Markdown syntax';
$string['submissionsize_link'] = 'mod/workshop/submission';
$string['submissionsize_desc'] = 'Default value for submission size. This value will be pre-filled and can be blah blah ...';
</syntaxhighlight>
Help string text should be short and simple - two small paragraphs maximum with no headings, no links, no bold, no tables etc.

[[:en:Markdown|Markdown]] can be used for some basic formatting of strings with the "_help" suffix. It is recommended to stick to paragraphs and lists only. Markdown can only be used for help strings, not for other language strings.
=== More help links ===
To display the optional ''More help...'' link at the bottom of the help popup, a string with the "_link" suffix must exist beside the relevant "_help" string. These link strings should consist of a short relative path like 'component/thing' (eg. 'mod/folder/view' or 'group/import'). This gets turned into a link to MoodleDocs in the user's language, and for the appropriate Moodle version.

Example:
<syntaxhighlight lang="php">
$string['groupimport_link'] = 'group/import';
</syntaxhighlight>
will make a link to something like <nowiki>'https://docs.moodle.org/32/en/group/import'</nowiki> (if you are an English user of Moodle 3.2). This page in turn should be a redirect page to the actual page providing the info. Note the <nowiki>'https://docs.moodle.org'</nowiki> part comes from $CFG->docroot.

The second option is that you can give an absolute link, if you want, and that will be used unmodified.
<syntaxhighlight lang="php">
$string['patronising_link'] = <nowiki>'http://lmgtfy.com/?q=Moodle+for+dummies';</nowiki>
</syntaxhighlight>
This case is detected by seeing if the string starts with <nowiki>http:// or https://</nowiki>. This option is useful if the documentation for your plugin is hosted at Github wiki, for example.

The third option is to start the link with <tt>%%WWWROOT%%</tt>. That is replaced by <tt>$CFG->wwwroot</tt>. This is useful for contributed plugins that want to keep the help within the plugin.
=== Words for roles ===
The following words should be used in the English (en) help strings whenever it is necessary to refer to people with particular roles in a generic sense.
* Participants - all people with a role
* Teachers - people with some sort of a facilitation/editing role
* Students - people primarily there to learn
* Users - people across the site
([[User:Martin Dougiamas|Martin Dougiamas]] 16:13, 20 April 2010 (UTC): I'm not entirely comfortable with this choice even though I just re-confirmed it, as I'd prefer to be using words like facilitator/moderator to subtly and continually promote a less transmissionist pedgagogy and I know many Moodle users would agree. However, I think these terms are far more commonly used in the community and in discussion, and from the point of view that we need to build a consistent system for usability they are less distracting and better for documentation.)
==Files==
String functions are defined in
# lib/moodlelib.php - locale related functions
# lib/textlib.class.php - general string functions (substr, strlen etc.)
==Functions and examples==
There are three main functions, used in moodle for getting/displaying the localised string, based on user preferred language.
===get_string()===
Returns a localised string for current user.

Example for displaying string "This is my plug-in" in any language that supports on your site, then you need to use the following identifier in language file (located in appropriate lang directory)
$string['plugintitle'] = 'This is my plug-in';
If you want to display this string in any supported language, then you would use this function.
echo get_string('plugintitle', 'module_pluginlangfilename');
If you want to substitute value in the language string then use '''{$a}''' for substituting value. $a is an object, string or number that can be used within translation strings. The variable has to be $a, and it has to be in single quotes. For example, if you want to display answer number in Drag & drop plugin then add
$string['answerno1'] = 'Answer {$a}'; //Substituting string/integer
$string['answerno2'] = 'Answer {$a->name}'; //Substituting object member
in qtype_dragdrop.php (Drag & Drop language file). And call it using
//Get string by substituting integer.
get_string('answerno1', 'qtype_dragdrop', $number);
//Get string by substituting object member integer
$user->number = 10;
get_string('answerno2', 'qtype_dragdrop', $user);
In Moodle 2.3 there is a new argument to this function $lazyload. Setting $lazyload to true causes get_string to return a lang_string object rather than the string itself.
$stringobject = get_string('answerno', 'qtype_dragdrop', $number, true);
The fetching of the string is then put off until the string object is first used. The object can be used by calling it's out method or by casting the object to a string, either directly e.g.
(string)$stringobject
or indirectly by using the string within another string or echoing it out e.g.
echo $stringobject;
return "{$stringobject}";
Note: using $lazyload and attempting to use the string as an array key will cause a fatal error as objects cannot be used as array keys.
===get_strings()===
Converts an array of string names to localised strings for a specific plugin. lazy loading is not supported in this function.
$txt = get_strings(array('enable', 'disable', 'up', 'down', 'none'), 'qtype_dragdrop');
echo $txt->up; //Display localised string for up
echo $txt->down //Display localised string for down
===print_string()===
Prints out a translated string by using '''get_string()''' function
===lang_string class===
In Moodle 2.3 a special class (lang_string) is used to create an object representation of a string request. In this case string processing doesn't occur until the object is first used. The class was created especially to aid performance in areas where strings were required to be generated but were not necessarily used. As an example the admin navigation tree when generated uses over 1500 strings, of which normally only 1/3 are ever actually printed at any time. The performance advantage is achieved by not actually processing strings that aren't being used, as such reducing the processing required for the page.

lang_string class can be used in two ways
# Setting $lazyload (fourth argument of the get_string function), to true.
$string = get_string('yes', 'qtype_dragdrop', null, true);
# Direct instantiation
$string = new lang_string('yes', 'qtype_dragdrop', null, 'en');
== String manager ==
Most of function listed above are just handful wrappers for the methods provided by the string manager class. Some strings related functionality is available only via directly calling the manager's methods.
<syntaxhighlight lang="php">
$stringman = get_string_manager();
</syntaxhighlight>
The factory function get_string_manager() returns singleton instance of <tt>core_string_manager_install</tt> in early stages of the site installation, or instance of <tt>core_string_manager_standard</tt> in all normal situations. These managers implement interface <tt>core_string_manager</tt>. Methods provided by the interface are
;get_string():Returns the given component string localised in the given language. Can be used to obtain the translation of the string in the other language than the current user's language.
;string_exists():Does the given string actually exist? This is typically checked by a code like
<syntaxhighlight lang="php">
if (get_string_manager()->string_exists('stringidentifier', 'component_name')) {
// Do something.
}
</syntaxhighlight>
;string_deprecated():Has the string been deprecated? See [[String deprecation]] for details.
;get_list_of_countries():Returns a localised list of all country names, sorted by country keys.
;get_list_of_languages():Returns a localised list of languages, sorted by code keys.
;translation_exists():Checks if the translation exists for the given language.
;get_list_of_translations():Returns localised list of installed language packs.
;get_list_of_currencies():Returns localised list of known currencies.
;load_component_strings():Loads all strings for one component.
;reset_caches():Invalidates all caches, should the manager use some.
;get_revision():Returns string revision counter.
=== Custom string managers ===
{{Moodle 2.9}}Plugins can provide custom implementation of the string manager. This is supposed to be used in experimental and/or development scenarios only, not in typical production environment. See MDL-49361 for use cases and implementation details. An example of such an implementation can be found at [https://github.com/mudrd8mz/moodle-local_stringman moodle-local_stringman.git] repository.
<syntaxhighlight lang="php">
// Custom string manager class can be defined in the main config.php file.
$CFG->customstringmanager = '\local_stringman\dummy_string_manager';
</syntaxhighlight>
==textlib (core_text) class==
textlib class provide pool of safe functions to operate on UTF-8 text. textlib provide set of static functions to operate on strings and gets included in setup.php

{{Moodle 2.6}}In Moodle 2.6 the textlib class was renamed to '''core_text'''.
===asort()===
Locale aware sorting, the key associations are kept, values are sorted alphabetically.
===code2utf8===
Returns the utf8 string corresponding to the unicode value
===convert===
Converts the text between different encodings. It uses iconv extension with //TRANSLIT parameter, fall back to typo3
===encode_mimeheader===
Generate a correct base64 encoded header to be used in MIME mail messages.
===entities_to_utf8===
Converts all the numeric entities &#nnnn; or &#xnnn; to UTF-8
===specialtoascii===
Converts upper unicode characters to plain ascii, the returned string may contain unconverted unicode characters.
===strlen===
Multibyte safe strlen() function, uses iconv for utf-8, falls back to typo3.
===strpos===
Find the position of the first occurrence of a substring in a string. UTF-8 ONLY safe strpos(), uses iconv.
===strrpos===
Find the position of the last occurrence of a substring in a string. UTF-8 ONLY safe strrpos(), uses iconv.
===strtolower===
Multibyte safe strtolower() function, uses mbstring, falls back to typo3.
===strtotitle===
Makes first letter of each word capital - words must be separated by spaces.
===strtoupper===
Multibyte safe strtoupper() function, uses mbstring, falls back to typo3.
===substr===
Multibyte safe substr() function, uses iconv for utf-8, falls back to typo3.
===trim_utf8_bom===
Removes the BOM from unicode string. [http://unicode.org/faq/utf_bom.html more info]
===utf8_to_entities===
Converts all Unicode chars > 127 to numeric entities &#nnnn; or &#xnnn;
==FAQ==
===When should I use a lang_string object?===
The lang_string object is designed to be used in any situation where a string may not be needed, but needs to be generated. The admin navigation tree is a good example of where lang_string objects should be used. A more practical example would be any class that requires strings that may not be printed (after all classes get renderer by renderers and who knows what they will do ;))
===When should I not use a lang_string object?===
Don't use lang_strings when you are going to use a string immediately. There is no need as it will be processed immediately and there will be no advantage, and in fact perhaps a negative hit as a class has to be instantiated for a lang_string object, however get_string won't require that.
===Limitation of lang_string===
lang_string object cannot be used as an array offset. Doing so will result in PHP throwing an error. (You can use it as an object property!)
===How to compare strings properties in two object===
collatorlib_property_comparison class can be used to compare properties of two objects
===Should the colon sign be hard-coded or included in a language string?===
The colon sign should not be hard-coded because languages may use a different symbol, or have a space before or after the colon sign. Instead, the colon sign may be included in a language string, though please consider first if it is really necessary. A colon sign in a field label is not recommended. (MDL-12192 is for removing existing hard-coded colon signs.)<syntaxhighlight lang="php">
// Do NOT hard code any punctuation:
$string['fieldname'] = 'Name';

echo get_string('fieldname', 'mod_foobar') . ': ' . $somevalue;

// Slightly better with no hard coded colon:
$string['fieldname'] = 'Name: ';

echo get_string('fieldname', 'mod_foobar') . $somevalue;

// Better with no assumed word order:
$string['fieldname'] = 'Name: {$a->name}';

echo get_string('fieldname', 'mod_foobar', ['name' => $somevalue]);

</syntaxhighlight>

===Changing or creating a new lang string?===
New strings were required in 1.x times when we did not have branches for translations. Since 2.0, for changes in master, structural and/or semantic modification of a string (such as adding or removing {$a} placeholders etc) is acceptable as such strings are highlighted in AMOS as outdated. Of course, if lang pack maintainers do not pay attention to it, mismatch can happen - but this is considered to be the lang pack bug.

When a new string is introduced, the old one should be [[String deprecation | deprecated or removed]] so that unused strings do not accumulate.
:A) Basically I see it as (for master only issues):
:
:# Any change is allowed (semantic or structural). It does not have sense to keep the old string because there won't be use for it when the new Moodle version is released. So we can forget about the old one 100%. Of course if the string is radically different, then it may have sense to create a new, completely different string (and proceed with next point).
:# If for any reason we stop using any string (because a feature is out or because we have decided to create a new different string instead of change existing)), then:
:#* if the old string is really specific (not suitable for reuse) we can safely proceed to delete it,.
:#* If the string belongs to a plugin, IMO we can also proceed to delete it (reuse of plugin strings should not be allowed).
:#* But, if the string is generic (may be reused) and it's not part of a plugin then the string must be deprecated.

:B) And, for issues involving stables, only small semantic changes are allowed. No structural, no deprecation and no deletion. Ever. If still something different must be shown it will be, always, via new string. When applied to master, these changes will also imply the [[String deprecation | standard old string deprecation]]. No CPY instruction will be performed between the old and the new strings (safest, easier, consistent behavior, no matter there are some cases where the instruction may be acceptable. Stop overthinking. Now!).
===User names placeholders===
Some strings may contain placeholders to display the user's name. Common examples include various email templates or welcome messages. Developers may be tempted to use just the user's firstname in such cases, to make the text feel informal and friendlier:
<syntaxhighlight lang="php">
// Do NOT do this.
$string['welcome'] = 'Hello {$a}! Welcome to the course.';
</syntaxhighlight>
But some cultures or institutions may want the full name or last name to be used, rather than the first name. To enable sites to easily customise such strings and have lastname and alternatename placeholders available, the following pattern should be used.

Let the language strings file for the given component define the default display of the name:
<syntaxhighlight lang="php">
$string['welcome'] = 'Hello {$a->firstname}! Welcome to the course.';
</syntaxhighlight>
Then in the file making use of the string:
<syntaxhighlight lang="php">
$namefields = [
'fullname' => fullname($USER),
'alternativefullname' => fullname($USER, true),
];

foreach (\core_user\fields::get_name_fields() as $namefield) {
$namefields[$namefield] = $USER->{$namefield};
}

echo get_string('welcome', 'xyz_component', $namefields);
</syntaxhighlight>
That way, the string can be locally customised and use any combination of the following placeholders as needed: firstnamephonetic, lastnamephonetic, middlename, alternatename, firstname, lastname, fullname and alternativefullname.
<syntaxhighlight lang="php">
// Locally customised variant of the string.
$string['welcome'] = 'Welcome to the course, {$a->fullname}';
</syntaxhighlight>
==See also==
* [[Output_functions]]
* [[Core APIs]]
* [[AMOS]]
* The section 'Language strings' in [[Coding style]]
* [[String deprecation]]
* [[Improving English language strings]]
[[Category:API]]
[[Category:Language]]

Check API

2022-05-11T03:07:30Z

Brendanheywood: /* The result summary */

{{Moodle 3.9}}
== Checks ==
https://tracker.moodle.org/browse/MDL-67818

A Check is a runtime test to make sure something is working well. You can think of Checks as similar and complimentary to the [[PHPUnit]] and [[Acceptance_testing]] but the next layer around them, and done at run time rather than dev time or build time. Like other forms of testing the tests themselves should easy to read and reason about and confirm as valid. As many types of runtime checks can't be unit tested, the checks '''are''' the test.

Checks can be used for a variety of purposes including:
* configuration checks
* security checks
* status checks
* performance checks
* health checks
Moodle has had various types of checks and reports for a long time but they were inconsistent and not machine readable. In Moodle 3.9 they were unified under a single Check API which also enabled plugins to cleanly define their own additional checks. Some examples include:
* a password policy plugin could add a security check
* a custom authentication plugin can add a check that the upstream identity system can be connected to
* a MUC store plugin could add a performance check
Having these centralized and with a consistent contract makes it much easier to ensure the whole system is running smoothly. This makes it possible for an external integration with monitoring systems such as Nagios / Icinga. The Check API is expose as an NPRE compliance cli script:
<syntaxhighlight lang="bash">
php admin/cli/checks.php
</syntaxhighlight>
== Result states of a check ==
{| class="wikitable" border="1"
|-
! Status
! Meaning
! Example
|-
| N/A
| This check doesn't apply - but we may still want to expose the check
| secure cookies setting is disabled because site is not https
|-
| Ok
| A component is configured, working and fast.
| ldap can bind and return value with low latency
|-
| Info
| A component is OK, and we may want to alert the admin to something non urgent such as a deprecation, or something which needs to be checked manually.
|
|-
| Unknown
| We don't yet know the state. eg it may be very expensive so it is run using the Task API and we are waiting for the answer. NOTE: unknown is generally a bad thing and is semantically treated as an error. It is better to have a result of Unknown until the first result happens, and from then on it is Ok, or perhaps Warning or Error if the last known result is getting stale. If you are caching or showing a stale result you should expose the time of this in the result summary text.
| A complex user security report is still running for the first time.
|-
| Warning
| Something is not ideal and should be addressed, eg usability or the speed of the site may be affected, but it may self heal (eg a load spike)
| auth_ldap could bind but was slower than normal
|-
| Error
| Something is wrong with a component and a feature is not working
| auth_ldap could not connect, so users cannot start new sessions
|-
| Critical
| An error which is affecting everyone in a major way
| Cannot read sitedata or the database, the whole site is down
|}
How the various states are then leveraged is a local decision. A typical policy might be that health checks with a status of 'Error' or 'Critical' will page a system administrator 24/7, while 'Warning' only pages during business hours.
== Check types and reports ==
Checks are broken down into types, which roughly map to a step life cycle of your Moodle System
=== Environmental checks (?) ===
/admin/environment.php

These are environmental checks to make sure a Moodle instance is fully setup. This page is potential candidate to move to the new Check API but it slightly more complex than the other checks so hasn't been tackled yet. It would be a deeper change and this is intrinsically part of the install and upgrade system. It is not as critical to refactor as it is already possible for a plugin to declare its own checks, via either declarative [[Environment_checking]] or programmatically with a custom check:

https://docs.moodle.org/dev/Environment_checking#Custom_checks
=== Configuration checks (?) ===
/admin/index.php?cache=1

The Admin notifications page performs a mixture of checks, including security, status, and performance but none are as exhaustive as the checks in the reports below. It also does checks such as whether the web services for the Moodle Mobile App are turned on, and whether the site has been registered. Long term this page could show a summary of all the other check reports, or perhaps just the checks which are not in an OK state.
=== Security checks (security) ===
These are checks to make sure a Moodle instance is hardened correctly for you needs.

/report/security/index.php

https://tracker.moodle.org/browse/MDL-67776
=== Status checks (status) ===
/report/status/index.php

A status check is an 'in the moment' test and covers operational tests such as 'can moodle connect to ldap'. The main core status checks are that cron is running regularly and there has been no failed tasks.

'''IMPORTANT:''' It is critical to understand that Status checks are conceptually defined at the level off the application and not at a lower host level such as a docker container or node in a cluster. Checks should be defined so that whichever instance you ask you should get a consistent answer. DO NOT use the Status Checks to detect containers which need reaped and restarted. If you do, any status error will mean all containers will simultaneously be marked for reaping.

An additional status check is likely the most common type of check a plugin would define. Especially a plugin that connects to a 3rd party service. If the concept of 'OK' requires some sort of threshold, eg network response within 500ms, then that threshold should be managed by the plugin and optionally exposed as a admin setting. The plugin may choose to have different thresholds for Warning / Error / Critical. When designing a new Status Check be mindful that it needs to be actionable, for instance if you are asserting that a remote domain is available and it goes down, which then alerts your infrastructure team, there isn't much they can do about it if it isn't their domain. If it is borderline then make things like this configurable so that each site has to option of tune their own policies of what should be considered an issue or not.

https://tracker.moodle.org/browse/MDL-47271
=== Performance checks (performance) ===
/report/performance/index.php

Each check might simply check for certain settings which are known to slow things down, or it might actually do some sort of test like multiple reads and writes to the db or filesystem to get a performance metric.
=== Health checks (health) ===
/admin/tool/health/

The 'health center' is currently unsupported and contains some very old code. It is conceptually similar to 'status' checks except larger more long terms issues, such as detecting corrupt records. Ideally it is improved and converted to the Check API, see https://tracker.moodle.org/browse/MDL-67228
== Implementing a new check ==
=== A check class ===
And make a new check class in mod/myplugin/classes/check/foobar.php and the only mandatory method is get_result(). By default it will use a set language string but you can override the get_name() method to reuse an existing string.
<syntaxhighlight lang="php">
$string['checkfoobar'] = 'Check the foos to make sure they are barred';
</syntaxhighlight>
<syntaxhighlight lang="php">
<?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {

public function get_action_link(): ?\action_link {
$url = new \moodle_url('/mod/myplugin/dosomething.php'),
return new \action_link($url, get_string('sitepolicies', 'admin'));
}

public function get_result() : result {
if (some_check()) {
$status = result::ERROR;
$summary = get_string('check_foobar_error', 'mod_myplugin');
} else {
$status = result::OK;
$summary = get_string('check_foobar_ok', 'mod_myplugin');
}
$details = get_string('check_details', 'mod_myplugin');
return new result($status, $summary, $details);
}
}
</syntaxhighlight>
=== The result summary ===
The summary could change depending on the result of the check but for a simple check might be a fixed string, not html. Try to keep the summary to 1 line as this might typically be the thing which gets passed through to a paging system and could be truncated.

=== The details ===
Unlike the summary the details is allowed and encouraged to be html. Often it will be a bullet list of a table of the things that it asserted which make up the check.

=== The action link ===
The action link is the place to go to help fix the issue. It should be as specific as possibly, such as a deep link into an admin settings page, and can include hash anchors.
=== lib.php callback ===
First decide if and when your new check(s) should be shown. Should it be present if your plugin is disabled? If you do not want it show if disabled then do not return it in callback below. If you do want it to show when disabled, but the check doesn't much sense then you can return a value of NA.

Next decide on what type of check it should be which determines what report it will be included in. Some checks might make sense to be reused with more than one report, eg it could be in both Status and Performance.

Implement the right callback in lib.php for the report you want to add it to, and return an array (usually with only 1 item) of check objects:

/mod/myplugin/lib.php
<syntaxhighlight lang="php">
function mod_myplugin_security_checks() {
return [new mod_myplugin\check\foobar()];
}
</syntaxhighlight>
=== Multiple instances of checks ===
Checks have been designed to be dynamic so you can return different checks depending on configuration, so auth_ldap would not return a check if the plugin is not enabled. Hypothetically if auth_ldap could be configured with 5 ldap servers then you could return 5 independent checks for each remote connection, each with different labels and information.

If you plan to return multiple instances of a check class, make sure that each instance has a unique id.
<syntaxhighlight lang="php">
function mod_myplugin_security_checks() {
return [
new mod_myplugin\check\foobar('one'),
new mod_myplugin\check\foobar('two'),
];
}
</syntaxhighlight>
Set the internal id in a way which is unique across all instances in your components namespace:
<syntaxhighlight lang="php"><?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {
protected $id = '';

public function __construct($id) {
$this->id = 'foobar' . $id;
}
public function get_id(): string {
return $this->id;
}
...
}
</syntaxhighlight>
=== Make checks as fast as practical ===
As many checks will be run and compiled into a report we want the checks themselves to be simple and as fast as possible. For instance an auth_ldap check while authenticating an end user could have a timeout of 60 seconds, and the check could warn if it takes more than 2 seconds. But the check could have a hard timeout of say 5 seconds and have a result status of ERROR for 5 or more seconds.

=== Lazy loading expensive result details ===
Checks can provide details on a check, such as the complete list of bad records. Generally this type of information might be expensive to produce so you can defer this lookup until get_details() is called specifically rather than setting this in the constructor. It will only be loaded on demand and shown when you drill down into the check details page.
<syntaxhighlight lang="php">
<?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {
public function get_result(): result {
return new foobar_result();
}
}
</syntaxhighlight>

<syntaxhighlight lang="php">
class foobar_result extends \core\check\result {
...
public function get_details(): string {
// Do expensive lookups in here.
}
}
</syntaxhighlight>
For a real example see:

https://github.com/moodle/moodle/blob/master/lib/classes/check/access/riskxss_result.php
=== Asynchronous checks ===
Some checks are by their nature asynchronous. For instance having moodle send an email to itself and then having it processed by the inbound mail handler to make it's properly configured (see https://tracker.moodle.org/browse/MDL-48800). In cases like these please make sure the age or staleness of the check is shown in the summary, and you should also consider turning the result status into a warning if the result is too old. If appropriate make the threshold a configurable admin setting.
==See also==
* [[:en:Performance overview|Performance overview]] user docs

Error pages

2022-04-19T04:10:33Z

Brendanheywood: /* 404 File not found - Web server errors */

This page is for any Moodle admins or developers who want to setup or customize error pages on their Moodle site. There is a variety of ways different errors can happen at different layers in a Moodle stack and because they each happen under different circumstance there cannot be a single way of handling them.
== 404 Normal Moodle errors and exceptions ==
These types of errors are things like you tried to access a course which no longer exists, or access a course which you don't have access to. Because these are "Moodle" pages they are themed correctly and normally there isn't much you want to do to improve them.

If you want to change the wording of the errors see:

https://docs.moodle.org/en/Language_customisation

If you want to change the theme, that is one and the same as normal theme customization:

https://docs.moodle.org/dev/Themes_overview

If a Moodle page throws an exception which isn't captured and results in an error page, and it will also helpfully, but blindly, link into this Moodle wiki where you might find more information on the error. This is great for sites in development or for admin type errors but typically less useful. You may want to divert users to your own support instead of off into a wiki page with generic Moodle information.

This links behavior can be changed and removed in your admin settings here:

/admin/settings.php?section=documentation

https://docs.moodle.org/dev/Exceptions

You can also simply hide the links using a capability too:

https://docs.moodle.org/38/en/Linking_Moodle_and_Docs#Removing_Moodle_Docs_links_for_teachers
== 403 Forbidden - Web server errors ==
For additional security you should not allow access to directory listings and you can also block access to certain files.

For more security you can serve a 404 instead of a 403, and if you prefer you can reuse the themed 404 page below.
=== Apache ===
<syntaxhighlight lang="php">
DirectoryIndex disabled
Options -Indexes
ErrorDocument 403 /error/index.php
</syntaxhighlight>
https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex
== 404 File not found - Web server errors ==
These are for errors which are technically outside of moodle such a 404 file not found at the server level rather than a 404 served by moodle. If you want to theme these then the approach is the same as you normally would for anything non Moodle, configure your web server to show a custom error page. But instead of writing your own, there is a built in Moodle error handler php script so they get the normal Moodle theme applied. You can see an example of what this error page looks like here:

https://moodle.org/errors/
=== Apache ===
You can configure this using:
<syntaxhighlight lang="php">
ErrorDocument 404 /error/index.php
</syntaxhighlight>
=== nginx ===
<syntaxhighlight lang="php">
error_page 404 /error/index.php;
</syntaxhighlight>
== 502 Bad Gateway - Proxy / CDN issues ==
This means some upstream of your moodle server could not connect, ie nginx, varnish, a load balancer, proxy or CDN.

These error pages cannot be themed by at the application level and how to customize them depends on what is proxying moodle.

NOTE: It is very important if you do customize these to make sure you are not overriding the 40x and 50x error pages that Moodle may serve.
== 503 Service unavailable - Fatal Moodle bootstrap errors ==
New in 4.0

These types of errors are more fundamental, and something has gone wrong so Moodle itself can't load, which means the error page can't access the database, MUC, or the language and strings API.

Previously these were difficult to style, and there are many edge cases to capture. This was fixed in this tracker:

https://tracker.moodle.org/browse/MDL-56041

These low level error pages resulted in the 'yellow box' and in 4.0 are now improved to more closely match the boost / bootstrap theme. You can customize this error page by editing this file:

error/plainpage.php

Note: this file cannot contain any Moodle API's and cannot link to files which are served by moodle such as theme files or plugin files. Ideally serve all dependencies such as css and even image inline.
== 503 Moodle under maintenance - Maintenance mode ==
During maintenance mode Moodle will serve a "503 Service Unavailable". This page can be customized:

https://docs.moodle.org/38/en/Maintenance_mode#CLI_maintenance_mode

Task API

2022-03-20T22:45:15Z

Brendanheywood: Reordered to match cron spec order

{{Infobox Project
|name = Task
|state = Integrated
|tracker = MDL-25505
|discussion = https://moodle.org/mod/forum/discuss.php?d=229139
|assignee = Damyon
}}
{{Moodle 2.7}}
== Tasks ==
A task is a unit of work that needs to be done later. Good uses for tasks:
* Run a slow operation in the background
* Run a maintenance task on a regular schedule
In general any operation that takes more than a few seconds should be a candidate for a task.
== Benefits ==
* Better user experience (give them feedback immediately, that their task has been queued)
* Prevent browser timeouts
* Better performance for clusters (tasks can be run on separate, non-webserving cluster node, tasks can run in parallel)
* Failed tasks will be retried
* A better user interface will prevent users queuing multiple tasks, because they thought it had "got stuck".
== Types of task ==
=== Scheduled tasks ===
Scheduled tasks are tasks that will run on a regular schedule. A default schedule can be set, but admins have the ability to change the default schedule if required. Note: Tasks will only run as often as cron is run in Moodle. In 2.7 it is recommended to run cron once per minute to get the benefit from the new task scheduling (don't worry - it will do much less work each time it runs).
=== Adhoc tasks ===
Adhoc tasks are for when you need to queue something to run in the background either immediately where they would be executed as soon as possible, or as a once off task at some future point in time. Adhoc tasks can contain custom data, specific to this specific instance of the task.
== Usage ==
=== Scheduled task usage ===
Scheduled tasks are created by subclassing \core\task\scheduled_task. They also require an entry in "db/tasks.php" for your plugin.

1. Create a subclass of \core\task\scheduled_task that contains your code to run in a schedule (within "<pluginroot>/classes/task/cut_my_toe_nails.php" for the autoloading mechanism to pick up).
<syntaxhighlight lang="php">
namespace mod_hygene\task;

/**
* An example of a scheduled task.
*/
class cut_my_toe_nails extends \core\task\scheduled_task {

/**
* Return the task's name as shown in admin screens.
*
* @return string
*/
public function get_name() {
return get_string('cutmytoenails', 'mod_hygene');
}

/**
* Execute the task.
*/
public function execute() {
// Apply fungus cream.
// Apply chainsaw.
// Apply olive oil.
}
}
</syntaxhighlight>
2. Create entry in db/tasks.php for your plugin (then a version bump to your plugin will install the task):
<syntaxhighlight lang="php">
$tasks = [
[
'classname' => 'mod_hygene\task\cut_my_toe_nails',
'blocking' => 0,
'minute' => '30',
'hour' => '17',
'day' => '*',
'month' => '1,7',
'dayofweek' => '0',
],
];
</syntaxhighlight>
The fields required in the db/tasks.php file are:
* classname - the fully namespaced classname of your task. This needs to comply with the autoloading rules.
* blocking - if this is set to 1, no other scheduled task will run at the same time as this task. Do not set this to 1 unless you really need it as it will impact the performance of the task queue.
* minute, hour, day, dayofweek, month - This is the default schedule for running the task. The syntax matches the syntax of unix cron. Starting with Moodle 2.8, the minute and hour values accept a special 'R' syntax which causes a random value to be set in the database at install time (useful to avoid overloading web services).
3. Increase the version number in version.php for your plugin and visit the Site "Administration -> Notifications" page to install the new task.

4. (Optional - since 2.8) Run this task even when the plugin is disabled. In rare cases, you may want the scheduled tasks for a plugin to run, even when the plugin is disabled. Some of the enrolment plugins do this to clean up data. If this is the case, the scheduled task must override the "get_run_if_component_disabled()" method and return true instead of false. If they do not do this, the scheduled task will not be run while the plugin is disabled.

When called from the command line for testing purposes errors can be hidden and a misleading error about locks can be displayed. To get at the details of an error Moodle 3.2 supports a showdebugging option that will show the actual errors. Read about it here
https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks

==== Cron syntax examples ====
* '''minute''' - Minute field for task schedule.
** "*" - Every minute
** "*/5" - Every 5 minutes
** "2-10" - Every minute between 2 and 10 minutes past the hour (inclusive)
** "2,6,9" - 2, 6 and 9 minutes past the hour
** "R" - A random value between 0 and 59 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* '''hour''' - Hour field for task schedule.
** "*" - Every hour
** "*/2" - Every 2 hours
** "2-10" - Every hour from 2am until 10am (inclusive)
** "2,6,9" - 2am, 6am and 9am
** "R" - A random value between 0 and 23 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* '''day''' - Day of month field for task schedule.
** "*" - Every day
** "*/2" - Every 2nd day
** "1" - The first of every month
** "1,15" - The first and fifteenth of every month
* '''month''' - Month field for task schedule.
** "*" - Every month
** "*/2" - Every second month
** "1" - Every January
** "1,5" - Every January and May
* '''dayofweek''' - Day of week field for task schedule.
** "*" - Every day
** "0" - Every Sunday
** "6" - Every Saturday
** "1,5" - Every Monday and Friday
** "1-5" - Every weekday
=== Adhoc task usage ===
This is even easier than scheduled tasks.

1. Create a subclass of \core\task\adhoc_task that contains your code to run in the background.
<syntaxhighlight lang="php">
class take_over_the_world extends \core\task\adhoc_task {
public function execute() {
// gain 100,000,000 friends on facebook.
// crash the stock market.
// run for president.

// Get the custom data.
$data = $this->get_custom_data();
}
}
</syntaxhighlight>
2. Create an instance of the task and queue it (adding custom data if required).
<syntaxhighlight lang="php">
// create the instance
$domination = new take_over_the_world();
// set blocking if required (it probably isn't)
// $domination->set_blocking(false);
// add custom data
$domination->set_custom_data(array(
'plansfortomorrownight' => 'The same thing we do every night, Pinky!'
));

// queue it
\core\task\manager::queue_adhoc_task($domination);

// profit
</syntaxhighlight>
3. There is no 3.

By default, your task will run as admin. If you would like it to run as a specific user, call:
<syntaxhighlight lang="php">
$task->set_userid($user->id);
</syntaxhighlight>
==== Set a task to run at a future time ====
<syntaxhighlight lang="php">
$task = new my_future_task();
$task->set_next_run_time($futuretime);
\core\task\manager::queue_adhoc_task($task);
</syntaxhighlight>
==== Ignore duplicate adhoc tasks ====
{{Moodle 3.3}}

If you pass true as the second argument $checkforexisting and a task with the same classname, component and customdata is already scheduled then it will not schedule a new task.
<syntaxhighlight lang="php">
\core\task\manager::queue_adhoc_task($task, true);
</syntaxhighlight>
==== Rescheduling an adhoc task ====
{{Moodle 3.7}}

A method to allow queuing or rescheduling of an existing scheduled task was added. This allows an existing task to be updated or queued as required. You can use this to implement "debounce" and its a good pattern do some expensive processing after some action, but the action maybe repeated several times and it keeps pushing back the task processing until there is lull in activity.
<syntaxhighlight lang="php">
$task = new heavy_debounced_task();
$task->set_next_run_time(time() + 5 * MINSECS);
\core\task\manager::reschedule_or_queue_adhoc_task($task);
</syntaxhighlight>
=== Failures ===
A task, either scheduled or adhoc can sometimes fail. An example would be updating an RSS field when the network is temporarily down. This is handled by the task system automatically - all the failing task needs to do is throw an exception. The task will be retried after 1 minute. If the task keeps failing, the retry algorithm will add more time between each successive attempts up to a max of 24 hours.
=== Caches ===
There is one special case that needs to be considered with this new system. If a particular scheduled or adhoc task runs for a long time and updates many DB records - particularly something related to enrolment - the next task in the queue may suffer because various API's in moodle use static caching to speed up requests, but assume that the data will not change much between the start and end of the request. In this case, you can force the cron to exit after running a task that has done many DB updates. The next cron will be run in the next minute, and will have all static caches cleared because it's a new process. To do this call \core\task\manager::clear_static_caches();
=== Security ===
When scheduling a task to run in the background - or creating a scheduled task, the task will run in the context of the cron user (see "cron_setup_user()"). If you need to perform access checks in your background task, you should pass the userid/context in custom_data and then pass that userid to the access check functions ("require_capability('moodle/course:update', $context, $userid)").
=== Legacy cron ===
The older syntax of cron.php or modname_cron() is still supported - but is not as good as this new API. This is because:
* the legacy cron functions run serially - a long running cron in one plugin will hold up the other plugins crons
* the legacy cron functions are fragile - a failure in one cron in one plugin will prevent the cron in other functions from running at all
* the scheduling cannot be changed by admins
Note that Legacy cron has been deprecated for Moodle 3.5 (MDL-52846) and will be deleted for Moodle 3.9 (MDL-61165).
=== Generating output ===
Since Moodle 3.5 it is safe to use the [[Output_API]] in cron tasks. Prior to this there may be cases where the Output API has not been initialised.
These issues were addressed in MDL-61800 and were also backported to Moodle 3.4.3, and Moodle 3.3.3.

In order to improve debugging information, it is good practice to call ''mtrace'' to log what's going on within a task execution:
<syntaxhighlight lang="php">
class my_task extends \core\task\scheduled_task {

public function execute() {
mtrace("My task started");

// do some work...

mtrace("My task finished");
}

}
</syntaxhighlight>
== For Admins ==
Admins have a new screen where they can adjust the schedules for any scheduled task. They can also reset any scheduled task to its default schedule.
[[File:task_admin_screenshot.png|Screenshot of admin page]]
NOTE: You can also run [https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks Scheduled tasks] via command line, individually.
== Specification ==
The specification for this feature is here: https://docs.moodle.org/dev/Scheduled_Tasks_Proposal

Task API

2022-03-20T22:41:35Z

Brendanheywood: Added weekday example

{{Infobox Project
|name = Task
|state = Integrated
|tracker = MDL-25505
|discussion = https://moodle.org/mod/forum/discuss.php?d=229139
|assignee = Damyon
}}
{{Moodle 2.7}}
== Tasks ==
A task is a unit of work that needs to be done later. Good uses for tasks:
* Run a slow operation in the background
* Run a maintenance task on a regular schedule
In general any operation that takes more than a few seconds should be a candidate for a task.
== Benefits ==
* Better user experience (give them feedback immediately, that their task has been queued)
* Prevent browser timeouts
* Better performance for clusters (tasks can be run on separate, non-webserving cluster node, tasks can run in parallel)
* Failed tasks will be retried
* A better user interface will prevent users queuing multiple tasks, because they thought it had "got stuck".
== Types of task ==
=== Scheduled tasks ===
Scheduled tasks are tasks that will run on a regular schedule. A default schedule can be set, but admins have the ability to change the default schedule if required. Note: Tasks will only run as often as cron is run in Moodle. In 2.7 it is recommended to run cron once per minute to get the benefit from the new task scheduling (don't worry - it will do much less work each time it runs).
=== Adhoc tasks ===
Adhoc tasks are for when you need to queue something to run in the background either immediately where they would be executed as soon as possible, or as a once off task at some future point in time. Adhoc tasks can contain custom data, specific to this specific instance of the task.
== Usage ==
=== Scheduled task usage ===
Scheduled tasks are created by subclassing \core\task\scheduled_task. They also require an entry in "db/tasks.php" for your plugin.

1. Create a subclass of \core\task\scheduled_task that contains your code to run in a schedule (within "<pluginroot>/classes/task/cut_my_toe_nails.php" for the autoloading mechanism to pick up).
<syntaxhighlight lang="php">
namespace mod_hygene\task;

/**
* An example of a scheduled task.
*/
class cut_my_toe_nails extends \core\task\scheduled_task {

/**
* Return the task's name as shown in admin screens.
*
* @return string
*/
public function get_name() {
return get_string('cutmytoenails', 'mod_hygene');
}

/**
* Execute the task.
*/
public function execute() {
// Apply fungus cream.
// Apply chainsaw.
// Apply olive oil.
}
}
</syntaxhighlight>
2. Create entry in db/tasks.php for your plugin (then a version bump to your plugin will install the task):
<syntaxhighlight lang="php">
$tasks = [
[
'classname' => 'mod_hygene\task\cut_my_toe_nails',
'blocking' => 0,
'minute' => '30',
'hour' => '17',
'day' => '*',
'month' => '1,7',
'dayofweek' => '0',
],
];
</syntaxhighlight>
The fields required in the db/tasks.php file are:
* classname - the fully namespaced classname of your task. This needs to comply with the autoloading rules.
* blocking - if this is set to 1, no other scheduled task will run at the same time as this task. Do not set this to 1 unless you really need it as it will impact the performance of the task queue.
* minute, hour, day, dayofweek, month - This is the default schedule for running the task. The syntax matches the syntax of unix cron. Starting with Moodle 2.8, the minute and hour values accept a special 'R' syntax which causes a random value to be set in the database at install time (useful to avoid overloading web services).
3. Increase the version number in version.php for your plugin and visit the Site "Administration -> Notifications" page to install the new task.

4. (Optional - since 2.8) Run this task even when the plugin is disabled. In rare cases, you may want the scheduled tasks for a plugin to run, even when the plugin is disabled. Some of the enrolment plugins do this to clean up data. If this is the case, the scheduled task must override the "get_run_if_component_disabled()" method and return true instead of false. If they do not do this, the scheduled task will not be run while the plugin is disabled.

When called from the command line for testing purposes errors can be hidden and a misleading error about locks can be displayed. To get at the details of an error Moodle 3.2 supports a showdebugging option that will show the actual errors. Read about it here
https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks

==== Cron syntax examples ====
* day - Day of month field for task schedule.
** "*" - Every day
** "*/2" - Every 2nd day
** "1" - The first of every month
** "1,15" - The first and fifteenth of every month
* dayofweek - Day of week field for task schedule.
** "*" - Every day
** "0" - Every Sunday
** "6" - Every Saturday
** "1,5" - Every Monday and Friday
** "1-5" - Every weekday
* hour - Hour field for task schedule.
** "*" - Every hour
** "*/2" - Every 2 hours
** "2-10" - Every hour from 2am until 10am (inclusive)
** "2,6,9" - 2am, 6am and 9am
** "R" - A random value between 0 and 23 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* minute - Minute field for task schedule.
** "*" - Every minute
** "*/5" - Every 5 minutes
** "2-10" - Every minute between 2 and 10 minutes past the hour (inclusive)
** "2,6,9" - 2, 6 and 9 minutes past the hour
** "R" - A random value between 0 and 59 is chosen at task install time (or when reset to default) [Supported since Moodle 2.8]
* month - Month field for task schedule.
** "*" - Every month
** "*/2" - Every second month
** "1" - Every January
** "1,5" - Every January and May
=== Adhoc task usage ===
This is even easier than scheduled tasks.

1. Create a subclass of \core\task\adhoc_task that contains your code to run in the background.
<syntaxhighlight lang="php">
class take_over_the_world extends \core\task\adhoc_task {
public function execute() {
// gain 100,000,000 friends on facebook.
// crash the stock market.
// run for president.

// Get the custom data.
$data = $this->get_custom_data();
}
}
</syntaxhighlight>
2. Create an instance of the task and queue it (adding custom data if required).
<syntaxhighlight lang="php">
// create the instance
$domination = new take_over_the_world();
// set blocking if required (it probably isn't)
// $domination->set_blocking(false);
// add custom data
$domination->set_custom_data(array(
'plansfortomorrownight' => 'The same thing we do every night, Pinky!'
));

// queue it
\core\task\manager::queue_adhoc_task($domination);

// profit
</syntaxhighlight>
3. There is no 3.

By default, your task will run as admin. If you would like it to run as a specific user, call:
<syntaxhighlight lang="php">
$task->set_userid($user->id);
</syntaxhighlight>
==== Set a task to run at a future time ====
<syntaxhighlight lang="php">
$task = new my_future_task();
$task->set_next_run_time($futuretime);
\core\task\manager::queue_adhoc_task($task);
</syntaxhighlight>
==== Ignore duplicate adhoc tasks ====
{{Moodle 3.3}}

If you pass true as the second argument $checkforexisting and a task with the same classname, component and customdata is already scheduled then it will not schedule a new task.
<syntaxhighlight lang="php">
\core\task\manager::queue_adhoc_task($task, true);
</syntaxhighlight>
==== Rescheduling an adhoc task ====
{{Moodle 3.7}}

A method to allow queuing or rescheduling of an existing scheduled task was added. This allows an existing task to be updated or queued as required. You can use this to implement "debounce" and its a good pattern do some expensive processing after some action, but the action maybe repeated several times and it keeps pushing back the task processing until there is lull in activity.
<syntaxhighlight lang="php">
$task = new heavy_debounced_task();
$task->set_next_run_time(time() + 5 * MINSECS);
\core\task\manager::reschedule_or_queue_adhoc_task($task);
</syntaxhighlight>
=== Failures ===
A task, either scheduled or adhoc can sometimes fail. An example would be updating an RSS field when the network is temporarily down. This is handled by the task system automatically - all the failing task needs to do is throw an exception. The task will be retried after 1 minute. If the task keeps failing, the retry algorithm will add more time between each successive attempts up to a max of 24 hours.
=== Caches ===
There is one special case that needs to be considered with this new system. If a particular scheduled or adhoc task runs for a long time and updates many DB records - particularly something related to enrolment - the next task in the queue may suffer because various API's in moodle use static caching to speed up requests, but assume that the data will not change much between the start and end of the request. In this case, you can force the cron to exit after running a task that has done many DB updates. The next cron will be run in the next minute, and will have all static caches cleared because it's a new process. To do this call \core\task\manager::clear_static_caches();
=== Security ===
When scheduling a task to run in the background - or creating a scheduled task, the task will run in the context of the cron user (see "cron_setup_user()"). If you need to perform access checks in your background task, you should pass the userid/context in custom_data and then pass that userid to the access check functions ("require_capability('moodle/course:update', $context, $userid)").
=== Legacy cron ===
The older syntax of cron.php or modname_cron() is still supported - but is not as good as this new API. This is because:
* the legacy cron functions run serially - a long running cron in one plugin will hold up the other plugins crons
* the legacy cron functions are fragile - a failure in one cron in one plugin will prevent the cron in other functions from running at all
* the scheduling cannot be changed by admins
Note that Legacy cron has been deprecated for Moodle 3.5 (MDL-52846) and will be deleted for Moodle 3.9 (MDL-61165).
=== Generating output ===
Since Moodle 3.5 it is safe to use the [[Output_API]] in cron tasks. Prior to this there may be cases where the Output API has not been initialised.
These issues were addressed in MDL-61800 and were also backported to Moodle 3.4.3, and Moodle 3.3.3.

In order to improve debugging information, it is good practice to call ''mtrace'' to log what's going on within a task execution:
<syntaxhighlight lang="php">
class my_task extends \core\task\scheduled_task {

public function execute() {
mtrace("My task started");

// do some work...

mtrace("My task finished");
}

}
</syntaxhighlight>
== For Admins ==
Admins have a new screen where they can adjust the schedules for any scheduled task. They can also reset any scheduled task to its default schedule.
[[File:task_admin_screenshot.png|Screenshot of admin page]]
NOTE: You can also run [https://docs.moodle.org/32/en/Administration_via_command_line#Scheduled_tasks Scheduled tasks] via command line, individually.
== Specification ==
The specification for this feature is here: https://docs.moodle.org/dev/Scheduled_Tasks_Proposal

Adding a web service to a plugin

2022-03-16T03:07:25Z

Brendanheywood: /* See also */

{{Moodle_2.0}}
= Quick start =
== File structure ==
The file structure is explained in these two documents:
* [[Web_services_API|Web services API]]
* [[External_functions_API|External function API]]
= Tutorial =
We will create a web service into a local plugin. This service will contain one ''local_myplugin_create_groups($groups)'' web service function. This web service function will create a group into a Moodle course.
== Write the specification documentation ==
Before starting coding, let's identify our needs writing some short specification documents.
=== functional specification===
''local_myplugin_create_groups($groups)'' will take a list of groups as parameters and it will return the same groups with their newly created id. If ever one group creation fails, the function will throw an exception, and no creation will happen.
=== technical specification ===
* '''the core function the external function will call''': ''groups_create_group()'' from [http://cvs.moodle.org/moodle/group/ /group/lib.php].
* '''the parameter types''': a list of object. This object are groups, with id/name/courseid.
* '''the returned value types''': a list of objects (groups) with their id.
* '''the user capabilities to check''': ''moodle/course:managegroups''
== Write a simple test client ==
The first thing you should code is a web service test client. You will often discover use cases that you didn't think about. We are not showing any test client code here, see [[Creating_a_web_service_client|How to create a web service client]].
== Declare the service ==
This step is optional. You can pre-build a service including any web service functions, so the Moodle administrator doesn't need to do it. Add into /local/myplugin/db/services.php:
<syntaxhighlight lang="php">
$services = array(
'mypluginservice' => array( // the name of the web service
'functions' => array ('local_myplugin_create_groups'), // web service functions of this service
'requiredcapability' => '', // if set, the web service user need this capability to access
// any function of this service. For example: 'some/capability:specified'
'restrictedusers' => 0, // if enabled, the Moodle administrator must link some user to this service
// into the administration
'enabled' => 1, // if enabled, the service can be reachable on a default installation
'shortname' => '', // optional – but needed if restrictedusers is set so as to allow logins.
'downloadfiles' => 0, // allow file downloads.
'uploadfiles' => 0 // allow file uploads.
)
);
</syntaxhighlight>
Note: it is not possible for an administrator to add/remove any function from a pre-built service.
== Declare the web service function ==
Following the [[Web_services_API|Web service API]], you must declare the web service function in the ''local/myplugin/db/services.php'' file.
<syntaxhighlight lang="php">
$functions = array(
'local_myplugin_create_groups' => array( //web service function name
'classname' => 'local_myplugin_external', //class containing the external function OR namespaced class in classes/external/XXXX.php
'methodname' => 'create_groups', //external function name
'classpath' => 'local/myplugin/externallib.php', //file containing the class/external function - not required if using namespaced auto-loading classes.
// defaults to the service's externalib.php
'description' => 'Creates new groups.', //human readable description of the web service function
'type' => 'write', //database rights of the web service function (read, write)
'ajax' => true, // is the service available to 'internal' ajax calls.
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE) // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
'capabilities' => '', // comma separated list of capabilities used by the function.
),
);
</syntaxhighlight>
Web service functions should match the [https://docs.moodle.org/dev/Web_services_Roadmap#Naming_convention naming convention].
== Write the external function descriptions ==
Every web service function is mapped to an external function. External function are described in the [[External_functions_API|External functions API]].
Each external function is written with two other functions describing the parameters and the return values. These description functions are used by web service servers to:
* validate the web service function parameters
* validate the web service function returned values
* build WSDL files or other protocol documents
These two description functions are located in the same file and the same class mentioned in local/myplugin/db/services.php.

Thus for the web service function '''local_myplugin_create_groups()''', we need write a class named '''local_myplugin_external''' in the file '''local/myplugin/externallib.php'''. The class will contain:
* create_groups(...)
* create_groups_parameters()
* create_groups_return()
=== create_groups_parameters() ===
<syntaxhighlight lang="php">
require_once("$CFG->libdir/externallib.php");

class local_myplugin_external extends external_api {

/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
)
);
}
</syntaxhighlight>
A web service function without parameters will have a parameter description function like that:
<syntaxhighlight lang="php">
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function functionname_parameters() {
return new external_function_parameters(
array(
//if I had any parameters, they would be described here. But I don't have any, so this array is empty.
)
);
}
</syntaxhighlight>
A parameter can be described as:
* a list => external_multiple_structure
* an object => external_single_structure
* a primary type => external_value

Our create_groups() function expects one parameter named ''groups'', so we will first write:
<syntaxhighlight lang="php">
/**
* Returns description of method parameters
* @return external_function_parameters
*/
public static function create_groups_parameters() {
return new external_function_parameters(
array(
'groups' => ...

)
);
}
</syntaxhighlight>
This ''groups'' parameter is a list of group. So we will write :
<syntaxhighlight lang="php">
'groups' => new external_multiple_structure(
...
)
</syntaxhighlight>
An external_multiple_structure object (list) can be constructed with:
* ''external_multiple_structure'' (list)
* ''external_single_structure'' (object)
* ''external_value'' (primary type).

For our function it will be a ''external_single_structure'':
<syntaxhighlight lang="php">
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
</syntaxhighlight>
Thus we obtain :
<syntaxhighlight lang="php">
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => ...,
'name' => ...,
'description' => ...,
'enrolmentkey' => ...,
)
)
)
</syntaxhighlight>
Each group values is a ''external_value'' (primary type):
* ''courseid'' is an integer
* ''name'' is a string (text only, not tag)
* ''description'' is a string (can be anything)
* ''enrolmentkey'' is also a string (can be anything)

We add them to the description :
<syntaxhighlight lang="php">
'groups' => new external_multiple_structure(
new external_single_structure(
array(
'courseid' => new external_value(PARAM_INT, 'id of course'), //the second argument is a human readable description text. This text is displayed in the automatically generated documentation.
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
)
</syntaxhighlight>
=== create_groups_returns() ===
It's similar to create_groups_parameters(), but instead of describing the parameters, it describes the return values.
<syntaxhighlight lang="php">
public static function create_groups_returns() {
return new external_multiple_structure(
new external_single_structure(
array(
'id' => new external_value(PARAM_INT, 'group record id'),
'courseid' => new external_value(PARAM_INT, 'id of course'),
'name' => new external_value(PARAM_TEXT, 'multilang compatible name, course unique'),
'description' => new external_value(PARAM_RAW, 'group description text'),
'enrolmentkey' => new external_value(PARAM_RAW, 'group enrol secret phrase'),
)
)
);
}
</syntaxhighlight>
=== Required, Optional or Default value ===
A value can be VALUE_REQUIRED, VALUE_OPTIONAL, or VALUE_DEFAULT. If not mentioned, a value is VALUE_REQUIRED by default.
<syntaxhighlight lang="php">
'yearofstudy' => new external_value(PARAM_INT, 'year of study',VALUE_DEFAULT, 1979),
</syntaxhighlight>
* VALUE_REQUIRED - if the value is not supplied => the server throws an error message
* VALUE_OPTIONAL - if the value is not supplied => the value is ignored. Note that VALUE_OPTIONAL can't be used in top level parameters, it must be used only within array/objects key definition. If you need top level Optional parameters you should use VALUE_DEFAULT instead.
* VALUE_DEFAULT - if the value is not supplied => the default value is used
Note: Because some web service protocols are strict about the number and types of arguments - it is not possible to specify an optional parameter as one of the top-most parameters for a function. Examples:

Not cool:
<syntaxhighlight lang="php">
public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'chocolatechips' => new external_value(PARAM_BOOL, PARAM_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ERROR! top level optional parameter!!!
)
);
}

</syntaxhighlight>
Cool:
<syntaxhighlight lang="php">

public static function get_biscuit_parameters() {
return new external_function_parameters(
array(
'ifeellike' => new external_single_structure(
array(
'chocolatechips' => new external_value(PARAM_BOOL, VALUE_REQUIRED),
'glutenfree' => new external_value(PARAM_BOOL, PARAM_DEFAULT, false),
'icingsugar' => new external_value(PARAM_BOOL, VALUE_OPTIONAL), // ALL GOOD!! We have nested the params in a external_single_structure.
)
)
)
);
}

</syntaxhighlight>
== Implement the external function ==
We declared our web service function and we defined the external function parameters and return values. We will now implement the external function:
<syntaxhighlight lang="php">
/**
* Create groups
* @param array $groups array of group description arrays (with keys groupname and courseid)
* @return array of newly created groups
*/
public static function create_groups($groups) { //Don't forget to set it as static
global $CFG, $DB;
require_once("$CFG->dirroot/group/lib.php");

$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));

$transaction = $DB->start_delegated_transaction(); //If an exception is thrown in the below code, all DB queries in this code will be rollback.

$groups = array();

foreach ($params['groups'] as $group) {
$group = (object)$group;

if (trim($group->name) == '') {
throw new invalid_parameter_exception('Invalid group name');
}
if ($DB->get_record('groups', array('courseid'=>$group->courseid, 'name'=>$group->name))) {
throw new invalid_parameter_exception('Group with the same name already exists in the course');
}

// now security checks
$context = get_context_instance(CONTEXT_COURSE, $group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);

// finally create the group
$group->id = groups_create_group($group, false);
$groups[] = (array)$group;
}

$transaction->allow_commit();

return $groups;
}
</syntaxhighlight>
=== Parameter validation ===
<syntaxhighlight lang="php">
$params = self::validate_parameters(self::create_groups_parameters(), array('groups'=>$groups));
</syntaxhighlight>
This ''validate_parameters'' function validates the external function parameters against the description. It will return an exception if some required parameters are missing, if parameters are not well-formed, and check the parameters validity. It is essential that you do this call to avoid potential hack.

'''Important:''' the parameters of the external function and their declaration in the description '''must be the same order'''. In this example we have only one parameter named $groups, so we don't need to worry about the order.
=== Context and Capability checks ===
<syntaxhighlight lang="php">
/// now security checks
$context = context_course::instance($group->courseid);
self::validate_context($context);
require_capability('moodle/course:managegroups', $context);
</syntaxhighlight>
Note: validate_context() is required in all external functions before operating on any data belonging to a context. This function does sanity and security checks on the context that was passed to the external function - and sets up the global $PAGE and $OUTPUT for rendering return values. Do NOT use require_login(), or $PAGE->set_context() in an external function.
=== Exceptions===
You can throw exceptions. These are automatically handled by Moodle web service servers.
<syntaxhighlight lang="php">
//Note: it is good practice to add detailled information in $debuginfo,
// and only send back a generic exception message when Moodle DEBUG mode < NORMAL.
// It's what we do here throwing the invalid_parameter_exception($debug) exception
throw new invalid_parameter_exception('Group with the same name already exists in the course');
</syntaxhighlight>
=== Correct return values ===
The return values will be validated by the Moodle web service servers:
* return values contain some values not described => these values will be skipped.
* return values miss some required values (VALUE_REQUIRED) => the server will return an error.
* return values types don't match the description (int != PARAM_ALPHA) => the server will return an error
'''Note:''' cast all your returned objects into arrays.
== Making web service accessible through Apache Thrift ==
This step is optional. If you wish to generate SDK for different programming languages and platforms using [http://thrift.apache.org Apache Thrift framework] then [https://bitbucket.org/hhteam/moodle_thrift_tools/wiki/Home Moodle Thrift tools] can help you.

Two steps should be performed:
* Generate .thrift files for Moodle API using thriftgenerator script.
* Generate thrift handlers for PHP and copy the php files generated to your plugin source tree.
It is also recommended to include thrift files into the distribution of your plugin in order to simplify creation of client bindings for the users of your API.

That's it. Now the web API of your plugin is accessible through Apache Thrift Framework.
== Bump the plugin version ==
Edit your local/myplugin/version.php and increase the plugin version. This should trigger a Moodle upgrade and the new web service should be available in the administration (''Administration > Plugins > Web Services > Manage services'')
== Deprecation ==
External functions deprecation process is slightly different from the standard deprecation. If you are interested in deprecating any of your external functions you should '''also''' (apart from the applicable points detailed in the [[Deprecation|standard deprecation docs]]) create a <tt>FUNCTIONNAME_is_deprecated()</tt> method in your external function class. Return true if the external function is deprecated. This is an example:
<syntaxhighlight lang="php"> /**
* Mark the function as deprecated.
* @return bool
*/
public static function create_groups_is_deprecated() {
return true;
}
</syntaxhighlight>
=See also=
* [[Web_services|Web services developer documentation]]
* [[:en:Web_services|Web services user documentation]]
* [[Creating_a_web_service_client|Implement a web service client]]
* Code example: [https://gist.github.com/timhunt/51987ad386faca61fe013904c242e9b4 Adding a web service, using APIs] by (Tim Hunt)
Specification:
* [[External services security]]
* [[External services description]]
* [[Session locks#Read only sessions in web services]]
[[Category:Web Services]]
[[Category:Plugins]]

Web services API

2022-03-16T03:07:10Z

Brendanheywood: /* See also */

==Overview==
The Web services API allows you to expose your plugin's functions (usually [[External functions API|external functions]]) as Web services.

Once you have done this, your plugin's functions will be accessible to other systems through Web services using one of a number of protocols, like XML-RPC, REST or SOAP.

Exposing functions as Web service functions is done in one file called services.php.
== services.php ==
* This file can be added to the '''db''' sub-folder of your [[Frankenstyle#Plugin_types|plugin]].
* This file contains one or two arrays. The first array declares your web service functions. Each of these declarations reference a function in your module (usually [[External functions API|an external function]]).
<syntaxhighlight lang="php">
<?php

$functions = array(
'local_PLUGINNAME_FUNCTIONNAME' => array( // local_PLUGINNAME_FUNCTIONNAME is the name of the web service function that the client will call.
'classname' => 'component\external[\optional\sub\namespaces\classname', // create this class in componentdir/classes/external
'methodname' => 'FUNCTIONNAME', // implement this function into the above class
'description' => 'This documentation will be displayed in the generated API documentation
(Administration > Plugins > Webservices > API documentation)',
'type' => 'write', // the value is 'write' if your function does any database change, otherwise it is 'read'.
'ajax' => true, // true/false if you allow this web service function to be callable via ajax
'capabilities' => 'moodle/xxx:yyy, addon/xxx:yyy', // List the capabilities required by the function (those in a require_capability() call) (missing capabilities are displayed for authorised users and also for manually created tokens in the web interface, this is just informative).
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE, 'local_mobile') // Optional, only available for Moodle 3.1 onwards. List of built-in services (by shortname) where the function will be included. Services created manually via the Moodle interface are not supported.
)
);
</syntaxhighlight>
* The second, optional array declares the pre-built services.
<syntaxhighlight lang="php">
// OPTIONAL
// During the plugin installation/upgrade, Moodle installs these services as pre-build services.
// A pre-build service is not editable by administrator.
$services = array(
'MY SERVICE' => array(
'functions' => array ('local_PLUGINNAME_FUNCTIONNAME'),
'restrictedusers' => 0, // if 1, the administrator must manually select which user can use this service.
// (Administration > Plugins > Web services > Manage services > Authorised users)
'enabled'=>1, // if 0, then token linked to this service won't work
'shortname'=>'myservice' //the short name used to refer to this service from elsewhere including when fetching a token
)
);
</syntaxhighlight>
Don't forget to increment the version number in the version.php file of your plugin whenever services.php changes, otherwise Moodle will not detect the changes.
== Detailed tutorial ==
A more detailed tutorial for this system can be found at the following page:
* [[Adding a web service to a plugin]]
Among other things, that tutorial explains how you define the parameters and return value of your function.
==See also==
* [[Core APIs]]
* [[External functions API]]
* [[Web services API Changes]]
* [[Session locks#Read only sessions in web services]]
[[Category:Web_Services]]
[[Category:API]]

Session locks

2022-03-16T03:04:43Z

Brendanheywood:

When you create a normal moodle page and include config.php then by default a large amount of moodle bootstrapping runs and you will have the $SESSION global setup for you. This is a safe starting assumption but when writing higher performance code it is better to reduce or eliminate the session locks where possible.
== Debugging session lock issues ==
If you have 'pages that are slow' and you profile them and see that you are waiting on a lock to free up then this is potentially an easy thing to fix to improve your overall performance.
$CFG->debugsessionlock = 5; // Time in seconds
When a session is locked for more N seconds a debugging call will be made with details of what the other page was which is holding onto the lock.
== Session unlocking ==
By default core assumes that you might need to mutate the $SESSION object so it will hold a lock on the session until the page finished and a shutdown handler will release the session lock.
If you are working on any page which is potentially long running, then you should cleanly separate logic which runs early which could mutate the session from the long running processin code and unlock the session.
\core\session\manager::write_close();
== Read only session in pages ==
For this to work, READONLY sessions must be enabled as well needing your code to support it. Not all session

If you know ahead of time that you will never mutate the session, but you still need to be able to read it, then you can declare your page to be read only. This will mean your page will never block the session in another http request.
define('READ_ONLY_SESSION', true);
== Read only sessions in web services ==
The same is possible in web services. When you declare your web service you can specify it will not need a session lock:
'core_message_get_unread_conversations_count' => array(
'classname' => 'core_message_external',
'methodname' => 'get_unread_conversations_count',
'classpath' => 'message/externallib.php',
'description' => 'Retrieve the count of unread conversations for a given user',
'type' => 'read',
'ajax' => true,
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE),
'''<nowiki>'readonlysession' => true, // We don't modify the session.</nowiki>'''
),
== No session at all ==
If your script doesn't actually need $SESSION in the first place then save even more processing and locks by declaring:
define('NO_MOODLE_COOKIES', true);
== No config is needed ==
Going to the absolute extreme, if you do not even need the full moodle bootstrap to run then you can skip it via:
define('ABORT_AFTER_CONFIG', true);

Performance and scalability

2022-03-16T02:45:25Z

Brendanheywood: /* Be cautious about other external calls */

'''Performance''' is about allowing Moodle to support as many users as possible with a certain amount of hardware.

Of course you can always always buy a bigger server. '''Scalability''' means ensuring that if you buy a server that is about twice as powerful, it can then handle about twice as much load.

This page is part of the [[Coding|Moodle coding guidelines]]
==Writing code that scales and performs==
===Every page should only use a fixed number of database queries===
* Be very suspicious if you ever see database code inside a loop.
** This is sometimes hard to spot if the database access is hidden in a function.
* Instead use JOINs and subqueries. (<tt>get_records_sql</tt>, <tt>get_recordset_sql</tt>, etc.).
** Or look for a Moodle API function that gets the information you want as efficiently as possible. (For example, <tt>get_users_by_capability</tt>)
** See the [[Database|Database guidelines]] for how to write SQL that will work on all our supported databases.

===Limit the amount of RAM each page requires to generate===
* Large reports should be broken into pages of a fixed size.
* Processing large amounts of data from the database should be handled with a recordset (when you cannot do all the processing in the database with SQL) and using [https://github.com/moodle/moodle/blob/master/lib/classes/dml/recordset_walk.php recordset_walk iterator], as there is no RAM benefit on using recordsets if you load all the results in a massive PHP array.

===Be cautious about other external calls===
Like a database query, there are other operations that are much slower than just executing PHP code. For example:
* running a shell script
* making a web-service call
* (to a lesser extent) working with files.
Whenever you are doing these things, worry about performance.

=== Limit the scope of session locks ===
If you don't need a session lock, or only need it for part of your page, then unlock the session. Full details here: [[Session locks]]

==How to improve the performance of your code==
===Measure during development===
* Turn on [[:en:Debugging#Performance_info|display of performance information]] (including counting database queries), so you are aware of what your code is doing.

* Use tools like [http://jakarta.apache.org/jmeter/ JMeter] to subject your new code to load.

* Use https://github.com/moodlehq/moodle-performance-comparison (Moodle 2.5 onwards) to compare performance. You can also use the [[Test site generator]] or [[Test course generator]] generators (also Moodle 2.5 onwards).

===Measure in production===
* If you're using postgres, there's a script that can parse the logs and output the top 10 slow queries, ready to be plugged into a cronjob to email you every day. It can be found here:
http://git.catalyst.net.nz/gw?p=pgtools.git;a=blob_plain;f=scripts/pg-log-process.pl;hb=refs/heads/pg-log-process-multidb

You need to configure postgres a bit to make it log things in the correct format. Instructions are in the file.
==How big can a Moodle site be?==
Looking at the [http://moodle.org/stats/ statistics], the largest sites in the world currently have
* Up to 1 000 000 users
* Up to 50 000 courses
* Up to 5 000 users per course
* Up to 50 roles
* Up to 100 course categories nested up to about 10 levels deep.
* Up to XXX activities in a course.
* Please add more things here.
When planning and testing your code, these are the sorts of numbers you should be contemplating. However, do not assume that Moodle sites will never get bigger than this.

Even if you can't test sites this big on your development server, you should use the generator script so you can test your code in a Moodle site that is not tiny.

==See also==
* [[Coding|Moodle coding guidelines]]
* [[Performance]] guidance for administrators
* [[Performance FAQ]]
[[Category:Coding guidelines|Performance]]
[[ja:パフォーマンスおよびスケーラビリティ]]

Core APIs

2022-03-16T02:44:00Z

Brendanheywood: /* See also */

Moodle has a number of core APIs that provide tools for Moodle scripts.

They are essential when writing [[Plugins|Moodle plugins]].

==Most-used General APIs==
These APIs are critical and will be used by nearly every Moodle plugin.
=== Access API (access) ===
The [[Access API]] gives you functions so you can determine what the current user is allowed to do, and it allows modules to extend Moodle with new capabilities.
=== Data manipulation API (dml) ===
The [[Data manipulation API]] allows you to read/write to databases in a consistent and safe way.
=== File API (files) ===
The [[File API]] controls the storage of files in connection to various plugins.
=== Form API (form) ===
The [[Form API]] defines and handles user data via web forms.
=== Logging API (log) ===
The [[Events API]] allows you to log events in Moodle, while [[Logging 2]] describes how logs are stored and retrieved.
=== Navigation API (navigation) ===
The [[Navigation API]] allows you to manipulate the navigation tree to add and remove items as you wish.
=== Page API (page) ===
The [[Page API]] is used to set up the current page, add JavaScript, and configure how things will be displayed to the user.
=== Output API (output) ===
The [[Output API]] is used to render the HTML for all parts of the page.
=== String API (string) ===
The [[String API]] is how you get language text strings to use in the user interface. It handles any language translations that might be available.
=== Upgrade API (upgrade) ===
The [[Upgrade API]] is how your module installs and upgrades itself, by keeping track of its own version.
=== Moodlelib API (core) ===
The [[Moodlelib API]] is the central library file of miscellaneous general-purpose Moodle functions. Functions can over the handling of request parameters, configs, user preferences, time, login, mnet, plugins, strings and others. There are plenty of defined constants too.
==Other General APIs==
=== Admin settings API (admin) ===
The [[Admin settings]] API deals with providing configuration options for each plugin and Moodle core.
=== Admin presets API (adminpresets) ===
The [[AdminPresetsAPI|Admin presets API]] allows plugins to make some decisions/implementations related to the Site admin presets.
=== Analytics API (analytics) ===
The [[Analytics API]] allow you to create prediction models and generate insights.
=== Availability API (availability) ===
The [[Availability API]] controls access to activities and sections.
=== Backup API (backup) ===
The [[Backup API]] defines exactly how to convert course data into XML for backup purposes, and the [[Restore API]] describes how to convert it back the other way.
=== Cache API (cache) ===
The [[The Moodle Universal Cache (MUC)]] is the structure for storing cache data within Moodle. [[Cache_API]] explains some of what is needed to use a cache in your code.
=== Calendar API (calendar) ===
The [[Calendar API]] allows you to add and modify events in the calendar for user, groups, courses, or the whole site.
=== Check API (check) ===
The [[Check API]] allows you to add security, performance or health checks to your site.
=== Comment API (comment) ===
The [[Comment API]] allows you to save and retrieve user comments, so that you can easily add commenting to any of your code.
=== Competency API (competency) ===
The [[Competency API]] allows you to list and add evidence of competencies to learning plans, learning plan templates, frameworks, courses and activities.
=== Data definition API (ddl) ===
The [[Data definition API]] is what you use to create, change and delete tables and fields in the database during upgrades.
=== Editor API ===
The [[Editor API]] is used to control HTML text editors.
=== Enrolment API (enrol) ===
The [[Enrolment API]] deals with course participants.
=== Events API (event) ===
The [[Events API]] allows to define "events" with payload data to be fired whenever you like, and it also allows you to define handlers to react to these events when they happen. This is the recommended form of inter-plugin communication. This also forms the basis for logging in Moodle.
=== Experience API (xAPI) ===
The Experience API (xAPI) is an e-learning standard that allows learning content and learning systems to speak to each other. The [[Experience API (xAPI)]]
allows any plugin to generate and handle xAPI standard statements.
=== External functions API (external) ===
The [[External functions API]] allows you to create fully parametrised methods that can be accessed by external programs (such as [[Web services]]).
=== Favourites API ===
The [[Favourites API]] allows you to mark items as favourites for a user and manage these favourites. This is often referred to as 'Starred'.
=== H5P API (h5p) ===
The [[H5P API]] allows plugins to make some decisions/implementations related to the [[H5P|H5P integration]].
=== Lock API (lock) ===
The [[Lock API]] lets you synchronise processing between multiple requests, even for separate nodes in a cluster.
=== Message API (message) ===
The [[Message API]] lets you post messages to users. They decide how they want to receive them.
=== Media API (media) ===
The [[Media_players#Using_media_players|Media]] API can be used to embed media items such as audio, video, and Flash.
=== My profile API ===
The [[My profile API]] is used to add things to the profile page.
=== OAuth 2 API (oauth2) ===
The [[OAuth 2 API]] is used to provide a common place to configure and manage external systems using OAuth 2.
=== Payment API (payment) ===
The [[Payment API]] deals with payments.
=== Preference API (preference) ===
The [[Preference API]] is a simple way to store and retrieve preferences for individual users.
=== Portfolio API (portfolio) ===
The [[Portfolio API]] allows you to add portfolio interfaces on your pages and allows users to package up data to send to their portfolios.
=== Privacy API (privacy) ===
The [[Privacy API]] allows you to describe the personal data that you store, and provides the means for that data to be discovered, exported, and deleted on a per-user basis.
This allows compliance with regulation such as the General Data Protection Regulation (GDPR) in Europe.
=== Rating API (rating) ===
The [[Rating API]] lets you create AJAX rating interfaces so that users can rate items in your plugin. In an activity module, you may choose to aggregate ratings to form grades.
=== Report builder API (reportbuilder) ===
The [[Report builder API]] allows you to create reports in your plugin, as well as providing custom reporting data which users can use to build their own reports.
=== RSS API (rss) ===
The [[RSS API]] allows you to create secure RSS feeds of data in your module.
=== Search API (search) ===
The [[Search API]] allows you to index contents in a search engine and query the search engine for results.
=== Tag API (tag) ===
The [[Tag API]] allows you to store tags (and a tag cloud) to items in your module.
=== Task API (task) ===
The [[Task API]] lets you run jobs in the background. Either once off, or on a regular schedule.
=== Time API (time) ===
The [[Time API]] takes care of translating and displaying times between users in the site.
=== Testing API (test) ===
The testing API contains the Unit test API ([[PHPUnit]]) and Acceptance test API ([[Acceptance testing]]). Ideally all new code should have unit tests written FIRST.
=== User-related APIs (user) ===
This is a rather informal grouping of miscellaneous [[User-related APIs]] relating to sorting and searching lists of users.
=== Web services API (webservice) ===
The [[Web services API]] allows you to expose particular functions (usually external functions) as web services.
=== Badges API (badges) ===
The [https://docs.moodle.org/dev/OpenBadges_User_Documentation Badges] user documentation (is a temp page until we compile a proper page with all the classes and APIs that allows you to manage particular badges and OpenBadges Backpack).
=== Custom fields API ===
The [[Custom fields API]] allows you to configure and add custom fields for different entities
== Activity module APIs ==
Activity modules are the most important plugin in Moodle. There are several core APIs that service only Activity modules.
=== Activity completion API (completion) ===
The [[Activity completion API]] is to indicate to the system how activities are completed.
=== Advanced grading API (grading) ===
The [[Advanced grading API]] allows you to add more advanced grading interfaces (such as rubrics) that can produce simple grades for the gradebook.
=== Conditional activities API (condition) - deprecated in 2.7 ===
The deprecated [[Conditional activities API]] used to provide conditional access to modules and sections in Moodle 2.6 and below. It has been replaced by the [[Availability API]].
=== Groups API (group) ===
The [[Groups API]] allows you to check the current activity group mode and set the current group.
=== Gradebook API (grade) ===
The [[Gradebook API]] allows you to read and write from the gradebook. It also allows you to provide an interface for detailed grading information.
=== Plagiarism API (plagiarism) ===
The [[Plagiarism API]] allows your activity module to send files and data to external services to have them checked for plagiarism.
=== Question API (question) ===
The [[Question API]] (which can be divided into the Question bank API and the Question engine API), can be used by activities that want to use questions from the question bank.
== See also ==
* [[Plugins]] - plugin types also have their own APIs
* [[Callbacks]] - list of all callbacks in Moodle
* [[Coding style]] - general information about writing PHP code for Moodle
* [[Session locks]]
[[ja:コアAPI]]

Session locks

2022-03-16T02:43:31Z

Brendanheywood:

When you create a normal moodle page and include config.php then by default a large amount of moodle bootstrapping runs and you will have the $SESSION global setup for you.

== Debugging session lock issues ==
If you have 'pages that are slow' and you profile them and see that you are waiting on a lock to free up then this is potentially an easy thing to fix to improve your overall performance.
$CFG->debugsessionlock = 5; // Time in seconds
When a session is locked for more N seconds a debugging call will be made with details of what the other page was which is holding onto the lock.

== Session unlocking ==
By default core assumes that you might need to mutate the $SESSION object so it will hold a lock on the session until the page finished and a shutdown handler will release the session lock.
If you are working on any page which is potentially long running, then you should cleanly separate logic which runs early which could mutate the session from the long running processin code and unlock the session.
\core\session\manager::write_close();
== Read only session in pages ==
For this to work, READONLY sessions must be enabled as well needing your code to support it. Not all session

If you know ahead of time that you will never mutate the session, but you still need to be able to read it, then you can declare your page to be read only. This will mean your page will never block the session in another http request.
define('READ_ONLY_SESSION', true);
== Read only sessions in web services ==
The same is possible in web services. When you declare your web service you can specify it will not need a session lock:
'core_message_get_unread_conversations_count' => array(
'classname' => 'core_message_external',
'methodname' => 'get_unread_conversations_count',
'classpath' => 'message/externallib.php',
'description' => 'Retrieve the count of unread conversations for a given user',
'type' => 'read',
'ajax' => true,
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE),
'''<nowiki>'readonlysession' => true, // We don't modify the session.</nowiki>'''
),
== No session at all ==
If your script doesn't actually need $SESSION in the first place then save even more processing and locks by declaring:
define('NO_MOODLE_COOKIES', true);
== No config is needed ==
Going to the absolute extreme, if you do not even need the full moodle bootstrap to run then you can skip it via:
define('ABORT_AFTER_CONFIG', true);

Session locks

2022-03-16T02:34:03Z

Brendanheywood:

When you create a normal moodle page and include config.php then by default a large amount of moodle bootstrapping runs and you will have the $SESSION global setup for you.

== Session unlocking ==
By default core assumes that you might need to mutate the $SESSION object so it will hold a lock on the session until the page finished and a shutdown handler will release the session lock.

If you are working on any page which is potentially long running, then you should cleanly separate logic which runs early which could mutate the session from the long running processin code and unlock the session.
\core\session\manager::write_close

== Read only session in pages ==
If you know ahead of time that you will never mutate the session, but you still need to be able to read it, then you can declare your page to be read only. This will mean your page will never block the session in another http request.
define('READ_ONLY_SESSION', true);

== Read only sessions in web services ==
The same is possible in web services. When you declare your web service you can specify it will not need a session lock:
'core_message_get_unread_conversations_count' => array(
'classname' => 'core_message_external',
'methodname' => 'get_unread_conversations_count',
'classpath' => 'message/externallib.php',
'description' => 'Retrieve the count of unread conversations for a given user',
'type' => 'read',
'ajax' => true,
'services' => array(MOODLE_OFFICIAL_MOBILE_SERVICE),
'''<nowiki>'readonlysession' => true, // We don't modify the session.</nowiki>'''
),

== No session at all ==
If your script doesn't actually need $SESSION in the first place then save even more processing and locks by declaring:
define('NO_MOODLE_COOKIES', true);

== No config is needed ==
Going to the absolute extreme, if you do not even need the full moodle bootstrap to run then you can skip it via:
define('ABORT_AFTER_CONFIG', true);

Session locks

2022-03-16T02:21:36Z

Brendanheywood:

When you create a new page by default you will have the $SESSION global setup for you.

\core\session\manager::write_close

READ_ONLY_SESSION

NO_MOODLE_COOKIES

ABORT_AFTER_CONFIG

Session locks

2022-03-16T02:20:57Z

Brendanheywood: Created page with " When you create a new page by default you will have the $SESSION global setup for you. \core\session\manager::write_close READ_ONLY_SESSION NO_MOODLE_COOKIES"

When you create a new page by default you will have the $SESSION global setup for you.

\core\session\manager::write_close

READ_ONLY_SESSION

NO_MOODLE_COOKIES

Check API

2022-03-15T22:35:48Z

Brendanheywood: /* Status checks (status) */

{{Moodle 3.9}}
== Checks ==
https://tracker.moodle.org/browse/MDL-67818

A Check is a runtime test to make sure something is working well. You can think of Checks as similar and complimentary to the [[PHPUnit]] and [[Acceptance_testing]] but the next layer around them, and done at run time rather than dev time or build time. Like other forms of testing the tests themselves should easy to read and reason about and confirm as valid. As many types of runtime checks can't be unit tested, the checks '''are''' the test.

Checks can be used for a variety of purposes including:
* configuration checks
* security checks
* status checks
* performance checks
* health checks
Moodle has had various types of checks and reports for a long time but they were inconsistent and not machine readable. In Moodle 3.9 they were unified under a single Check API which also enabled plugins to cleanly define their own additional checks. Some examples include:
* a password policy plugin could add a security check
* a custom authentication plugin can add a check that the upstream identity system can be connected to
* a MUC store plugin could add a performance check
Having these centralized and with a consistent contract makes it much easier to ensure the whole system is running smoothly. This makes it possible for an external integration with monitoring systems such as Nagios / Icinga. The Check API is expose as an NPRE compliance cli script:
<syntaxhighlight lang="bash">
php admin/cli/checks.php
</syntaxhighlight>
== Result states of a check ==
{| class="wikitable" border="1"
|-
! Status
! Meaning
! Example
|-
| N/A
| This check doesn't apply - but we may still want to expose the check
| secure cookies setting is disabled because site is not https
|-
| Ok
| A component is configured, working and fast.
| ldap can bind and return value with low latency
|-
| Info
| A component is OK, and we may want to alert the admin to something non urgent such as a deprecation, or something which needs to be checked manually.
|
|-
| Unknown
| We don't yet know the state. eg it may be very expensive so it is run using the Task API and we are waiting for the answer. NOTE: unknown is generally a bad thing and is semantically treated as an error. It is better to have a result of Unknown until the first result happens, and from then on it is Ok, or perhaps Warning or Error if the last known result is getting stale. If you are caching or showing a stale result you should expose the time of this in the result summary text.
| A complex user security report is still running for the first time.
|-
| Warning
| Something is not ideal and should be addressed, eg usability or the speed of the site may be affected, but it may self heal (eg a load spike)
| auth_ldap could bind but was slower than normal
|-
| Error
| Something is wrong with a component and a feature is not working
| auth_ldap could not connect, so users cannot start new sessions
|-
| Critical
| An error which is affecting everyone in a major way
| Cannot read sitedata or the database, the whole site is down
|}
How the various states are then leveraged is a local decision. A typical policy might be that health checks with a status of 'Error' or 'Critical' will page a system administrator 24/7, while 'Warning' only pages during business hours.
== Check types and reports ==
Checks are broken down into types, which roughly map to a step life cycle of your Moodle System
=== Environmental checks (?) ===
/admin/environment.php

These are environmental checks to make sure a Moodle instance is fully setup. This page is potential candidate to move to the new Check API but it slightly more complex than the other checks so hasn't been tackled yet. It would be a deeper change and this is intrinsically part of the install and upgrade system. It is not as critical to refactor as it is already possible for a plugin to declare its own checks, via either declarative [[Environment_checking]] or programmatically with a custom check:

https://docs.moodle.org/dev/Environment_checking#Custom_checks
=== Configuration checks (?) ===
/admin/index.php?cache=1

The Admin notifications page performs a mixture of checks, including security, status, and performance but none are as exhaustive as the checks in the reports below. It also does checks such as whether the web services for the Moodle Mobile App are turned on, and whether the site has been registered. Long term this page could show a summary of all the other check reports, or perhaps just the checks which are not in an OK state.
=== Security checks (security) ===
These are checks to make sure a Moodle instance is hardened correctly for you needs.

/report/security/index.php

https://tracker.moodle.org/browse/MDL-67776
=== Status checks (status) ===
/report/status/index.php

A status check is an 'in the moment' test and covers operational tests such as 'can moodle connect to ldap'. The main core status checks are that cron is running regularly and there has been no failed tasks.

'''IMPORTANT:''' It is critical to understand that Status checks are conceptually defined at the level off the application and not at a lower host level such as a docker container or node in a cluster. Checks should be defined so that whichever instance you ask you should get a consistent answer. DO NOT use the Status Checks to detect containers which need reaped and restarted. If you do, any status error will mean all containers will simultaneously be marked for reaping.

An additional status check is likely the most common type of check a plugin would define. Especially a plugin that connects to a 3rd party service. If the concept of 'OK' requires some sort of threshold, eg network response within 500ms, then that threshold should be managed by the plugin and optionally exposed as a admin setting. The plugin may choose to have different thresholds for Warning / Error / Critical. When designing a new Status Check be mindful that it needs to be actionable, for instance if you are asserting that a remote domain is available and it goes down, which then alerts your infrastructure team, there isn't much they can do about it if it isn't their domain. If it is borderline then make things like this configurable so that each site has to option of tune their own policies of what should be considered an issue or not.

https://tracker.moodle.org/browse/MDL-47271
=== Performance checks (performance) ===
/report/performance/index.php

Each check might simply check for certain settings which are known to slow things down, or it might actually do some sort of test like multiple reads and writes to the db or filesystem to get a performance metric.
=== Health checks (health) ===
/admin/tool/health/

The 'health center' is currently unsupported and contains some very old code. It is conceptually similar to 'status' checks except larger more long terms issues, such as detecting corrupt records. Ideally it is improved and converted to the Check API, see https://tracker.moodle.org/browse/MDL-67228
== Implementing a new check ==
=== A check class ===
And make a new check class in mod/myplugin/classes/check/foobar.php and the only mandatory method is get_result(). By default it will use a set language string but you can override the get_name() method to reuse an existing string.
<syntaxhighlight lang="php">
$string['checkfoobar'] = 'Check the foos to make sure they are barred';
</syntaxhighlight>
<syntaxhighlight lang="php">
<?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {

public function get_action_link(): ?\action_link {
$url = new \moodle_url('/mod/myplugin/dosomething.php'),
return new \action_link($url, get_string('sitepolicies', 'admin'));
}

public function get_result() : result {
if (some_check()) {
$status = result::ERROR;
$summary = get_string('check_foobar_error', 'mod_myplugin');
} else {
$status = result::OK;
$summary = get_string('check_foobar_ok', 'mod_myplugin');
}
$details = get_string('check_details', 'mod_myplugin');
return new result($status, $summary, $details);
}
}
</syntaxhighlight>
=== The result summary ===
The summary could change depending on the result of the check but for a simple check might be a fixed string, not html. Try to keep the summary to 1 line as this might typically be the thing which gets passed through to a paging system and could be truncated.
=== The action link ===
The action link is the place to go to help fix the issue. It should be as specific as possibly, such as a deep link into an admin settings page, and can include hash anchors.
=== lib.php callback ===
First decide if and when your new check(s) should be shown. Should it be present if your plugin is disabled? If you do not want it show if disabled then do not return it in callback below. If you do want it to show when disabled, but the check doesn't much sense then you can return a value of NA.

Next decide on what type of check it should be which determines what report it will be included in. Some checks might make sense to be reused with more than one report, eg it could be in both Status and Performance.

Implement the right callback in lib.php for the report you want to add it to, and return an array (usually with only 1 item) of check objects:

/mod/myplugin/lib.php
<syntaxhighlight lang="php">
function mod_myplugin_security_checks() {
return [new mod_myplugin\check\foobar()];
}
</syntaxhighlight>
=== Multiple instances of checks ===
Checks have been designed to be dynamic so you can return different checks depending on configuration, so auth_ldap would not return a check if the plugin is not enabled. Hypothetically if auth_ldap could be configured with 5 ldap servers then you could return 5 independent checks for each remote connection, each with different labels and information.

If you plan to return multiple instances of a check class, make sure that each instance has a unique id.
<syntaxhighlight lang="php">
function mod_myplugin_security_checks() {
return [
new mod_myplugin\check\foobar('one'),
new mod_myplugin\check\foobar('two'),
];
}
</syntaxhighlight>
Set the internal id in a way which is unique across all instances in your components namespace:
<syntaxhighlight lang="php"><?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {
protected $id = '';

public function __construct($id) {
$this->id = 'foobar' . $id;
}
public function get_id(): string {
return $this->id;
}
...
}
</syntaxhighlight>
=== Make checks as fast as practical ===
As many checks will be run and compiled into a report we want the checks themselves to be simple and as fast as possible. For instance an auth_ldap check while authenticating an end user could have a timeout of 60 seconds, and the check could warn if it takes more than 2 seconds. But the check could have a hard timeout of say 5 seconds and have a result status of ERROR for 5 or more seconds.

=== Lazy loading expensive result details ===
Checks can provide details on a check, such as the complete list of bad records. Generally this type of information might be expensive to produce so you can defer this lookup until get_details() is called specifically rather than setting this in the constructor. It will only be loaded on demand and shown when you drill down into the check details page.
<syntaxhighlight lang="php">
<?php
namespace mod_myplugin\check;
use core\check\check;

class foobar extends check {
public function get_result(): result {
return new foobar_result();
}
}
</syntaxhighlight>

<syntaxhighlight lang="php">
class foobar_result extends \core\check\result {
...
public function get_details(): string {
// Do expensive lookups in here.
}
}
</syntaxhighlight>
For a real example see:

https://github.com/moodle/moodle/blob/master/lib/classes/check/access/riskxss_result.php
=== Asynchronous checks ===
Some checks are by their nature asynchronous. For instance having moodle send an email to itself and then having it processed by the inbound mail handler to make it's properly configured (see https://tracker.moodle.org/browse/MDL-48800). In cases like these please make sure the age or staleness of the check is shown in the summary, and you should also consider turning the result status into a warning if the result is too old. If appropriate make the threshold a configurable admin setting.
==See also==
* [[:en:Performance overview|Performance overview]] user docs

Security:Cross-site request forgery

2022-02-09T11:35:56Z

Brendanheywood: /* Frontend leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like below and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.
<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>
===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Backend leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Frontend leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user in a way that could then leak to a 3rd party. An example might be accidentally disclosing it during a screen sharing session or even at a desktop being watched or filmed. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the sesskey check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param in a GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T11:33:46Z

Brendanheywood: /* Frontend leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like below and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.
<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>
===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Backend leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Frontend leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user in a way that could then leak to a 3rd party. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the sesskey check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param in a GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:53:47Z

Brendanheywood: /* Session key */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like below and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.
<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>
===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Backend leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Frontend leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the sesskey check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param in a GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:50:51Z

Brendanheywood: /* Guidelines for removing the sesskey from visible urls */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Backend leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Frontend leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the sesskey check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param in a GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:49:08Z

Brendanheywood: /* Ensure your code does not expose the sesskey inadvertently */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Backend leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Frontend leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param ina GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:48:36Z

Brendanheywood: /* Back end leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this generally cannot be addressed by a developer in the code of moodle or your plugin, instead it is best addressed at the infrastructure level for example by stripping it out these params from urls before they are logged in your server configuration. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param ina GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:47:08Z

Brendanheywood: /* Ensure your code does not expose the sesskey inadvertently */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of risks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin., it can only be address at the infrastructure level for example by stripping it out these params from urls before they are logged. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param ina GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:46:21Z

Brendanheywood: /* Back end leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin., it can only be address at the infrastructure level for example by stripping it out these params from urls before they are logged. A similar level of care needs to be taken when logging other things, for example logging the cookies in your access logs has a risk of allowing a session take over by anyone who has access to those logs.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param ina GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T05:41:50Z

Brendanheywood: /* Guidelines for removing the sesskey from visible urls */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# A sesskey param ina GET request then it is ok as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead. If it takes a long time then it runs the risk of the url being visible and lengthens the window of opportunity for a leak.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:29:23Z

Brendanheywood: /* Guidelines for removing the sesskey from visible urls */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## This page doesn't load any sub resources on another domain and where the sesskey could leak through the referer header
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:27:20Z

Brendanheywood: /* Guidelines for removing the sesskey from visible urls */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param. For example any urls for pluginfile.php should not have a sesskey param.
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:23:20Z

Brendanheywood: /* Guidelines for removing the sesskey from visible urls */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.

=== Examples of sesskey fixed in core ===
Some examples to help guide how to fix these issues:

https://tracker.moodle.org/browse/MDL-68292

https://tracker.moodle.org/browse/MDL-73295

==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:19:44Z

Brendanheywood: /* Ensure your code does not expose the sesskey inadvertently */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important hole is leaking the sesskey as part of the url in the referrer header when linking or interacting from another domain.
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:18:42Z

Brendanheywood: /* Back end leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be addressed in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:18:08Z

Brendanheywood: /* See also */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be address in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
* https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:13:31Z

Brendanheywood: /* Back end leaking */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern. Either way this cannot be address in the source of moodle or your plugin.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:12:44Z

Brendanheywood: /* Ensure your code does not expose the sesskey inadvertently */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.
== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be otherwise be mitigated by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.
=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern.
=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important
=== Guidelines for removing the sesskey from visible urls ===
# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.
==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
[[Category:Security]]

Security:Cross-site request forgery

2022-02-09T01:08:17Z

Brendanheywood: /* What you need to do as an administrator */

This page forms part of the [[Security|Moodle security guidelines]].

==What is the danger?==
When you put a web application on the internet, you are making it available so that anyone can send requests to it, and any request can be simply encoded as a URL.

Suppose that in Moodle, the way for an Administrator to delete a user was to click a '''Delete''' button in their user profile, and then click '''Yes''' on an confirmation page. Suppose that as a result of that, the Administrator's web browser sends a POST request to <nowiki>http://example.com/moodle/user/delete.php</nowiki>, with post data ?id=123&confirm=1.

Now suppose that Evil Hacker knows this, and wants to trick the administrator into deleting another user. (If Evil Hacker makes this request himself, he will see a permission denied error.)

All the Hacker Needs to do is to put the link <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki> somewhere where the administrator will click on it. For example, they could send the Administrator an email with a link saying "Look at this cool YouTube video" but where the link actually goes to the delete URL. The Administrator may click on the link without checking where it goes, and when the Administrator clicks that link, user 123 really will be deleted.

Or, more seriously, the student could put a post in a forum that Administrators will read, and in the forum post, put an <img src="<nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1</nowiki>" />. That way, the moment the Administrator reads the forum, user 123 will be deleted.

It is also possible to fake POST requests, you can simple put the form on external site and post a link pointing to that site on your Moodle server, it is also possible to use external flash instead of forms.

It may be a bit surprising, but this type of attack may be used against servers behind firewall on private network. It is not important where is the exploiting code, you can attack any server users may access from their browsers.

==How Moodle avoids this problem==
===Session key===
The most important protection is the concept of a '''sesskey''', short for session key.

When you log in, Moodle adds a random string to your session. Whever it prints a link or a button to perform a significant action, it adds the sesskey value to the submitted data. Before performing the action, it checks the sesskey value in the request with the one in the session, and the action is only performed if the two match.

Therefore, the request to delete a user is actually something like <nowiki>http://example.com/moodle/user/delete.php?id=123&confirm=1&sesskey=E8i5BCxLJR</nowiki>, and there is no way for Evil Hacker to know what the sesskey is, so they cannot construct an URL that tricks the admin into deleting a user.

===Use HTTP correctly===
Web applications use HTTP to encode requests from the user. In HTTP, there are various types of request. The two most important are GET and POST.

GET requests should be used for getting information. So, for example, viewing a user's profile should be a GET request.

POST requests should be used for changing things in the application. For example deleting a user should be a POST request.

When you click a link or load an image, it is always a GET request. When you submit a form, it is either a POST or a GET request, depending on the form.

Moodle should only process changes in response to a POST request. If that is the case, then it does Evil Hacker no good to trick a user into clicking on a link or viewing an embedded image. They have to trick a user into clicking a form submit button, which is harder.
==What you need to do in your code==
Use the [[Form API]] whenever possible for handling HTML forms. This automatically checks the sesskey and request method for you.

There are valid cases when using forms is not appropriate and you need to perform an action based on a parameter submitted via GET request - such as various action links. In this case, you have to manually include the sesskey among submitted parameters, and then make sure the submitted sesskey value is valid.

Include the sesskey among the submitted parameters:
<syntaxhighlight lang="php">
$action = new moodle_url('/admin/tool/do/something.php', ['delete' => $id, 'sesskey' => sesskey()]);
echo html_writer::link($action, get_string('delete'));
</syntaxhighlight>
And in the target script, make sure to check the submitted sesskey is correct before executing the operation:
<syntaxhighlight lang="php">
$delete = optional_param('delete', null, PARAM_INT);

if ($delete) {
require_sesskey();
// Do whatever you need to, like $DB->delete_records(...) etc.
}
</syntaxhighlight>
Note that when using standard elements like $OUTPUT->continue_button() and other elements based on the single_button widget submitted via POST method, the sesskey can be implicitly added to submitted parameters. Still, it is your duty to explicitly check the submitted value is valid.

== Ensure your code does not expose the sesskey inadvertently ==
There are various ways that the sesskey could be leaked, and if this happens then it opens the door to types of attacks that would be prevented by the sesskey.

https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html

There are broadly two ways it could be leaked, in the frontend and in the backend.

=== Back end leaking ===
This is things such as the sesskey appearing in various server logs. If your infrastructure is locked down well this should not be a major concern.

=== Front end leaking ===
The frontend includes ever showing the sesskey in the browser url bar or anywhere else visible to the end user. An example might be accidentally disclosing it during a screen sharing session. Another important

=== Guidelines for removing the sesskey from visible urls ===

# First don't remove the requirement for checking the sesskey if it is actually needed
# If the page doesn't change any state on the server then the check can be removed along with the query param
# If it does change state and is not a GET request, eg a post then it's ok as is
# If it is a GET request then it is ok as is as long as:
## It is not the primary http call, ie it is an ajax call or a sub request like an iframe
## If the request will ALWAYS do something very quickly and then redirect away. But generally speaking these should be a http post instead.

==What you need to do as an administrator==
* This is really only a code issue, but try not to fall for Evil Hacker's tricks ;-).

==See also==
* [[Security]]
* [[Coding]]
[[Category:Security]]

Serving files

2022-01-31T04:30:14Z

Brendanheywood: /* Icons / pix */

There are a lot of ways that files of various types can be served in Moodle and various API to help do it correctly. This page aims to link together all the various ways in one place with the pros and cons and best practice of each way.
== Factors to consider ==
=== Security ===
If your files must only be visible to people with certain roles then that is a clear indication that it should use the pluginfile API.
=== Performance and caching ===
One of the driving reasons for this page is making sure that any public files which can be cached are done so in the correct way so that they can be offloaded to a CDN like Cloudflare or Cloudfront or reverse proxy such as Varnish or nginx.
=== Maintenance ===
Some ways of serving files have a lower maintenance cost than others so it is better to use the best practice way of doing things below.
== Ways of serving files ==
=== Javascript files ===
For all javascript files whether files you have written yourself or 3rd party bundle libraries see: [[Javascript Modules]]
=== CSS files ===
[[Themes overview#CSS]]
=== Font files ===
[[Themes overview#Adding custom fonts]]
=== Icons / pix inside a theme ===
[[Themes overview#Making use of images]]

=== Images in a plugin ===
TBA

=== File API and pluginfile.php ===
For any files which have been uploaded into the Moodle File API and need to be served see [[File API#Serving files to users]]

Note that this API can also be used for files which are not in the File API, such as files on disk, or files which are generated on demand.
=== Data format API ===
If you need to generate a structured file on demand such as a CSV or Excel it is generally best to use the [[Data formats]] API. The Data format API can also be used in conjunction with the pluginfile API.
=== RSS and Calendar export ===
There are dedicated API's for serving [[RSS API]] and [[Calendar API]] data.
=== Custom download.php endpoint ===
It is easy and temping to implement your own custom script which downloads or serves a file. In the majority of cases the same functionality could be done via the pluginfile.php API so you can remove a lot of boilerlate and get a lot of functionality for free by avoiding this approach.
=== File with a .well-known or preset url structure ===
Often to implement some sort of protocol which another service with interact with, the url you serve on must have a specific location. For example:
/.well-known/password-change
Moodle currently does not yet have any proper generic routing framework to accept 'clean urls' and divert them to the appropriate plugin for handling.
=== Direct from Apache / nginx ===
This is only included for completeness and in general you should almost never serve files of any type directly from your web server. As a rule of thumb you should be able to comment out the native file serving directives in your nginx config and everything should still work because all files are served through php rather than directly. This may seem like it would be worse for performance in some cases, but it is actually better as Moodle is designed to sit behind a reverse proxy caching server or a CDN like Cloudflare, Cloudfront etc.

Serving files

2022-01-31T04:14:47Z

Brendanheywood: /* Custom download.php endpoint */

There are a lot of ways that files of various types can be served in Moodle and various API to help do it correctly. This page aims to link together all the various ways in one place with the pros and cons and best practice of each way.
== Factors to consider ==
=== Security ===
If your files must only be visible to people with certain roles then that is a clear indication that it should use the pluginfile API.
=== Performance and caching ===
One of the driving reasons for this page is making sure that any public files which can be cached are done so in the correct way so that they can be offloaded to a CDN like Cloudflare or Cloudfront or reverse proxy such as Varnish or nginx.
=== Maintenance ===
Some ways of serving files have a lower maintenance cost than others so it is better to use the best practice way of doing things below.
== Ways of serving files ==
=== Javascript files ===
For all javascript files whether files you have written yourself or 3rd party bundle libraries see: [[Javascript Modules]]
=== CSS files ===
[[Themes overview#CSS]]
=== Font files ===
[[Themes overview#Adding custom fonts]]
=== Icons / pix ===
[[Themes overview#Making use of images]]
=== File API and pluginfile.php ===
For any files which have been uploaded into the Moodle File API and need to be served see [[File API#Serving files to users]]

Note that this API can also be used for files which are not in the File API, such as files on disk, or files which are generated on demand.
=== Data format API ===
If you need to generate a structured file on demand such as a CSV or Excel it is generally best to use the [[Data formats]] API. The Data format API can also be used in conjunction with the pluginfile API.
=== RSS and Calendar export ===
There are dedicated API's for serving [[RSS API]] and [[Calendar API]] data.
=== Custom download.php endpoint ===
It is easy and temping to implement your own custom script which downloads or serves a file. In the majority of cases the same functionality could be done via the pluginfile.php API so you can remove a lot of boilerlate and get a lot of functionality for free by avoiding this approach.

=== File with a .well-known or preset url structure ===
Often to implement some sort of protocol which another service with interact with, the url you serve on must have a specific location. For example:
/.well-known/password-change
Moodle currently does not yet have any proper generic routing framework to accept 'clean urls' and divert them to the appropriate plugin for handling.

=== Direct from Apache / nginx ===
This is only included for completeness and in general you should almost never serve files of any type directly from your web server. As a rule of thumb you should be able to comment out the native file serving directives in your nginx config and everything should still work because all files are served through php rather than directly. This may seem like it would be worse for performance in some cases, but it is actually better as Moodle is designed to sit behind a reverse proxy caching server or a CDN like Cloudflare, Cloudfront etc.

Serving files

2022-01-31T04:11:09Z

Brendanheywood: /* Performance and caching */

There are a lot of ways that files of various types can be served in Moodle and various API to help do it correctly. This page aims to link together all the various ways in one place with the pros and cons and best practice of each way.
== Factors to consider ==
=== Security ===
If your files must only be visible to people with certain roles then that is a clear indication that it should use the pluginfile API.
=== Performance and caching ===
One of the driving reasons for this page is making sure that any public files which can be cached are done so in the correct way so that they can be offloaded to a CDN like Cloudflare or Cloudfront or reverse proxy such as Varnish or nginx.

=== Maintenance ===
Some ways of serving files have a lower maintenance cost than others so it is better to use the best practice way of doing things below.

== Ways of serving files ==
=== Javascript files ===
For all javascript files whether files you have written yourself or 3rd party bundle libraries see: [[Javascript Modules]]
=== CSS files ===
[[Themes overview#CSS]]
=== Font files ===
[[Themes overview#Adding custom fonts]]
=== Icons / pix ===
[[Themes overview#Making use of images]]
=== File API and pluginfile.php ===
For any files which have been uploaded into the Moodle File API and need to be served see [[File API#Serving files to users]]

Note that this API can also be used for files which are not in the File API, such as files on disk, or files which are generated on demand.
=== Data format API ===
If you need to generate a structured file on demand such as a CSV or Excel it is generally best to use the [[Data formats]] API. The Data format API can also be used in conjunction with the pluginfile API.
=== RSS and Calendar export ===
There are dedicated API's for serving [[RSS API]] and [[Calendar API]] data.
=== Custom download.php endpoint ===
It is easy and temping to implement your own custom script which downloads or serves a file. In the majority of cases the same functionality could be done via the pluginfile.php API so you can remove a lot of boilerlate and get a lot of functionality for free by avoiding this approach.
=== Direct from Apache / nginx ===
This is only included for completeness and in general you should almost never serve files of any type directly from your web server. As a rule of thumb you should be able to comment out the native file serving directives in your nginx config and everything should still work because all files are served through php rather than directly. This may seem like it would be worse for performance in some cases, but it is actually better as Moodle is designed to sit behind a reverse proxy caching server or a CDN like Cloudflare, Cloudfront etc.

Serving files

2022-01-31T04:08:23Z

Brendanheywood: /* Factors to consider */

There are a lot of ways that files of various types can be served in Moodle and various API to help do it correctly. This page aims to link together all the various ways in one place with the pros and cons and best practice of each way.
== Factors to consider ==
=== Security ===
If your files must only be visible to people with certain roles then that is a clear indication that it should use the pluginfile API.

=== Performance and caching ===
One of the driving reasons for this page is making sure that any public files which can be cached are done so in the correct way so that they can be offloaded to a CDN like Cloudflare or Cloudfront or reverse proxy such as Varnish or nginx.

== Ways of serving files ==
=== Javascript files ===
For all javascript files whether files you have written yourself or 3rd party bundle libraries see: [[Javascript Modules]]
=== CSS files ===
[[Themes overview#CSS]]
=== Font files ===
[[Themes overview#Adding custom fonts]]
=== Icons / pix ===
[[Themes overview#Making use of images]]
=== File API and pluginfile.php ===
For any files which have been uploaded into the Moodle File API and need to be served see [[File API#Serving files to users]]

Note that this API can also be used for files which are not in the File API, such as files on disk, or files which are generated on demand.
=== Data format API ===
If you need to generate a structured file on demand such as a CSV or Excel it is generally best to use the [[Data formats]] API. The Data format API can also be used in conjunction with the pluginfile API.
=== RSS and Calendar export ===
There are dedicated API's for serving [[RSS API]] and [[Calendar API]] data.
=== Custom download.php endpoint ===
It is easy and temping to implement your own custom script which downloads or serves a file. In the majority of cases the same functionality could be done via the pluginfile.php API so you can remove a lot of boilerlate and get a lot of functionality for free by avoiding this approach.
=== Direct from Apache / nginx ===
This is only included for completeness and in general you should almost never serve files of any type directly from your web server. As a rule of thumb you should be able to comment out the native file serving directives in your nginx config and everything should still work because all files are served through php rather than directly. This may seem like it would be worse for performance in some cases, but it is actually better as Moodle is designed to sit behind a reverse proxy caching server or a CDN like Cloudflare, Cloudfront etc.

Serving files

2022-01-31T04:02:55Z

Brendanheywood: /* Custom download.php endpoint */

There are a lot of ways that files of various types can be served in Moodle and various API to help do it correctly. This page aims to link together all the various ways in one place with the pros and cons and best practice of each way.
== Factors to consider ==
=== Security ===
=== Performance and caching ===
One of the driving
== Ways of serving files ==
=== Javascript files ===
For all javascript files whether files you have written yourself or 3rd party bundle libraries see: [[Javascript Modules]]
=== CSS files ===
[[Themes overview#CSS]]
=== Font files ===
[[Themes overview#Adding custom fonts]]
=== Icons / pix ===
[[Themes overview#Making use of images]]
=== File API and pluginfile.php ===
For any files which have been uploaded into the Moodle File API and need to be served see [[File API#Serving files to users]]

Note that this API can also be used for files which are not in the File API, such as files on disk, or files which are generated on demand.
=== Data format API ===
If you need to generate a structured file on demand such as a CSV or Excel it is generally best to use the [[Data formats]] API. The Data format API can also be used in conjunction with the pluginfile API.
=== RSS and Calendar export ===
There are dedicated API's for serving [[RSS API]] and [[Calendar API]] data.
=== Custom download.php endpoint ===
It is easy and temping to implement your own custom script which downloads or serves a file. In the majority of cases the same functionality could be done via the pluginfile.php API so you can remove a lot of boilerlate and get a lot of functionality for free by avoiding this approach.
=== Direct from Apache / nginx ===
This is only included for completeness and in general you should almost never serve files of any type directly from your web server. As a rule of thumb you should be able to comment out the native file serving directives in your nginx config and everything should still work because all files are served through php rather than directly. This may seem like it would be worse for performance in some cases, but it is actually better as Moodle is designed to sit behind a reverse proxy caching server or a CDN like Cloudflare, Cloudfront etc.

Serving files

2022-01-31T03:58:33Z

Brendanheywood: /* Custom download.php endpoint */