MoodleDocs - Användarbidrag [sv]

Cache definitions

2014-06-12T03:32:01Z

Samhemelryk:

This page contains information on the cache definitions within core. This information will likely be of most use to those administrators who are really trying to get the most out the caching system in Moodle. The information about each cache should help in deciding which backends to use for which definitions.

For small sites, or those running on a single server likely adopting a single backend such as memcache is going to give you an immediate performance boost. 
However with a bit of planning and some careful mapping likely you can improve things further.

==What you will find in this document==
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* If shared between sites please be aware that any language customisations will also be shared.

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache definitions

2014-06-12T03:31:28Z

Samhemelryk:

This page contains information on the cache definitions within core. This information will likely be of most use to those administrators who are really trying to get the most out the caching system in Moodle. The information about each cache should help in deciding which backends to use for which definitions. 
For small sites, or those running on a single server likely adopting a single backend such as memcache is going to give you an immediate performance boost. 
However with a bit of planning and some careful mapping likely you can improve things further.

Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* If shared between sites please be aware that any language customisations will also be shared.

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache definitions

2014-06-12T03:30:53Z

Samhemelryk:

This page contains information on the cache definitions within core. This information will likely be of most use to those administrators who are really trying to get the most out the caching system in Moodle. The information about each cache should help in deciding which backends to use for which definitions.

For small sites, or those running on a single server likely adopting a single backend such as memcache is going to give you an immediate performance boost. 
However with a bit of planning and some careful mapping likely you can improve things further.

==What can be found here==
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* If shared between sites please be aware that any language customisations will also be shared.

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Caching

2014-06-12T03:24:00Z

Samhemelryk: /* More information */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use those servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache. 
You are presented with several drop-downs that allow you to map one or more cached. All mapped caches get interacted with. The "Primary" store is the store that will be used first when interacting with the cache.
The "Final" mapped store will be the last cache store interacted with. 
How this interaction occurs is documented below.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

===More on performance testing===

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

===Performance advice for load-balanced web servers===

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==Troubleshooting==

Have you encountered a problem, or found yourself in a conundrum? Perhaps the answer is in this section. If its not when you find have an answer how about sharing it here.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-12T03:21:54Z

Samhemelryk: /* Other performance advice for load-balanced web servers */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use those servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache. 
You are presented with several drop-downs that allow you to map one or more cached. All mapped caches get interacted with. The "Primary" store is the store that will be used first when interacting with the cache.
The "Final" mapped store will be the last cache store interacted with. 
How this interaction occurs is documented below.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

===More on performance testing===

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

===Performance advice for load-balanced web servers===

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-12T03:21:23Z

Samhemelryk: /* Other performance testing */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use those servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache. 
You are presented with several drop-downs that allow you to map one or more cached. All mapped caches get interacted with. The "Primary" store is the store that will be used first when interacting with the cache.
The "Final" mapped store will be the last cache store interacted with. 
How this interaction occurs is documented below.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

===More on performance testing===

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-11T23:58:43Z

Samhemelryk: /* Mapping a cache to a store instance */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use those servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache. 
You are presented with several drop-downs that allow you to map one or more cached. All mapped caches get interacted with. The "Primary" store is the store that will be used first when interacting with the cache.
The "Final" mapped store will be the last cache store interacted with. 
How this interaction occurs is documented below.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-11T23:49:52Z

Samhemelryk: /* Memcache */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use those servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-11T23:49:10Z

Samhemelryk: /* Memcached */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use a dedicated Memcached servers and to '''NOT''' configure any other software to use it. Doing so may lead to performance deprecation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

'''Important implementation notes'''

* The Memcached extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. 
Because of this it is important to note that when you purge a Memcached store within Moodle it deletes ALL entries in the Memcached server. Not just those relating to Moodle. 
For that reason it is highly recommended to use dedicated Memcached servers and to '''NOT''' configure any other software to use the same servers. Doing so may lead to performance depreciation and adverse affects.
* Likewise if you want to use Memcached for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-11T23:47:49Z

Samhemelryk: /* Memcache */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

'''Important implementation notes'''

* The Memcache extension does not provide a means of deleting a set or entries. Either a single entry is deleted, or all entries are deleted. Because of this it is important to note that when you purge a Memcache store within Moodle it deletes ALL entries in the Memcache backend. Not just those relating to Moodle. For that reason it is highly recommended to use a dedicated Memcached servers and to '''NOT''' configure any other software to use it. Doing so may lead to performance deprecation and adverse affects.
* Likewise if you want to use Memcache for caching and for sessions in Moodle it is essential to use two Memcached servers. One for sessions, and one for caching. Otherwise a cache purge in Moodle will purge your sessions!

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-11T23:33:25Z

Samhemelryk: /* Stores used when no mapping is present */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are no specific mappings in place. 
To simplify that, this show the default cache stores for each type.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Cache definitions

2014-06-11T23:24:20Z

Samhemelryk: /* Language string cache */

This page contains information on the cache definitions within core. 
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* If shared between sites please be aware that any language customisations will also be shared.

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Caching

2014-06-11T23:21:03Z

Samhemelryk: Added note about language customisations in regards to sharing.

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar with, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that get used when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it. A cache by nature is volatile and any information that is absolutely mission critical should be a more permanent data store likely the database.

Nonetheless there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share in many situations, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites. 
It is important to consider with the language cache that by sharing it between sites any language customisations will also be shared.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. It is useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve your sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mongodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column select the link for '''Edit mappings'''.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Scenario:
You have have a web server that has a Moodle site as well as other sites.
You also have a Memcache server that is used by several sites including Moodle.

Memcache has a limited size cache, that when full and requested to store more information frees space by dropping the least used cache entries.

You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.

Solution:
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.

Explanation:
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step-by-step guide to this.

How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered are some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way.

If you are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Fil:caching-27-08-add-memcache-store.png

2014-06-09T01:46:21Z

Samhemelryk: Screenshot of adding a Memcache store instance.

Screenshot of adding a Memcache store instance.

Fil:caching-27-12-default-store-mapping.png

2014-06-09T01:45:41Z

Samhemelryk: Screenshot of default store mappings.

Screenshot of default store mappings.

Fil:caching-27-11-store-mapping.png

2014-06-09T01:45:18Z

Samhemelryk: Screenshot of store mappings.

Screenshot of store mappings.

Fil:caching-27-10-add-mongodb-store.png

2014-06-09T01:45:03Z

Samhemelryk: Screenshot of adding a Mongodb store instance.

Screenshot of adding a Mongodb store instance.

Fil:caching-27-09-add-memcached-store.png

2014-06-09T01:44:53Z

Samhemelryk: Screenshot of adding a Memcached store instance.

Screenshot of adding a Memcached store instance.

Fil:caching-27-07-add-file-cache-store.png

2014-06-09T01:44:29Z

Samhemelryk: Screenshot of adding a new file cache store instance.

Screenshot of adding a new file cache store instance.

Fil:caching-27-06-stores-used-when-no-mapping-is-present.png

2014-06-09T01:44:05Z

Samhemelryk: Screenshot of default stores in cache configuration screen.

Screenshot of default stores in cache configuration screen.

Fil:caching-27-05-summary-of-cache-lock-instances.png

2014-06-09T01:43:30Z

Samhemelryk: Screenshot of cache lock instances in cache configuration screen.

Screenshot of cache lock instances in cache configuration screen.

Fil:caching-27-04-known-cache-definitions.png

2014-06-09T01:43:05Z

Samhemelryk: Screenshot of known cache definitions.

Screenshot of known cache definitions.

Fil:caching-27-03-configured-store-instances.png

2014-06-09T01:42:50Z

Samhemelryk: Screenshot of installed cache store instances.

Screenshot of installed cache store instances.

Fil:caching-27-02-installed-cache-stores.png

2014-06-09T01:42:29Z

Samhemelryk: Screenshot of installed cache stores

Screenshot of installed cache stores

Caching

2014-06-09T01:41:48Z

Samhemelryk: Update images

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:caching-27-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:caching-27-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:caching-27-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:caching-27-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:caching-27-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:caching-27-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:caching-27-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:caching-27-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:caching-27-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:caching-27-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:caching-27-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:caching-27-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Fil:caching-27-01-configuration-screen.png

2014-06-09T01:39:58Z

Samhemelryk: Cache configuration screen in Moodle 2.7

Cache configuration screen in Moodle 2.7

Caching

2014-06-09T01:33:38Z

Samhemelryk: /* Configuring caching for your site */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:26-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:26-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:26-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:26-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:26-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:26-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:26-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:26-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:26-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:26-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:26-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:26-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-09T01:32:35Z

Samhemelryk: /* Known cache definitions */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:26-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:26-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:26-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:26-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

The idea of a cache definition hasn't been discussed here yet. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

Information on specific cache definitions can be found on the [[Cache definitions]] page.

===Summary of cache lock instances===

[[Image:26-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:26-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:26-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:26-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:26-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:26-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:26-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:26-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-09T01:30:35Z

Samhemelryk: /* More information */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:26-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:26-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:26-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:26-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

This is a list of all the caches within Moodle that have a definition. 
The idea of a cache definition hasn't been discussed here. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

===Summary of cache lock instances===

[[Image:26-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:26-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:26-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:26-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:26-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:26-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:26-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:26-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Cache definitions

2014-06-09T01:30:05Z

Samhemelryk: /* More information */

This page contains information on the cache definitions within core. 
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API|Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference|Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)|The Moodle Universal Cache (MUC)]] The original cache specification.

Cache definitions

2014-06-09T01:29:18Z

Samhemelryk: /* Repositories instances data */

This page contains information on the cache definitions within core. 
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)]] The original cache specification.

Cache definitions

2014-06-09T01:28:25Z

Samhemelryk: Better formatting

This page contains information on the cache definitions within core. 
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

'''Since:''' When this cache definition was first introduced into Moodle.
 '''Component/Area:''' The code component this cache belongs to and the area (unique simple name) given to it.
 '''Growth:''' Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
 '''Who:''' Which users can expect to benefit from the cache.
 '''Priority:''' An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

'''Since:''' Moodle 2.6
 '''Component/Area:''' core, coursemodinfo
 '''Growth:''' exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
 '''Who:''' everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
 '''Priority:''' 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

'''Since:''' Moodle 2.5
 '''Component/Area:''' core, calendar_subscriptions
 '''Growth:''' determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
 '''Who:''' everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
 '''Priority:''' 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

'''Since:''' Moodle 2.7 (MDL-44366)
 '''Component/Area:''' mod_glossary, concepts
 '''Growth:''' exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
 '''Who:''' everyone when the glossary filter is enabled.
 '''Priority:''' 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

'''Since:''' Moodle 2.4
 '''Component/Area:''' core, config
 '''Growth:''' fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
 '''Who:''' everyone, on every page, without fail.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecattree
 '''Growth:''' fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
 '''Who:''' everyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

'''Since:''' Moodle 2.5 (MDL-34398)
 '''Component/Area:''' core, groupdata
 '''Growth:''' fixed, the number of courses and groups within those courses determines the size of this cache.
 '''Who:''' all users accessing a course in which groups are used.
 '''Priority:''' 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, databasemeta
 '''Growth:''' fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
 '''Who:''' everyone
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, eventinvalidation
 '''Growth:''' fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
 '''Who:''' everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
 '''Priority:''' 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

'''Since:''' Moodle 2.6 (MDL-39846)
 '''Component/Area:''' core, observers
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

'''Since:''' Moodle 2.5.2, 2.6 (MDL-40924)
 '''Component/Area:''' core, externalbadges
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability_grade, items
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

'''Since:''' Moodle 2.4.2, 2.5 (MDL-36297)
 '''Component/Area:''' core, htmlpurifier
 '''Growth:''' exponential, cleaned content is usually stored once for each user + context combination.
 '''Who:''' everyone
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

'''Since:''' Moodle 2.4 (MDL-25290)
 '''Component/Area:''' core, string
 '''Growth:''' fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
 '''Who:''' everyone, on every browsable page within Moodle.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

'''Since:''' Moodle 2.6 (MDL-41019)
 '''Component/Area:''' core, langmenu
 '''Growth:''' fixed, determined by the number of languages installed on the site.
 '''Who:''' everyone
 '''Priority:''' 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

'''Since:''' Moodle 2.5 (MDL-38596)
 '''Component/Area:''' core, coursecontacts
 '''Growth:''' the number of courses and the number of users with course contact roles determines this size of this cache.
 '''Who:''' everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
 '''Priority:''' 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

'''Since:''' Moodle 2.5 (MDL-34401)
 '''Component/Area:''' core, plugin_manager
 '''Growth:''' fixed, the number of plugins determines the size of this cache.
 '''Who:''' Administrators primarily.
 '''Priority:''' 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

'''Since:''' Moodle 2.4 (MDL-34399)
 '''Component/Area:''' core, questiondata
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

'''Since:'''
 '''Component/Area:''' core, gradecondition
 '''Growth:'''
 '''Who:'''
 '''Priority:'''

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

'''Since:''' Moodle 2.7 (MDL-44070)
 '''Component/Area:''' availability, scores
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

'''Since:''' Moodle 2.5 (MDL-38391)
 '''Component/Area:''' core, yuimodules
 '''Growth:''' fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
 '''Who:''' everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
 '''Priority:''' 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecat
 '''Growth:''' the number of categories and courses on the site that the user can see will determine the size of this for the current user.
 '''Who:''' everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

'''Since:''' Moodle 2.6 (MDL-42299)
 '''Component/Area:''' core, userselections
 '''Growth:''' exponential, depends upon how much the user interacts with things like expading categories.
 '''Who:''' logged in users.

===Folder name cache [repository_skydrive]===

?

'''Since:''' Moodle 2.6 (MDL-30740)
 '''Component/Area:''' repository_skydrive, foldername
 '''Growth:''' ?
 '''Who:''' ?
 '''Priority:''' ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

'''Since:''' Moodle 2.5 (MDL-38147)
 '''Component/Area:''' core, coursecatrecords
 '''Growth:''' fixes, the number of categories on the site visible to the user will determine this.
 '''Who:''' everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

'''Since:''' Moodle 2.6 (MDL-13114)
 '''Component/Area:''' tool_uploadcourse, helper
 '''Growth:''' ?
 '''Who:''' Administrators
 '''Priority:''' ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

'''Since:''' Moodle 2.5 (MDL-34346)
 '''Component/Area:''' core, repositories
 '''Growth:''' ?
 '''Who:''' logged in users with one or more accessible repositories.
 '''Priority:''' ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

Cache definitions

2014-06-09T01:25:01Z

Samhemelryk: Created page with "This page contains information on the cache definitions within core. Each cache definition found within core will be under its own heading and will include a description..."

This page contains information on the cache definitions within core. 
Each cache definition found within core will be under its own heading and will include a description of the cache. 
As well as the name and description the following information will also be details:

; Since : When this cache definition was first introduced into Moodle.
; Component/Area : The code component this cache belongs to and the area (unique simple name) given to it.
; Growth : Will state if this cache is expected to be of fixed size, or can be expected to grow as the site data grows. Perhaps some information on how the cache will grow.
; Who : Which users can expect to benefit from the cache.
; Priority : An indication of the anticipated cache use on a site. A value between 1 and 5. If the cache is of fixed size and is accessed expected to be utilised on every page it will be given a 5 for priority as it can be expected that assigning that cache to the fastest backend available would be the good idea. In contrast a priority of 1 would be given to a cache that is expected to grow quickly, is only accessed on specific pages, and only applies to some users (e.g. teachers, or the admin). This cache should be the least of your concerns when deciding upon how to implement caching on your site.

==A note about cache configuration on large sites==
Each different cache can be configured independently, allowing admins to "tune" their setup for particular systems. 
By default these caches are all set to use the file system, which is usually fine on a small one-server system.

On a cluster, however, these defaults can cause problems because shared filesystems are slow, so in these cases we recommend you use a faster shared caching backend like memcached instead. Note that most of these caches operating under the assumption that they are shared. 
In some cases you can choose to use a non-shared cache like the local filesystem however in these instances you be careful to purge caches MANUALLY as part of system administration.

==Application caches==
Application caches are shared caches.

===Accumulated information about modules and sections for each course===

Accumulated information about course modules, and sections used to print the course page as well as in calls to ''get_fast_modinfo()''. 
This cache gets forcefully reset within ''rebuild_course_cache()''.

This is an excellent cache to optimise caching for on a production site. 
It is accessed primarily for the course view page, and very frequently hit and often expensive page and it is utilised within ''get_fast_modinot()'' a function that is called often in Moodle when querying or displaying information on a course, section or activity.

; Since : Moodle 2.6
; Component/Area : core, coursemodinfo
; Growth : exponential, the number of courses, sections within those courses, and activities within each section will determine the size of this cache.
; Who : everyone. This cache isn't accessed on every page, however as its associated with courses you can expect its accces to still be high.
; Priority : 4

===Calendar subscriptions===

Caches calendar subscriptions. 
This is a very simple cache that stores the calendar subscription records from the database for quick, specific access.

This should be considered a low priority cache unless your users are likely to be creating many calendar subscriptions. 
The actual data being stored is going to be very small, should you have lots of calendar subscriptions mapping this cache definition to a fast, but small backend would be ideal.

; Since : Moodle 2.5
; Component/Area : core, calendar_subscriptions
; Growth : determined by the number of calendar subscriptsion created by users. Entirely dependent on your users.
; Who : everyone who has a calendar subscript, on calendar pages, or pages where the calendar block is being displayed.
; Priority : 1

'''Notes for advanced, multi-server sites'''

* What is cached:- Record entries from 'event_subscriptions' table, representing various calendar subscriptions.
* When the cache is updated:- When a calendar subscription is updated or deleted.
* How often it is hit:- Everytime a calendar subscription detail is fetched.
* When should the cache be purged completely:- This should not be needed.

===Concept linking [mod_glossary]===

Caches concepts for the glossary filter.

Concepts are cached in relation to either the site or a course. The course ID is used as the key if they are for a course, 0 is used as the key for site concepts. 
Each entry in the cache is an array consisting of two items, the first is an array of glossaries, the second an array of concepts. 
The actual data being stored is minimal, even on a large site the storage requirement for this cache should be relatively small.

'''Important''' This cache CANNOT be a local cache, it must be shared.

; Since : Moodle 2.7 (MDL-44366)
; Component/Area : mod_glossary, concepts
; Growth : exponential, the number of glossaries, and the number of terms within those glossaries will determine the size of this cache.
; Who : everyone when the glossary filter is enabled.
; Priority : 3

===Config settings===

Caches the configuration for the site. This is the administration settings in both core and all plugins.

This cache is going to be accessed on every single page within Moodle, regradless of the users state or what is being done. 
You should map this cache to the fastest backend you've got available.

; Since : Moodle 2.4
; Component/Area : core, config
; Growth : fixed, the number of items is the number of settings in core and all installed plugins. This will only increase as settings are introduced or removed.
; Who : everyone, on every page, without fail.
; Priority : 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* This cache is hit quite often for getting config settings.

===Course categories tree===

This cache stores the full course categories tree.

It is often used when navigating the course category structure and you can expect it to be hit in situations where either the full tree, or part of the tree is displayed. 
This includes some elements that can be configured to display on the front page. 
Its use will be determinent on how you have configured Moodle and what is set to be displayed.

; Since : Moodle 2.5 (MDL-38147)
; Component/Area : core, coursecattree
; Growth : fixed, will be limited to the number of course categories on your site. During course creation this site will obviously grow, but in day to day use it should remain static.
; Who : everyone.
; Priority : 3

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Gets invalidated when changesincoursecat event is triggered.

===Course group information===

Caches grouping information for courses.

Information is cached in relation to a course and is very simple. It is only used in situations where a course has defined groups and those groups are being used. 
This cache can be expected to save 1 - 2 queries per page. 

; Since : Moodle 2.5 (MDL-34398)
; Component/Area : core, groupdata
; Growth : fixed, the number of courses and groups within those courses determines the size of this cache.
; Who : all users accessing a course in which groups are used.
; Priority : 2

'''Notes for advanced, multi-server sites'''

* Gets updated when groups data is fetched for a course.
* Requires shared cache.

===Database meta information===

Caches meta information on database tables including information about columns.

This cache is a priority cache, it is accessed on nearly every page within Moodle and its operation can be expensive. 
Each entry uses the table name as the key and the cached data is the structure of the database table.

; Since : Moodle 2.4 (MDL-25290)
; Component/Area : core, databasemeta
; Growth : fixed, the number of database tables determines the size of this cache and may only change during upgrade and installation/deletion of plugins.
; Who : everyone
; Priority : 5

'''Notes for advanced, multi-server sites'''
* Requires shared cache
* If not shared, caches need to be invalidated after any DB structure change including creation of temporary tables.

===Event invalidation===

This cache is used to manage event invalidation for the MUC API. This is an internal API and has no relation what so ever to the Event API introduced in 2.7.

This is not often used within Moodle and when it is used occurs when editing happens and multiple caches need to purged, not all of which may be shared and some which may be cleared when the user next touches the cache.
The overall cache size should be relatively small, and as checking often occurs for these caches on page access it is recommended to map this cache to a fast backend.

; Since : Moodle 2.4 (MDL-25290)
; Component/Area : core, eventinvalidation
; Growth : fixed, events are subscribed to by definitinos and will only change during upgrade and the installation and deletion of plugins.
; Who : everyone, events get triggered by action and in disconnected caches this be not be reflected until the user next hits the page.
; Priority : 4

'''Notes for advanced, multi-server sites'''
* Whenever something is invalidated, it is purged immediately and an event record is created with the timestamp.
* Requires shared cache.

===Event observers===

This cache is used for storing list of event observers.

It is updated on install/update while initialising list of event observers. 
This cache is accessed, when event is trigged.

; Since : Moodle 2.6 (MDL-39846)
; Component/Area : core, observers
; Growth : ?
; Who : ?
; Priority : ?

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

===External badges for particular user===

Used to store external badges.

; Since : Moodle 2.5.2, 2.6 (MDL-40924)
; Component/Area : core, externalbadges
; Growth : ?
; Who : ?
; Priority : ?

===Grade items cached for evaluating conditional availability [availability_grade, items]===

Used to cache course grade items for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

; Since : Moodle 2.7 (MDL-44070)
; Component/Area : availability_grade, items
; Growth : ?
; Who : ?
; Priority : ?

===HTML Purifier - cleaned content===

This is a cache of texts (forum posts, resources, intros etc etc) from all parts of Moodle, after it has been cleaned of possible malicious data.

Caches cleaned text in relation to user + context of the text being cleaned.

'''Tip:''' This data may be safetly stored in local caches on clusted nodes.

; Since : Moodle 2.4.2, 2.5 (MDL-36297)
; Component/Area : core, htmlpurifier
; Growth : exponential, cleaned content is usually stored once for each user + context combination.
; Who : everyone
; Priority : 3

'''Notes for advanced, multi-server sites'''

In Moodle 2.5,
* This expects a shared cache
* If not shared, must be manually purged after every upgrade or change of $CFG->allowobjectembed setting

In Moodle 2.6 and later,
* This works fine with local or shared node caches, you don't have to do anything special.

===Language string cache===

The language string cache is one of the most essential caches within Moodle, it caches each and every language file used within Moodle. 
Its of fixed size as there are a finite number of languages and language files and as it is accessed one pretty much every page within Moodle. 
This is a prime cache to configure, map it to the fastest backend you've got available.

The string cache uses the a hash of the revision, language, and component of a language file as the key, and the array found within the corrosponding file is the data. 
Static acceleration is used on this cache to speed up subsequent request for a string within a given language file. This is essential as string are usually requested one by one and often only from a handful of language files.

'''Tip:''' Cache usage differs greatly between Admins + managers and teacher + students. Admins and managers being able to complete more actions and often across difference components and plugins often require that many more language files be accessed for a page than a user such as a teacher or student require. 
When testing your cache configuration ensure you test as a student.

; Since : Moodle 2.4 (MDL-25290)
; Component/Area : core, string
; Growth : fixed size, each language string file is cached here, its size will be fixed but will be determined by the number of languages available on your site.
; Who : everyone, on every browsable page within Moodle.
; Priority : 5

'''Notes for advanced, multi-server sites'''

In Moodle 2.4 and Moodle 2.6:
* Expects shared cache.
* If not shared it must be manually purged after any language string change such as editing of local lang packs, updating of lang packs during upgrade, installation or uninstallation of languages.

In Moodle 2.6 and later:
* Works fine with local or shared node caches, you don't have to do anything special.

===List of available languages===

Caches a list of available languages.

This list of languages is used every time the list of languages is requested in a page. 
Because of this the cache will be hit by on many pages within Moodle on any site with more than one language installed.
As its a small, fixed size cache on a site with multiple languages installed its a good idea to map this to the fastest backend you've got available.

; Since : Moodle 2.6 (MDL-41019)
; Component/Area : core, langmenu
; Growth : fixed, determined by the number of languages installed on the site.
; Who : everyone
; Priority : 4

===List of course contacts===

Used to cache course contacts.

Course contacts are displayed in several places throughout Moodle, they are in most situations considered public information (like course names) and can be seeing by all users. 
The process of listing these course contacts can be expensive, however the course contacts are shown only on select pages.

; Since : Moodle 2.5 (MDL-38596)
; Component/Area : core, coursecontacts
; Growth : the number of courses and the number of users with course contact roles determines this size of this cache.
; Who : everyone, course contacts are public information and may be displayed in several places throughout Moodle that can be accessed by anyone.
; Priority : 3

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering/searching course
* Requires shared cache.

===Plugin info manager===

Caches information on installed plugins, enabled plugins, and present plugins.

This cache is heavily used when managing plugins for site through the admin interfaces. 
Primarily people accessing this page benefit from the existence of this cache.

'''Note:''' This must be a shared cache.

; Since : Moodle 2.5 (MDL-34401)
; Component/Area : core, plugin_manager
; Growth : fixed, the number of plugins determines the size of this cache.
; Who : Administrators primarily.
; Priority : 2

====Plugin info - base====
* This cache is used by plugininfo_base class and stores plugin information.
* This is accessed while loading/checking plugin versions from disk.
* Requires shared cache.

====Plugin info - activity modules====
* This cache is used by plugininfo_mod class and provide access to records in modules table.
* This is accessed while loading/checking modules.
* Requires shared cache.

====Plugin info - blocks====
* This cache is used by plugininfo_block class and provide access to records in block table.
* This is accessed while loading/checking blocks.
* Requires shared cache.

====Plugin info - filters====
* This cache is used by plugininfo_filter class and stores names of all filters installed.
* This is accessed while loading/checking installed filters.
* Requires shared cache.

====Plugin info - repositories====
* This cache is used by plugininfo_repositories class and provide access to records in repository table.
* This is accessed while loading enabled repositories.
* Requires shared cache.

====Plugin info - portfolios====
* This cache is used by plugininfo_portfolio class and stores list of enabled portfolio plugins.
* This is accessed while checking if portfolio is enabled.
* Requires shared cache.

===Question definitions===

Caches question definitions. This is used by the question bank class.

; Since : Moodle 2.4 (MDL-34399)
; Component/Area : core, questiondata
; Growth : ?
; Who : ?
; Priority : ?

'''Notes for advanced, multi-server sites'''
* This gets updated when question is loaded or edited.
* Doesn't require data guarantee.
* Requires shared cache.

===User grades cached for evaluating conditional availability===

; Since :
; Component/Area : core, gradecondition
; Growth :
; Who :
; Priority :

===User grades cached for evaluating conditional availability [availability_grade, scores]===

Used to cache user grades for conditional availability purposes.

'''Note:''' This cache sets a TTL (Time To Live) of 3600 (1 hour).

; Since : Moodle 2.7 (MDL-44070)
; Component/Area : availability, scores
; Growth : ?
; Who : ?
; Priority : ?

===YUI Module definitions===

Caches information on shifter YUI modules used within Moodle when JS caching is enabled.

; Since : Moodle 2.5 (MDL-38391)
; Component/Area : core, yuimodules
; Growth : fixes, the number of Moodle YUI modules within core and plugins. This will only change during upgrade, or installation/removal of plugins.
; Who : everyone, this is used when ever Moodle YUI modules are used on a page and that is most pages as of 2.8.
; Priority : 5

'''Notes for advanced, multi-server sites'''

* Requires shared cache.

==Session caches==
Data cached here belongs to the user browsing the site.

===Course categories lists for particular user===

Used to store data for course categories visible to current user. Helps to browse list of categories. 
This is also used during course category management.

; Since : Moodle 2.5 (MDL-38147)
; Component/Area : core, coursecat
; Growth : the number of categories and courses on the site that the user can see will determine the size of this for the current user.
; Who : everyone, it is used within several front page elements.

'''Notes for advanced, multi-server sites'''

* Requires shared cache.
* Is invalidated when changesincoursecat or changesincourse event is trigged.

===Data used to persist user selections throughout Moodle===

Caches user selections that should persist for the lifetime of the users log in. This includes things like which categories the user has expanded in the course category management page. 
Think of them like user preferences but with limited lifetime.

; Since : Moodle 2.6 (MDL-42299)
; Component/Area : core, userselections
; Growth : exponential, depends upon how much the user interacts with things like expading categories.
; Who : logged in users.

===Folder name cache [repository_skydrive]===

?

; Since : Moodle 2.6 (MDL-30740)
; Component/Area : repository_skydrive, foldername
; Growth : ?
; Who : ?
; Priority : ?

==Request caches==
Caches here last only for the life time of the request and are only available to the browsing user.

===Course categories records===

Caches a list of course categories visible to the user.

; Since : Moodle 2.5 (MDL-38147)
; Component/Area : core, coursecatrecords
; Growth : fixes, the number of categories on the site visible to the user will determine this.
; Who : everyone

'''Notes for advanced, multi-server sites'''

* This is accessed while rendering course category.
* Is invalidated when changesincoursecat event is triggered.
* Require local cache.

===Helper caching [tool_uploadcourse]===

; Since : Moodle 2.6 (MDL-13114)
; Component/Area : tool_uploadcourse, helper
; Growth : ?
; Who : Administrators
; Priority : ?

===Repositories instances data===

Used to cache data on configured repositories to avoid repetitive database calls to load repositories.

; Since : Moodle 2.5 (MDL-34346)
; Component/Area : core, repositories
; Growth : ?
; Who : logged in users with one or more accessible repositories.
; Priority : ?

'''Notes for advanced, multi-server sites'''

* Requires local cache.

Caching

2014-06-09T01:24:33Z

Samhemelryk: /* More information */

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:26-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:26-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:26-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:26-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

This is a list of all the caches within Moodle that have a definition. 
The idea of a cache definition hasn't been discussed here. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

===Summary of cache lock instances===

[[Image:26-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:26-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:26-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:26-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:26-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:26-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:26-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:26-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Caching

2014-06-09T01:23:25Z

Samhemelryk: Massive changes to really beef up the caching documentation

{{Performance}}

A cache is a collection of processed data that is kept on hand and re-used in order to avoid costly repeated database queries.

Moodle 2.4 saw the implementation of MUC, the Moodle Universal Cache. This new system allows certain functions of Moodle (eg string fetching) take advantage of different installed cache services (eg files, ram, memcached).

In future versions of Moodle we will continue expanding the number of Moodle functions that use MUC, which will continue improving performance, but you can already start using it to improve your site.

==General approach to performance testing==

Here is the general strategy you should be taking:

# Build a test environment that is as close to your real production instance as possible (eg hardware, software, networking, etc)
# Make sure to remove as many uncontrolled variables as you can from this environment (eg other services)
# Use a tool to place a realistic, but simulated and repeatable load upon you server. (eg jmeter or selenium).
# Decide on a way to measure performance of the server by capturing data (ram, load, time taken, etc)
# Run your load and measure a baseline performance result.
# Change one variable at a time, and re-run the load to see if performance gets better or worse. Repeat as necessary.
# When you discover settings that result in a consistent performance improvement, apply to your production site.

==Cache configuration in Moodle==

Since Moodle 2.4, Moodle has provided a caching plugin framework to give administrators the ability to control where Moodle stores cached data. For most Moodle sites the default configuration should be sufficient and it is not necessary to change the configuration. For larger Moodle sites with multiple servers, administrators may wish to use memcached, mongodb or other systems to store cache data. The cache plugin screen provides administrators with the ability to configure what cache data is stored where. 
Caching in Moodle is controlled by what is known as the Moodle Universal Cache. Commonly referred to MUC.

This document explains briefly what MUC is before proceeding into detail about the concepts and configuration options it offers.

==The basic cache concepts in Moodle==
Caching in Moodle isn't as complex as it first appears. A little background knowledge will go a long way in understanding how cache configuration works.

===Cache types===
Lets start with cache types (sometimes referred to as mode). There are three basic types of caches in Moodle.

The first is the application cache. This is by far the most commonly used cache type in code. Its information is shared by all users and its data persists between requests. Information stored here is usually cached for one of two reasons, either it is required information for the majority of requests and saves us a one of more database interactions or it is information that is accessed less frequently but is resource intensive to generate. 
By default this information is stored in an organised structure within your Moodle data directory.

The second cache type is the session cache. This is just like the PHP session that you will already be familiar, in fact it uses the PHP session by default. You may be wondering why we have this cache type at all, but the answer is simple. MUC provides a managed means of storing, and removing information that is required between requests. It offers developers a framework to use rather than having to re-invent the wheel and ensures that we have access to a controlled means of managing the cache as required. 
Its important to note that this isn't a frequently used cache type as by default session cache data is stored in the PHP session and the PHP session is stored in the database. Uses of the session cache type are limited to small datasets as we don't want to bloat sessions and thus put strain on the database.

The third and final type is the request cache. Data stored in this cache type only persists for the lifetime of the request. If you're a PHP developer think of it like a managed static variable. 
This is by far the least used of the three cache types, uses are often limited to information that will be accessed several times within the same request, usually by more than area of code.
Cached information is stored in memory by default.

==== Cache types and multiple-server systems ====

If you have a system with multiple front-end web servers, the application cache must be shared between the servers. In other words, you cannot use fast local storage for the application cache, but must use shared storage or some other form of shared cache such as a shared memcache.

The same applies to session cache, unless you use a 'sticky sessions' mechanism to ensure that within a session, users always access the same front-end server.

===Cache backends===
Cache backends are where data actually gets stored. These include things like the file system, php session, Memcached, and memory. 
By default just file system, php session, and memory are used within Moodle. 
We don't require that a site has access to any other systems such a Memcached. Instead that is something you are responsible for installing and configuring yourself. 
When cache backends are mentioned think of systems outside of Moodle that can be used to store data. The MongoDB server, the Memcache server and similiar "server" applications.

===Cache stores===
Cache stores are a plugin type within Moodle. They facilitate connecting Moodle to the cache backends discussed above. 
Moodle ships with the three defaults mentioned above as well as Memcache, Memcached, and MongoDB. 
You can find other cache store plugins in the [https://moodle.org/plugins/browse.php?list=category&id=48 plugins database]. 
The code for these is located within cache/stores in your Moodle directory root.

Within Moodle you can configure as many cache stores as your architecture requires. If you have several Memcache servers for instance you can create an cache store instance for each. 
Moodle by default contains three cache store instances that gets when you've made no other configuration.
* A file store instance is created which gets used for all application caches. It stores its data in your moodledata directory.
* A session store instance is created which gets used for all session caches. It stores its data in the PHP session, which by default is stored if your database.
* A static memory store instance is created which gets used for all request cache types. Data exists in memory for just the lifetime of a request.

===Caches: what happens in code===
Caches are created in code and are used by the developer to store data they see a need to cache. 
Lets keep this section nice and short because perhaps you are not a developer. There is one very important point you must know about. 
The developer does not get any say in where the data gets cached. They must specify the following information when creating a cache to use.
# The type of cache they require.
# The area of code this cache will belong to (the API if you will).
# The name of the cache, something they make up to describe in one word what the cache stores.

There are several optional requirements and settings they can specify as well, but don't worry about that at this point. 
The important point is that they can't choose which cache backend to use, they can only choose the type of cache they want from the three detailed above.

===How it ties together===

This is best described in relation to roles played in an organisation.

# The system administrator installs the cache backends you wish to use. Memcache, XCache, APC and so on. Moodle doesn't know about these, they are outside of Moodle's scope and purely the responsibility of your system administrator.
# The Moodle administrator creates a cache store instance in Moodle for each backend the site will make use of. There can be one or more cache stores instances for each backend. Some backends like Memcached have settings to create separated spaces within one backend.
# The developer has created caches in code and is using them to store data. He doesn't know anything about how you will use your caches, he just creates a "cache" and tells Moodle what type it is best for it.
# The Moodle administrator creates a mapping between a cache store instance and a cache. That mapping tells Moodle to use the backend you specify to store the data the developer wants cached.

In addition to that you can take things further still.
* You can map many caches to a single cache store instance.
* You can map multiple cache store instances to a single cache with priority (primary ... final)
* You can map a cache store instance to be the default store used for all caches of a specific type that don't otherwise have specific mappings.

If this is the first time you are reading about the Moodle Universal Cache this probably sounds pretty complex but don't worry it will be discussed in better detail as we work through how to configure the caching in Moodle.

==Advanced concepts==
These concepts are things that most sites will not need to know or concern themselves about.

You should only start looking here if you are looking to maximise performance on large sites running over clustered services with shared cache backends, or on multi-site architecure again where information is being shared between sites.

===Locking===

The idea of locking is nothing new, it is the process of controlling access in order to avoid concurrency issues.

MUC has a second type of plugin, a cache lock plugin that gets used when caches require it. To date no caches have required it, a cache by nature is volatlie and any information that is absolutely mission critical should be a more permanent data store likely the database. 
None the less there is a locking system that cache definitions can require within their options and that will be applied when interacting with a cache store instance.

===Sharing===

Every bit of data that gets stored within a cache has a calculated unique key associated with it. 
By default part of that key is the site identifier making any content stored in the cache specific to the site that stored it. For most sites this is exactly what you want. 
However in some situations its beneficial to allow mutiple sites, or somehow linked sites to share cached data. 
Of course not all caches can be shared, however some certainly can and by sharing you can further reduce load and increase performance by maximising resource use.

This is an advanced feature, if you choose to configure sharing please do so carefully.

To make use of sharing you need to first configure identical cache store instances in the sites you want to share information, and then on each site set the sharing for the cache to the same value.

; Sites with the same site ID : This is the default, it allows for sites with the same site ID to share cached information. It is the most restrictive but is going to work for all caches. All other options carry an element of risk in that you have to ensure the information in the cache is applicable to all sites that will be accessing it.
; Sites running the same version : All sites accessing the backend that have the same Moodle version can share the information this cache has stored in the cache store.
; Custom key : For this you manually enter a key to use for sharing. You must then enter the exact same key into the other sites you want to share information.
; Everyone : The cached data is accessible to all other sites accessing the data. This option puts the ball entirely in the Moodle administrators court.

As an example if you had several Moodle sites all the same version running on server with APC installed you could decide to map the language cache to the APC store and configure sharing for all sites running the same version. 
The language cache for sites on the same version is safe to share, it is used on practically every page, and APC is extremely fast. These three points may result in a nice wee performance boost for your sites.

==The cache configuration screen==

The cache configuration screen is your one stop shop for configuring caching in Moodle. 
It gives you an overview of how caching is currently configured for your site and it provides links to all of the actions you can perform to configure caching to your specific needs.

===Accessing the cache configuration screen===

[[Image:26-01-configuration-screen.png|thumb|500px|The cache configuration screen in Moodle 2.6]]

The cache configuration screen can only be accessed by users with the ''moodle/site:config'' capability. By default this is only admins. 
Once logged in the configuration screen can be found in the Settings block under '''Site Administration > Plugins > Caching > Configuration'''.

===Installed cache stores===

[[Image:26-02-installed-cache-stores.png|thumb|500px|Installed cache stores screenshot]]

This is showing you a list of cache store plugins that you have installed. 
For each plugin you can quickly see whether it is ready to be used (any php requirements have been met), how many store instances already exist on this site, the cache types that this store can be used for, what features it supports (advanced) and any actions you can perform relating to this store.

Often the only action available is to create a new store instance. 
Most stores support having multiple instances, however not all as you will see that that the session cache and static request cache do not. For those two stores it does not make sense to have multiple instances.

===Configured store instances===

[[Image:26-03-configured-store-instances.png|thumb|500px|Configured store instances screenshot]]

Here you get a list of the cache store instances on this site.

; '''Store name''' : The name given to this cache store instance when it is created so that you can recognise it. It can be anything you want and is only used so that you can identify the store instance.
; '''Plugin''' : The cache store plugin of which this is an instance of.
; '''Ready''' : A tick gets shown when all PHP requirements have been met as well as any connection or set-up requirements have been verified.
; '''Store mappings''' : The number of caches this store instance has been mapped to explicitly. Does not include any uses through default mappings (discussed below).
; '''Modes''' : The modes that this cache store instance can serve.
; '''Supports''' : ''Advanced''. The features supported by this cache store instance.
; '''Locking mechanism''' : ''Advanced''. The locking mechanism this cache store instance will make use of. We've not discussed this yet, but read on and you'll find information on it.
; '''Actions''' : Any actions that can be performed against this cache store instance.

===Known cache definitions===

[[Image:26-04-known-cache-definitions.png|thumb|500px|Known cache definitions screenshot]]

This is a list of all the caches within Moodle that have a definition. 
The idea of a cache definition hasn't been discussed here. It is something controlled by the developer. When they create a cache they can do so in two ways, the first is by creating a cache definition. This is essentially telling Moodle about the cache they've created. The second way is to create an Adhoc cache. Developers are always encouraged to use the first method. Only caches with a definition can be mapped and further configured by the admin. Adhoc caches will make use of default settings only. 
Typically Adhoc caches are only permitted in situations where the cache is small and configuring it beyond defaults would provide no benefit to administrators. Really its like saying you the administrator doesn't need to concern yourself with it.

For each cache shown here you get the following information:

; '''Definition''' : A concise description of this cache.
; '''Mode''' : The cache type this cache is designed for.
; '''Component''' : The code component the cache is associated with.
; '''Area''' : The area of code this cache is serving within the component.
; '''Store mappings''' : The store or stores that will be used for this cache.
; '''Sharing''' : How is sharing configured for this site.
; '''Actions''' : Any actions that can be performed on the cache. Typically you can edit the cache store instance mappings, edit sharing, and purge the cache.

You'll also find at the bottom of this table a link title "Rescan definitions". Clicking this link will cause Moodle to go off an check all core components, and installed plugins looking for changes in the cache definitions. 
This happens by default during upgrade, and if a new cache definition is encountered. However should you find yourself looking for a cache that isn't there this may be worth a try. 
It is also handy for developers as it allows them to quickly apply changes when working with caches. Its useful when tweaking cache definitions to find what works best.

===Summary of cache lock instances===

[[Image:26-05-summary-of-cache-lock-instances.png|thumb|500px|Summary of cache lock instances screenshot]]

As mentioned above cache locking is an advanced concept in MUC. 
The table here shows information on the configured locking mechanisms available to MUC. By default just a single locking mechanism is available, file locking.
At present there are no caches that make use of this and as such I won't discuss it further here.

===Stores used when no mapping is present===

[[Image:26-06-stores-used-when-no-mapping-is-present.png|thumb|500px|Mapping of default stores screenshot]]

This table quickly shows which cache store instances are going to be used for cache types if there are specific mappings in place. 
Of simply this shows the default cache store instances.

At the bottom you will notice there is a link "Edit mappings" that takes you to a page where you can configure this.

==Adding cache store instances==

The default configuration is going to work for all sites, however you may be able to improve you sites performance by making use of various caching backends and techniques. The first thing you are going to want to do is add cache store instances configured to connect to/use the cache backends you've set up.

===File cache===

[[Image:26-07-add-file-cache-store.png|thumb|300px|Adding a file cache store screenshot]]

When on the cache configuration screen within the ''Installed cache stores'' table you should be able to see the File cache plugin, click ''`Add instance`'' to start the process of adding a file cache store instance.

When creating a file cache there is in fact only one required param, the store name. The store name is used to identify the file store instance in the configuration interface and must be unique to the site.
It can be anything you want, but we would advice making it something that describes you intended use of the file store.

The following properties can also be specified, customising where the file cache will be located, and how it operates.

; '''Cache path''' : Allows you to specify a directory to use when storing cache data on the file system. Of course the user the webserver is running as must have read/write access to this directory. By default (blank) the Moodledata directory will be used.
; '''Auto create directory''' : If enabled when the cache is initialised if the specified directory does not exist Moodle will create it. If this is specified and the directory does not exist the the cache will be deemed not ready and will not be used.
; '''Single directory store''' : By default the file store will create a subdirectory structure to store data in. The first 3 characters of the data key will be used as a directory. This is useful in avoiding file system limits it the file system has a maximum number of files per directory. By enabling this option the file cache will not use subdirectories for storage of data. This leads to a flat structure but one that is more likely hit file system limits. Use with care.
; '''Prescan directory''' : One of the features the file cache provides is to prescan the storage directory when the cache is first used. This leads to faster checks of files at the expense of an in-depth read.

The file cache store is the default store used for application caches and by default the moodledata directory gets used for the cache.
File access can be a taxing resource in times of increased load and the following are some ideas about configuring alternative file stores in order to improve performance.

First up is there a faster file system available for use on your server. 
Perhaps you've an SSD installed but are not using it for your moodledata directory because space is a premium. 
You should consider creating a directory or small partition on your SSD and creating a file store to use that instead of your Moodle data directory.

Next you've not got a faster drive available for use, but you do have plenty of free space. 
Something that may be worth giving a shot would be to create a small partition on the drive you've got installed that uses a performance orientated file system. 
Many linux installations these days for example use EXT4, a nice file system but one that has overheads due to the likes of journalling. 
Creating a partition and using a file system that has been optimised for performance may give you that little boost you are looking for. Remember caches are designed to be volatile and choosing a file system for a cache is different decision to choosing a file system for your server. 

Finally if you're ready to go to lengths and have an abundance of memory on your server you could consider creating a ramdisk/tmpfs and pointing a file store at that.
Purely based in memory, it is volatile exactly like the cache is, and file system performance just isn't going to get much better than this. 
Of course you will be limited in space and you are essentially taking that resource away from your server.

Please remember with all of these ideas that they are just ideas. 
What ever you choose - test, test, test, be sure of the decision you make.

===Memcache===

[[Image:26-08-add-memcache-store.png|thumb|300px|Add Memcache store screenshot]]

Before you can add a Memcache store instance you must first have a Memcached server you can access and have the Memcache php extension installed and enabled on your web server.

Like the file store you must provide a the store name. It is used to identify the store instance in the configuration interface and must be unique to the site. 
For a Memcache store you must also enter the Memcache server, or servers you wish it to make use of.
Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

Optionally you can also specify a key prefix to use. What you enter here will prefixed to all keys before accessing the server and can be used to effectively partition the Memcache space in a recognisable way. This can be handy if you have a management tool for you Memcached server that you use to inspect what is stored there.

===Memcached===

[[Image:26-09-add-memcached-store.png|thumb|300px|Add Memcached store screenshot]]

Like the Memcache store you must first have a Memcached server you can access and have the Memcached php extension installed and enabled on your server.

Also like the Memcache store there are two required parameters in configuring a Memcached store.

; '''Store name''' : It is used to identify the store instance in the configuration interface and must be unique to the site.
; '''Servers''' : The servers you wish this cache store use. See below for details.

Servers should be added one per line and each line can contain 1 to 3 properties separated by colons.
# The URL or IP address of the server (required)
# The port the server is listening on (optional)
# The weight to give this server (optional)

For example, if you had two Memcached instances running on your server, one configured for the default port, and one configured for 11212 you would use the following:
<code>
127.0.0.1
127.0.0.1:11212
</code>

There are also several optional parameters you can set when creating a Memcached store.

; '''Use compression''' : Defaults to true, but can be disabled if you wish.
; '''Use serialiser''' : Allows your to select which serialiser gets used when communicating with the Memcache server. By default the Memcached extension and PHP only provide one serialised, however there is a couple of others available for installation if you go looking for them. One for example is the igbinary found at https://github.com/igbinary/igbinary.
; '''Prefix key''' : Allows you to set some characters that will be prefixed to all keys before interacting with the server.
; '''Hash method''' : The hash method provided by the Memcached extension is used by default here. However you can select to use an alternative if you wish. http://www.php.net/manual/en/memcached.constants.php provides a little information on the options available. Please note if you wish to you can also override the default hash function PHP uses within your php.ini.
; '''Buffer writes''' : Disabled by default, and for good reason. Turning on buffered writes will minimise interaction with the Memcached server by buffering io operations. The downside to this is that on a system with any concurrency there is a good chance multiple requests will end up generating the data because no one had pushed it to the Memcached server when they first requested it. Enabling this can be advantageous for caches that are only accessed in capability controlled areas for example where multiple interaction is taking a toll on network resources or such. But that is definitely on the extreme tweaking end of the scale.

===MongoDB===

[[Image:26-10-add-mongodb-store.png|thumb|300px|Add MongoDB store screenshot]]

MongoDB is an open source document orientated NoSQL database. Check out their website www.mogodb.org for more information.

; '''Store name''' : Used to identify the store instance in the configuration interface and must be unique to the site.
; '''Server''' : This is the connection string for the server you want to use. Multiple servers can be specified using a comma-separated list.
; '''Database''' : The name of the database to make use of.
; '''Username''' : The username to use when making a connection.
; '''Password''' : The password of the user being used for the connection.
; '''Replica set''' : The name of the replica set to connect to. If this is given the master will be determined by using the ismaster database command on the seeds, so the driver may end up connecting to a server that was not even listed.
; '''Use''' : If enabled the usesafe option will be used during insert, get, and remove operations. If you've specified a replica set this will be forced on anyway.
; '''Use safe value''' : You can choose to provide a specific value for use safe. This will determine the number of servers that operations must be completed on before they are deemed to have been completed.
; '''Use extended keys''' : If enabled full key sets will be used when working with the plugin. This isn't used internally yet but would allow you to easily search and investigate the MongoDB plugin manually if you so choose. Turning this on will add an small overhead so should only be done if you require it.

==Mapping a cache to a store instance==

[[Image:26-11-store-mapping.png|thumb|300px|Cache definition store mapping screenshot]]

Mapping a store instance to a cache tells Moodle to use that store instance when the cache is interacted with. This allows the Moodle administrator to control where information gets stored and to most importantly optimise performance of your site by making the most of the resources available to your site.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Known cache definitions'' table and within it find the cache you'd like to map.
In the actions column you should see a link to'''Edit mappings''', click this.

The screen you are presented allows you to map one or more cache store instances to be used by this cache.

If no stores are mapped for the cache then the default stores are used. Have a look at the section below for information on changing the default stores.

If a single store instance is mapped to the cache the following occurs:
; ''Getting data from the cache'' : Moodle asks the cache to get the data. The cache attempts to get it from the store. If the store has it it gives it to the cache, and the cache gives it to Moodle so that it can use the data. If the store doesn't have it the a fail is returned and Moodle will have to generate the data and will most likely then send it to the cache.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, and the cache will give it to the cache store.

If multiple store instances are mapped to the cache the following occurs:
; ''Getting data from a store'' : Moodle asks the cache to get the data. The cache attempts to get it from the first store. If the first store has it then it returns the data to the cache and the cache returns it to Moodle. If the first store doesn't have the data then it attempts to get the data from the second store. If the second store has it it returns it to the first store that then stores it itself before returning it to the cache. If it doesn't then the next store is used. This continue until either the data is found or there are no more stores to check.
; ''Storing data in the cache'' : Moodle will ask the cache to store some data, the cache will give it to every mapped cache store for storage.

The main advantage to assigning multiple stores is that you can introduce cache redundancy. Of course this introduces an overhead so it should only be used when actually required.
The following is an example of when mapping multiple stores can provide an advantage:
<pre>
Imagine if you will that you have have a web server that has a Moodle site as well as other sites. You also have a Memcache server that is used by several sites including Moodle. 
Memcache is a limited size cache, that when full and requested to store more information frees space by dropping least used cache entries.
You want to use Memcache for your Moodle site because it is fast, however you are aware that it may introduce more cache misses because it is a heavily used Memcache server.
To get around this you map two stores to caches you wish to use Memcache.
You make Memcache the primary store, and you make the default file store the final cache store.
By doing this you've created redundancy, when something is requested Moodle first tries to get it from Memcache (the fastest store) and if its not there it proceeds to check the file cache.
</pre>

Just a couple more points of interest:

* Mapping multiple caches will introduce overhead, the more caches mapped the more overhead.
* Consider the cache stores you are mapping to, if data remains there once set then there is no point mapping any further stores after it. This technique is primarily valuable in situations where data is not guaranteed to remain after being set.
* Always test your configuration. Enable the display of performance information and then watch which stores get used when interacting with Moodle in such a way as to trigger the cache.

==Setting the stores that get used when no mapping is present==

[[Image:26-12-default-store-mapping.png|thumb|300px|Setting which stores get used when no mapping is present screenshot]]

This is really setting the default stores that get used for a cache type when there is not a specific mapping that has been made for it.

To set a mapping first browse to the cache configuration screen. 
Proceed to find the ''Stores used when no mapping is present'' table. 
After the table you will find a link '''Edit mappings''', click this.

On the screen you are presented with you can select one store for each cache type to use when a cache of the corresponding type gets initialised and there is not an explicit mapping for it.

Note that on this interface the drop downs only contain store instances that are suitable for mapping to the type.
Not all instances will necessarily be shown. If you have a store instance you don't see then it is not suitable for '''ALL''' the cache definitions that exist. 
You will not be able to make that store instance the default, you will instead need to map it explicitly to each cache you want/can use it for.

==Configuring caching for your site==

This is where it really gets tricky, and unfortunately there is no step by step guide to this. 
How caching can be best configured for a site comes down entirely to the site in question and the resources available to it.

What can be offered is some tips and tricks to get you thinking about things and to perhaps introduce ideas that will help you along the way. 
If yu are reading this document and you've learnt a thing or two about configuring caching on your site share your learnings by adding to the points here.

* Plan it. It's a complex thing. Understand your site, understand your system, and really think how users will be using it all.
* If you've got a small site the gains aren't likely to be significant, if you've got a large site getting this right can lead to a substantial boost in performance.
* When looking at cache backends really research the advantages and disadvantages of each. Keep your site in mind when thinking about them. Depending upon your site you may find that no one cache backend is going to meet the entire needs of your site and that you will benefit from having a couple of backends at your disposal.
* Things aren't usually as simple as installing a cache backend and then using it. Pay attention to configuration and try to optimise it for your system. Test it separately and have an understanding of its performance before tell Moodle about it. The cache allows you to shift load off the database and reduce page request processing. If for instance you have Memcache installed but your connection has not been optimised for it you may well find yourself in a losing situation before you even tell Moodle about the Memcache server.
* When considering your default store instances keep in mind that they must operate with data sets of varying sizes and frequency. For a large site really your best bet is to look at each cache definition and map it to a store that is best suited for the data it includes and the frequency of access. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Again when mapping store instances to caches really think about the cache you are mapping and make a decision based upon what you understand of your site and what you know about the cache. Cache definitions have been documented [[User:Sam_Hemelryk/Cache definitions]].
* Test your configuration. If you can stress test it even better! If you turn on performance information Moodle will also print cache access information at the bottom of the screen. You can use this to visually check the cache is being used as you expect, and it will give you an indication of where misses etc are occurring.
* Keep an eye on your backend. Moodle doesn't provide a means of monitoring a cache backend and that is certainly something you should keep an eye on. Memcache for instance drops least used data when full to make room for new entries. APC on the other hand stops accepting data when full. Both will impact your performance if full and you're going to encounter misses. However APC when full is horrible, but it is much faster.

==Other performance testing==

Two links that might be useful to anyone considering testing performance on their own servers:

* [http://www.iteachwithmoodle.com/2012/10/12/moodle-performance-testing-how-much-more-horsepower-do-each-new-versions-of-moodle-require/ Moodle performance testing: how much more horsepower do each new versions of Moodle require?]
* [http://www.iteachwithmoodle.com/2012/10/11/how-to-stress-test-your-moodle-server-using-loadstorm/ How to load test your Moodle server using Loadstorm]

==Other performance advice for load-balanced web servers==

# In Moodle 2.4 onwards with load-balanced web servers, don't use the default caching option that stores the data in moodledata on a shared network drive. Use memcached instead. See Tim Hunt's article on http://tjhunt.blogspot.de/2013/05/performance-testing-moodle.html
# In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc.

==More information==
* [[:dev:User:Sam Hemelryk/Cache definitions]] Information on the cache definitions found within Moodle.
* [[:dev:Cache API]] Details of the Cache API.
* [[:dev:Cache API - Quick reference]] A short, code focused page of on the Cache API.
* [[:dev:The Moodle Universal Cache (MUC)]] The original cache specification.

Cache related forum discussions that may help in understanding MUC:
* [https://moodle.org/mod/forum/discuss.php?d=217195 MUC is here, now what?]
* [https://moodle.org/mod/forum/discuss.php?d=226123 Status of MUC?]
* [https://moodle.org/mod/forum/discuss.php?d=222250 Putting cachedir on local disks in cluster]
* [https://moodle.org/mod/forum/discuss.php?d=232122 moodle cachestore_file]

Other:
* [http://jfilip.ca/2013/08/20/moodle-2-4-5-vs-2-5-1-performance-and-muc-apc-cache-store/ Moodle 2.4.5 vs. 2.5.1 performance and MUC APC cache store] blog post by Justin Filip

[[de:Caching]]
[[es:Cacheando]]

Session handling

2014-05-29T22:43:48Z

Samhemelryk: /* Memcached */

{{Server settings}}
An administrator can change the following settings in ''Administration > Site administration > Server > Session Handling''.

Once someone logs in to your Moodle server, the server starts a session. The session data allows the server to track users as they access different pages.

==Use database for session information==

Moodle needs to store the session data in some storage. By default either file or database session storage is selected, this option allows admin to change it. Large installation should use memcached driver described below.

Note that this option disappears after setting the $CFG->session_handler_class in config.php file.

==Timeout==

If users don't load a new page during the amount of time set here, Moodle will end their session and log them out.

Be sure this time frame is long enough to cover the longest test your teachers may offer. If a student is logged out while they are taking a test, their responses to the test questions may be lost.

==Cookie prefix==

Most of the time, you can leave this blank, unless you are running more than one Moodle site on the same server. In this case, you will want to customize the name of the cookie each Moodle site uses to track the session. This enables you to be logged into more than one Moodle site at the same time.

Note: If you change "Cookie prefix" or "Cookie path" you will need to login again as the changes take effect immediately.

==Cookie path==

The relative path to this Moodle installation, this may be used to force sending of Moodle session cookie to parent directories. Invalid values are ignored automatically.

==Cookie domain==

This can be used to send session cookies to higher domains instead of just the server domain. This may be useful for some SSO solutions. Invalid values are ignored automatically.

==Session drivers==
User sessions may be stored in different backends. Session drivers can be configured only in config.php file - see examples in config-dist.php file.

===Memcached===
Memcached session driver is the fastest driver, it requires external memcached server and memcached PHP extension. Server cluster nodes must use shared session storage.

Configuration options in config.php:
<code php>
$CFG->session_handler_class = '\core\session\memcached';
$CFG->session_memcached_save_path = '127.0.0.1:11211';
$CFG->session_memcached_prefix = 'memc.sess.key.';
$CFG->session_memcached_acquire_lock_timeout = 120;
$CFG->session_memcached_lock_expire = 7200; // Ignored if memcached extension <= 2.1.0
</code>

Notes:
* Make sure the memcached server has enough memory.
* Use different prefix when storing sessions from multiple Moodles in one server.
* If memcached extension <= 2.1.0 the locking works differently from other drivers, the lock is expired/released at the end of timeout - see MDL-42485.
* Don't use the same memcached server for both sessions and MUC. Events triggering MUC caches to be purged leads to MUC purging the memcached server.

For windows users, PHP.net only supplies binaries for memcache, (and not memcached). (http://windows.php.net/downloads/pecl/releases/)

(As of 2.7, two different contribs exist for memcache session handling - see MDL-42011 - it seems the OU one doesn't use prefix/lock_expire for some reason... possibly better to use the catalyst patch, where the only difference to the above config.php is the spelling of memcache(d))

===Files===
This driver is used by default in new installation.

Configuration options in config.php:
<code php>
$CFG->session_handler_class = '\core\session\file';
$CFG->session_file_save_path = $CFG->dataroot.'/sessions';
</code>

Notes:
* File based sessions require file system that supports file locking.

===Database===
This type of driver was used by default in Moodle 2.0-2.5

<code php>
$CFG->session_handler_class = '\core\session\database';
$CFG->session_database_acquire_lock_timeout = 120;
</code>

Notes:
* DB sessions are not compatible with MyISAM database engine.
* If you are using MySQL/MariaDB make sure that \'max_allowed_packet\' in my.cnf (or my.ini) is at least 4M.
* The performance is relatively low, it is not recommended for large sites.

==See also==

*[[Sessions FAQ]]

[[cs:admin/setting/sessionhandling]]
[[ja:セッションハンドリング]]
[[de:Sitzungsinformationen]]

Session handling

2014-05-29T22:43:09Z

Samhemelryk: /* Memcached */

Användare:Sam Hemelryk

2011-09-23T02:47:09Z

Samhemelryk: Replaced content with "Hi, I maintain my profile on the dev wiki. You can find it here: https://docs.moodle.org/dev/User:Sam_Hemelryk"

Hi, I maintain my profile on the dev wiki.
You can find it here:

https://docs.moodle.org/dev/User:Sam_Hemelryk

Användare:Sam Hemelryk/My Moodle Git workflow

2011-09-23T02:30:50Z

Samhemelryk: Replaced content with "Hi, this doc has previously existed in a couple of locations and has now been consolidated to just one location. You can now find it here: https://docs.moodle.org/dev/User:Sa..."

Hi, this doc has previously existed in a couple of locations and has now been consolidated to just one location.
You can now find it here:

https://docs.moodle.org/dev/User:Sam_Hemelryk/My_Moodle_Git_workflow

PLEASE DO NOT EDIT THIS PAGE, feel free to edit the linked page above.

Cheers
Sam

Themes 2.0 adding courses and categories to the custom menu

2011-05-06T03:17:56Z

Samhemelryk: /* A word of warning */

Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. 
The reason for writing this tutorial is simply that this seems to be a request that is being made over and over again.

==Before we get started==
Please please please make sure you are familiar with the other Themes 2.0 tutorials before attempting this tutorial.

In particular I would recommend being familiar with the following documents and tutorials as I'm going to move through this tutorial at a fast pace.

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]

Providing you are familiar with the above you should manage just fine with this tutorial and if you ever get stuck don't forget to ask in the theme's forums, there is always someone around who can help.

===Two words of warning===
'''First''' Just a quick warning here, this tutorial adds all categories and courses to the navigation, on a site with a large number of categories and/or courses this will be a performance hit. I would strongly suggest that if you are looking to implement something such as this on a large site that you talk to your system admin and/or resident developer about setting up a shared cache and caching the result of the get_course_category_tree method... it is a technical task but doing so will improve the performance of this extension on a large site.

'''Second''' If you have categories nested 2 or more deep then there is a chance that you will run into display problems with the custom menu on some themes... not too much you can do about it but if you encounter it with a core theme and you fix it perhaps you would be kind enough to create an issue in our bug tracker and share the fix. That'd being said this is only a hunch I have all of the theme's may be fine :)

==Laying the ground work==
Alright I don't want this tutorial to drag out so get ready. For this tutorial I am just going to extend the standard theme, I'm going to make absolutely no changes to the design and layout of the theme or the custom menu I am ONLY going to add a list of courses and categories to it.

So to begin with within your Moodle themes directory create a new directory coursecategorymenu in which I want you to create three files, the first config.php which is obviously required, the second renderers.php as we are going to achieve this by overriding a renderer, and the third the English language file for this theme.

You should end up with:

* moodle/theme/coursecategorymenu
* moodle/theme/coursecategorymenu/config.php
* moodle/theme/coursecategorymenu/renderers.php
* moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php

==config.php==
Obviously because all I want to base this theme off the standard theme and only override a renderer the config.php file is going to be VERY basic, so much so that I'm not going to go into details about it, as you've read the recommended tutorials you will already be completely familiar with everything it is doing.

<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

==renderers.php==
This is of course where the magic is going to happen.

In my case I want to add a branch to the end of the custom menu titled ''Courses'' and then I want to add the category and course structure to that branch.

So like the [[Development:Themes 2.0 extending the custom menu|extending the custom menu]] tutorial I will be overriding the core renderer and I will also be overriding the render_custom_menu_method.

So the code for this:
<code php>
class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

You'll notice that is identical to the starting code for the extending custom menu tutorial.

The difference comes in the code that we are going to populate it with:

<code php>
require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
</code>

So lets work through this code line by line:

<code php>
require_once($CFG->dirroot.'/course/lib.php');
</code>

This we need to do because we are going to use part of the course API, in particular a function called ''get_course_category_tree()'' that we'll look at shortly.
The function we want to use exists within the course lib file and in order to be sure its always available we need to include this file.

<code php>
$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);
</code>

This line adds the ''Courses'' branch to the custom menu into which we will add the category and course structure.

<code php>
$categorytree = get_course_category_tree();
</code>

This line is the main one to note, here we are calling a Moodle function that will return us a category and course tree, because all courses need to be within a category the initial return is an array of categories, each category in the array will have the properties of the category as well as two additional properties, the first property '''categories''' is an array containing all of this categories sub categories, and the second property '''courses''' is also an array containing all of the courses within this category.

<code php>
$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}
</code>

Now these lines of code deal with the initial array of categories returned by the ''get_course_category_tree'' function.

In this case we iterate of the initial categories and for each one we call a method of the renderer '''add_category_to_custommenu''' now those of you who are clued in will realise that this method doesn't exist yet... and you are correct - we need to write this method which is what we will look at next. 
As a heads up the reason that we need to write another method is because we need a method that we can call recursivily as there are potentially infinite category branches all contains sub categories and courses.

<code php>
return parent::render_custom_menu($menu);
</code>

The final line of code simply calls the original render_custom_menu method now that we have extended the custom menu as we want.

Now that we have looked at the render_custom_menu lets write the function we just talked about '''add_category_to_custommenu'''. 
This function as discussed above is special in that we intend to call it recursively, that means we want to call it once for EVERY category that exists in the tree. We will need to give it the categories parent menu item as well as the category object we want added.

Lets look at the code for this method:

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}
</code>

Ok so that doesn't look too bad, as you've already read the extending custom menu tutorial you will be able to spot some of the things it is doing, none the less lets walk through the code.

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
.....
}
</code>

First up the function definition, this function is going to be private because only we need it and it is going to take two arguments, the first $parent has to be a custom_menu_item, the second $category has to be the category object.

The objective of this method is to add a category node to the custom menu, in regards to our arguments we want to add $category as a child of $parent.

<code php>
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
</code>

This line looks very familiar, here we are adding a node for the category to the $parent, we are also collecting the newly added node as $branch.

<code php>
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
</code>

Now this code is responsible for add the sub categories of this category to the menu as well, first we need to check that the categories property isn't empty (if it is there is nothing to add).
Assuming there are sub categories we need go through each of them and call this method on them so that they get added to the custom menu as well. 
This is called recursion.

<code php>
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
</code>
This code is very similar to the above code except that we are looking at the courses within this category. 
Also note that this bit of code doesn't call add_category_to_custommenu recursivily, it doesn't need to as courses are the last bit we want to display on the menu within the category.

And that is it, time to tidy up and we're done.

==Finishing up==
All of the code is written now there is only last thing to do in order to tidy up however and that is to add the ''courses'' string that we used earlier... did you spot it?

This is of course very easy, open up '''moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php''' and add the following:

<code php>
<?php

$string['courses'] = 'Courses';
</code>

And we're done. If you now browse to your site and change to the new theme you should see the Courses branch at the end of your custom menu that contains all of the categories, sub categories and courses for your site.

==Full source==
===config.php===
<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>
===renderers.php===
<code php>
<?php

class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
global $CFG;

require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
}

protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}

}
</code>
===lang/en/theme_coursecategorymenu.php===
<code php>
<?php

$string['courses'] = 'Courses';
</code>

==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Themes 2.0 adding courses and categories to the custom menu

2011-05-06T03:11:50Z

Samhemelryk: /* Before we get started */

Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. 
The reason for writing this tutorial is simply that this seems to be a request that is being made over and over again.

==Before we get started==
Please please please make sure you are familiar with the other Themes 2.0 tutorials before attempting this tutorial.

In particular I would recommend being familiar with the following documents and tutorials as I'm going to move through this tutorial at a fast pace.

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]

Providing you are familiar with the above you should manage just fine with this tutorial and if you ever get stuck don't forget to ask in the theme's forums, there is always someone around who can help.

===A word of warning===
Just a quick warning here, this tutorial adds all categories and courses to the navigation, on a site with a large number of categories and/or courses this will be a performance hit. I would strongly suggest that if you are looking to implement something such as this on a large site that you talk to your system admin and/or resident developer about setting up a shared cache and caching the result of the get_course_category_tree method... it is a technical task but doing so will improve the performance of this extension on a large site.

==Laying the ground work==
Alright I don't want this tutorial to drag out so get ready. For this tutorial I am just going to extend the standard theme, I'm going to make absolutely no changes to the design and layout of the theme or the custom menu I am ONLY going to add a list of courses and categories to it.

So to begin with within your Moodle themes directory create a new directory coursecategorymenu in which I want you to create three files, the first config.php which is obviously required, the second renderers.php as we are going to achieve this by overriding a renderer, and the third the English language file for this theme.

You should end up with:

* moodle/theme/coursecategorymenu
* moodle/theme/coursecategorymenu/config.php
* moodle/theme/coursecategorymenu/renderers.php
* moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php

==config.php==
Obviously because all I want to base this theme off the standard theme and only override a renderer the config.php file is going to be VERY basic, so much so that I'm not going to go into details about it, as you've read the recommended tutorials you will already be completely familiar with everything it is doing.

<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

==renderers.php==
This is of course where the magic is going to happen.

In my case I want to add a branch to the end of the custom menu titled ''Courses'' and then I want to add the category and course structure to that branch.

So like the [[Development:Themes 2.0 extending the custom menu|extending the custom menu]] tutorial I will be overriding the core renderer and I will also be overriding the render_custom_menu_method.

So the code for this:
<code php>
class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

You'll notice that is identical to the starting code for the extending custom menu tutorial.

The difference comes in the code that we are going to populate it with:

<code php>
require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
</code>

So lets work through this code line by line:

<code php>
require_once($CFG->dirroot.'/course/lib.php');
</code>

This we need to do because we are going to use part of the course API, in particular a function called ''get_course_category_tree()'' that we'll look at shortly.
The function we want to use exists within the course lib file and in order to be sure its always available we need to include this file.

<code php>
$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);
</code>

This line adds the ''Courses'' branch to the custom menu into which we will add the category and course structure.

<code php>
$categorytree = get_course_category_tree();
</code>

This line is the main one to note, here we are calling a Moodle function that will return us a category and course tree, because all courses need to be within a category the initial return is an array of categories, each category in the array will have the properties of the category as well as two additional properties, the first property '''categories''' is an array containing all of this categories sub categories, and the second property '''courses''' is also an array containing all of the courses within this category.

<code php>
$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}
</code>

Now these lines of code deal with the initial array of categories returned by the ''get_course_category_tree'' function.

In this case we iterate of the initial categories and for each one we call a method of the renderer '''add_category_to_custommenu''' now those of you who are clued in will realise that this method doesn't exist yet... and you are correct - we need to write this method which is what we will look at next. 
As a heads up the reason that we need to write another method is because we need a method that we can call recursivily as there are potentially infinite category branches all contains sub categories and courses.

<code php>
return parent::render_custom_menu($menu);
</code>

The final line of code simply calls the original render_custom_menu method now that we have extended the custom menu as we want.

Now that we have looked at the render_custom_menu lets write the function we just talked about '''add_category_to_custommenu'''. 
This function as discussed above is special in that we intend to call it recursively, that means we want to call it once for EVERY category that exists in the tree. We will need to give it the categories parent menu item as well as the category object we want added.

Lets look at the code for this method:

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}
</code>

Ok so that doesn't look too bad, as you've already read the extending custom menu tutorial you will be able to spot some of the things it is doing, none the less lets walk through the code.

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
.....
}
</code>

First up the function definition, this function is going to be private because only we need it and it is going to take two arguments, the first $parent has to be a custom_menu_item, the second $category has to be the category object.

The objective of this method is to add a category node to the custom menu, in regards to our arguments we want to add $category as a child of $parent.

<code php>
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
</code>

This line looks very familiar, here we are adding a node for the category to the $parent, we are also collecting the newly added node as $branch.

<code php>
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
</code>

Now this code is responsible for add the sub categories of this category to the menu as well, first we need to check that the categories property isn't empty (if it is there is nothing to add).
Assuming there are sub categories we need go through each of them and call this method on them so that they get added to the custom menu as well. 
This is called recursion.

<code php>
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
</code>
This code is very similar to the above code except that we are looking at the courses within this category. 
Also note that this bit of code doesn't call add_category_to_custommenu recursivily, it doesn't need to as courses are the last bit we want to display on the menu within the category.

And that is it, time to tidy up and we're done.

==Finishing up==
All of the code is written now there is only last thing to do in order to tidy up however and that is to add the ''courses'' string that we used earlier... did you spot it?

This is of course very easy, open up '''moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php''' and add the following:

<code php>
<?php

$string['courses'] = 'Courses';
</code>

And we're done. If you now browse to your site and change to the new theme you should see the Courses branch at the end of your custom menu that contains all of the categories, sub categories and courses for your site.

==Full source==
===config.php===
<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>
===renderers.php===
<code php>
<?php

class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
global $CFG;

require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
}

protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}

}
</code>
===lang/en/theme_coursecategorymenu.php===
<code php>
<?php

$string['courses'] = 'Courses';
</code>

==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Användare:Sam Hemelryk

2011-05-06T03:03:21Z

Samhemelryk:

Hello, I am a senior developer at Moodle and have been with the project since the June 2009.

For those interested I have written a tutorial on my Moodle development process with Git: [[User:Sam_Hemelryk/My_Moodle_Git_workflow]]

A short list of helpful documents I have created:

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]] - A quick step by step guide to creating your first theme.
* [[Development:Themes 2.0 overriding a renderer]] - A tutorial on creating a custom renderer and changing the HTML Moodle produces.
* [[Development:Themes 2.0 How to use images within your theme]] - Explains how to use and override images within your theme.
* [[Development:Themes 2.0 adding a settings page]] - Looks at how to add a setting page making your theme easily customisable.
* [[Development:Themes 2.0 extending the custom menu]] - Customising the custom menu.
* [[Development:Themes 2.0 adding courses and categories to the custom menu]] - Extending the custom menu further adding all categories + courses
* [[Development:Themes 2.0 how to make the dock horizontal]] - Modifying the dock to make it horizontal.
* [[Development:Themes 2.0 adding upgrade code]]
* [[Development:Styling and customising the dock]] - How to style and customise the dock.
* [[Development:Using jQuery with Moodle 2.0]]

Themes 2.0 adding courses and categories to the custom menu

2011-05-06T03:02:43Z

Samhemelryk: /* See also */

Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. 
The reason for writing this tutorial is simply that this seems to be a request that is being made over and over again.

==Before we get started==
Please please please make sure you are familiar with the other Themes 2.0 tutorials before attempting this tutorial.

In particular I would recommend being familiar with the following documents and tutorials as I'm going to move through this tutorial at a fast pace.

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]

Providing you are familiar with the above you should manage just fine with this tutorial and if you ever get stuck don't forget to ask in the theme's forums, there is always someone around who can help.

==Laying the ground work==
Alright I don't want this tutorial to drag out so get ready. For this tutorial I am just going to extend the standard theme, I'm going to make absolutely no changes to the design and layout of the theme or the custom menu I am ONLY going to add a list of courses and categories to it.

So to begin with within your Moodle themes directory create a new directory coursecategorymenu in which I want you to create three files, the first config.php which is obviously required, the second renderers.php as we are going to achieve this by overriding a renderer, and the third the English language file for this theme.

You should end up with:

* moodle/theme/coursecategorymenu
* moodle/theme/coursecategorymenu/config.php
* moodle/theme/coursecategorymenu/renderers.php
* moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php

==config.php==
Obviously because all I want to base this theme off the standard theme and only override a renderer the config.php file is going to be VERY basic, so much so that I'm not going to go into details about it, as you've read the recommended tutorials you will already be completely familiar with everything it is doing.

<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

==renderers.php==
This is of course where the magic is going to happen.

In my case I want to add a branch to the end of the custom menu titled ''Courses'' and then I want to add the category and course structure to that branch.

So like the [[Development:Themes 2.0 extending the custom menu|extending the custom menu]] tutorial I will be overriding the core renderer and I will also be overriding the render_custom_menu_method.

So the code for this:
<code php>
class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

You'll notice that is identical to the starting code for the extending custom menu tutorial.

The difference comes in the code that we are going to populate it with:

<code php>
require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
</code>

So lets work through this code line by line:

<code php>
require_once($CFG->dirroot.'/course/lib.php');
</code>

This we need to do because we are going to use part of the course API, in particular a function called ''get_course_category_tree()'' that we'll look at shortly.
The function we want to use exists within the course lib file and in order to be sure its always available we need to include this file.

<code php>
$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);
</code>

This line adds the ''Courses'' branch to the custom menu into which we will add the category and course structure.

<code php>
$categorytree = get_course_category_tree();
</code>

This line is the main one to note, here we are calling a Moodle function that will return us a category and course tree, because all courses need to be within a category the initial return is an array of categories, each category in the array will have the properties of the category as well as two additional properties, the first property '''categories''' is an array containing all of this categories sub categories, and the second property '''courses''' is also an array containing all of the courses within this category.

<code php>
$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}
</code>

Now these lines of code deal with the initial array of categories returned by the ''get_course_category_tree'' function.

In this case we iterate of the initial categories and for each one we call a method of the renderer '''add_category_to_custommenu''' now those of you who are clued in will realise that this method doesn't exist yet... and you are correct - we need to write this method which is what we will look at next. 
As a heads up the reason that we need to write another method is because we need a method that we can call recursivily as there are potentially infinite category branches all contains sub categories and courses.

<code php>
return parent::render_custom_menu($menu);
</code>

The final line of code simply calls the original render_custom_menu method now that we have extended the custom menu as we want.

Now that we have looked at the render_custom_menu lets write the function we just talked about '''add_category_to_custommenu'''. 
This function as discussed above is special in that we intend to call it recursively, that means we want to call it once for EVERY category that exists in the tree. We will need to give it the categories parent menu item as well as the category object we want added.

Lets look at the code for this method:

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}
</code>

Ok so that doesn't look too bad, as you've already read the extending custom menu tutorial you will be able to spot some of the things it is doing, none the less lets walk through the code.

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
.....
}
</code>

First up the function definition, this function is going to be private because only we need it and it is going to take two arguments, the first $parent has to be a custom_menu_item, the second $category has to be the category object.

The objective of this method is to add a category node to the custom menu, in regards to our arguments we want to add $category as a child of $parent.

<code php>
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
</code>

This line looks very familiar, here we are adding a node for the category to the $parent, we are also collecting the newly added node as $branch.

<code php>
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
</code>

Now this code is responsible for add the sub categories of this category to the menu as well, first we need to check that the categories property isn't empty (if it is there is nothing to add).
Assuming there are sub categories we need go through each of them and call this method on them so that they get added to the custom menu as well. 
This is called recursion.

<code php>
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
</code>
This code is very similar to the above code except that we are looking at the courses within this category. 
Also note that this bit of code doesn't call add_category_to_custommenu recursivily, it doesn't need to as courses are the last bit we want to display on the menu within the category.

And that is it, time to tidy up and we're done.

==Finishing up==
All of the code is written now there is only last thing to do in order to tidy up however and that is to add the ''courses'' string that we used earlier... did you spot it?

This is of course very easy, open up '''moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php''' and add the following:

<code php>
<?php

$string['courses'] = 'Courses';
</code>

And we're done. If you now browse to your site and change to the new theme you should see the Courses branch at the end of your custom menu that contains all of the categories, sub categories and courses for your site.

==Full source==
===config.php===
<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>
===renderers.php===
<code php>
<?php

class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
global $CFG;

require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
}

protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}

}
</code>
===lang/en/theme_coursecategorymenu.php===
<code php>
<?php

$string['courses'] = 'Courses';
</code>

==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Themes 2.0 extending the custom menu

2011-05-06T03:02:19Z

Samhemelryk: /* See also */

{{Template:Themes}}{{Moodle 2.0}}This is a quick tutorial that will quickly run you through extending the custom menu by overriding your theme's renderer.

This tutorial is primarily PHP coding, and ranges from is a moderate to advanced difficulty.

==Before we begin==
First things first you need to know a little something about the custom menu. As you are undoubtedly aware the custom menu is populated from a specially crafted lines that you enter within the custommenu admin setting on the theme settings page under appearance in the settings block. Once the admin has entered the custom menu items into the setting and saved them the following steps are what they go through in order to be displayed:
# The theme calls '''''$OUTPUT->custom_menu()''''' which is the core_renderers method responsible for producing the custom menu.
# core_renderer::custom_menu() does the following:
## Checks to make sure the admin has added custom menu items.
## Creates a new '''''custom_menu''''' object. When the custom menu object is created it turns what ever the admin entered into a structured array of information.
## Calls core_renderer::render_custom_menu and gives it the custom_menu object.
# core_renderer::render_custom_menu is where the magic happens, it does the following.
## First it checks to make sure custom menu contains items.
## Initialises the JavaScript for the custom menu, this is the YUI3 menunode component.
## Creates the HTML base of the custom menu
## Then iterates through all of the items and calls the custom menu and passes them to another function that will turn them into HTML correctly.

I'm going to stop there, as there is no need at this point to go into any more detail. Don't worry if you are a little lost, it will get clearer as we start working through code.

In this tutorial will we look at how to extend the theme's renderer in two different way.
# Add a dynamic My Courses branch to the custom menu that will display all of the courses a user is enrolled in.
# Get the custom menu to fetch strings from the language files rather than just static strings you type, allowing for a translatable custom menu.

Before you start into this tutorial you should have read the following tutorials I have written, or at least have a good knowledge of the the Moodle 2.0 theme engine and Moodle development.
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]

As this is a quick tutorial I am going to assume you have created a theme, tested it and got it all working, and are already well on your way to becoming a theme guru.

Because the pressure is on to get Moodle 2.0 out the door this will be a quick tutorial, please if you have any questions or need a hand ask in the theme's forum.

==Getting started==
Alright, as you have already have your theme ready to go preparation is pretty simple, we are going to go through and create (if you haven't already) the following files:
* theme/themename/lib.php
* theme/themename/renderers.php
* theme/themename/lang/en/themename.php

Next step open up your themes config.php file and add the following configuration option to it (at the bottom):
<code php>
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

If you've already got a custom renderer you will already have this line.

And thats it! now we move on to extending the custom menu.

==Adding the My Courses branch==
So the point of this extension is to add a '''My Courses''' branch to the end of the custom menu.

The plan is to have the my courses branch and then within that branch have an entry for every course the user is enrolled in, much the same as the my courses branch of the navigation.

In order to achieve this we need to add some items to the custom menu, the my courses branch and all of the courses within it. We can do this overriding the core_renderers '''render_custom_menu''' method, of more accuratly create our own render_custom_menu method that adds our items and then calls the original. Remember we only need to add items, we don't need to change what is being produced.

So within our renderers.php file add the following code:
<code php>
class theme_themename_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

So that is pretty simple right?!

Here we are defining our core_renderer which will contain our overridden method ''render_custom_menu'' which we have also defined there.
Note the definition of the render_custom_menu must be the same as the original, this means it must be '''protected''' and it must take one argument '''custom_menu $menu'''.

Next step is to write the code for our render_custom_menu method. As stated above we want to add a branch and courses to the end of it which we can do with the following bit of code:
<code php>
$mycourses = $this->page->navigation->get('mycourses');

if (isloggedin() && $mycourses && $mycourses->has_children()) {
$branchlabel = get_string('mycourses');
$branchurl = new moodle_url('/course/index.php');
$branchtitle = $branchlabel;
$branchsort = 10000;

$branch = $menu->add($branchlabel, $branchurl, $branchtitle, $branchsort);

foreach ($mycourses->children as $coursenode) {
$branch->add($coursenode->get_content(), $coursenode->action, $coursenode->get_title());
}
}

return parent::render_custom_menu($menu);
</code>

So how this all works....
<code php>
$mycourses = $this->page->navigation->get('mycourses');
</code>
This line is a little bit of smarts, what we are doing is getting the mycourses branch from the navigation. We could make all of the database calls and work it all out ourselves but this will be much better for performance and is easier!

The next line of code is an ''if'' statement that checks three things
# The user is logged in, you must be logged in to see your courses.
# That we have a mycourses object, if the user isn't enrolled in anything they won't have a mycourses object.
# The the mycourses object has children, if it exists it should but it is better to be safe than get errors.

Within the if statement we are doing two main things happening, the first is to add the branch to the menu:
<code php>
$branchlabel = get_string('mycourses');
$branchurl = new moodle_url('/course/index.php');
$branchtitle = $branchlabel;
$branchsort = 10000;

$branch = $menu->add($branchlabel, $branchurl, $branchtitle, $branchsort);
</code>
When adding a new branch we need to give it three things:
# A label '''$branchlabel'''.
# A URL '''$branchurl'''.
# A title '''$branchtitle''' in this case I have just set it to the same as $branchlabel because I am lazy, you can make it anything you want.
# A sort order '''$branchsort''' this just needs to be high enough that it ends up on last place where we want it. Make it small to put it at the front.

The final line of code from above simply adds it to the menu and collects a reference to it so that we can add courses to it.

The second thing being done in the if statement is iterate through all of the courses in the mycourses object and add them to the branch. That is done with the following bit of code:
<code php>
foreach ($mycourses->children as $coursenode) {
$branch->add($coursenode->get_content(), $coursenode->action, $coursenode->get_title());
}
</code>
So here we go through all of the mycourses objects children (which we know will be courses) and for each one we add an item to the branch.
When adding items here, like above, we need to give it the following:
# A label '''$coursenode->get_content()''' returns us the text in the navigation node.
# A url '''$coursenode->action''' which is the URL for the node.
# A title '''$coursenode->get_title()'''.

In this case we don't need to set a sort order because the navigation has already ordered them correctly!

So now we are through the if statement and there is only one line of code left:
<code php>
return parent::render_custom_menu($menu);
</code>

This line of code simply calls the original '''core_renderer::render_custom_menu''' method to do all of the remaining work. Because we don't need to change the display at all we don't need to redo what it does, we can just use it!.

How easy way that?! it's all done, if you open you site in your browser and log in the custom menu should now contain a my courses branch (providing you are enrolled in courses).

==Loading labels from language files==
If you run a site that is available in more than one language '''and''' you want to make use of the custom menu then you will probably be very interested in this.

The idea behind this extension is to load the labels and titles that get shown in the custom menu from the theme's language files rather than having just the fixed strings you type into the admin setting.

In order to achieve this we need to do somehow override the custom_menu_item class so that it loads the strings from the database if they match a special pattern. We can then update the admin setting to use our patter and wallah! things should load from the language files.

What makes this tricky is that we can't just override the custom_menu_item class, we also need to override the thing that makes create custom_menu_item instances which in this case is the core_renderer::render_custom_menu_item method.

===Overriding the custom_menu_item class===
Within your themes lib.php file add the following lines of code:
<code php>
class transmuted_custom_menu_item extends custom_menu_item {
public function __construct(custom_menu_item $menunode) {
// Our code will go here
}
}
</code>
What we have done here is create a overridden custom_menu_item class called '''transmuted_custom_menu_item'''.
Within that class we are overriding the constructor, the constructor will be given the original menu item and we will need to instatiate this object with the properties of the original.
Once we have instantiated it with the properties of the original we can alter then as we like.

Next we need to write the contents of the ''__construct'' method:
<code php>
parent::__construct($menunode->get_text(), $menunode->get_url(), $menunode->get_title(), $menunode->get_sort_order(), $menunode->get_parent());
$this->children = $menunode->get_children();

$matches = array();
if (preg_match('/^\[\[([a-zA-Z0-9\-\_\:]+)\]\]$/', $this->text, $matches)) {
try {
$this->text = get_string($matches[1], 'theme_themename');
} catch (Exception $e) {
$this->text = $matches[1];
}
}

$matches = array();
if (preg_match('/^\[\[([a-zA-Z0-9\-\_\:]+)\]\]$/', $this->title, $matches)) {
try {
$this->title = get_string($matches[1], 'theme_themename');
} catch (Exception $e) {
$this->title = $matches[1];
}
}
</code>

So there are essentially four things going on here.

Firs we need to populate this object with the properties of the original item. We do this by calling the original constructor with $menunode's properties as shown below.
<code php>
parent::__construct($menunode->get_text(), $menunode->get_url(), $menunode->get_title(), $menunode->get_sort_order(), $menunode->get_parent());
</code>

Next we need to copy the properties that arn't included in the original constructor, in this case there is only one '''$this->children'''. This should be an array of all of this items children (sub items).
<code php>
$this->children = $menunode->get_children();
</code>

Now that we have all the properties set up we can modify them. In our case we want to check if the items label and title match a special pattern and if they do we want to fetch them from the language file instead.
The special pattern is '''<nowiki>[[stringname]]</nowiki>''' where stringname is the name of the string that we want to get from the language file.
<code php>
$matches = array();
if (preg_match('/^\[\[([a-zA-Z0-9\-\_\:]+)\]\]$/', $this->text, $matches)) {
try {
$this->text = get_string($matches[1], 'theme_themename');
} catch (Exception $e) {
$this->text = $matches[1];
}
}
</code>
In the code above we do the following:
# Create an array $matches which will hold the stringname if the label matches the pattern.
# We attempt to match the label to the pattern. If it does the $matches array now contains the stringname.
# If it did match we call get string as shown.

The third and final thing is to do the above again but this time for the title.

With that done our transmutable_custom_menu_item class is complete. Next step is to make use of it.

===Overriding render_custom_menu_item===
The next step is make use of the transmutable_custom_menu_item class we created previously.

We can do this by overriding the '''core_renderer::render_custom_menu_item''' method within our renderers.php file as shown below.
<code php>
protected function render_custom_menu_item(custom_menu_item $menunode) {
$transmutedmenunode = new transmuted_custom_menu_item($menunode);
return parent::render_custom_menu_item($transmutedmenunode);
}
</code>
This is pretty simply, essentially we are using our transmuted_custom_menu_item class like a façade to the original custom_menu_item class by creating a new transmuted instance using the original.
Once we have the transmuted object we can call the original render_custom_menu_item method with it.

And that is it! Congratulations if you got it this far.

The only thing left to do is add strings to your themes language file as required!

==Complete source==
<code php>
// theme/themename/renderers.php

class theme_themename_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {

$mycourses = $this->page->navigation->get('mycourses');

if (isloggedin() && $mycourses && $mycourses->has_children()) {
$branchlabel = get_string('mycourses');
$branchurl = new moodle_url('/course/index.php');
$branchtitle = $branchlabel;
$branchsort = 10000;

$branch = $menu->add($branchlabel, $branchurl, $branchtitle, $branchsort);

foreach ($mycourses->children as $coursenode) {
$branch->add($coursenode->get_content(), $coursenode->action, $coursenode->get_title());
}
}

return parent::render_custom_menu($menu);
}

protected function render_custom_menu_item(custom_menu_item $menunode) {
$transmutedmenunode = new transmuted_custom_menu_item($menunode);
return parent::render_custom_menu_item($transmutedmenunode);
}

}
</code>

<code php>
// theme/themename/lib.php

class transmuted_custom_menu_item extends custom_menu_item {
public function __construct(custom_menu_item $menunode) {
parent::__construct($menunode->get_text(), $menunode->get_url(), $menunode->get_title(), $menunode->get_sort_order(), $menunode->get_parent());
$this->children = $menunode->get_children();

$matches = array();
if (preg_match('/^\[\[([a-zA-Z0-9\-\_\:]+)\]\]$/', $this->text, $matches)) {
try {
$this->text = get_string($matches[1], 'theme_themename');
} catch (Exception $e) {
$this->text = $matches[1];
}
}

$matches = array();
if (preg_match('/^\[\[([a-zA-Z0-9\-\_\:]+)\]\]$/', $this->title, $matches)) {
try {
$this->title = get_string($matches[1], 'theme_themename');
} catch (Exception $e) {
$this->title = $matches[1];
}
}
}
}
</code>
==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 adding courses and categories to the custom menu]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Themes 2.0

2011-05-06T03:01:57Z

Samhemelryk: /* See also */

{{Template:Themes}}{{Moodle 2.0}}Welcome to the new world of themes within Moodle 2.0!

This document explains how themes work in Moodle and is intended to help you create or modify most themes for Moodle 2.0.

==Introduction==

Moodle themes are responsible for much of the "look" of a Moodle site. They provide the CSS for colours, layouts, fonts and so on, and can also change the structural XHTML code below that.

A theme can either contain all the definitions (completely stand alone) or it can extend an existing theme with one or more customizations.

Most theme developers use the second method and simply add a few new CSS or layout definitions to their new theme. If a definition or element is not found in the new theme, it looks to the "parent" (or up the hierarchy of themes) to find one. It an easy way to achieve just about any look you want for a theme.

==What's new in 2.0==

The theme system was completely redesigned in Moodle 2.0. Known issues have been addressed and new features have been added to meet community requests.

Unfortunately it was not possible to maintain backward compatibility, so all Moodle 1.x themes need to be recreated for Moodle 2.0.

Major changes include:
* Clearer and more consistent CSS classes and IDs throughout all pages in Moodle
* Introduction of layout files (templates) describing overall layout HTML for many different types of pages in Moodle.
* Introduction of renderers, which produce the smaller "parts" of a HTML page. Advanced themes can choose to override these too if they choose.
* Introduction of standard methods for adding Javascript to themes.
* Easier control over icons and images in Moodle.
* The old "standard" theme has been split into two themes:
**'''base''' - contains absolutely basic layout, and
**'''standard''' - which adds CSS to the base theme to make it look like the old standard theme.
* Performance tuning: In normal production mode CSS files are combined into a single optimised file, and both CSS and JavaScript files are minimised to ensure there are no wasted connections or traffic. Files are heavily cached, but also versioned, so that users never need to clear their caches.

==The structure of a theme==

Some important things to know when building good themes:

# '''config.php''' - this file is required in every theme. It defines configuration settings and definitions required to make the theme work in Moodle. These include theme, file, region, default region and options.
# '''Layouts and layout files''' - in config.php there is one definition for each page type (see [[#theme_layouts_table|Appendix A: Theme layouts]] for a list of over 12 types). Each page type definition tells Moodle which layout file will be used, what block regions this page type should display and so on. The layout file contains the HTML and the minimum PHP required to display basic structure of pages. (If you know Moodle 1.9, it's like a combination of header.html and footer.html).
# '''The base theme''' - is not intended to be used for production sites. It sets up the simplest possible generic layout and includes only CSS essential to that layout ''or'' to Moodle as a whole. It tries not to make any unnecessary rules and makes as few assumptions as possible. It's the perfect base on which to start designing a theme, as there are very few colours, borders, margins, and alignments to override. You can just start adding what you need.

===Files and folders===
A theme's files are placed in a folder with under moodle/theme folder and have subfolders. They are laid out like this:

{| class="nicetable"
|-
! Directory
! File
! Description
|-
|
| /config.php
| Contains all of the configuration and definitions for each theme
|-
|
| /lib.php
| Contains speciality classes and functions that are used by theme
|-
|
| /renderers.php
| Contains any custom renderers for the theme.
|-
|
| /settings.php
| Contains custom theme settings. These local settings are defined by the theme allowing the theme user to easily alter something about the way it looks or operates. (eg a background colour, or a header image)
|-
| /javascript/
|
| All specialty JavaScript files the theme requires should be located in here.
|-
| /lang/
|
| Any special language files the theme requires should be located in here.
|-
| /layout/
|
| Contains the layout files for the theme.
|-
| /pix/
|
| Contains any images the theme makes use of either in CSS or in the layout files.
|-
| /pix
| /favicon.ico
| The favicon to display for this theme.
|-
| /pix
| /screenshot.jpg
| A screenshot of the theme to be displayed in on the theme selection screen.
|-
| /style
|
| Default location for CSS files.
|-
|
|/*.css
|CSS files the theme requires
|}

There are also several other places that stylesheets can be included from (see the CSS how and why section below).

==Theme options==
All theme options are set within the config.php file for the theme. The settings that are most used are: parents, sheets, layouts, and javascripts. Have a look at the '''[[#theme_options_table|theme options table]]''' for a complete list of theme options which include lesser used specialised or advanced settings.

===Basic theme config example===
Lets have a look at a basic theme configuration file and the different bits that make it up:
<code php>
$THEME->name = 'newtheme';

$THEME->parents = array(
'base'
);

$THEME->sheets = array(
'admin',
'blocks',
'calendar',
'course',
'grade',
'message',
'question',
'user'
);

$THEME->layouts = array(
'base' => array(
'theme' => 'newtheme',
'file' => 'general.php',
'regions' => array(),
),
'standard' => array(
'theme' => 'newtheme',
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post',
)
//.......
);

$THEME->javascripts_footer = array(
'navigation'
);
</code>

===Basic theme example settings explained===
First up you will notice everything is added to $THEME. This is the theme's configuration object, it is created by Moodle using default settings and is then updated by whatever settings you add to it.

The first setting, $THEME->name is the theme's name. This should simply be whatever your theme's name is, most likely whatever you named your theme directory.

$THEME->parents defines the themes that the theme will extend. In this case it is extending only the base theme.

After this is the $THEME->sheets array containing the names of the CSS stylesheets to include for this theme. Note that it is just the name of the stylesheet and does not contain the directory or the file extension. Moodle assumes that the theme's stylesheets will be located in the styles directory of the theme and have .css as an extension.

Next we see the $THEME->layouts definition. In this example, two layouts have been defined to override the layouts from the base theme. For more information see the [[#Layouts|layouts]] section below.

The final setting is to include a JavaScript file, as $THEME->javascripts_footer. Much like stylesheets, you only need to provide the files name. Moodle will assume it is in your themes JavaScript directory and be a .js file.

'''''Note''''': When you first begin writing themes, make sure you take a look at the configuration files of the other themes that get shipped with Moodle. You will get a good picture of how everything works, and what is going on in a theme, simply by reading it and taking notice of what it is including or excluding.

==CSS==
===Locations of CSS files===
First lets look at where CSS can be included from within Moodle:
; \theme\themename\styles\*.css : This is the default location for all of the stylesheets that are used by a theme and the place which should be used by a theme designer.

New theme developers should note that the order in which CSS files are found and included creates a hierarchy. This order ensures that the rules, within a theme's style sheets, take precedence over identical rules in other files that may have been introduced before. This can both extend another files definitions (see parent array in the config file) and also ensures that the current theme's CSS rules/definitions have the last say.

There are other locations that can be used (although very rarely) to include CSS in a page. A developer of a php file can manually specify a stylesheet from anywhere within Moodle, like the database. Usually, if code is doing this, it is because there is a non-theme config or plugin setting that contains information requires special CSS information. As a theme designer you should be aware of, but not have to worry about, these locations of CSS files. Here are some examples:

; {pluginpath}\styles.css e.g. \block\blockname\styles.css or \mod\modname\styles.css : Every plugin can have its own styles.css file. This file should only contain the required CSS rules for the module and should not add anything to the look of the plugin such as colours, font sizes, or margins other than those that are truly required. Theme specific styles for a plugin should be located within the themes styles directory.
; {pluginpath}\styles_themename.css : This should only ever be used by plugin developers. It allows them to write CSS that is designed for a specific theme without having to make changes to that theme. You will notice that this is never used within Moodle and is designed to be used only by contributed code.

As theme designers, we will only use the first method of introducing CSS: adding rules to a stylesheet file located in the themes styles directory.

===Moodle's core CSS organisation===
The next thing to look at is the organisation of CSS and rules within a theme. Although as a theme designer it is entirely up to you as to how you create and organise your CSS. Please note that within the themes provided in the standard install by Moodle there is a very clear organisation of CSS.

First is the pagelayout.css file. This contains the CSS required to give the layouts their look and feel. It doesn't contain any rules that affect the content generated by Moodle.

Next is the core.css file. If you open up core you will notice that it contains all manner of general (usually simple) rules that don't relate to a specific section of Moodle but to Moodle as a whole.

There can also be rules that relate to specific sections. However, this is done only when there are only a handful of rules for that section. These small clusters of rules are grouped together and separated by comments identifying for which section each relates.

Finally there are all the other CSS files, you will notice that there is a file for each section of Moodle that has a significant collection of rules.

:For those who are familiar with Moodle 1.9 theme's, this organisation will be a big change. In 1.9, CSS was organised by its nature (for example: colours, layout, other).

===How to write effective CSS rules within Moodle===
In Moodle 2.0, writing good CSS rules is an incredibly important.

Due to performance requirements and browser limitations, all of the CSS files are combined into a single CSS file that gets included every time. This means that rules need to be written in such a way as to minimise the chances of a collision leading to unwanted styles being applied. Whilst writing good CSS is something most designers strive for we have implemented several new body classes and put more emphasis on developers for using classes more appropriately.

====The body tag====
As of Moodle 2.0 the ID tag that gets applied to the body will always be a representation of the URI. For example if you are looking at a forum posting and the URI is '/mod/forum/view.php' then the body tags ID will be '#page-mod-forum-view'.

As well as the body's ID attribute the URI is also exploded to form several CSS classes that get added to the body tag, so in the above example '/mod/forum/view' you would end up with the following classes being added to the body tag '.path-mod', '.path-mod-forum'. Note that '.path-mod-forum-view' is not added as a class, this is intentionally left out to lessen confusion and duplication as rules can relate directly to the page by using the ID and do not require the final class.

The body ID and body classes described above will form the bread and butter for many of the CSS rules you will need to write for your theme, however there are also several other very handy classes that get added to the body tag that will be beneficial to you once you start your journey down the rabbit hole that is themeing. Some of the more interesting classes are listed below.

* If JavaScript is enabled then 'jsenabled' will be added as a class to the body tag allowing you to style based on JavaScript being enabled or not.
* Either 'dir-rtl' or 'dir-ltr' will be added to the body as a class depending on the direction of the language pack: rtl = right to left, ltr = left to right. This allows you to determine your text-alignment based on language if required.
* A class will be added to represent the language pack currently in use, by default en_utf8 is used by Moodle and will result in the class 'lang-en_utf8' being added to the body tag.
* The wwwroot for Moodle will also be converted to a class and added to the body tag allowing you to stylise your theme based on the URL through which it was reached. e.g. http://sam.moodle.local/moodle/ will become '.sam-moodle-local—moodle'
* If the current user is not logged then '.notloggedin' will be added to the body tag.

What does all of this look like in practise? Well using the above example /mod/forum/view.php you would get at least the following body tag:
<code html4strict><body id=”page-mod-forum-view” class=”path-mod path-mod-forum” /></code>

====Writing your rules====
The best use of body ids and classes and writing selectors will reduce problems.

There are many specific classes used within Moodle. We try to put them everywhere where anyone may want to apply their own styles. It is important to recognise that no one developer can be aware of the all of the class names that have been used all throughout Moodle, let alone within all of the different contributed bits and pieces available for Moodle. It is up to the theme developer to write good rules and minimise the chances of a collision between rules because in this case good CSS is FAR more effective.

When starting to write rules make sure that you have a good understanding of where you want those rules to be applied, it is a good idea to make the most of the body classes mentioned above.
If you want to write a rule for a specific page make use of the body tag's ID, e.g.:

<code css>#page-mod-forum-view .forumpost {border:1px solid orange;)</code>

If you want to write a rule that will be applied all throughout the forum.:

<code css>.path-mod-forum .forumpost {border:1px solid orange;}</code>

The other very important thing to take into consideration is the structure leading up to the tag you want to style. Browsers apply conflicting styles with priority on the more specific selectors. It can be very beneficial to keep this in mind and write full selectors that rely on the structure of the tags leading to the tag you wish to style.

By making use of body id's and classes and writing selectors to take into account the leading structure you can greatly minimise the chance of a collision both with Moodle now and in the future.

==Layouts==
All themes are required to define the layouts they wish to be responsible for as well as create; however, many layout files are required by those layouts. If the theme is overriding another theme then it is a case of deciding which layouts this new theme should override. If the theme is a completely fresh start then you will need to define a layout for each of the different possibilities. For both situations these layouts should be defined within config.php.

It is also important to note that a new theme that will base itself on another theme (overriding it) does not need to define any layouts or use any layout files if there are no changes that it wishes to make to the layouts of the existing theme. The standard theme in Moodle is a good example of this as it extends the base theme simply adding CSS to achieve its look and feel.

So layouts... as mentioned earlier layouts are defined in config.php within $THEME->layouts. The following is an example of one such layout definition:
<code php>
$THEME->layouts = array(
// Standard layout with blocks, this is recommended for most pages with general information
'standard' => array(
'theme' => 'base',
'file' => 'general.php',
'regions' => array('side-pre', 'side-post'),
'defaultregion' => 'side-post'
)
)
</code>
The first thing Moodle looks at is the name of the layout, in this case it is `standard` (the array key in PHP), it then looks at the settings for the layout, this is the theme, file, regions, and default region. There are also a couple of other options that can be set by a layout.

; theme : is the theme the layout file exists in. That's right you can make use of layouts from other installed themes. ''Optional''
; file : is the name of the layout file this layout wants to use. ''Required''
; regions : is the different block regions (places you can put blocks) within the theme. ''Required''
; defaultregion : is the default location when adding new blocks. '''Required if regions is non-empty, otherwise optional'''
; options : an array of layout specific options described in detail below. '''Optional'''

The '''theme''' is optional. Normally the the layout file is looked for in the current theme, or, if it is not there, in the parent theme. However, you can use a layout file from any other theme by giving the theme name here.

You can define whatever regions you like. You just need to pick an name for each one. Most themes just use one or both of '''side_pre''' and '''side_post''', which is like 'left side' and 'right side', except in right to left languages, when they are reversed. If you say in config.php that your the layout provides regions called 'fred' and 'barney', then you must call $OUTPUT->blocks_for_region('fred') and $OUTPUT->blocks_for_region('barney') somewhere in the layout file.

The final setting '''options''' is a special case that only needs to be set if you want to make use of it. This setting allows the theme designer to specify special options that they would like to create that can be later accessed within the layout file. This allows the theme to make design decisions during the definition and react upon those decisions in what ever layout file is being used.

One such place this has been used is infact within the base theme. If you take a look first at theme/base/config.php you will notice that several layouts specify options '''nonavbar''' and '''nofooter''' which can both be set to either true or false. Then if we take a look at theme/base/layout/general.php you will spot lines like the following:
<code php>
<?php
$hasnavbar = (empty($PAGE->layout_options['nonavbar']) && $PAGE->has_navbar());
$hasfooter = (empty($PAGE->layout_options['nofooter']));
?>
............
<?php if ($hasnavbar) { ?>
<div class="navbar clearfix">
<div class="breadcrumb"><?php echo $OUTPUT->navbar(); ?></div>
<div class="navbutton"> <?php echo $PAGE->button; ?></div>
</div>
<?php } ?>
</code>
What you are seeing here is the use of those settings from the layout within the layout file. In this case it is being used to toggle the display of the navigation bar and page footer.

==Layout files==
A layout file is a file that contains the core HTML structure for a layout including the header, footer, content and block regions.
For those of you who are familiar with themes in Moodle 1.9 this is simply header.html and footer.html combined.
Of course it is not all HTML, there are bits of HTML and content that Moodle needs to put into the page, within each layout file this will be done by a couple of VERY simple PHP calls to get bits and pieces including content.

The following is a very simple layout file to illustrate the different bits that make it up:
<code php>
<?php echo $OUTPUT->doctype() ?>
<html <?php echo $OUTPUT->htmlattributes() ?>>
<head>
<title><?php echo $PAGE->title ?></title>
<?php echo $OUTPUT->standard_head_html() ?>
</head>
<body id="<?php p($PAGE->bodyid) ?>" class="<?php p($PAGE->bodyclasses) ?>">
<?php echo $OUTPUT->standard_top_of_body_html() ?>
<table id="page">
<tr>
<td colspan="3">
<h1 class="headermain"><?php echo $PAGE->heading ?></h1>
<div class="headermenu"><?php echo $OUTPUT->login_info(); echo $PAGE->headingmenu; ?></div>
</td>
</tr>
<tr>
<td>
<?php echo $OUTPUT->blocks_for_region('side-pre') ?>
</td>
<td>
<?php echo core_renderer::MAIN_CONTENT_TOKEN ?>
</td>
<td>
<?php echo $OUTPUT->blocks_for_region('side-post') ?>
</td>
</tr>
<tr>
<td colspan="3">
<?php echo page_doc_link(get_string('moodledocslink')) ?>
<?php
echo $OUTPUT->login_info();
echo $OUTPUT->home_link();
echo $OUTPUT->standard_footer_html();
?>
</td>
</tr>
</table>
<?php echo $OUTPUT->standard_end_of_body_html() ?>
</body>
</html>
</code>

Lets assume you know a enough HTML to understand the basic structure above, what you probably don't understand are the PHP calls so lets look at them.
<code php>
<?php echo $OUTPUT->doctype() ?>
</code>
This occurs at the VERY top of the page, it must be the first bit of output and is responsible for adding the (X)HTML document type definition to the page. This of course is determined by the settings of the site and is one of the things that the theme designer has no control over.

<code php>
<html <?php echo $OUTPUT->htmlattributes() ?>>
</code>
Here we have started writing the opening html tag and have asked Moodle to give us the HTML attributes that should be applied to it. This again is determined by several settings within the actual HTML install.

<code php>
<?php echo $PAGE->title ?>
</code>
This gets us the title for the page.

<code php>
<?php echo $OUTPUT->standard_head_html() ?>
</code>
This very important call gets us the standard head HTML that needs to be within the HEAD tag of the page. This is where CSS and JavaScript requirements for the top of the page will be output as well as any special script or style tags.

<code php>
<body id="<?php p($PAGE->bodyid); ?>" class="<?php p($PAGE->bodyclasses); ?>">
</code>
Much like the html tag above we have started writing the body tag and have asked for Moodle to get us the desired ID and classes that should be applied to it.

<code php>
<h1 class="headermain"><?php echo $PAGE->heading; ?></h1>
<div class="headermenu"><?php echo $OUTPUT->login_info(); echo $PAGE->headingmenu; ?></div>
</code>
Here we are creating the header for the page. In this case we want the heading for the page, we want to display the login information which will be the current users username or a link to log in if they are not logged in, and we want the heading menu if there is one.

<code php>
<?php echo $OUTPUT->blocks_for_region('side-pre') ?>
</code>
Here we get the HTML to display the blocks that have been added to the page. In this case we have asked for all blocks that have been added to the area labelled ''side-pre''.

<code php>
<?php echo core_renderer::MAIN_CONTENT_TOKEN ?>
</code>
This is one of the most important calls within the file, it determines where the actual content for the page gets inserted.

<code php>
<?php echo $OUTPUT->blocks_for_region('side-post') ?>
</code>
Here we get the HTML to display the blocks that have been added to the page. In this case we have asked for all blocks that have been added to the area labelled ''side-post''.

<code php>
<?php
echo $OUTPUT->login_info();
echo $OUTPUT->home_link();
echo $OUTPUT->standard_footer_html();
?>
</code>
This final bit of code gets the content for the footer of the page. It gets the login information which is the same as in the header, a home link, and the standard footer HTML which like the standard head HTML contains all of the script and style tags required by the page and requested to go in the footer.

'''''Note''''': Within Moodle 2.0 most of the JavaScript for the page will be included in the footer. This greatly helps reduce the loading time of the page.

When writing layout files think about the different layouts and how the HTML that each makes use of will differ. You will most likely find you do not need a different layout file for each layout, most likely you will be able to reuse the layout files you create across several layouts. You can of course make use of layout options as well to further reduce the number of layout files you need to produce.

Of course as mentioned above if you are customising an existing theme then you may not need to create any layouts or layout files at all.

'''$OUTPUT''' is an instance of the '''core_renderer''' class which is defined in lib/outputrenderers.php. Each method is clearly documented there, along with which is appropriate for use within the layout files.

'''$PAGE''' is an instance of the '''moodle_page''' class defined in lib/pagelib.php. Most of the things you will want to use are the properties that are all documented at the top of the file. If you are not familiar with PHP properties, you access them like $PAGE->activityname, just like fields of an ordinary PHP object. (However, behind the scenes, some magic is going on, and the value you get is produced by calling a function. Also, you cannot change these values, they are read-only. However, you don't need to understand all that if you are just using these properties in your theme.)

==Making use of images==
Right at the start when listing the features of the new themes system one of the features mentioned was the ability to override any of the standard images within Moodle from within your theme. At this point we will look at both how to make use of your own images within your theme, and secondly how to override the images being used by Moodle.
So first up a bit about images within Moodle,

# Images you want to use within your theme '''need''' to be located within your theme's pix directory.
# You can use sub directories within the pix directory of your theme.
# Images used by Moodle's core are located within the pix directory of Moodle.
# Modules, blocks and other plugins should also store there images within a pix directory.

So making use of your own images first up. Lets assume you have added two image files to the pix directory of your theme.

* /theme/yourthemename/pix/imageone.jpg
* /theme/yourthemename/pix/subdir/imagetwo.png

Notice that one image is a JPEG image, and the second is a PNG. Also the second image is in a subdirectory.

The following code snippet illustrates how to make use of your images within HTML, such as if you wanted to use them within a layout file.
<code php>
<img src="<?php echo $OUTPUT->pix_url('imageone', 'theme');?>" alt="" />
<img src="<?php echo $OUTPUT->pix_url('subdir/imagetwo', 'theme');?>" alt="" />
</code>

'''DO NOT''' include the image file extension. Moodle will work it out automatically and it will not work if you do include it.

In this case rather than writing out the URL to the image we use a method of Moodle's output library. Its not too important how that functions works but it is important that we use it as it is what allows images within Moodle to be over-rideable.

The following is how you would use the images from within CSS as background images.
<code css>
.divone {background-image:url([[pix:theme|imageone]]);}
.divtwo {background-image:url([[pix:theme|subdir/imagetwo]]);}
</code>
If this case we have to use some special notations that Moodle looks for. Whenever Moodle hands out a CSS file it first searches for all ''[[something]]'' tags and replaces them with what is required.

The final thing to notice with both of the cases above is that at no point do we include the images file extension.
The reason for this leads us into the next topic, how to override images.

From within a theme you can VERY easily override any standard image within Moodle by simply adding the replacement image to the theme's pix directory in the same sub directory structure as it is in Moodle.
So for instance we wanted to override the following two images:
# /pix/moodlelogo.gif
# /pix/i/user.gif
We would simply need to add our replacement images to the theme in the following locations
# /theme/themename/pix/moodlelogo.gif
# /theme/themename/pix/i/user.gif
How easy is that!

Now the other very cool thing to mention is that Moodle looks for not just replacements of the same image type (jpg, gif, etc...) but also replacements in any image format. This is why above when working with our images we never specified the images file extension.
This means that the following would also work:
# /theme/themename/pix/moodlelogo.png
# /theme/themename/pix/i/user.bmp

For a more detailed description of how this all works see the page on [[Development:Themes_2.0_How_to_use_images_within_your_theme|using images within your theme]]

==Unobvious Things==
===Getting Your Theme to Appear Correctly in Theme Selector===
If you follow the examples on this page to the letter, when you go to the Theme Selector page you may be discouraged to find that your theme does not appear like the other themes do. In fact, instead of your theme's name, you will see something along the lines of <nowiki>[[pluginname]]</nowiki>.

To correct this, you must add the /lang/en/theme_THEMENAME.php file, where THEMENAME is the name of the theme folder. Inside that file, add the string "$string['pluginname'] = 'THEMENAME'; ". Make THEMENAME the name of your theme, however you want it displayed in the Theme selector.

The screenshot for the theme should be about 500x400 px.

===Required theme divs===

Some parts of Moodle may rely on particular divs, for example the div with id 'page-header'.

Consequently all themes must include at least the divs (with the same ids) that are present in the 'base' theme.

Missing out these elements may result in unexpected behaviour within specific modules or other plugins.

==Appendix A==
===Theme options as of April 28th, 2010===
{| class="nicetable" id="theme_options_table"
|-
! Setting
! Effect
|-
| $THEME->'''csspostprocess'''
| Allows the user to provide the name of a function that all CSS should be passed to before being delivered.
|-
| $THEME->'''editor_sheets'''
| An array of stylesheets to include within the body of the editor.
|-
| $THEME->'''enable_dock'''
| If set to true the side dock is enabled for blocks
|-
| $THEME->'''hidefromselector'''
| Used to hide a theme from the theme selector (unless theme designer mode is on). Accepts true or false.
|-
| $THEME->'''filter_mediaplugin_colors'''
| Used to control the colours used in the small media player for the filters
|-
| $THEME->'''javascripts'''
| An array containing the names of JavaScript files located in /javascript/ to include in the theme. (gets included in the head)
|-
| $THEME->'''javascripts_footer'''
| As above but will be included in the page footer.
|-
| $THEME->'''larrow'''
| Overrides the left arrow image used throughout Moodle
|-
| $THEME->'''layouts'''
| An array setting the layouts for the theme
|-
| $THEME->'''name'''
| Name of the theme. Most likely the name of the directory in which this file resides.
|-
| $THEME->'''parents'''
| An array of themes to inherit from
|-
| $THEME->'''parents_exclude_javascripts'''
| An array of JavaScript files NOT to inherit from the themes parents
|-
| $THEME->'''parents_exclude_sheets'''
| An array of stylesheets not to inherit from the themes parents
|-
| $THEME->'''plugins_exclude_sheets'''
| An array of plugin sheets to ignore and not include.
|-
| $THEME->'''rarrow'''
| Overrides the right arrow image used throughout Moodle
|-
| $THEME->'''renderfactory'''
| Sets a custom render factory to use with the theme, used when working with custom renderers.
|-
| $THEME->'''resource_mp3player_colors'''
| Controls the colours for the MP3 player
|-
| $THEME->'''sheets'''
| An array of stylesheets to include for this theme. Should be located in the theme's style directory.
|}
===The different layouts as of August 17th, 2010===
{| class="nicetable" id="theme_layouts_table"
|-
! Layout
! Purpose
|-
| base
| Most backwards compatible layout without the blocks - this is the layout used by default.
|-
| standard
| Standard layout with blocks, this is recommended for most pages with general information.
|-
| course
| Main course page.
|-
| coursecategory
| Use for browsing through course categories.
|-
| incourse
| Default layout while browsing a course, typical for modules.
|-
| frontpage
| The site home page.
|-
| admin
| Administration pages and scripts.
|-
| mydashboard
| My dashboard page.
|-
| mypublic
| My public page.
|-
| login
| The login page.
|-
| popup
| Pages that appear in pop-up windows - no navigation, no blocks, no header.
|-
| frametop
| Used for legacy frame layouts only. No blocks and minimal footer.
|-
| embedded
| Embeded pages, like iframe/object embedded in moodleform - it needs as much space as possible
|-
| maintenance
| Used during upgrade and install. This must not have any blocks, and it is good idea if it does not have links to other places - for example there should not be a home link in the footer.
|-
| print
| Used when the page is being displayed specifically for printing.
|}

==See also==

* [[Development:Themes 2.0 creating your first theme]] - A quick step by step guide to creating your first theme.
* [[Development:Themes 2.0 overriding a renderer]] - A tutorial on creating a custom renderer and changing the HTML Moodle produces.
* [[Development:Themes 2.0 How to use images within your theme]] - Explains how to use and override images within your theme.
* [[Development:Themes 2.0 adding a settings page]] - Looks at how to add a setting page making your theme easily customisable.
* [[Development:Themes 2.0 extending the custom menu]] - Customising the custom menu.
* [[Development:Themes 2.0 adding courses and categories to the custom menu]] - Extending the custom menu further adding all categories + courses
* [[Development:Themes 2.0 how to make the dock horizontal]] - Modifying the dock to make it horizontal.
* [[Development:Themes 2.0 adding upgrade code]]
* [[Development:Styling and customising the dock]] - How to style and customise the dock.
* [[Development:Theme changes in 2.0]]
* [[Development:Using jQuery with Moodle 2.0]]
* [[Development:Themes 2.0 how to clone a Moodle 2.0 theme]]
* [http://www.youtube.com/watch?v=OvaU54uh-qA New themes in Moodle 2.0 video]

[[de:Designs 2.0]]
[[es:Temas 2.0]]

Themes 2.0 adding courses and categories to the custom menu

2011-05-06T02:55:54Z

Samhemelryk: /* lang/en/theme_ */

Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. 
The reason for writing this tutorial is simply that this seems to be a request that is being made over and over again.

==Before we get started==
Please please please make sure you are familiar with the other Themes 2.0 tutorials before attempting this tutorial.

In particular I would recommend being familiar with the following documents and tutorials as I'm going to move through this tutorial at a fast pace.

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]

Providing you are familiar with the above you should manage just fine with this tutorial and if you ever get stuck don't forget to ask in the theme's forums, there is always someone around who can help.

==Laying the ground work==
Alright I don't want this tutorial to drag out so get ready. For this tutorial I am just going to extend the standard theme, I'm going to make absolutely no changes to the design and layout of the theme or the custom menu I am ONLY going to add a list of courses and categories to it.

So to begin with within your Moodle themes directory create a new directory coursecategorymenu in which I want you to create three files, the first config.php which is obviously required, the second renderers.php as we are going to achieve this by overriding a renderer, and the third the English language file for this theme.

You should end up with:

* moodle/theme/coursecategorymenu
* moodle/theme/coursecategorymenu/config.php
* moodle/theme/coursecategorymenu/renderers.php
* moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php

==config.php==
Obviously because all I want to base this theme off the standard theme and only override a renderer the config.php file is going to be VERY basic, so much so that I'm not going to go into details about it, as you've read the recommended tutorials you will already be completely familiar with everything it is doing.

<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

==renderers.php==
This is of course where the magic is going to happen.

In my case I want to add a branch to the end of the custom menu titled ''Courses'' and then I want to add the category and course structure to that branch.

So like the [[Development:Themes 2.0 extending the custom menu|extending the custom menu]] tutorial I will be overriding the core renderer and I will also be overriding the render_custom_menu_method.

So the code for this:
<code php>
class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

You'll notice that is identical to the starting code for the extending custom menu tutorial.

The difference comes in the code that we are going to populate it with:

<code php>
require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
</code>

So lets work through this code line by line:

<code php>
require_once($CFG->dirroot.'/course/lib.php');
</code>

This we need to do because we are going to use part of the course API, in particular a function called ''get_course_category_tree()'' that we'll look at shortly.
The function we want to use exists within the course lib file and in order to be sure its always available we need to include this file.

<code php>
$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);
</code>

This line adds the ''Courses'' branch to the custom menu into which we will add the category and course structure.

<code php>
$categorytree = get_course_category_tree();
</code>

This line is the main one to note, here we are calling a Moodle function that will return us a category and course tree, because all courses need to be within a category the initial return is an array of categories, each category in the array will have the properties of the category as well as two additional properties, the first property '''categories''' is an array containing all of this categories sub categories, and the second property '''courses''' is also an array containing all of the courses within this category.

<code php>
$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}
</code>

Now these lines of code deal with the initial array of categories returned by the ''get_course_category_tree'' function.

In this case we iterate of the initial categories and for each one we call a method of the renderer '''add_category_to_custommenu''' now those of you who are clued in will realise that this method doesn't exist yet... and you are correct - we need to write this method which is what we will look at next. 
As a heads up the reason that we need to write another method is because we need a method that we can call recursivily as there are potentially infinite category branches all contains sub categories and courses.

<code php>
return parent::render_custom_menu($menu);
</code>

The final line of code simply calls the original render_custom_menu method now that we have extended the custom menu as we want.

Now that we have looked at the render_custom_menu lets write the function we just talked about '''add_category_to_custommenu'''. 
This function as discussed above is special in that we intend to call it recursively, that means we want to call it once for EVERY category that exists in the tree. We will need to give it the categories parent menu item as well as the category object we want added.

Lets look at the code for this method:

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}
</code>

Ok so that doesn't look too bad, as you've already read the extending custom menu tutorial you will be able to spot some of the things it is doing, none the less lets walk through the code.

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
.....
}
</code>

First up the function definition, this function is going to be private because only we need it and it is going to take two arguments, the first $parent has to be a custom_menu_item, the second $category has to be the category object.

The objective of this method is to add a category node to the custom menu, in regards to our arguments we want to add $category as a child of $parent.

<code php>
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
</code>

This line looks very familiar, here we are adding a node for the category to the $parent, we are also collecting the newly added node as $branch.

<code php>
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
</code>

Now this code is responsible for add the sub categories of this category to the menu as well, first we need to check that the categories property isn't empty (if it is there is nothing to add).
Assuming there are sub categories we need go through each of them and call this method on them so that they get added to the custom menu as well. 
This is called recursion.

<code php>
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
</code>
This code is very similar to the above code except that we are looking at the courses within this category. 
Also note that this bit of code doesn't call add_category_to_custommenu recursivily, it doesn't need to as courses are the last bit we want to display on the menu within the category.

And that is it, time to tidy up and we're done.

==Finishing up==
All of the code is written now there is only last thing to do in order to tidy up however and that is to add the ''courses'' string that we used earlier... did you spot it?

This is of course very easy, open up '''moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php''' and add the following:

<code php>
<?php

$string['courses'] = 'Courses';
</code>

And we're done. If you now browse to your site and change to the new theme you should see the Courses branch at the end of your custom menu that contains all of the categories, sub categories and courses for your site.

==Full source==
===config.php===
<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>
===renderers.php===
<code php>
<?php

class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
global $CFG;

require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
}

protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}

}
</code>
===lang/en/theme_coursecategorymenu.php===
<code php>
<?php

$string['courses'] = 'Courses';
</code>

==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Themes 2.0 adding courses and categories to the custom menu

2011-05-06T02:55:38Z

Samhemelryk: Created page with "Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. The reason for writing this tutorial is simply that this see..."

Hello this is going to be a very brief tutorial on how to add a courses and categories tree to the custom menu. 
The reason for writing this tutorial is simply that this seems to be a request that is being made over and over again.

==Before we get started==
Please please please make sure you are familiar with the other Themes 2.0 tutorials before attempting this tutorial.

In particular I would recommend being familiar with the following documents and tutorials as I'm going to move through this tutorial at a fast pace.

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [[Development:Themes 2.0 extending the custom menu]]

Providing you are familiar with the above you should manage just fine with this tutorial and if you ever get stuck don't forget to ask in the theme's forums, there is always someone around who can help.

==Laying the ground work==
Alright I don't want this tutorial to drag out so get ready. For this tutorial I am just going to extend the standard theme, I'm going to make absolutely no changes to the design and layout of the theme or the custom menu I am ONLY going to add a list of courses and categories to it.

So to begin with within your Moodle themes directory create a new directory coursecategorymenu in which I want you to create three files, the first config.php which is obviously required, the second renderers.php as we are going to achieve this by overriding a renderer, and the third the English language file for this theme.

You should end up with:

* moodle/theme/coursecategorymenu
* moodle/theme/coursecategorymenu/config.php
* moodle/theme/coursecategorymenu/renderers.php
* moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php

==config.php==
Obviously because all I want to base this theme off the standard theme and only override a renderer the config.php file is going to be VERY basic, so much so that I'm not going to go into details about it, as you've read the recommended tutorials you will already be completely familiar with everything it is doing.

<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>

==renderers.php==
This is of course where the magic is going to happen.

In my case I want to add a branch to the end of the custom menu titled ''Courses'' and then I want to add the category and course structure to that branch.

So like the [[Development:Themes 2.0 extending the custom menu|extending the custom menu]] tutorial I will be overriding the core renderer and I will also be overriding the render_custom_menu_method.

So the code for this:
<code php>
class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
// Our code will go here shortly
}

}
</code>

You'll notice that is identical to the starting code for the extending custom menu tutorial.

The difference comes in the code that we are going to populate it with:

<code php>
require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
</code>

So lets work through this code line by line:

<code php>
require_once($CFG->dirroot.'/course/lib.php');
</code>

This we need to do because we are going to use part of the course API, in particular a function called ''get_course_category_tree()'' that we'll look at shortly.
The function we want to use exists within the course lib file and in order to be sure its always available we need to include this file.

<code php>
$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);
</code>

This line adds the ''Courses'' branch to the custom menu into which we will add the category and course structure.

<code php>
$categorytree = get_course_category_tree();
</code>

This line is the main one to note, here we are calling a Moodle function that will return us a category and course tree, because all courses need to be within a category the initial return is an array of categories, each category in the array will have the properties of the category as well as two additional properties, the first property '''categories''' is an array containing all of this categories sub categories, and the second property '''courses''' is also an array containing all of the courses within this category.

<code php>
$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}
</code>

Now these lines of code deal with the initial array of categories returned by the ''get_course_category_tree'' function.

In this case we iterate of the initial categories and for each one we call a method of the renderer '''add_category_to_custommenu''' now those of you who are clued in will realise that this method doesn't exist yet... and you are correct - we need to write this method which is what we will look at next. 
As a heads up the reason that we need to write another method is because we need a method that we can call recursivily as there are potentially infinite category branches all contains sub categories and courses.

<code php>
return parent::render_custom_menu($menu);
</code>

The final line of code simply calls the original render_custom_menu method now that we have extended the custom menu as we want.

Now that we have looked at the render_custom_menu lets write the function we just talked about '''add_category_to_custommenu'''. 
This function as discussed above is special in that we intend to call it recursively, that means we want to call it once for EVERY category that exists in the tree. We will need to give it the categories parent menu item as well as the category object we want added.

Lets look at the code for this method:

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}
</code>

Ok so that doesn't look too bad, as you've already read the extending custom menu tutorial you will be able to spot some of the things it is doing, none the less lets walk through the code.

<code php>
protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
.....
}
</code>

First up the function definition, this function is going to be private because only we need it and it is going to take two arguments, the first $parent has to be a custom_menu_item, the second $category has to be the category object.

The objective of this method is to add a category node to the custom menu, in regards to our arguments we want to add $category as a child of $parent.

<code php>
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
</code>

This line looks very familiar, here we are adding a node for the category to the $parent, we are also collecting the newly added node as $branch.

<code php>
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
</code>

Now this code is responsible for add the sub categories of this category to the menu as well, first we need to check that the categories property isn't empty (if it is there is nothing to add).
Assuming there are sub categories we need go through each of them and call this method on them so that they get added to the custom menu as well. 
This is called recursion.

<code php>
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
</code>
This code is very similar to the above code except that we are looking at the courses within this category. 
Also note that this bit of code doesn't call add_category_to_custommenu recursivily, it doesn't need to as courses are the last bit we want to display on the menu within the category.

And that is it, time to tidy up and we're done.

==Finishing up==
All of the code is written now there is only last thing to do in order to tidy up however and that is to add the ''courses'' string that we used earlier... did you spot it?

This is of course very easy, open up '''moodle/theme/coursecategorymenu/lang/en/theme_coursecategorymenu.php''' and add the following:

<code php>
<?php

$string['courses'] = 'Courses';
</code>

And we're done. If you now browse to your site and change to the new theme you should see the Courses branch at the end of your custom menu that contains all of the categories, sub categories and courses for your site.

==Full source==
===config.php===
<code php>
<?php

$THEME->name = 'coursecategorymenu';
$THEME->parents = array('standard', 'base');
$THEME->rendererfactory = 'theme_overridden_renderer_factory';
</code>
===renderers.php===
<code php>
<?php

class theme_coursecategorymenu_core_renderer extends core_renderer {

protected function render_custom_menu(custom_menu $menu) {
global $CFG;

require_once($CFG->dirroot.'/course/lib.php');

$branch = $menu->add(get_string('courses', 'theme_coursecategorymenu'), null, null, 10000);

$categorytree = get_course_category_tree();
foreach ($categorytree as $category) {
$this->add_category_to_custommenu($branch, $category);
}

return parent::render_custom_menu($menu);
}

protected function add_category_to_custommenu(custom_menu_item $parent, stdClass $category) {
$branch = $parent->add($category->name, new moodle_url('/course/category.php', array('id' => $category->id)));
if (!empty($category->categories)) {
foreach ($category->categories as $subcategory) {
$this->add_category_to_custommenu($branch, $subcategory);
}
}
if (!empty($category->courses)) {
foreach ($category->courses as $course) {
$branch->add($course->shortname, new moodle_url('/course/view.php', array('id' => $course->id)), $course->fullname);
}
}
}

}
</code>
===lang/en/theme_===
<code php>
<?php

$string['courses'] = 'Courses';
</code>

==See also==
* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]]
* [[Development:Themes 2.0 overriding a renderer]]
* [http://developer.yahoo.com/yui/3/node-menunav/ YUI 3 Menunav component the custom menu uses]

Användare:Sam Hemelryk

2011-04-19T03:28:14Z

Samhemelryk:

Hello, I am a senior developer at Moodle and have been with the project since the June 2009.

For those interested I have written a tutorial on my Moodle development process with Git: [[User:Sam_Hemelryk/My_Moodle_Git_workflow]]

A short list of helpful documents I have created:

* [[Development:Themes 2.0]]
* [[Development:Themes 2.0 creating your first theme]] - A quick step by step guide to creating your first theme.
* [[Development:Themes 2.0 overriding a renderer]] - A tutorial on creating a custom renderer and changing the HTML Moodle produces.
* [[Development:Themes 2.0 How to use images within your theme]] - Explains how to use and override images within your theme.
* [[Development:Themes 2.0 adding a settings page]] - Looks at how to add a setting page making your theme easily customisable.
* [[Development:Themes 2.0 extending the custom menu]] - Customising the custom menu.
* [[Development:Themes 2.0 how to make the dock horizontal]] - Modifying the dock to make it horizontal.
* [[Development:Themes 2.0 adding upgrade code]]
* [[Development:Styling and customising the dock]] - How to style and customise the dock.
* [[Development:Using jQuery with Moodle 2.0]]

Användare:Sam Hemelryk/My Moodle Git workflow

2011-04-19T03:23:23Z

Samhemelryk: /* What to do when your work gets rejected */

This document is a sort of tutorial/inspection of my personal development workflow when working on Moodle with Git.

==Creating the repositories==
Ok so the very first step is to create several Moodle git repositories, one for each major branch of Moodle being worked on.

At the time of writing the major branches are MOODLE_19_STABLE, MOODLE_20_STABLE and master.

The reason for this is that that way we have a separate site for each Moodle branch that we can test on easily. Yes you will have three sites but you won't have to worry about swtiching your site when you change branches.

===Step 1: Create a directory to create the repositories within===

Personally I use Ubuntu (Linux) and I chose to create my repositories within a folder I created in /var/www. If you are using windows I would instead suggest creating C:/www/

<pre>
cd /var/www
mkdir repositories
cd repositories
</pre>

===Step 2: Create a master repository===
This is the easiest of the branches to create a repository for.

Within your repositories directory create a new directory called master and move into it:

<pre>
mkdir master
cd master
</pre>

Next we want to clone the Moodle source from the offical Moodle git repository as follows:

<pre>
git clone git://git.moodle.org/moodle.git moodle
</pre>

This will create a new directory called moodle within the master directory we created just before.

That new moodle directory should contain the Moodle source code as a git repository.

Now that we've got our main git repository we need to add our github account as a remote so that we can push to it later:

<pre>
cd moodle
git remote add github git@github.com:yourname/moodle.git
</pre>

This adds your moodle repository on github as a remote called github that you can push to later.

Finally before we move onto the next repository we need to create a data directory because we know we will need that:

<pre>
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 3: Create a MOODLE_20_STABLE repository===

The next repository we create will be for the MOODLE_20_STABLE branch.
To start with get back to the repositories directory that we created earlier and create a directory called MOODLE_20_STABLE within that.

<pre>
cd /var/www/repositories
mkdir MOODLE_20_STABLE
cd MOODLE_20_STABLE
</pre>

Now create a new clone of the main Moodle repository like we did for the master branch above.

<pre>
git clone git://git.moodle.org/moodle.git moodle
</pre>

Once we have done that move into the moodle directory.

By default the master branch is the branch that is created, however we don't want that for this repository. Instead we want the MOODLE_20_STABLE branch.

To do this we checkout local branch based upon the official MOODLE_20_STABLE branch as follows:

<pre>
git checkout -b MOODLE_20_STABLE origin/MOODLE_20_STABLE
</pre>

Once we have created that branch we can delete the local master branch so that we don't accidentally ever work on it.

<pre>
git branch -D master
</pre>

Then like we did for master we add our github repository as a remote:

<pre>
git remote add github git@github.com:yourname/moodle.git
</pre>

And finally create a data directory we can use when we have install the site:

<pre>
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 4: Create a MOODLE_19_STABLE repository===

This step is identical to the above step except we are using the MOODLE_19_STABLE branch instead of the MOODLE_20_STABLE branch.
The commands for this are as follows:

<pre>
cd /var/www/repositories
mkdir MOODLE_19_STABLE
cd MOODLE_19_STABLE
git clone git://git.moodle.org/moodle.git moodle
git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE
git branch -D master
git remote add github git@github.com:yourname/moodle.git
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 5: Install the sites===
Now I can't tell you how to do this on your own machine however what I would recommend is that you create a link from your web root to the moodle directory of each branch.

If you are using linux and your webroot is /var/www/localhost you would do it as follows:

<pre>
cd /var/www/localhost/
ln -s /var/www/repositories/master/moodle master
ln -s /var/www/repositories/MOODLE_20_STABLE/moodle MOODLE_20_STABLE
ln -s /var/www/repositories/MOODLE_19_STABLE/moodle MOODLE_19_STABLE
</pre>

With that done you should be able to browse to each repository with the following URL's

<pre>
http://localhost/master/
http://localhost/MOODLE_20_STABLE/
http://localhost/MOODLE_19_STABLE/
</pre>

Once that is done you need to browse to each site and complete the installation.

Under this method your will need to ensure you set the configuration option sessioncookiepath to the URI of your site. You can either do this in the admin interfaces or add the following to your config.php file for each site.

<code php>
// For the master site
$CFG->sessioncookiepath = '/master/';
// For the MOODLE_20_STABLE site
$CFG->sessioncookiepath = '/MOODLE_20_STABLE/';
// For the MOODLE_19_STABLE site
$CFG->sessioncookiepath = '/MOODLE_19_STABLE/';
</code>

Failure to do this will mean that when you log in at one site and then visit the next you will have to log in again and your session on the first site will no longer work.

Now that you have got 3 repositories, one for each main branch, and set up a site for each you are ready to start development.

==Before you start work each day==
Before you start work each day you should update each of the repositories you have created to ensure that you are always working with up to date versions of Moodle.

You should also take this opportunity to update the repositories on your github account to ensure they are always up to date.

To start with lets update the master repository:

<pre>
cd /var/www/repositories/master/moodle
git fetch --all --prune
</pre>

This moves to the master repository and then tells git for fetch all of the changes from your remote repositories and clean up any old branches it is aware of.

Once that operation has completed you should update your master branch

<pre>
git checkout master
git pull
</pre>

Before you do that it pays to make sure you don't have any uncommit changes, if you do DON'T TRY IT. There would be a good chance you will encounter problems. Instead finish what you are working on, commit your changes and then update your master branch.

Next we will update the github repositories by running the following command:

<pre>
git push github refs/remotes/origin/master:master
</pre>

This tells git to push the master branch from origin (the offical Moodle repository) to the master branch on our github account.

Now that you've updated the master repository and your github account its time to update the MOODLE_20_STABLE and MOODLE_19_STABLE branches.

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_20_STABLE
git pull
git push github refs/remotes/origin/MOODLE_20_STABLE:MOODLE_20_STABLE
</pre>

And

<pre>
cd /var/www/repositories/MOODLE_19_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_19_STABLE
git pull
git push github refs/remotes/origin/MOODLE_19_STABLE:MOODLE_19_STABLE
</pre>

==Working on a Moodle issue==

This part of the document looks at how I go about working on an MDL issue from the Moodle tracker.

At Moodle HQ we use the scrum methodology we sees us choose several issues to include in a scrum which we then work on for the period of the scrum. As a community member its more likely you will just be choosing issues that you are passionate about.

Either way once you have found an issue to work on you are ready to start.

For the purpose of this part of the document lets say I am going to work on bug MDL-12345.

==Fixing a bug on the master repository - MDL-12345==

The first step is to create a local branch to make changes on, to do this we will start our work on the master repository.

When fixing an issue I find it easiest to fix the issue on the master branch first and then move my changes to the other branches.

The VERY first thing you should do when you start working on a Moodle issue is make sure you have assigned it to yourself and then click the start progress button.

This way other users know that you are activly working on the bug and no one from HQ will steal it from you.

===Step 1: Creating a local branch===

So first get to the master repository:

<pre>
cd /var/www/repositories/master/moodle
</pre>

Once there we create a new branch as follows:

<pre>
git checkout -b wip-MDL-12345-master origin/master
</pre>

This command checks out a new branch called wip-MDL-12345-master that is based upon origin/master.

origin/master is of course the master branch on the official Moodle repository (git.moodle.org/moodle.git).

The name of the branch is also very important. It is essentially telling us three things.

; wip : This stands for work in progress, it helps people see that you are currently working on this branch and that they shouldn't base there work on it because it will likely change.
; MDL-12345 : This is of course the Moodle bug number. It helps people quickly identify what you are working on.
; master : This is the branch we are making changes on. It helps people quickly understand where the changes are being made.

You can optionally add more to the end of the branch name such as a key word that identifies the area or a date at which you started work e.g.

* wip-MDL-12345-master-navigation-changes
* wip-MDL-12345-master-20110418

That is up to you, personally I'm not a fan of it, however the rest is all required to help integrate your work.

Either way now you have a branch, mine is called wip-MDL-12345-master.

===Step 2: Make changes===

Now that you have created you branch you are ready to make changes.

I'll let you make whatever changes you are need, once you have made changes move onto the next step.

Before you do move on however make sure that your code meets the high quality of code Moodle requires.

You can find information about that on the moodle docs here: https://docs.moodle.org/en/Development:Coding_style

The following are common mistakes:

* Incorrect white space
** Tabs instead of spaces
** Extra spaces at the end of lines
** Multiple new lines
** Forgetting to put spaces between function arguments
* Incorrect commit messages (read on to find out about writing a good commit message)
* Incorrect variable or function names

Making these sort of mistakes can lead to your code being rejected despite it working correctly.

I would '''strongly''' suggest while learning Moodle development that you use [[User:Tim Hunt|Tim Hunt's]] fantastic [https://github.com/timhunt/moodle-local_codechecker Code Checker] plugin. 
You'll find information on how to use it within its README file and it will pick up many of the things that may lead to your work being rejected.

Or course it will be reviewed in regards to security, usability and the other important factors of code development.

===Step 3: Commit your changes===

Now that you have made your changes you are getting ready to commit them.

The first thing to do is check your changes. To do this run the following command:

<pre>
git status
</pre>

You should see all of the files you have changes highlighted in red.

Next you need to stage all of the files you want to commit.

To do this you add the files to the stage using the following command.

<pre>
git add lib/modifiedfile.php
git add lib/newfile.php
git add theme/base/style/core.css
</pre>

Why not just commit all changes. Many of the git guides you read will tell you above git commit -a which commits all of the changes you have made.

We've hardly being using git here at Moodle and already I've seen several people accidentally commit changes that they didn't mean to.

Staging files like this ENSURES you only commit things you intend to.

Now that you have staged your changes/new files you are ready to commit. The following command commits your changes:

<pre>
git commit -m "MDL-12345 enrol - Fixed up a couple of enrolment bugs"
</pre>

This command commits your changes with the commit message "MDL-12345 enrol - Fixed up a couple of enrolment bugs".
It's very important to Moodle that you format your commit messages like this.
They consist of three parts:

# The MDL bug number in this case MDL-12345
# The area you made changes to in this case enrol
# A short description of what you did, in this case I fixed a couple of enrolment bugs.

When you commit your message you will see something like the following:

<pre>
[master 19a484e] changes
1 files changed, 16 insertions(+), 18 deletions(-)
</pre>

There is one thing here you need to note down for later, that is the number that appears within the square brackets after the branch name, in my case it is 19a484e. This is the commit id of the commit I just made.

===Step 4: Pushing your changes to your github account===

Now that you have made changes to your local branch you should push it to your github account so that it can be reviewed and hopefully integrated into Moodle.

Doing this is very simple just run the following command:

<pre>
git push github wip-MDL-12345-master
</pre>

Here we are telling git to push the branch wip-MDL-12345-master to the remote called github which we added when setting up our git repositories.
And just like that you've made a branch that contains your work and pushed it to github ready to get it reviewed.

==Moving your changes to another branch==
Now that you have made changes to the master branch you need to decide whether it is appropriate make those changes on any of the other branches.
Most likely if you are fixing a bug you will need to make the changes on the MOODLE_20_STABLE branch as well, and perhaps the MOODLE_19_STABLE branch if it is a security bug.

Lets assume for MDL-12345 that I fixed earlier that it should be ported to MOODLE_20_STABLE as well.

===Step 1: Creating a local MOODLE_20_STABLE branch===

To start with I need to move to the MOODLE_20_STABLE repository as follows:

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
</pre>

Once there I need to create a new branch based upon the MOODLE_20_STABLE branch that is going to contain my changes.

<pre>
git checkout -b wip-MDL-12345-MOODLE_20_STABLE origin/MOODLE_20_STABLE
</pre>

This is very similar to the branch we created for the changes to the master branch except that I have subsituted MOODLE_20_STABLE in place of master. This is because the new branch we are creating is based upon the MOODLE_20_STABLE branch on the offical Moodle git repository and contains changes for that branch.

===Step 2: Cherry-picking my changes===
Once we have a branch for our changes I am ready to make them again. There is however with git an easy way to do this, cherry-picking.

Cherry-picking is the process of copying a single commit from one branch to another. In this case I want to cherry pick the commit I made earlier onto the branch I've just created.

However before I can do this I need to fetch the changes I made from the github repository. I need to do this because I made that changes of a totally separate local repository remember.

So to update my repository with the latest changes I run the following command.

<pre>
git fetch --all --prune
</pre>

This command tells git to look at each remote and get any new changes (it also removes any branches that have been deleted).
Once it has completed you will be ready to cherry pick the commit. To do so you use the following command but replace my commit is with yours.

<pre>
git cherry-pick 19a484e
</pre>

Providing there are no conflicts your branch will now contain a copy of the changes you made to the master branch.
If there are conficts that you will need to resolve them and commit your changes however you can search for another tutorial about how to do that,

Now I know at this point that there will be some of you who are saying 'Whoops I forgot what my commit id was!'. If this is you don't worry, its pretty easy to find it providing you used a proper commit message like I described above.
Simply run the following command:

<pre>
git log --oneline origin/master..github/wip-MDL-12345-master
</pre>

This command shows you all of the commits that are in the wip-MDL-12345-master at your github account but are not in the master branch at the offical Moodle repository.
You can then cherry-pick the commits shown there (the commit ID's should be highlighted in yellow).

Once you have cherry-picked your commits its time to check that everything worked, run the command:

<pre>
git status
</pre>

You should see something like the following:

<pre>
# On branch wip-MDL-12345-MOODLE_20_STABLE
# Your branch is ahead of 'origin/MOODLE_20_STABLE' by 1 commit.
#
nothing to commit (working directory clean)
</pre>

This is very handy as it tells you that your branch is ahead of the offical MOODLE_20_STABLE branch by one commit.
You can also check that your commit contains the changes you think by running the following command:

<pre>
git diff origin/MOODLE_20_STABLE
</pre>

This command shows you all of the changes you have made as a diff.

Now that you know that your commit is there you and that it is correct it is time to push it to your github account.

<pre>
git push github wip-MDL-12345-MOODLE_20_STABLE
</pre>

And thats it.

You github account should now have two branches on it:

; wip-MDL-12345-master : This branch is the changes you have made for the master branch
; wip-MDL-12345-MOODLE_20_STABLE : This branch is those same changes but for the MOODLE_20_STABLE branch

Now you are ready to create PULL requests so that your work gets reviewed and hopefully integrated.

==Creating PULL requests==
Now that you've got a branch with changes for your MDL issue its time to create some PULL requests to get it integrated.

===Creating a PULL request for the master branch===

So you have successfully fixed the bug MDL-12345, you have created two branches one for master and one for MOODLE_20_STABLE. It is time to create a PULL request for each of these branches.

First lets create a PULL request for the master branch:

# Browse to tracker.moodle.org/browse/MDL-12345
# When the page loads log in if you haven't already.
# Click on the button in the top right labelled `Create issue`
# Change Project to `Pull Requests`
# Change Issue type to `Pull Request`
# Click create

This takes you to a screen where you can enter the details for the PULL request.

You should fill it out in the following way:

====Summary====
This is the summary for the PULL request, you should copy and paste the summary from the MDL issue here.
For MDL-12345 the summary is `alphabetization of UI different on left and right sides` so that's what I will use.

====Affects Versions====
This is the version that your branch is changing, in the case of this PULL request I will select `master`.

====Security level====
If the issue you are working on is a security issue you should set this to the same level that the issue is set to. If its not a security issue or you don't know just leave it as none.

====Pull from repository====
This is the repository that the branch is at, for you this will be your github accont. If you head to http://github.com/yourname/moodle you will see a box that has three options SSH, HTTP, and Git read only, click Git Read Only and then copy the contents of the text box into the repositry field of the PULL request.
For me that is git://github.com/yourname/moodle.git

====Pull branch====
This is the name of the branch that contains your changes. For me that is wip-MDL-12345-master.

====Pull Diff URL====
This is the URL to a page that shows the changes that you've made on your branch. Github is nice in that it has a pretty interface for that. To get to the interface follow these steps:

# Browse to https://github.com/youraccount/moodle
# Click `Switch Branches` and then click on the branch you changes are based upon, in my case master.
# Click the button labelled `Branch List`
# Locate your branch in the list and click the `Compare button` for me this was the compare button to the right of wip-MDL-12345-master.
# Copy the URL in your browse and paste it into the Pull Diff URL of the PULL request.

For me this URL was: https://github.com/yourname/moodle/compare/master...wip-MDL-23532-master
For the MOODLE_20_STABLE branch this would be: https://github.com/yourname/moodle/compare/MOODLE_20_STABLE...wip-MDL-23532-MOODLE_20_STABLE

====Description====
The description should contain several bits of important information:

# A short description of what your patch does
# Notes about that changes you've made that might help the integrator understand what you did when he/she is reviewing it.
# Testing instructions that guide the testers through testing your changes.

====Components====
Select general from the components list, general is the only option.

====Finishing up====
Once you have filled in the fields as above click `Create`. This should create you PULL request and then take you to view it.

Its important to remember that there is still one thing you need to do and that is link to the original issue.
While you are still logged in click the `More Actions` button and select `Link issue`, in the dialoug that pops up change the `This issue` field to `will help resolve`, type MDL-12345 into the issues field, and then type `Linking to MDL-12345` into the comments box. Once you've done that simple click Link. I should add that the comment isn't really required. It's just something I personally like doing.

Time to create the PULL request for the other branches.

===Creating the PULL request for MOODLE_20_STABLE===
Once you've created the PULL request for the master branch it is time to create the PULL request for the MOODLE_20_STABLE branch.
This is as easy as can be, click Create Issue in the top right, select PULL project and choose Pull Request and then click create.
On the next screen enter the following details:

; Summary : Copy the summary from the previous PULL Request
; Affects Versions : Select MOODLE_20_STABLE
; Security level : Same as the MDL
; Pull from repository : Copy from the previous PULL request
; Pull Branch : wip-MDL-12345-MOODLE_20_STABLE
; Pull Diff URL : https://github.com/yourname/moodle/compare/MOODLE_20_STABLE...wip-MDL-23532-MOODLE_20_STABLE
; Description : Copy from the previous PULL request
; Component : general

Then click create to create the PULL request.

Once the PULL request has been create don't forget to link to the MDL issue like we did for the previous PULL request.

==Resolving the issue==
The final step for the MDL issue now that you have created the two PULL requests is to Resolve the issue (do not close it, just resolve).

You can do this by logging into tracker, browsing to the MDL issue, and clicking the resolve button.

If your fixes are integrated the tester will close the issue, otherwise if it doesn't get integrated someone will reopen the MDL.

==Rebasing after the weekly release==
As many of you will be aware on Wednesday after the latest weekly has been released a comment will be added to all PULL requests that are still open.
<pre>
The main moodle.git repository has just been updated with latest weekly modifications.
You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

TIA and ciao :)
</pre>
This is a message that is bulk added as is done so as a polite request. While it's not essential for you to rebase your work it really does help the integrators as its much easier to see what is going on and greatly reduces the chance of conflicts (because you'll solve them when you rebase).

The good news is that rebasing your work is EASY!

For the following example lets assume I created the two PULL requests above on Wednesday morning and the weekly was released on Wednesday afternoon. This means that my branch doesn't have the changes that have just been released and I need to rebase them.

===Step 1: Rebasing the wip-MDL-12345-master===

The first step is to rebase my work that is based upon the master branch. So first move to the master repository.

<pre>
cd /var/www/repositories/master/moodle
</pre>

The next thing we have to do is get all of the changes from the remote repositories, this will fetch down all of the latest changes.

<pre>
git fetch --all --prune
</pre>

Now before the next step make sure that you don't have any uncommit changes, if you do finish that work, commit it, then proceed to the next step which is updating your local master.

<pre>
git checkout master
git pull
git push master github
</pre>

Now that your local master is up to date, and has you've updated you master branch at github you are ready to rebase!
At this point we want to checkout the branch we are going to rebase, in this case wip-MDL-12345-master and then we are going to rebase origin/master which is the official Moodle master branch (and the branch our branch is based upon).

<pre>
git checkout wip-MDL-12345-master
git rebase origin/master
</pre>

Once that command has completed your branch wip-MDL-12345-master will be completely up to date. Before you are done however you need to push the rebased wip-MDL-12345-master up to your github account so that its the branch the integrator sees.

<pre>
git push -f github wip-MDL-12345-master
</pre>

Now that command is just about identical to the command we used when we first pushed our branch to our github account, there is however one VERY important difference, the '''-f''' option.
The '''-f''' option tells git to force the push, this is required because when you rebase all of your commit id's will change and unless you tell git to force it will see that they have changed and not allow you to push your work.

Thats it! you've now rebased your work and the integrators will be happy again :)

===Step 2: Rebasing wip-MDL-12345-MOODLE_20_STABLE===
Now that we've rebased our work that was based upon the master branch it is time to rebase the MOODLE_20_STABLE version.
This step is just about identical to the previous step except we substitute ''master'' for ''MOODLE_20_STABLE''.

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_20_STABLE
git pull
git push github MOODLE_20_STABLE
git checkout wip-MDL-12345-MOODLE_20_STABLE
git rebase origin/MOODLE_20_STABLE
git push -f github wip-MDL-12345-MOODLE_20_STABLE
</pre>

Done!

==So your work got rejected... what now?==
First up don't give up, we (Moodle integrators) don't just reject code willy-nilly, we always provide a reason.

There are a couple of different paths that this may lead you down:
# More work needed: This happens if you have fixed half of the bug, or some of the bugs, but not all/completely. In this case you just need to finish off the MDL and then create new PULL requests.
# Bugs + white space issues: If this is the case you need to fix up what ever was spotted and then create new PULL requests.
# Not appropriate for a stable branch: This happens if the integrator decides that your work shouldn't get into the STABLE branch but is still OK for the master branch. Things like new features or improvements are likely to head down this path. If this is the case then everything is done, your work will have been integrated into master and you will just have to wait until the next major release (2.1, 2.2 etc).
# Not appropriate for core: If this happens it means that the integrator has decided that the work you've done isn't suitable to go into the main Moodle code base. If this is the case and you still want to use your branches that is fine, just leave them up on your github account and whenever you need to use them just merge your branch into your sites repository.

==The extra mile==
Now as many of you may have guessed by the time you are working on several different issues and managing several repositories the process I've described here seems VERY repetitive. And it '''IS!'''.

If you talk to any HQ developer they will tell you they deal with it in their own way, anything from scripts, to custom config files, through to virtual machine snapshots.

Personally I have a couple of bash scripts that I wrote to help me manage my workflow.

The first script I use I run after each weekly release and it does the following things:

* Moves to each repository and does the following
*# Fetches all remote changes
*# Checks whether it is safe to update my local main release branches and does that if it is safe
*# Updates all main release branches on my github account

The second script I run on every site that I have installed, it does the following

* Goes to each site I have installed and does the following
*# Overwrites the database with a stable snapshot
*# Overwrites the moodle data directory with a stable snapshot taken at the same time as the database one.
*# Upgrades the moodle site if required
*# Takes a snapshot of the database and Moodle data directory.

The third script I run whenever I start working on a new issue and it restores the stable snapshot of a sites database and moodle data directory.

These three scripts allow me to very easily ensure I am working with the latest changes and that I can easily restore to a point I know is stable and safe.

There was a fourth script I used to use but don't any more that also rebased any unmerged branches however I found it to be far to problematic and I prefer to do it myself so that I know exactly what is going on.

==Useful links==
The following are links I found useful while learning Git.

* [http://help.github.com Github help (http://help.github.com)] Provides EXCELLENT help tutorials that very clearly explain the basic through to the advanced.
* [http://cheat.errtheblog.com/s/git/ $ cheat git] This is my favourite cheat sheet, straight forward to the point, if you know the concept you'll find the answer here.
* [http://gitster.livejournal.com/28309.html Gitsters journal - Fun with FETCH_HEAD] David Mudrak pointed me at this entry which I found INCREDIBLY useful when reviewing other peoples work.
* [[Development:Git tips]] Moodle docs page with a few handy tips about using Git for Moodle development.

I you're looking to read/buy a book on Git I'd strongly recommend [http://progit.org/ Pro Git]. I certainly found that the most useful book I read.

[[Category:Git]]

Användare:Sam Hemelryk/My Moodle Git workflow

2011-04-19T03:16:40Z

Samhemelryk: /* Step 2: Rebasing wip-MDL-12345-MOODLE_20_STABLE */

This document is a sort of tutorial/inspection of my personal development workflow when working on Moodle with Git.

==Creating the repositories==
Ok so the very first step is to create several Moodle git repositories, one for each major branch of Moodle being worked on.

At the time of writing the major branches are MOODLE_19_STABLE, MOODLE_20_STABLE and master.

The reason for this is that that way we have a separate site for each Moodle branch that we can test on easily. Yes you will have three sites but you won't have to worry about swtiching your site when you change branches.

===Step 1: Create a directory to create the repositories within===

Personally I use Ubuntu (Linux) and I chose to create my repositories within a folder I created in /var/www. If you are using windows I would instead suggest creating C:/www/

<pre>
cd /var/www
mkdir repositories
cd repositories
</pre>

===Step 2: Create a master repository===
This is the easiest of the branches to create a repository for.

Within your repositories directory create a new directory called master and move into it:

<pre>
mkdir master
cd master
</pre>

Next we want to clone the Moodle source from the offical Moodle git repository as follows:

<pre>
git clone git://git.moodle.org/moodle.git moodle
</pre>

This will create a new directory called moodle within the master directory we created just before.

That new moodle directory should contain the Moodle source code as a git repository.

Now that we've got our main git repository we need to add our github account as a remote so that we can push to it later:

<pre>
cd moodle
git remote add github git@github.com:yourname/moodle.git
</pre>

This adds your moodle repository on github as a remote called github that you can push to later.

Finally before we move onto the next repository we need to create a data directory because we know we will need that:

<pre>
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 3: Create a MOODLE_20_STABLE repository===

The next repository we create will be for the MOODLE_20_STABLE branch.
To start with get back to the repositories directory that we created earlier and create a directory called MOODLE_20_STABLE within that.

<pre>
cd /var/www/repositories
mkdir MOODLE_20_STABLE
cd MOODLE_20_STABLE
</pre>

Now create a new clone of the main Moodle repository like we did for the master branch above.

<pre>
git clone git://git.moodle.org/moodle.git moodle
</pre>

Once we have done that move into the moodle directory.

By default the master branch is the branch that is created, however we don't want that for this repository. Instead we want the MOODLE_20_STABLE branch.

To do this we checkout local branch based upon the official MOODLE_20_STABLE branch as follows:

<pre>
git checkout -b MOODLE_20_STABLE origin/MOODLE_20_STABLE
</pre>

Once we have created that branch we can delete the local master branch so that we don't accidentally ever work on it.

<pre>
git branch -D master
</pre>

Then like we did for master we add our github repository as a remote:

<pre>
git remote add github git@github.com:yourname/moodle.git
</pre>

And finally create a data directory we can use when we have install the site:

<pre>
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 4: Create a MOODLE_19_STABLE repository===

This step is identical to the above step except we are using the MOODLE_19_STABLE branch instead of the MOODLE_20_STABLE branch.
The commands for this are as follows:

<pre>
cd /var/www/repositories
mkdir MOODLE_19_STABLE
cd MOODLE_19_STABLE
git clone git://git.moodle.org/moodle.git moodle
git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE
git branch -D master
git remote add github git@github.com:yourname/moodle.git
cd ..
mkdir data
sudo chmod -R 777 data
</pre>

===Step 5: Install the sites===
Now I can't tell you how to do this on your own machine however what I would recommend is that you create a link from your web root to the moodle directory of each branch.

If you are using linux and your webroot is /var/www/localhost you would do it as follows:

<pre>
cd /var/www/localhost/
ln -s /var/www/repositories/master/moodle master
ln -s /var/www/repositories/MOODLE_20_STABLE/moodle MOODLE_20_STABLE
ln -s /var/www/repositories/MOODLE_19_STABLE/moodle MOODLE_19_STABLE
</pre>

With that done you should be able to browse to each repository with the following URL's

<pre>
http://localhost/master/
http://localhost/MOODLE_20_STABLE/
http://localhost/MOODLE_19_STABLE/
</pre>

Once that is done you need to browse to each site and complete the installation.

Under this method your will need to ensure you set the configuration option sessioncookiepath to the URI of your site. You can either do this in the admin interfaces or add the following to your config.php file for each site.

<code php>
// For the master site
$CFG->sessioncookiepath = '/master/';
// For the MOODLE_20_STABLE site
$CFG->sessioncookiepath = '/MOODLE_20_STABLE/';
// For the MOODLE_19_STABLE site
$CFG->sessioncookiepath = '/MOODLE_19_STABLE/';
</code>

Failure to do this will mean that when you log in at one site and then visit the next you will have to log in again and your session on the first site will no longer work.

Now that you have got 3 repositories, one for each main branch, and set up a site for each you are ready to start development.

==Before you start work each day==
Before you start work each day you should update each of the repositories you have created to ensure that you are always working with up to date versions of Moodle.

You should also take this opportunity to update the repositories on your github account to ensure they are always up to date.

To start with lets update the master repository:

<pre>
cd /var/www/repositories/master/moodle
git fetch --all --prune
</pre>

This moves to the master repository and then tells git for fetch all of the changes from your remote repositories and clean up any old branches it is aware of.

Once that operation has completed you should update your master branch

<pre>
git checkout master
git pull
</pre>

Before you do that it pays to make sure you don't have any uncommit changes, if you do DON'T TRY IT. There would be a good chance you will encounter problems. Instead finish what you are working on, commit your changes and then update your master branch.

Next we will update the github repositories by running the following command:

<pre>
git push github refs/remotes/origin/master:master
</pre>

This tells git to push the master branch from origin (the offical Moodle repository) to the master branch on our github account.

Now that you've updated the master repository and your github account its time to update the MOODLE_20_STABLE and MOODLE_19_STABLE branches.

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_20_STABLE
git pull
git push github refs/remotes/origin/MOODLE_20_STABLE:MOODLE_20_STABLE
</pre>

And

<pre>
cd /var/www/repositories/MOODLE_19_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_19_STABLE
git pull
git push github refs/remotes/origin/MOODLE_19_STABLE:MOODLE_19_STABLE
</pre>

==Working on a Moodle issue==

This part of the document looks at how I go about working on an MDL issue from the Moodle tracker.

At Moodle HQ we use the scrum methodology we sees us choose several issues to include in a scrum which we then work on for the period of the scrum. As a community member its more likely you will just be choosing issues that you are passionate about.

Either way once you have found an issue to work on you are ready to start.

For the purpose of this part of the document lets say I am going to work on bug MDL-12345.

==Fixing a bug on the master repository - MDL-12345==

The first step is to create a local branch to make changes on, to do this we will start our work on the master repository.

When fixing an issue I find it easiest to fix the issue on the master branch first and then move my changes to the other branches.

The VERY first thing you should do when you start working on a Moodle issue is make sure you have assigned it to yourself and then click the start progress button.

This way other users know that you are activly working on the bug and no one from HQ will steal it from you.

===Step 1: Creating a local branch===

So first get to the master repository:

<pre>
cd /var/www/repositories/master/moodle
</pre>

Once there we create a new branch as follows:

<pre>
git checkout -b wip-MDL-12345-master origin/master
</pre>

This command checks out a new branch called wip-MDL-12345-master that is based upon origin/master.

origin/master is of course the master branch on the official Moodle repository (git.moodle.org/moodle.git).

The name of the branch is also very important. It is essentially telling us three things.

; wip : This stands for work in progress, it helps people see that you are currently working on this branch and that they shouldn't base there work on it because it will likely change.
; MDL-12345 : This is of course the Moodle bug number. It helps people quickly identify what you are working on.
; master : This is the branch we are making changes on. It helps people quickly understand where the changes are being made.

You can optionally add more to the end of the branch name such as a key word that identifies the area or a date at which you started work e.g.

* wip-MDL-12345-master-navigation-changes
* wip-MDL-12345-master-20110418

That is up to you, personally I'm not a fan of it, however the rest is all required to help integrate your work.

Either way now you have a branch, mine is called wip-MDL-12345-master.

===Step 2: Make changes===

Now that you have created you branch you are ready to make changes.

I'll let you make whatever changes you are need, once you have made changes move onto the next step.

Before you do move on however make sure that your code meets the high quality of code Moodle requires.

You can find information about that on the moodle docs here: https://docs.moodle.org/en/Development:Coding_style

The following are common mistakes:

* Incorrect white space
** Tabs instead of spaces
** Extra spaces at the end of lines
** Multiple new lines
** Forgetting to put spaces between function arguments
* Incorrect commit messages (read on to find out about writing a good commit message)
* Incorrect variable or function names

Making these sort of mistakes can lead to your code being rejected despite it working correctly.

I would '''strongly''' suggest while learning Moodle development that you use [[User:Tim Hunt|Tim Hunt's]] fantastic [https://github.com/timhunt/moodle-local_codechecker Code Checker] plugin. 
You'll find information on how to use it within its README file and it will pick up many of the things that may lead to your work being rejected.

Or course it will be reviewed in regards to security, usability and the other important factors of code development.

===Step 3: Commit your changes===

Now that you have made your changes you are getting ready to commit them.

The first thing to do is check your changes. To do this run the following command:

<pre>
git status
</pre>

You should see all of the files you have changes highlighted in red.

Next you need to stage all of the files you want to commit.

To do this you add the files to the stage using the following command.

<pre>
git add lib/modifiedfile.php
git add lib/newfile.php
git add theme/base/style/core.css
</pre>

Why not just commit all changes. Many of the git guides you read will tell you above git commit -a which commits all of the changes you have made.

We've hardly being using git here at Moodle and already I've seen several people accidentally commit changes that they didn't mean to.

Staging files like this ENSURES you only commit things you intend to.

Now that you have staged your changes/new files you are ready to commit. The following command commits your changes:

<pre>
git commit -m "MDL-12345 enrol - Fixed up a couple of enrolment bugs"
</pre>

This command commits your changes with the commit message "MDL-12345 enrol - Fixed up a couple of enrolment bugs".
It's very important to Moodle that you format your commit messages like this.
They consist of three parts:

# The MDL bug number in this case MDL-12345
# The area you made changes to in this case enrol
# A short description of what you did, in this case I fixed a couple of enrolment bugs.

When you commit your message you will see something like the following:

<pre>
[master 19a484e] changes
1 files changed, 16 insertions(+), 18 deletions(-)
</pre>

There is one thing here you need to note down for later, that is the number that appears within the square brackets after the branch name, in my case it is 19a484e. This is the commit id of the commit I just made.

===Step 4: Pushing your changes to your github account===

Now that you have made changes to your local branch you should push it to your github account so that it can be reviewed and hopefully integrated into Moodle.

Doing this is very simple just run the following command:

<pre>
git push github wip-MDL-12345-master
</pre>

Here we are telling git to push the branch wip-MDL-12345-master to the remote called github which we added when setting up our git repositories.
And just like that you've made a branch that contains your work and pushed it to github ready to get it reviewed.

==Moving your changes to another branch==
Now that you have made changes to the master branch you need to decide whether it is appropriate make those changes on any of the other branches.
Most likely if you are fixing a bug you will need to make the changes on the MOODLE_20_STABLE branch as well, and perhaps the MOODLE_19_STABLE branch if it is a security bug.

Lets assume for MDL-12345 that I fixed earlier that it should be ported to MOODLE_20_STABLE as well.

===Step 1: Creating a local MOODLE_20_STABLE branch===

To start with I need to move to the MOODLE_20_STABLE repository as follows:

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
</pre>

Once there I need to create a new branch based upon the MOODLE_20_STABLE branch that is going to contain my changes.

<pre>
git checkout -b wip-MDL-12345-MOODLE_20_STABLE origin/MOODLE_20_STABLE
</pre>

This is very similar to the branch we created for the changes to the master branch except that I have subsituted MOODLE_20_STABLE in place of master. This is because the new branch we are creating is based upon the MOODLE_20_STABLE branch on the offical Moodle git repository and contains changes for that branch.

===Step 2: Cherry-picking my changes===
Once we have a branch for our changes I am ready to make them again. There is however with git an easy way to do this, cherry-picking.

Cherry-picking is the process of copying a single commit from one branch to another. In this case I want to cherry pick the commit I made earlier onto the branch I've just created.

However before I can do this I need to fetch the changes I made from the github repository. I need to do this because I made that changes of a totally separate local repository remember.

So to update my repository with the latest changes I run the following command.

<pre>
git fetch --all --prune
</pre>

This command tells git to look at each remote and get any new changes (it also removes any branches that have been deleted).
Once it has completed you will be ready to cherry pick the commit. To do so you use the following command but replace my commit is with yours.

<pre>
git cherry-pick 19a484e
</pre>

Providing there are no conflicts your branch will now contain a copy of the changes you made to the master branch.
If there are conficts that you will need to resolve them and commit your changes however you can search for another tutorial about how to do that,

Now I know at this point that there will be some of you who are saying 'Whoops I forgot what my commit id was!'. If this is you don't worry, its pretty easy to find it providing you used a proper commit message like I described above.
Simply run the following command:

<pre>
git log --oneline origin/master..github/wip-MDL-12345-master
</pre>

This command shows you all of the commits that are in the wip-MDL-12345-master at your github account but are not in the master branch at the offical Moodle repository.
You can then cherry-pick the commits shown there (the commit ID's should be highlighted in yellow).

Once you have cherry-picked your commits its time to check that everything worked, run the command:

<pre>
git status
</pre>

You should see something like the following:

<pre>
# On branch wip-MDL-12345-MOODLE_20_STABLE
# Your branch is ahead of 'origin/MOODLE_20_STABLE' by 1 commit.
#
nothing to commit (working directory clean)
</pre>

This is very handy as it tells you that your branch is ahead of the offical MOODLE_20_STABLE branch by one commit.
You can also check that your commit contains the changes you think by running the following command:

<pre>
git diff origin/MOODLE_20_STABLE
</pre>

This command shows you all of the changes you have made as a diff.

Now that you know that your commit is there you and that it is correct it is time to push it to your github account.

<pre>
git push github wip-MDL-12345-MOODLE_20_STABLE
</pre>

And thats it.

You github account should now have two branches on it:

; wip-MDL-12345-master : This branch is the changes you have made for the master branch
; wip-MDL-12345-MOODLE_20_STABLE : This branch is those same changes but for the MOODLE_20_STABLE branch

Now you are ready to create PULL requests so that your work gets reviewed and hopefully integrated.

==Creating PULL requests==
Now that you've got a branch with changes for your MDL issue its time to create some PULL requests to get it integrated.

===Creating a PULL request for the master branch===

So you have successfully fixed the bug MDL-12345, you have created two branches one for master and one for MOODLE_20_STABLE. It is time to create a PULL request for each of these branches.

First lets create a PULL request for the master branch:

# Browse to tracker.moodle.org/browse/MDL-12345
# When the page loads log in if you haven't already.
# Click on the button in the top right labelled `Create issue`
# Change Project to `Pull Requests`
# Change Issue type to `Pull Request`
# Click create

This takes you to a screen where you can enter the details for the PULL request.

You should fill it out in the following way:

====Summary====
This is the summary for the PULL request, you should copy and paste the summary from the MDL issue here.
For MDL-12345 the summary is `alphabetization of UI different on left and right sides` so that's what I will use.

====Affects Versions====
This is the version that your branch is changing, in the case of this PULL request I will select `master`.

====Security level====
If the issue you are working on is a security issue you should set this to the same level that the issue is set to. If its not a security issue or you don't know just leave it as none.

====Pull from repository====
This is the repository that the branch is at, for you this will be your github accont. If you head to http://github.com/yourname/moodle you will see a box that has three options SSH, HTTP, and Git read only, click Git Read Only and then copy the contents of the text box into the repositry field of the PULL request.
For me that is git://github.com/yourname/moodle.git

====Pull branch====
This is the name of the branch that contains your changes. For me that is wip-MDL-12345-master.

====Pull Diff URL====
This is the URL to a page that shows the changes that you've made on your branch. Github is nice in that it has a pretty interface for that. To get to the interface follow these steps:

# Browse to https://github.com/youraccount/moodle
# Click `Switch Branches` and then click on the branch you changes are based upon, in my case master.
# Click the button labelled `Branch List`
# Locate your branch in the list and click the `Compare button` for me this was the compare button to the right of wip-MDL-12345-master.
# Copy the URL in your browse and paste it into the Pull Diff URL of the PULL request.

For me this URL was: https://github.com/yourname/moodle/compare/master...wip-MDL-23532-master
For the MOODLE_20_STABLE branch this would be: https://github.com/yourname/moodle/compare/MOODLE_20_STABLE...wip-MDL-23532-MOODLE_20_STABLE

====Description====
The description should contain several bits of important information:

# A short description of what your patch does
# Notes about that changes you've made that might help the integrator understand what you did when he/she is reviewing it.
# Testing instructions that guide the testers through testing your changes.

====Components====
Select general from the components list, general is the only option.

====Finishing up====
Once you have filled in the fields as above click `Create`. This should create you PULL request and then take you to view it.

Its important to remember that there is still one thing you need to do and that is link to the original issue.
While you are still logged in click the `More Actions` button and select `Link issue`, in the dialoug that pops up change the `This issue` field to `will help resolve`, type MDL-12345 into the issues field, and then type `Linking to MDL-12345` into the comments box. Once you've done that simple click Link. I should add that the comment isn't really required. It's just something I personally like doing.

Time to create the PULL request for the other branches.

===Creating the PULL request for MOODLE_20_STABLE===
Once you've created the PULL request for the master branch it is time to create the PULL request for the MOODLE_20_STABLE branch.
This is as easy as can be, click Create Issue in the top right, select PULL project and choose Pull Request and then click create.
On the next screen enter the following details:

; Summary : Copy the summary from the previous PULL Request
; Affects Versions : Select MOODLE_20_STABLE
; Security level : Same as the MDL
; Pull from repository : Copy from the previous PULL request
; Pull Branch : wip-MDL-12345-MOODLE_20_STABLE
; Pull Diff URL : https://github.com/yourname/moodle/compare/MOODLE_20_STABLE...wip-MDL-23532-MOODLE_20_STABLE
; Description : Copy from the previous PULL request
; Component : general

Then click create to create the PULL request.

Once the PULL request has been create don't forget to link to the MDL issue like we did for the previous PULL request.

==Resolving the issue==
The final step for the MDL issue now that you have created the two PULL requests is to Resolve the issue (do not close it, just resolve).

You can do this by logging into tracker, browsing to the MDL issue, and clicking the resolve button.

If your fixes are integrated the tester will close the issue, otherwise if it doesn't get integrated someone will reopen the MDL.

==Rebasing after the weekly release==
As many of you will be aware on Wednesday after the latest weekly has been released a comment will be added to all PULL requests that are still open.
<pre>
The main moodle.git repository has just been updated with latest weekly modifications.
You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

TIA and ciao :)
</pre>
This is a message that is bulk added as is done so as a polite request. While it's not essential for you to rebase your work it really does help the integrators as its much easier to see what is going on and greatly reduces the chance of conflicts (because you'll solve them when you rebase).

The good news is that rebasing your work is EASY!

For the following example lets assume I created the two PULL requests above on Wednesday morning and the weekly was released on Wednesday afternoon. This means that my branch doesn't have the changes that have just been released and I need to rebase them.

===Step 1: Rebasing the wip-MDL-12345-master===

The first step is to rebase my work that is based upon the master branch. So first move to the master repository.

<pre>
cd /var/www/repositories/master/moodle
</pre>

The next thing we have to do is get all of the changes from the remote repositories, this will fetch down all of the latest changes.

<pre>
git fetch --all --prune
</pre>

Now before the next step make sure that you don't have any uncommit changes, if you do finish that work, commit it, then proceed to the next step which is updating your local master.

<pre>
git checkout master
git pull
git push master github
</pre>

Now that your local master is up to date, and has you've updated you master branch at github you are ready to rebase!
At this point we want to checkout the branch we are going to rebase, in this case wip-MDL-12345-master and then we are going to rebase origin/master which is the official Moodle master branch (and the branch our branch is based upon).

<pre>
git checkout wip-MDL-12345-master
git rebase origin/master
</pre>

Once that command has completed your branch wip-MDL-12345-master will be completely up to date. Before you are done however you need to push the rebased wip-MDL-12345-master up to your github account so that its the branch the integrator sees.

<pre>
git push -f github wip-MDL-12345-master
</pre>

Now that command is just about identical to the command we used when we first pushed our branch to our github account, there is however one VERY important difference, the '''-f''' option.
The '''-f''' option tells git to force the push, this is required because when you rebase all of your commit id's will change and unless you tell git to force it will see that they have changed and not allow you to push your work.

Thats it! you've now rebased your work and the integrators will be happy again :)

===Step 2: Rebasing wip-MDL-12345-MOODLE_20_STABLE===
Now that we've rebased our work that was based upon the master branch it is time to rebase the MOODLE_20_STABLE version.
This step is just about identical to the previous step except we substitute ''master'' for ''MOODLE_20_STABLE''.

<pre>
cd /var/www/repositories/MOODLE_20_STABLE/moodle
git fetch --all --prune
git checkout MOODLE_20_STABLE
git pull
git push github MOODLE_20_STABLE
git checkout wip-MDL-12345-MOODLE_20_STABLE
git rebase origin/MOODLE_20_STABLE
git push -f github wip-MDL-12345-MOODLE_20_STABLE
</pre>

Done!

==What to do when your work gets rejected==

==The extra mile==
Now as many of you may have guessed by the time you are working on several different issues and managing several repositories the process I've described here seems VERY repetitive. And it '''IS!'''.

If you talk to any HQ developer they will tell you they deal with it in their own way, anything from scripts, to custom config files, through to virtual machine snapshots.

Personally I have a couple of bash scripts that I wrote to help me manage my workflow.

The first script I use I run after each weekly release and it does the following things:

* Moves to each repository and does the following
*# Fetches all remote changes
*# Checks whether it is safe to update my local main release branches and does that if it is safe
*# Updates all main release branches on my github account

The second script I run on every site that I have installed, it does the following

* Goes to each site I have installed and does the following
*# Overwrites the database with a stable snapshot
*# Overwrites the moodle data directory with a stable snapshot taken at the same time as the database one.
*# Upgrades the moodle site if required
*# Takes a snapshot of the database and Moodle data directory.

The third script I run whenever I start working on a new issue and it restores the stable snapshot of a sites database and moodle data directory.

These three scripts allow me to very easily ensure I am working with the latest changes and that I can easily restore to a point I know is stable and safe.

There was a fourth script I used to use but don't any more that also rebased any unmerged branches however I found it to be far to problematic and I prefer to do it myself so that I know exactly what is going on.

==Useful links==
The following are links I found useful while learning Git.

* [http://help.github.com Github help (http://help.github.com)] Provides EXCELLENT help tutorials that very clearly explain the basic through to the advanced.
* [http://cheat.errtheblog.com/s/git/ $ cheat git] This is my favourite cheat sheet, straight forward to the point, if you know the concept you'll find the answer here.
* [http://gitster.livejournal.com/28309.html Gitsters journal - Fun with FETCH_HEAD] David Mudrak pointed me at this entry which I found INCREDIBLY useful when reviewing other peoples work.
* [[Development:Git tips]] Moodle docs page with a few handy tips about using Git for Moodle development.

I you're looking to read/buy a book on Git I'd strongly recommend [http://progit.org/ Pro Git]. I certainly found that the most useful book I read.

[[Category:Git]]