Note: You are currently viewing documentation for Moodle 3.11. Up-to-date documentation for the latest stable version of Moodle may be available here: Cron with Unix or Linux.

Cron with Unix or Linux: Difference between revisions

From MoodleDocs
Main page ► Installation ► Cron with Unix or Linux
Installation
(Logging and monitoring, Low latency adhoc tasks, Scaling up cron with multiple processes moved to Cron as they apply to Windows too https://moodle.org/mod/forum/discuss.php?d=410599#p1682139)
 
(3 intermediate revisions by 2 users not shown)
Line 44: Line 44:
==Using the crontab program on Unix/Linux==
==Using the crontab program on Unix/Linux==


Once you have selected (and tested!) an appropriate command to invoke the Moodle cron it must be added to the web users 'crontab' to schedule it to run regularly. 'Crontab' is both a file containing the user's cron commands and is also the name of the (command line) program used to edit it. Use the following command (as root) substituting the correct user in place of 'www-data' (e.g. 'apache' for Centos, 'www-data' for Debian/Ubuntu, '_www' for macOS—Google will know!)
Once you have selected (and tested!) an appropriate command to invoke the Moodle cron it must be added to the web users 'crontab' to schedule it to run regularly. 'Crontab' is both a file containing the user's cron commands and is also the name of the (command line) program used to edit it. Use the following command (as root) substituting the correct user in place of 'www-data' (e.g. 'Apache' for Centos, 'www-data' for Debian/Ubuntu, '_www' for macOS—Google will know!)
<pre>
<pre>
# crontab -u www-data -e
# crontab -u www-data -e
Line 50: Line 50:
This will bring up an editor window (the first time it may ask you which editor to use). Add the command onto the end of the file in this way (it may be empty or it may have some instructional comments):
This will bring up an editor window (the first time it may ask you which editor to use). Add the command onto the end of the file in this way (it may be empty or it may have some instructional comments):
<pre>
<pre>
  */1 * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php
  * * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php
</pre>
</pre>
The first five entries specify the times, followed by the command, to run. This says to run the command every minute which is normally ok. On a hosted system you may get complaints if you do not run it a lot less often (e.g. to run every two hours use '0 */2 * * *' for the first five entries). If you want to use the wget/curl version, the first five entries remain the same - just change the command part.
The first five * entries specify the times, followed by the command, to run. This says to run the command as often as possible, ie every minute.
 
== Low latency adhoc tasks ==
 
Each time cron is run, after the scheduled tasks the adhoc tasks are also run. While scheduled tasks can run at most once a minute, adhoc tasks can be queued at any moment, and generally you want them processed as soon as possible and to not have to wait for the scheduled task to run first. If you only run the normal admin/cli/cron.php then not only might it have to wait to process all the scheduled tasks first, if it has already finished you will have to wait until the next minute for cron to start again for it to be processed.
 
Instead you can run one or more dedicated adhoc task processors which run in parallel to the main cron process.
 
<pre>
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/adhoc_task.php --execute --keep-alive=59
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/adhoc_task.php --execute --keep-alive=59
...
</pre>
 
Starting from Moodle 3.9 the --keep-alive option runs like a daemon so when the queue is empty rather then exiting it waits for new tasks to be queued so it can start processing as soon as possible.
 
== Scaling up cron with multiple processes ==
 
As your site grows many of the scheduled tasks will take longer to complete, and also there will be more adhoc tasks queued which need to be processed. The cron system is designed to work in parallel but each individual process can only process one task at a time so you must run multiple cron cli's. You can generally run a fairly high number of cron processes on a dedicated cron instance before needing to run multiple cron instances. To run more than one process simply spawn multiple cron processes each minute:
 
<code>
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php
...
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/adhoc_task.php --execute --keep-alive=59
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/adhoc_task.php --execute --keep-alive=59
* * * * * /usr/bin/php  /path/to/moodle/admin/cli/adhoc_task.php --execute --keep-alive=59
...
</code>
 
It can be especially important to increase the number of adhoc_task.php processes as certain plugins and systems can generate very large numbers of adhoc tasks, or tasks that can take a long time to process. Especially tasks like document conversions and automated backups can build up more quickly than they are processed if they are left on default settings.
 
By default only 3 scheduled tasks and 3 adhoc tasks can be run concurrently so as you add more processes you need to increase the level of allowed concurrency:
 
Site administration > Server > Tasks > Task processing
 
<code php>
config.php:
$CFG->task_scheduled_concurrency_limit = 20; // Defaults to 3
$CFG->task_adhoc_concurrency_limit = 50;  // Defaults to 3
</code>
 
Whatever you set these too make sure the server(s) hosting them can comfortably handle this many processes. Often the bottleneck will be a shared service, usually the database.
 
You may find that certain types of very long running tasks will consume all the available task processes which means no other tasks will run. eg if you have 5 cli processes, but in the task queue there are 20 adhoc tasks for an automated backup, each of which will take ten minutes to complete, then very quickly all 5 processes will be consumed by backups and nothing else. Other small very fast and light tasks like a document conversion or forum emails will not get sent until the backups are complete and a process frees up. To manage this you can limit the concurrency of specific types of adhoc tasks. A good rule of thumb is that if all of the 'heavy' tasks consume all of their own limits then you should still have another few processes standing by on idle waiting for anything else which may be added to the queue.
 
Automated backups are the worst known offender, so hypothetically if you are running 50 adhoc task processes concurrently a reasonable restriction might be to cap the backups to consume no more than half of those processes, ie 25 at most:


<code php>
See https://en.wikipedia.org/wiki/Cron#CRON_expression for details on cron expressions.
config.php:
$CFG->task_concurrency_limit = [
    'core\task\course_backup_task' => 25,
    'core_course\task\course_delete_modules' => 5,
];
</code>


== See also ==
== See also ==
Line 113: Line 60:
* [http://linuxweblog.com/node/24 A basic crontab tutorial]  
* [http://linuxweblog.com/node/24 A basic crontab tutorial]  
* [http://www.freebsd.org/cgi/man.cgi?query=crontab&apropos=0&sektion=5&manpath=FreeBSD+6.0-RELEASE+and+Ports&format=html Online version of the man page]
* [http://www.freebsd.org/cgi/man.cgi?query=crontab&apropos=0&sektion=5&manpath=FreeBSD+6.0-RELEASE+and+Ports&format=html Online version of the man page]
* [http://www.easycron.com/predictor Predicting Cron job's run time]


[[es:Cron con Unix o Linux]]
[[es:Cron con Unix o Linux]]

Latest revision as of 13:58, 5 February 2021

On Unix and Linux use the built in cron program which is standard on nearly all systems. You are required to add a command to the 'crontab' (the table that holds cron commands) for the web server user.

There are two different methods that can be used to invoke the Moodle cron process:

NOTE: The commands shown need to be added to the crontab to function (described in a moment). However, you can - and should - run them on the command line to check they work first.

Method 1: The command line (cli) cron

If you have a choice, this is normally the best way to run Moodle cron.

PHP is also capable of running programs directly from the command line. Your system needs to be set up to do this; specifically you need the 'CLI' version of PHP to be installed. Most systems with PHP installed will have this by default. If you have the PHP CLI version installed then this is the recommended method of invoking cron. The correct command will be something like... 

/usr/bin/php /path/to/moodle/admin/cli/cron.php

(substitute the correct path to moodle and for php as required)

You can simply type this on the command line this to see if it works. If you are not sure about the path to PHP you can type "which php".

Tip:: If you have problems, see the PHP page. In particular, suspect an alternate php.ini for the CLI PHP command which may not have suitable settings.

Method 2: Web based cron

NOTE: In order to use the web based cron script you must first check Cron settings to make sure this method is permitted.

The idea is to call the following web page (you can try this from your browser): 

http://url.of.your/moodle/admin/cron.php

A command line (text based) browser is needed to run this on the server. Possibilities are as follows (OSX, for example, only ships with curl)... 

/usr/bin/wget -q -O /dev/null/ http://url.of.your/moodle/admin/cron.php

(no output is displayed - remove the -O /dev/null/ to test)

...OR... 

/usr/bin/curl http://url.of.your/moodle/admin/cron.php -o /dev/null/ -silent

(no output is displayed - remove the -o /dev/null/ -silent to test)

Using the crontab program on Unix/Linux

Once you have selected (and tested!) an appropriate command to invoke the Moodle cron it must be added to the web users 'crontab' to schedule it to run regularly. 'Crontab' is both a file containing the user's cron commands and is also the name of the (command line) program used to edit it. Use the following command (as root) substituting the correct user in place of 'www-data' (e.g. 'Apache' for Centos, 'www-data' for Debian/Ubuntu, '_www' for macOS—Google will know!) 

# crontab -u www-data -e

This will bring up an editor window (the first time it may ask you which editor to use). Add the command onto the end of the file in this way (it may be empty or it may have some instructional comments): 

 * * * * * /usr/bin/php  /path/to/moodle/admin/cli/cron.php

The first five * entries specify the times, followed by the command, to run. This says to run the command as often as possible, ie every minute.

See https://en.wikipedia.org/wiki/Cron#CRON_expression for details on cron expressions.

See also

Retrieved from "https://docs.moodle.org/311/en/index.php?title=Cron_with_Unix_or_Linux&oldid=139610"
Powered by MediaWiki