Performance and scalability: Difference between revisions
From MoodleDocs
| (9 intermediate revisions by 6 users not shown) | |||
| Line 4: | Line 4: | ||
This page is part of the [[Coding|Moodle coding guidelines]] | This page is part of the [[Coding|Moodle coding guidelines]] | ||
==Writing code that scales and performs== | ==Writing code that scales and performs== | ||
===Every page should only use a fixed number of database queries=== | ===Every page should only use a fixed number of database queries=== | ||
* Be very suspicious if you ever see database code inside a loop. | * Be very suspicious if you ever see database code inside a loop. | ||
** This is sometimes hard to spot if the database access is hidden in a function. | ** This is sometimes hard to spot if the database access is hidden in a function. | ||
| Line 17: | Line 14: | ||
===Limit the amount of RAM each page requires to generate=== | ===Limit the amount of RAM each page requires to generate=== | ||
* Large reports should be broken into pages of a fixed size. | * Large reports should be broken into pages of a fixed size. | ||
* Processing large amounts of data from the database should be handled with a recordset (when you cannot do all the processing in the database with SQL). | * Processing large amounts of data from the database should be handled with a recordset (when you cannot do all the processing in the database with SQL) and using [https://github.com/moodle/moodle/blob/master/lib/classes/dml/recordset_walk.php recordset_walk iterator], as there is no RAM benefit on using recordsets if you load all the results in a massive PHP array. | ||
===Be cautious about other external calls=== | ===Be cautious about other external calls=== | ||
Like a database query, there are other operations that are much slower than just executing PHP code. For example: | Like a database query, there are other operations that are much slower than just executing PHP code. For example: | ||
* running a shell script | * running a shell script | ||
| Line 29: | Line 24: | ||
* (to a lesser extent) working with files. | * (to a lesser extent) working with files. | ||
Whenever you are doing these things, worry about performance. | Whenever you are doing these things, worry about performance. | ||
=== Limit the scope of session locks === | |||
If you don't need a session lock, or only need it for part of your page, then unlock the session. Full details here: [[Session locks]] | |||
==How to improve the performance of your code== | ==How to improve the performance of your code== | ||
===Measure during development=== | ===Measure during development=== | ||
* Turn on [[:en:Debugging#Performance_info|display of performance information]] (including counting database queries), so you are aware of what your code is doing. | |||
* Turn on [[Debugging#Performance_info|display of performance information]] (including counting database queries), so you are aware of what your code is doing. | |||
* Use tools like [http://jakarta.apache.org/jmeter/ JMeter] to subject your new code to load. | * Use tools like [http://jakarta.apache.org/jmeter/ JMeter] to subject your new code to load. | ||
* Use | * Use https://github.com/moodlehq/moodle-performance-comparison (Moodle 2.5 onwards) to compare performance. You can also use the [[Test site generator]] or [[Test course generator]] generators (also Moodle 2.5 onwards). | ||
===Measure in production=== | ===Measure in production=== | ||
* If you're using postgres, there's a script that can parse the logs and output the top 10 slow queries, ready to be plugged into a cronjob to email you every day. It can be found here: | |||
http://git.catalyst.net.nz/gw?p=pgtools.git;a=blob_plain;f=scripts/pg-log-process.pl;hb=refs/heads/pg-log-process-multidb | |||
You need to configure postgres a bit to make it log things in the correct format. Instructions are in the file. | |||
==How big can a Moodle site be?== | |||
==How big a site | |||
Looking at the [http://moodle.org/stats/ statistics], the largest sites in the world currently have | Looking at the [http://moodle.org/stats/ statistics], the largest sites in the world currently have | ||
* Up to 1 000 000 users | * Up to 1 000 000 users | ||
| Line 56: | Line 51: | ||
* Up to XXX activities in a course. | * Up to XXX activities in a course. | ||
* Please add more things here. | * Please add more things here. | ||
When planning and testing your code, these are the sorts of numbers you should be contemplating. However, do not assume that Moodle sites will never get bigger than this. | When planning and testing your code, these are the sorts of numbers you should be contemplating. However, do not assume that Moodle sites will never get bigger than this. | ||
| Line 63: | Line 57: | ||
==See also== | ==See also== | ||
* [[Coding|Moodle coding guidelines]] | * [[Coding|Moodle coding guidelines]] | ||
* [[Performance]] guidance for administrators | * [[Performance]] guidance for administrators | ||
* [[Performance FAQ]] | * [[Performance FAQ]] | ||
[[Category:Coding guidelines|Performance]] | |||
[[Category:Performance]] | [[ja:パフォーマンスおよびスケーラビリティ]] | ||
Latest revision as of 02:45, 16 March 2022
Performance is about allowing Moodle to support as many users as possible with a certain amount of hardware.
Of course you can always always buy a bigger server. Scalability means ensuring that if you buy a server that is about twice as powerful, it can then handle about twice as much load.
This page is part of the Moodle coding guidelines
Writing code that scales and performs
Every page should only use a fixed number of database queries
- Be very suspicious if you ever see database code inside a loop.
- This is sometimes hard to spot if the database access is hidden in a function.
- Instead use JOINs and subqueries. (get_records_sql, get_recordset_sql, etc.).
- Or look for a Moodle API function that gets the information you want as efficiently as possible. (For example, get_users_by_capability)
- See the Database guidelines for how to write SQL that will work on all our supported databases.
Limit the amount of RAM each page requires to generate
- Large reports should be broken into pages of a fixed size.
- Processing large amounts of data from the database should be handled with a recordset (when you cannot do all the processing in the database with SQL) and using recordset_walk iterator, as there is no RAM benefit on using recordsets if you load all the results in a massive PHP array.
Be cautious about other external calls
Like a database query, there are other operations that are much slower than just executing PHP code. For example:
- running a shell script
- making a web-service call
- (to a lesser extent) working with files.
Whenever you are doing these things, worry about performance.
Limit the scope of session locks
If you don't need a session lock, or only need it for part of your page, then unlock the session. Full details here: Session locks
How to improve the performance of your code
Measure during development
- Turn on display of performance information (including counting database queries), so you are aware of what your code is doing.
- Use tools like JMeter to subject your new code to load.
- Use https://github.com/moodlehq/moodle-performance-comparison (Moodle 2.5 onwards) to compare performance. You can also use the Test site generator or Test course generator generators (also Moodle 2.5 onwards).
Measure in production
- If you're using postgres, there's a script that can parse the logs and output the top 10 slow queries, ready to be plugged into a cronjob to email you every day. It can be found here:
You need to configure postgres a bit to make it log things in the correct format. Instructions are in the file.
How big can a Moodle site be?
Looking at the statistics, the largest sites in the world currently have
- Up to 1 000 000 users
- Up to 50 000 courses
- Up to 5 000 users per course
- Up to 50 roles
- Up to 100 course categories nested up to about 10 levels deep.
- Up to XXX activities in a course.
- Please add more things here.
When planning and testing your code, these are the sorts of numbers you should be contemplating. However, do not assume that Moodle sites will never get bigger than this.
Even if you can't test sites this big on your development server, you should use the generator script so you can test your code in a Moodle site that is not tiny.
See also
- Moodle coding guidelines
- Performance guidance for administrators
- Performance FAQ
