London hackfest April 2017 - Live monitoring moodle sites

Part of London hackfest April 2017.

How do people monitor their live Moodle sites?

Some of those present shard how they monitor their live Moodle site, or do load-testing before upgrades.

A lot of Nagios, but also some other things like ???.

There are at least two plugins for make more information from Moodle available in Nagios: https://moodle.org/plugins/tool_heartbeat & https://github.com/University-of-Strathclyde-LTE-Team/moodle-local_nagios.

A lot of people have moved from original MySQL to clones like Maria DB or Percona. Others use Postgres.

Most people doing load-testing seem to be using JMeter. There are scripts that can generate a JMeter script for users interacting with a particular course. A weakness is that JMeter only does the original PHP/HTML load + images, CSS & JS. It will not simulate any HTTP requests that are done by AJAX. You need to script those manually to get a realistic test.

It turns out that having JMeter to simulate realistic load is useful when you need to investigate & reproduce why your Moodle site crashed ...

How has your Moodle site crashed?

The above discussion also involved people sharing their horror stories of when their Moodle site crashed. Database tuning seems key. (And, if you are running many sites, one huge shared DB server is probably not the best option.)

Cash purging at busy times is also probably not a good thing. (Particularly bad with Memcache.)

Also, a cat (or book) sat on the keyboard (F5 - refresh) tends to be very bad. Probably best prevented by some rate-limiting on your load-balancer.

Better monitoring for web services

The final thing we talked about was how to improve the monitoring of web services. At the moment, all AJAX requests look like /lib/ajax/service.php?sesskey=xxxxxx, at least in the logs. Two things that are problematic with that:

• If you have some badly performing requests (but others are fast) you don't really know what is going on. The proposal is to put the method name into the URL (even though that would then be ignored by Moodle because the method name that is actually used is in the POST data). This is now MDL-58555.
• One original page-load can have many linked AJAX requests. There is no way to link those together, to get the total back-end performance impact of one original page-load. No tracker issue yet.