Logging 2: Difference between revisions
From MoodleDocs
| Line 58: | Line 58: | ||
* --> Action logging improvements META - MDL-28443 | * --> Action logging improvements META - MDL-28443 | ||
== | ==Solution summary== | ||
We can't actually predict all the use cases so we must think generically and plan for worst cases. | We can't actually predict all the use cases so we must think generically and plan for worst cases. | ||
| Line 69: | Line 69: | ||
* add_to_log retained for backward compatibility, make it a wrapper for new logger function. | * add_to_log retained for backward compatibility, make it a wrapper for new logger function. | ||
* provide hooks to expose logs everywhere. For example, each page should have a link that shows logs and stats for that page. | * provide hooks to expose logs everywhere. For example, each page should have a link that shows logs and stats for that page. | ||
===Moodle -> Logs === | ===Moodle -> Logs === | ||
Revision as of 07:33, 17 December 2012
Why is Logging an important topic?
Logging is fundamental to research, reporting and analytics.
- Not much of new Moodle development is based on measured data. We need to
- Prove there is a problem
- Prove that a given change is an improvement.
- Data comes from logging
- Better data would allow researchers to study what happens in online teaching.
- Better data would give admins more feedback on how to run their site (both technically and process)
- Better data would give teachers better feedback to improve their teaching process.
- Better data would give students better feedback to improve the learning process.
- We can't predict all the analysis that might happen in future.
Problems
- Logging is added adhoc by developers, has spotty coverage. No admin logging!
- Log tables are joined in many popular queries and are slow.
- No archiving, so have to delete old logs
- Deletion is wholesale, no control.
- No synchronization with server logs to help debugging performance issues
- No storage of traces when there are problems, so we can debug Moodle issues
Users of log table at the moment
A lot of the time-critical uses of the log table can be moved to new tables specifically for this purpose. It will speed things up a lot.
- recent activity (block, myMoodle) EVENTS -> NEW TABLE
- conditional access EVENTS -> NEW TABLE
- failed logins EVENTS -> NEW TABLE
- files via clamav EVENTS -> NEW TABLE
Other things can can continue to query the main logs:
- course activity report, participation report, user outline report
- stats
- cron
- automated backup locking/heartbeats (probably)
- auth/mnet/auth.php
- backup/restore (target logs and own logging)
- grade history (currently separate log)
And some future things that will need the full log:
- Engagement reporting
Places where we need more logs
- Errors and warnings
- People report errors, but cannot duplicate it.
- Logging history with stack trace would be very useful
- Also dump session variables and all other data that can be useful for debugging
- Admin activities
- Micro activities in a page (more than one per load)
- AJAX calls (eg on course page)
- Should be careful about logging confidential information (e.g. do not log any POST data)
- Shouldn’t delete logs when course is deleted
- All events should all be logged
- Add more events (they are cheap and very useful)
- Need more standardisation of event data for all triggered events
- Add a catch-all handler that pushes events to the logging system
- --> Action logging improvements META - MDL-28443
Solution summary
We can't actually predict all the use cases so we must think generically and plan for worst cases.
- Things that don't need to use logs should not use logs. (Recent activity, etc)
- Make logging calls as cheap as possible
- Use MUC-like plug-ins to determine where to put logs permanently (NoSQL, SAS, file...). Default will be to the current log table.
- Log everything we possibly can think of.
- Define log level settings (on each logging call) so different sites can choose what they want logged and so control size, speed etc
- add_to_log retained for backward compatibility, make it a wrapper for new logger function.
- provide hooks to expose logs everywhere. For example, each page should have a link that shows logs and stats for that page.
Moodle -> Logs
- MUC-like plugins to interface between our logging calls and log backends
- File
- logsstash
- fluentd
- memcached?
- http://logging.apache.org/log4php/ ?
Logs -> Moodle
- Each plugin implements some methods to query and extract logs back to Moodle
- Do we auto-sync anything back to Moodle tables for convenience?
See also
Examples of data-based learning technology development:
