reportbuilder/Totara thoughts on integration
| Warning: This page is no longer in use. The information contained on the page should NOT be seen as relevant or reliable. |
Having previously given some thought to merging reportbuilder into Moodle, and as the original author I thought I'd put down some thoughts ideas that might help with the process.
Areas that may need some work
1. Group Concat
Report builder has some code to provide a cross-platform version of the mysql only aggregate function group_concat (which lets you aggregate text data by concatenation). As you spotted there is postgres specific code in totara/reportbuilder/db/install.php which creates a new aggregate function. In 2.6 (seedlings code) we added support for MSSQL which proved to be quite difficult. See here for how that is done:
https://github.com/totara/seedlings/blob/totara-seedlings-2.6/totara/mssql/db/install.php
Both the postgres and mssql solutions might be considered a bit "custom" for core moodle.
Also Totara doesn't support Oracle at all so currently there is no Oracle compatibility. There is certainly some work there if you decide that group concat support is wanted. If not you would just need to remove any existing uses of it (activity and tag lists in course source, cohort filters, multiselect custom field type). A search for sql_group_concat should find them all.
2. Embedded reports
Currently all embedded report classes are stored in totara/reportbuilder/embedded/. That's fine for us but in Moodle you will want plugins to be able to define their own embedded reports which they make use of. That will probably mean using autoloading but you will also need a way for the system to discover which embedded reports exist (so you can provide a list on totara/reportbuilder/index.php).
3. Extending sources
This is something that would help us since once report builder is in core we'll need to be able to put back our totara-specific additions. It would be helpful if it was doable without us having to make too many core changes. One thing we will be able to do is extend the existing sources like this:
But it would be great if this could be kept in mind as I'm sure lots of organisations would appreciate an easy way to add their own functionality into existing report sources.
Potential changes to consider
The following are all things we've intended to do but not got around to yet. I thought I'd mention them here as they involve API changes, and so if you consider any of them important it would be better to get them in from the start rather than having to make breaking changes later (requiring plugin authors to update their sources).
1. Update filters to use GET instead of POST
This would allow people to use a URL instead of having to create a saved search to point to a particular results set. At the same time I would refactor saved searches to use a table with key/value pairs rather than how it is done right now (serializing filter data in session/database).
2. Modify filter UI
Currently the report creator defines which filters are available and they are all displayed every time (with some set to advanced). We would like to change it so the user can see a list of all filters and then chooses which ones to add (more like JIRA basic search). The main advantages are a cleaner UI and it would make it possible to add extra features such as AND/OR logic and reusing the same filter multiple times in a single report.
3. Refactor Content restrictions by merging with filters.
For historical reasons content restrictions have completely separate code from filters. However they have ended up converging and I think the system would be more flexible if they were combined. All existing content restrictions could be converted to filters fairly easily (they are a bit more dynamic but basically the same). If item 1 above was done then the content tab could just list all available filters and the report creator would add them as a kind of pre-filter. The only difference would be that report users couldn't remove filters that the admin had setup. This would instantly make all the current filters available as content restrictions which would be great!
4. More fine-grained management of reports.
Currently there is one capability that controls the ability to manage all reports. As they get used more its becoming evident that this needs to be more fine-grained. Perhaps reports should use contexts to control who can manage them, or maybe each report should be "owned" by the creator and only them and site admins can edit them. I'm not sure of the best approach but it is clear that the current system is a bit limited.
Related is the list of embedded reports on totara/reportbuilder/index.php. This is already getting quite long for us as it is a site-wide list of all embedded reports. It seems like it will soon need to be split out in some way, either with categories, or separate per-component management areas.
5. Improved support for reusing predefined "groups" of reporting functionality
As you know report sources are quite prescriptive in that the developer must specify exactly what is available. Early on we found the need to reuse some common fields (such as user profile fields) across multiple reports and ended up with helper functions like add_user_fields_to_columns() in rb_base_source.php to achieve that. While predefined sources have got us a long way, we are starting to see requests for more flexibility - people wanting to pull in data from multiple existing sources.
I've been considering how it might be possible to define a group of fields/filters along with any foreign keys that they can join against to make it easier to reuse common code.
Other potential future features
Here are a few ideas that we are looking at adding to future versions of report builder.
1. Graphical reporting
It's long been on our roadmap to add support for graphical reports using a front-end javascript graphing library. This is a high priority for us this year so may be something we tackle soon. With report builder going into core we would most likely do this as a contribution to Moodle so would need to go through the normal process for code contributions. Just wanted to give you a heads up that it's on the cards.
2. Separate reporting database connection
Particularly on large sites, big reports can take a while to generate if there's huge amounts of data and multiple complex joins. In 2.5 we added report caching which provides a way to get faster reporting at the expense of real-time results. Another approach we investigated was supported a separate database connection to a read-only slave database specifically for reporting. This has the advantage of reducing load on the main database server. We have some code that does this but haven't integrated it yet. Happy to share what we have if it sounds of interest.
