Analytics settings

The Moodle learning analytics system requires some initial configuration before it can be used. You can access Analytics settings from Site administration > Analytics > Analytics settings.

Site information

Site information will be used to help learning analytics models take characteristics of the institution into account. This information is also reported as part of site data collection when you register your site. This will allow HQ to understand which areas in learning analytics are seeing the most use and prioritize development resources appropriately.

Configure learning analytics settings

Analytics may be disabled from Site administration / Advanced features

They may then be configured from Site administration / Analytics.

Predictions processor

Predictions processor selection

Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. Moodle core includes 2 prediction processors:

PHP predictions processor

The PHP processor is the default. There are no other system requirements to use this processor.

Python predictions processor

The Python processor is more powerful and it generates graphs that explain the model performance. It requires setting up extra tools: Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodlemlbackend python package. The package can be installed in the web server (in all the nodes if in a clustered environment) or in a separate server.

Versions

• Moodle 3.8 and up uses package 2.3.* (minimum 2.3.1)
• Moodle 3.9.11 and up uses package 2.6.* (minimum 2.6.4)
• Moodle 3.10.8 and up uses package 2.6.* (minimum 2.6.4)
• Moodle 3.11.4 and up uses package 2.6.* (minimum 2.6.4)
• Moodle 4.0 and up uses package 3.0.* (minimum 3.0.2)

Installed in the web server

The latest version of the packages for Moodle 3.8 and up are compatible with Python 3.4, 3.5, 3.6 and 3.7. Note that the package should be available for both the Command Line Interface (CLI) user and the user who runs the web server (e.g. www-data).

1. If necessary, install Python 3 (and pip for Python 3)
2. Ensure that you use Python 3 to install the moodlemlbackend package (for VERSION, see the previous section):

   sudo -H python3 -m pip install "moodlemlbackend==VERSION"

1. You must also enter the path to the Python 3 executable in Site administration -> Server -> System paths:

Enter system path for Python 3

Installed in a separate server

To install the python package in a separate server instead of installing it on the web server/s have some advantages:

• Keeps the python ML backend as an external service
• To keep a separated control of the resources the web server/s dedicate to serving Moodle and the resources dedicated to the python ML backend
• You can reuse the same ML server for multiple Moodle sites. Easier to setup and maintain than to install/upgrade the python package in all nodes in the cluster
• You can install the package as a new docker container in your dockerized environment
• You can serve the ML backend from AWS through the API gateway and AWS lambda, storing the trained model files in S3

On the other hand, it is expected that there can be some added latency in connecting to the python ML backend server.

The python backend is exposed as a Flask application. The Flask application is part of the official 'moodlemlbackend' Python package and its FLASK_APP script is 'webapp', in the root of the package. You are free to use the setup that better suits your existing infrastructure.

New server in your infrastructure

The python ML backend is exposed as a Flask application, which uses a WSGI server (https://wsgi.readthedocs.io/en/latest/what.html) to be exposed to the www. The official documentation on how to deploy a Flask app can be found in https://flask.palletsprojects.com/en/1.0.x/tutorial/deploy/.

• Use MOODLE_MLBACKEND_PYTHON_USERS environment var to set a list of users and password (comma-separated). The value is 'default:sshhhh' (user: default, password: sshhhh).
• Set MOODLE_MLBACKEND_PYTHON_DIR to the path you want to use to store the data generated by the package

Docker

https://hub.docker.com/r/moodlehq/moodle-mlbackend-python is the official moodle-mlbackend-python docker image. We use it internally at Moodle HQ for internal testing and you can use it as well. You may want more control over the image, if that is the case https://github.com/moodlehq/moodle-docker-mlbackend-python/blob/master/Dockerfile can serve as an example of what is needed to get the python moodlemlbackend package working.

• Use MOODLE_MLBACKEND_PYTHON_USERS environment var to set a list of users and passwords (comma-separated). The value is 'default:sshhhh' (user: default, password: sshhhh).

To run the docker container locally you can execute (For VERSION, see the Versions section above):

docker pull moodlehq/moodle-mlbackend-python:VERSION-python3.7.5
docker run -d -p 5000:5000 --name=mlbackendpython --rm --add-host=mlbackendpython:0.0.0.0 moodlehq/moodle-mlbackend-python:VERSION-python3.7.5

Note that you need to add --network=moodledocker_default if your are using moodle-docker and you want this container to be visible from the web server.

Then add this to your config.php file:

$CFG->pathtopython = 'python';
define('TEST_MLBACKEND_PYTHON_HOST', 'localhost'); // Change to "mlbackendpython" if your are using moodle-docker.
define('TEST_MLBACKEND_PYTHON_PORT', 5000);
define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');

To check if it works, this command should not Skip any test:

vendor/bin/phpunit analytics/tests/prediction_test.php --verbose

AWS serverless service

You can serve the Flask application as a serverless application using the AWS API gateway and AWS lambda. The easiest way to do it is using Zappa https://github.com/Miserlou/Zappa to deploy the Flask application contained in the python ML package.

• Use MOODLE_MLBACKEND_PYTHON_USERS environment var to set a list of users and password (comma-separated). The value is 'default:sshhhh' (user: default, password: sshhhh).
• Set MOODLE_MLBACKEND_PYTHON_DIR to the path you want to use to store the data generated by the package
• Set these environment variables below to setup the S3 access:
  • MOODLE_MLBACKEND_PYTHON_S3_BUCKET_NAME to the bucket name
  • AWS_ACCESS_KEY_ID as usual
  • AWS_SECRET_ACCESS_KEY as usual

Once this has been done, you can select the Python prediction processor as the default or for an individual model:

Predictions processor default

Predictions processor selection for an individual model

Log store

From Moodle version 2.7 and up, the “Standard logstore” is the default. If for some reason you also have data in the older “legacy logs,” you can enable the Moodle Learning Analytics system to access them instead.

Analysis intervals

Analysis intervals determine how often insights will be generated, and how much information to use for each calculation. Using proportional analysis intervals allows courses of different lengths to be used to train a single model.

Several analysis intervals are available for models in the system. In this setting, the analysis intervals that will be used to evaluate models are defined, e.g. so the best analysis interval identified by the evaluation process can be selected for the model. This setting does not restrict the analysis intervals that can be used for specific models.

Analysis intervals

Each analysis interval divides the course duration into segments. At the end of each defined segment, the predictions engine will run and generate insights. It is recommended that you only enable the analysis intervals you are interested in using; the evaluation process will iterate through all enabled analysis intervals, so the more analysis intervals enabled, the slower the evaluation process will be.

Models output directory

Models output directory

This setting allows you to define a directory where machine learning backends data is stored. Be sure this directory exists and is writable by the web server. This setting can be used by Moodle sites with multiple frontend nodes (a cluster) to specify a shared directory across nodes. This directory can be used by machine learning backends to store trained algorithms (its internal variables weights and stuff like that) to use them later to get predictions. Moodle cron lock will prevent multiple executions of the analytics tasks that train machine learning algorithms and get predictions from them.

Scheduled tasks

Most analytics API processes are executed through scheduled tasks. These processes usually read the activity log table and can require some time to finish. You can find Train models and Predict models scheduled tasks listed in Administration > Site administration > Server > Scheduled tasks. It is recommended to edit the tasks schedule so they run nightly.

Capabilities

There are two analytics capabilities:

• Manage models - allowed for the default role of manager only
• List insights - allowed for the default roles of manager, teacher and non-editing teacher

To receive notifications and view insights, a user must have the list insights capability within the context used as the "Analysable" for the model. For example, the Students at risk of dropping out model operates within the context of a course. Insights will be generated for each enrolment within any course matching the criteria of the model (courses with a start date in the past and an end date in the future, with at least one teacher and student), and these insights will be sent to anyone with the list insights capability in that course.

Some models (e.g. the No teaching model) generate insights at the Site level. To receive insights from these models, the user must have a role assignment at the System level which includes the list insights capability.

Note: Site administrators do not automatically receive insight notifications, though they can choose to view details of any insight notifications on the system. To enable site administrators to receive notifications of insights, assign an additional role that includes the list insights capability to the site administrator at the system level.