Analytics settings: Difference between revisions

From MoodleDocs
Line 18: Line 18:
Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. Moodle core includes 2 prediction processors:
Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. Moodle core includes 2 prediction processors:


* The PHP processor is the default. There are no other system requirements to use this processor.
==== PHP predictions processor ====
* The Python processor is more powerful and it generates [https://www.tensorflow.org/get_started/summaries_and_tensorboard graphs that explain the model performance]. It requires setting up extra tools: Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodlemlbackend python package. The latest version of the package for '''Moodle 3.8''' is compatible with '''Python 3.4'''. Note that the package should be available for both the Command Line Interface (CLI) user and the user who runs the web server (e.g. www-data).
 
The PHP processor is the default. There are no other system requirements to use this processor.
 
==== Python predictions processor ====
 
The Python processor is more powerful and it generates [https://www.tensorflow.org/get_started/summaries_and_tensorboard graphs that explain the model performance]. It requires setting up extra tools: Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodlemlbackend python package. The package can be installed in the web server (in all the codes if in a clustered environment) or in a separate server.
 
===== Installed in the web server =====
 
The latest version of the package for '''Moodle 3.8''' is compatible with '''Python 3.4, 3.5, 3.6 and 3.7'''. Note that the package should be available for both the Command Line Interface (CLI) user and the user who runs the web server (e.g. www-data).


# If necessary, install Python 3 (and pip for Python 3)
# If necessary, install Python 3 (and pip for Python 3)
Line 27: Line 36:
# You must also enter the path to the Python 3 executable in Site administration -> Server -> System paths:
# You must also enter the path to the Python 3 executable in Site administration -> Server -> System paths:
[[File:path_to_python_3.png|frame|center|Enter system path for Python 3]]
[[File:path_to_python_3.png|frame|center|Enter system path for Python 3]]
===== Installed in a separate server =====
To install the python package in a separate server instead of installing it on the web server/s have some advantages:
* Keeps the python ML backend as an external service
* To keep a separated control of the resources the web server/s dedicate to serving Moodle and the resources dedicated to the python ML backend
* You can reuse the same ML server for multiple Moodle sites. Easier to setup and maintain than to install/upgrade the python package in all nodes in the cluster
* You can install the package as a new docker container in your dockerized environment
* You can serve the ML backend from AWS through the API gateway and AWS lambda, storing the trained model files in S3
On the other hand, it is expected that there can be some added latency in connecting to the python ML backend server.
The python backend is exposed as a Flask application. The Flask application is part of the official 'moodlemlbackend' Python package and its FLASK_APP script is 'webapp', in the root of the package. You are free to use the setup that better suits your existing infrastructure.
====== New server in your infrastructure ======
The python ML backend is exposed as a Flask application, which uses a WSGI server (https://wsgi.readthedocs.io/en/latest/what.html) to be exposed to the www. The official documentation on how to deploy a Flask app can be found in https://flask.palletsprojects.com/en/1.0.x/tutorial/deploy/.
====== Docker ======
https://hub.docker.com/repository/docker/moodlehq/moodle-mlbackend-python is the official moodle-mlbackend-python docker image. We use it internally at Moodle HQ for internal testing and you can use it as well. You may want more control over the image, if that is the case https://github.com/moodlehq/moodle-docker-mlbackend-python/blob/master/Dockerfile can serve as an example of what is needed to get the package working.
 
====== AWS serverless service ======
You can serve the Flask application as a serverless application using the AWS API gateway and AWS lambda. The easiest way to do it is using Zappa https://github.com/Miserlou/Zappa to deploy the Flask application contained in the python ML package.


Once this has been done, you can select the Python prediction processor as the default or for an individual model:
Once this has been done, you can select the Python prediction processor as the default or for an individual model:

Revision as of 08:50, 16 December 2019

Template:Learning analytics The Moodle learning analytics system requires some initial configuration before it can be used. You can access Analytics settings from Site administration > Analytics > Analytics settings.

Site information

Site information will be used to help learning analytics models take characteristics of the institution into account. This information is also reported as part of site data collection when you register your site. This will allow HQ to understand which areas in learning analytics are seeing the most use and prioritize development resources appropriately.

Configure learning analytics settings

New feature
in Moodle 4.1!
Analytics may be disabled from Site administration / Advanced features

They may then be configured from Site administration / Analytics.

Predictions processor

Predictions processor selection

Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. Moodle core includes 2 prediction processors:

PHP predictions processor

The PHP processor is the default. There are no other system requirements to use this processor.

Python predictions processor

The Python processor is more powerful and it generates graphs that explain the model performance. It requires setting up extra tools: Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodlemlbackend python package. The package can be installed in the web server (in all the codes if in a clustered environment) or in a separate server.

Installed in the web server

The latest version of the package for Moodle 3.8 is compatible with Python 3.4, 3.5, 3.6 and 3.7. Note that the package should be available for both the Command Line Interface (CLI) user and the user who runs the web server (e.g. www-data).

  1. If necessary, install Python 3 (and pip for Python 3)
  2. Ensure that you use Python 3 to install the moodlemlbackend package:
   sudo -H python3 -m pip install moodlemlbackend
  1. You must also enter the path to the Python 3 executable in Site administration -> Server -> System paths:
Enter system path for Python 3
Installed in a separate server

To install the python package in a separate server instead of installing it on the web server/s have some advantages:

  • Keeps the python ML backend as an external service
  • To keep a separated control of the resources the web server/s dedicate to serving Moodle and the resources dedicated to the python ML backend
  • You can reuse the same ML server for multiple Moodle sites. Easier to setup and maintain than to install/upgrade the python package in all nodes in the cluster
  • You can install the package as a new docker container in your dockerized environment
  • You can serve the ML backend from AWS through the API gateway and AWS lambda, storing the trained model files in S3

On the other hand, it is expected that there can be some added latency in connecting to the python ML backend server.

The python backend is exposed as a Flask application. The Flask application is part of the official 'moodlemlbackend' Python package and its FLASK_APP script is 'webapp', in the root of the package. You are free to use the setup that better suits your existing infrastructure.

New server in your infrastructure

The python ML backend is exposed as a Flask application, which uses a WSGI server (https://wsgi.readthedocs.io/en/latest/what.html) to be exposed to the www. The official documentation on how to deploy a Flask app can be found in https://flask.palletsprojects.com/en/1.0.x/tutorial/deploy/.

Docker

https://hub.docker.com/repository/docker/moodlehq/moodle-mlbackend-python is the official moodle-mlbackend-python docker image. We use it internally at Moodle HQ for internal testing and you can use it as well. You may want more control over the image, if that is the case https://github.com/moodlehq/moodle-docker-mlbackend-python/blob/master/Dockerfile can serve as an example of what is needed to get the package working.

AWS serverless service

You can serve the Flask application as a serverless application using the AWS API gateway and AWS lambda. The easiest way to do it is using Zappa https://github.com/Miserlou/Zappa to deploy the Flask application contained in the python ML package.

Once this has been done, you can select the Python prediction processor as the default or for an individual model:

Predictions processor default
Predictions processor selection for an individual model

Log store

From Moodle version 2.7 and up, the “Standard logstore” is the default. If for some reason you also have data in the older “legacy logs,” you can enable the Moodle Learning Analytics system to access them instead.

Analysis intervals

Analysis intervals determine how often insights will be generated, and how much information to use for each calculation. Using proportional analysis intervals allows courses of different lengths to be used to train a single model.

Several analysis intervals are available for models in the system. In this setting, the analysis intervals that will be used to evaluate models are defined, e.g. so the best analysis interval identified by the evaluation process can be selected for the model. This setting does not restrict the analysis intervals that can be used for specific models.

Analysis intervals

Each analysis interval divides the course duration into segments. At the end of each defined segment, the predictions engine will run and generate insights. It is recommended that you only enable the analysis intervals you are interested in using; the evaluation process will iterate through all enabled analysis intervals, so the more analysis intervals enabled, the slower the evaluation process will be.

Models output directory

Models output directory

This setting allows you to define a directory where machine learning backends data is stored. Be sure this directory exists and is writable by the web server. This setting can be used by Moodle sites with multiple frontend nodes (a cluster) to specify a shared directory across nodes. This directory can be used by machine learning backends to store trained algorithms (its internal variables weights and stuff like that) to use them later to get predictions. Moodle cron lock will prevent multiple executions of the analytics tasks that train machine learning algorithms and get predictions from them.

Scheduled tasks

Most analytics API processes are executed through scheduled tasks. These processes usually read the activity log table and can require some time to finish. You can find Train models and Predict models scheduled tasks listed in Administration > Site administration > Server > Scheduled tasks. It is recommended to edit the tasks schedule so they run nightly.

Defining roles

Moodle learning analytics makes use of a number of capabilities. These can be added or removed from roles at the site level or within certain contexts to customise who can view insights.

To receive notifications and view insights, a user must have the analytics:listinsights capability within the context used as the "Analysable" for the model. For example, the Students at risk of dropping out model operates within the context of a course. Insights will be generated for each enrolment within any course matching the criteria of the model (courses with a start date in the past and an end date in the future, with at least one teacher and student), and these insights will be sent to anyone with the listinsights capability in that course. By default, the roles of Teacher, Non-editing teacher, and Manager have this capability.

Some models (e.g. the No teaching model) generate insights at the Site level. To receive insights from these models, the user must have a role assignment at the System level which includes the listinsights capability. By default, this is included in the Manager role if assigned at the site level.

Note: Site administrators do not automatically receive insight notifications, though they can choose to view details of any insight notifications on the system. To enable site administrators to receive notifications of insights, assign an additional role that includes the listinsights capability to the site administrator at the system level (e.g. the Manager role).