Note: You are currently viewing documentation for Moodle 3.6. Up-to-date documentation for the latest stable version of Moodle is likely available here: Analytics.

Analytics

From MoodleDocs
Revision as of 14:26, 9 November 2017 by David Monllaó (talk | contribs)

New feature
in Moodle 3.6!


Overview

This page describes analytics features in Moodle core. Open source, transparent next-generation learning analytics using machine learning backends that go beyond simple descriptive analytics to provide predictions of learner success to learners and teachers.

Settings

You can access Analytics settings from Site administration > Analytics > Analytics settings.

Predictions processor

Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. Moodle core includes 2 prediction processors:

  • The PHP one is the default. There are no other system requirements to use this processor.
  • The Python one is more powerful and it generates graphs with the model performance but it requires setting up extra stuff like Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodlemlbackend package.
   pip install moodlemlbackend

Time splitting methods

The time splitting method divides the course duration in parts, the predictions engine will run at the end of these parts. It is recommended that you only enable the time splitting methods you could be interested on using; the site contents analyser will calculate all indicators using each of the enabled time splitting methods. The more enabled time splitting methods the slower the evaluation process will be.

Models output directory

This setting allows you to define a directory where machine learning backends data is stored. Be sure this directory exists and is writable by the web server.

Models management

Moodle can support multiple prediction models at once. Moodle core comes with two prediction models Students at risk of dropping out and No teaching, you can add your own prediction models using Moodle analytics API. Each model is based on the prediction of a single, specific "target," or outcome (whether desirable or undesirable), based on a number of selected indicators.

You can manage your system models from Site Administration > Analytics > Analytics models.

prediction-models-list.jpeg

These are some of the actions you can perform on a model:

  • Get predictions: Train machine learning algorithms with the new data available on the system and get predictions for ongoing courses. Predictions are not limited to ongoing courses, this depends on the model.
  • Evaluate: Evaluate the prediction model by getting all the training data available on the site, calculating all the indicators and the target and passing the resulting dataset to machine learning backends, they will split the dataset into training data and testing data and calculate its accuracy. Note that the evaluation process use all information available on the site, even if it is very old, the accuracy returned by the evaluation process will generally be lower than the real model accuracy as indicators are more reliably calculated straight after training data is available because the site state changes along time. The metric used as accuracy is the Matthew’s correlation coefficient (good metric for binary classifications)
  • Log: View previous evaluations log, including the model accuracy as well as other technical information generated by the machine learning backends like ROC curves, learning curves graphs, the tensorboard log dir or the model's Matthew’s correlation coefficient. The information available will depend on the machine learning backend in use.
  • Edit: You can edit the models by modifying the list of indicators or the time-splitting method. All previous predictions will be deleted when a model is modified. Models based on assumptions (static models) can not be edited.
  • Enable / Disable: The scheduled task that trains machine learning algorithms with the new data available on the system and gets predictions for ongoing courses skips disabled models. Previous predictions generated by disabled models are not available until the model is enabled again.
  • Export: Export your site training data to share it with your partner institutions.
  • Invalid site elements: Reports on what elements in your site can not be analysed by this model
  • Clear predictions: Clears all the model predictions and training data

model-evaluation.jpeg

Core models

Students at risk of dropping out

This model predicts students who are at risk of non-completion (dropping out) of a Moodle course, based on low student engagement. The prediction model uses the Community of Inquiry model of student engagement, consisting of three parts:

  • Cognitive presence
  • Social presence
  • Teacher presence

This prediction model is able to analyze and draw conclusions from a wide variety of courses, and apply those conclusions to make predictions about new courses. The model is not limited to making predictions about student success only in perfect copies of courses offered in the past. However, there are some limitations:

  1. This prediction model assumes that courses have fixed start and end dates, and is not designed to be used with rolling enrollment courses. Models that support a wider range of course types will be included in future versions of Inspire. Because of this model design assumption, it is very important to properly set course start and end dates for each course to use this model. If both past courses and ongoing courses start and end dates are not properly set predictions cannot be accurate. Because the course end date field was only introduced in Moodle 3.2 and some courses may not have set a course start date in the past, we include a command line interface script (https://github.com/moodlehq/moodle-tool_inspire/blob/master/cli/guess_course_start_and_end.php) that attempts to estimate past course start and end dates by looking at the student enrolments and students' activity logs. After running this script, please check that the estimated start and end dates script results are reasonably correct.
  2. This model requires a certain amount of in-Moodle data with which to make predictions. At the present time, only core Moodle activities are included in the indicator set (see below). Courses which do not include several core Moodle activities per “time slice” will have poor predictive support in this model. This prediction model will be most effective with fully online or “hybrid” or “blended” courses with substantial online components.

No teaching

This model insights will inform site managers of which courses with a close start date will not have teaching activity. This is a simple model and it does not use machine learning backend to return predictions. It bases the predictions on assumptions like there is no teaching if there are no students.

Predictions and Insights

Models will start generating predictions at different point in time, depending on the site prediction models and the site courses start and end dates.

Each model defines which predictions will generate insights and which predictions will be ignored. This is an example of Students at risk of dropping out prediction model; if a student is predicted as not at risk no insight is generated as what is interesting is to know which students are at risk of dropping out of courses, not which students are not at risk.

prediction-model-insights.jpeg