Inspire

Overview

The Inspire plugin implements open source, transparent next-generation learning analytics using machine learning backends that go beyond simple descriptive analytics to provide predictions of learner success, and ultimately diagnosis and prescriptions (advisements) to learners and teachers.

Features

• A built-in prediction model: Students at risk of dropping out of courses.
• A set of student engagement indicators based on the Community of Inquiry.
• Built-in tools to evaluate models against your site's data
• Proactive notifications for instructors using Events
• Instructors can easily send messages to students identified by the model, or jump to the Outline report for that student for more detail about student activity
• An API to build indicators and prediction models for third-party Moodle plugins
• Machine learning backend plugin type - supports PHP and Python, and can be extended to implement other ML backends

Limitations

The initial 3.3-r1 version of the Inspire plugin has some limitations:

• This version is installed as a third-party plugin. Future versions will become part of Moodle Core.
• This version only reads activity logs from the standard log store. A log store selector will be added in future versions of the plugin.
• This version requires a site with previous completed course, ideally using the Moodle course completion feature. This data is used to train the prediction engine.
• The prediction model included with this version requires that courses have fixed start and end dates, and is not designed to be used with rolling enrollment courses. Models that support a wider range of course types will be included in future versions of Inspire.
• Models and predictions are only visible to teachers and administrators at present.

We are continuing to enhance Inspire, and expanded capabilities will be released going forward. To help contribute to our progress, please join the conversation at the Project Inspire Community. In particular, we need data sets from a wide variety of Moodle-using institutions in order to be able to ship a working prediction model that does not depend on local site data before it can be used.

Installation

See Installing plugins#installing a plugin.

If you are manually installing, this plugin should be unpacked to admin/tool/inspire.

Plugin configuration

After installation completes, you will be prompted to configure the settings for the Inspire plugins. You can also access these options at any time within the Site Administration menu:

Dashboard / Site administration / Plugins / Admin tools / Inspire

Or at:

http://your.moodle.site/moodle/admin/settings.php?section=inspiremanagement

Predictions processor

Predictions processor selection

Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets and return predictions. This version of Inspire is shipped with 2 prediction processors:

• The PHP processor is the default. There are no other system requirements to use this processor.
• The Python processor is more powerful and it generates graphs with the model performance but it requires setting up extra tools like Python itself (https://wiki.python.org/moin/BeginnersGuide/Download) and the moodleinspire python package:

$ pip install moodleinspire

Time splitting methods

The time splitting method divides each course into segments, based on the “Course start date” and “Course end date” settings per course. The predictions engine will run at the end of each segment. It is recommended that you only enable the time splitting methods you plan to use; the site contents analyser will calculate all indicators using each of the enabled time splitting methods. The more enabled time splitting methods, the slower the evaluation process will be.

The use of these time splitting methods allows insights generated from one course to be used on another course, even if the two are not exactly the same length.

Models output directory

This setting allows you to define a directory where evaluation data is stored. Be sure this directory exists and is writable by the web server.

Teacher roles

Inspire needs to understand which users should be treated as “teachers” within a course or other context. The actions of teachers are handled differently from the actions of students in the prediction models.

Student roles

Inspire needs to understand which users should be treated as “students” within a course or other context. These roles will be recognized by prediction models. For example, the built-in model, “Identify students at risk,” generates predictions only for users identified as students within courses.

Model configuration

Inspire can support multiple prediction models at once. Each model requires a certain amount of configuration before use. Each model is based on the prediction of a single, specific "target," or outcome (whether desirable or undesirable), based on a number of selected indicators.

Examples of prediction targets:

• Student will not complete a course using defined course completion criteria
• Student will not achieve a passing grade as defined in the course gradebook
• Student will not attain the competencies attached to the course before the course end date
• An unacceptable proportion of students in a given course will not achieve a passing grade, based on the course design
• An unacceptable proportion of students in a given course will not achieve a passing grade, based on teacher actions in the course

Models apply to specific contexts. For example, a model predicting student success in a course is operating in the context of a course. A model could be defined to apply to a cohort of students receiving a specific intervention, or to categories of courses (e.g. to compare courses in different academic subjects or using different teaching methods).

Models

This version of Inspire (3.3-r1) contains one model:

• Students at risk of dropping out

Additional models will be implemented in the future. An API is also provided for community model development.

Students at risk of dropping out

This model predicts students who are at risk of non-completion (dropping out) of a Moodle course, based on low student engagement. The prediction model uses the Community of Inquiry model of student engagement, consisting of three parts:

• Cognitive presence
• Social presence
• Teacher presence

This prediction model is able to analyze and draw conclusions from a wide variety of courses, and apply those conclusions to make predictions about new courses. The model is not limited to making predictions about student success only in perfect copies of courses offered in the past. However, there are some limitations:

1. This prediction model assumes that courses have fixed start and end dates, and is not designed to be used with rolling enrollment courses. Models that support a wider range of course types will be included in future versions of Inspire. Because of this model design assumption, it is very important to properly set course start and end dates for each course to use this model. If both past courses and ongoing courses start and end dates are not properly set predictions cannot be accurate. Because the course end date field was only introduced in Moodle 3.2 and some courses may not have set a course start date in the past, we include a command line interface script (https://github.com/moodlehq/moodle-tool_inspire/blob/master/cli/guess_course_start_and_end.php) that attempts to estimate past course start and end dates by looking at the student enrolments and students' activity logs. After running this script, please check that the estimated start and end dates script results are reasonably correct.
2. This model requires a certain amount of in-Moodle data with which to make predictions. At the present time, only core Moodle activities are included in the indicator set (see below). Courses which do not include several core Moodle activities per “time slice” will have poor predictive support in this model. This prediction model will be most effective with fully online or “hybrid” or “blended” courses with substantial online components.

Indicators

In this model, indicators are defined for the cognitive presence and social presence of each of the Core activity plugins, along with a few other indicators.

Additional indicators will be incorporated in future versions of Inspire. The API also supports community development of indicators.

The Web and CLI interfaces

Large production sites require intense, resource-heavy processing when a model is first enabled and evaluated. Inspire allows operations to be executed via a web-based interface or from the command line.

• The web-based interface is appropriate for smaller sites, e.g. test or prototype sites.

Dashboard / Site administration / Reports / Inspire models

• The command-line interface is appropriate for larger, production sites or copies of production sites.

$ cd <moodle install directory>/admin/tool/inspire/cli

For performance reasons, we do not recommend attempting to use the web-based interface for model configuration and evaluation on production sites.

Enable the model

Before you can evaluate a model, you must select the time splitting method and enable it.

Enable the model using the CLI

$ cd <moodle install directory>/admin/tool/inspire/cli
$ php enable_model.php --modelid=1 --timesplitting=\"\\tool_inspire\\local\\time_splitting\\quarters\"

Enable the model using the web interface

Navigate to the Inspire Models administration page: Dashboard / Site administration / Reports / Inspire models

Select a model from the list of available models, using the Edit menu item:

Use the “Edit” Action menu to view and configure the settings for this model, and to enable it.

In this screen you can choose from among the time splitting methods enabled in the plugin settings, and you can also disable (or restore) some of the default indicators.

Evaluate the model

After enabling a model with a specific time-splitting method and list of indicators, evaluate the model against your existing site data (courses whose end date is in the past). This will allow you to determine which time splitting method works best with your courses. You may wish to return to the “enabling” step to alter the timesplitting method, then re-evaluate.

Evaluate the model using the CLI

$ cd <moodle install directory>/admin/tool/inspire/cli
$ php evaluate_model.php --modelid=1 --non-interactive

Analysing the site The site contents are calculated and stored in batches, during evaluation you can stop the process at any moment, the next time you run it it will continue from the point you stopped it. !! There is no data to evaluate the model !!

Evaluate the model using the Web-based interface

Navigate to the Inspire Models administration page: Dashboard / Site administration / Reports / Inspire models

Select a model from the list of available models, using the Evaluate menu item:

Execute the model

Once the model has been evaluated against past course data, the model can be executed to begin to make predictions about current and future courses (courses with start dates in the past or future, and end dates in the future).

Execute the model using the CLI

These are the 2 tasks that will run through cron regularly. You can force their execution.

php admin/tool/task/cli/schedule_task.php --execute=\\tool_inspire\\task\\train_models
php admin/tool/task/cli/schedule_task.php --execute=\\tool_inspire\\task\\predict_models

The “train” process reviews data from previously completed courses. The “predict” process generates predictions using current courses (or future courses that have not yet started, if the model is designed for that purpose).

Execute the model using the web interface

Navigate to the Inspire Models administration page: Dashboard / Site administration / Reports / Inspire models

Select a model from the list of available models, using the Execute menu item:

View predictions

Predictions are viewed within the web-based interface and are automatically sent to the teacher of a course using notifications. Note that you will first need to enable and execute the model.

Access predictions as a manager/admin

Go to Site administration > Reports > Inspire models Select a context from the predictions list menu, e.g. a specific course. Available contexts depend on the model definition.

Access predictions as a teacher

Go to an ongoing course and Course administration > Reports > Insights A user with tool/inspire:listinsights capability will also automatically receive notifications when new predictions are available for them. e.g. Course teachers will receive a notification about their students at risk of dropping out.

From this screen, the teacher can view more details or send a message directly to the student:

View prediction details

The prediction details show which of the indicator values were used in the prediction, and what the student’s values for those indicators are.

Again, the teacher can send a message to the student at this point, or view the Outline report.

Developer’s documentation

Architecture overview (mostly for developers / systems administrators / researchers): https://docs.moodle.org/dev/Project_Inspire_API

How to request a new feature

• Check in the open issues if your feature have been already requested https://github.com/moodlehq/moodle-tool_inspire/issues
• https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20component%20%3D%20Analytics
• If not, create a New issue Project: Moodle (MDL)
• Select the component Analytics and your Moodle version
• Explain your needs in the Description field
• Vote and Watch the issue you have created

Credits

• David Monllaó - Lead Maintainer
• Elizabeth Dalton - Research Analyst

See Also

• Inspire tool in Plugins database
• Project Inspire Community
• Learning analytics documentation
• Analytics and reporting forum