Engagement Analytics Plugin

Introduction

Engagement Analytics Block example

The Engagement Analytics block provides information about student progress against a range of indicators. As the name suggests the block provides feedback on the level of "engagement" of a student, in this plugin "engagement" refers to activities which have been identified by current research to have an impact on student success in an online course. The plugin was developed as part of a NetSpot Innovation Fund project by Monash University (Project manager: Dr Phillip Dawson), with code by NetSpot developers (Ashley Holman and Adam Olley).

From the project plan: "We intend to implement a block that teachers can add to their Moodle course that will provide them with a quick graphical snapshot of which students are at risk." (Dr Phillip Dawson)

It is important to note that the purpose of the plugin is to provide teaching with information only, it does not automatically take any action based on the indicators eg NO email or notification is sent to students automatically. If desired the teacher would follow up on the information themselves, based on what they know about the student and their other communications.

Currently the plugin has three indicators: - Forum activity - Login activity - Assessment activity

The block has been constructed to allow additional indicators to be developed to extend it's current functionality.

Each of these indicators measures different metrics which are combined using configurable thresholds and weightings to produce an overall risk for that indicator. The individual indicators are then combined together (with more configurable weightings) to produce an overall risk for the student on that course. The risk is reflected in the block as a percentage (100% being maximum risk of failure, 0% being minimum risk of failure). The colour codes change from green to red as the risk increases.

The Engagement Analytics block, includes a block and reports that are only visible to teachers, or higher.

This page describes the configuration and use of this plugin.

Engagement Analytics Report example

Plugin release date

Now available 20/08/2012 [http://moodle.org/plugins/browse.php?list=set&id=20 ]

Installation

Please see Installing Moodle Plugins for information about how to install plugins.

Terminology

• Indicator: An indicator is an element of LMS interaction which may determine the success of the student. This plugin has been developed so that additional indicators can be added. The indicators currently included are: Assessment Activity, Forum Activity and Login Activity.
• Weighting: The weighting of an indicator determines its importance and therefore how much it influences the measurement of a student's "risk". The higher the percent in the weighting, the more important the indicator. For example a course may be blended, where discussions are predominantly held face to face, the course might not require participation in Forums, therefore forum activity would have a low weighting.
• Algorithm: The formula used to calculate the risk, based on the settings for each indicator.

About Indicators

The indicators have been selected based on evidence provided by current research. Each indicator is calculated based on an algorithm, which has been developed by Dr Phillip Dawson in conjunction with the Project team and Project reference group. For more information about the development of the algorithm please contact Dr Phillip Dawson (email: phillip.dawson@monash.edu). The plugin has been built so that additional indicators can be developed and added. Each indicator has default values, based on the current research, which can be modified per course to suit the delivery and outcomes of the course.

The indicators are displayed to the teacher in a block, which uses as traffic light system to indicate visually the level of risk - Green for low or no risk, through to Red for high risk. The teacher can view a course report or click on an individual student to see how each of the indicators is contributing to their risk result.

Course weighting

The weighting allows you to set the importance of each of the indicators for the course.

• In your course, select > Reports > Engagement Analytics

Course weighting

Forum activity indicator

The forum activity indicator, by default:

• Returns 0 (Green) if fa student is at no risk and 1 (red) if a student is a maximum risk.
• Maximum risk = a student has not read any forum posts and has not written any forum posts.
• Minimum risk = a student has read all forum posts and has written many replies and new posts.

How to:

• In your course, select > Reports > Engagement Analytics
• Scroll down to Forum Activity

Forum Activity Settings

Login activity indicator

The login activity indicator, by default:

• Returns 0 if student at no risk; 1 if student at maximum risk.
• Maximum risk = a student has not logged in.
• Minimum risk = a student has logged in frequently and spends time in the course.
• Course-specific on a Moodle instance with multiple courses this indicator is interested in student behaviour on a particular course.

How to:

• In your course, select > Reports > Engagement Analytics
• Scroll down to Login Activity

Login Activity Settings

Assessment activity indicator

The Assessment activity indicator, by default:

• Returns 0 if student at no risk; 1 if student at maximum risk.
• Maximum risk = a student has not submitted any assessment, and at least one assessment task is overdue by the maximum number of days.
• Minimum risk = a student has submitted all assessments on time so far (or no assessment has been due).
• It is only concerned with assessments whose due dates have passed.

How to:

• In your course, select > Reports > Engagement Analytics
• Scroll down to Assessment activity

Assessment Activity Settings

Page status

Complete, any feedback or suggestions please feel free to email me (kim.edgar@netspot.com.au).