Note: You are currently viewing documentation for Moodle 3.11. Up-to-date documentation for the latest stable version of Moodle may be available here: Learning analytics indicators.

Learning analytics indicators: Difference between revisions

From MoodleDocs
Main page ► Managing a Moodle course ► Tracking progress ► Analytics ► Learning analytics indicators
Analytics
(→‎General indicators: correction for '''User profile is completed''')
m (Analytics template)
 
Line 1: Line 1:
{{Learning analytics}}
{{Analytics}}
== Overview ==
== Overview ==
Models can contain multiple indicators (predictors). Conceptually, indicators collect and aggregate values from a set of data and will combine with targets to make predictions using the selected machine learning algorithms. Indicators can be re-used between models, or can be created specifically for a new model. However, it is best to understand the target you are trying to predict before selecting indicators. [[File:indicators.png|thumb]]
Models can contain multiple indicators (predictors). Conceptually, indicators collect and aggregate values from a set of data and will combine with targets to make predictions using the selected machine learning algorithms. Indicators can be re-used between models, or can be created specifically for a new model. However, it is best to understand the target you are trying to predict before selecting indicators. [[File:indicators.png|thumb]]
Line 7: Line 7:
How the data points are processed involves important assumptions that affect the indicators. In particular, indicators can be absolute, meaning that the value of the indicator stays the same no matter what other samples are in the context, or relative, meaning that the indicator compares the sample to others in the context.
How the data points are processed involves important assumptions that affect the indicators. In particular, indicators can be absolute, meaning that the value of the indicator stays the same no matter what other samples are in the context, or relative, meaning that the indicator compares the sample to others in the context.


Currently, Moodle does not filter the available indicators for a model according to the analyser (controled by the [[Learning analytics targets | target]] of the model). However, not all indicators are designed to work with all targets. For example, an indicator about a user will not perform well in a model about course design. In the future, Moodle may offer more guidance about how to select indicators that match the target.
Currently, Moodle does not filter the available indicators for a model according to the analyser (controlled by the [[Learning analytics targets | target]] of the model). However, not all indicators are designed to work with all targets. For example, an indicator about a user will not perform well in a model about course design. In the future, Moodle may offer more guidance about how to select indicators that match the target.


== General indicators ==
== General indicators ==
Line 13: Line 13:
* '''Course accessed before start date''' -- This binary indicator is set to 1 if the user has accessed the course before the start date.
* '''Course accessed before start date''' -- This binary indicator is set to 1 if the user has accessed the course before the start date.
* '''Any write action''' -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) anywhere on the site.
* '''Any write action''' -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) anywhere on the site.
* '''Any write action in the course''' -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) in the enroled course.
* '''Any write action in the course''' -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) in the enrolled course.
* '''Read actions amount''' -- This indicator estimates the amount of content the user has accessed (read) on a scale of -1 to 1.
* '''Read actions amount''' -- This indicator estimates the amount of content the user has accessed (read) on a scale of -1 to 1.
* '''Activities due''' -- This binary indicator is set to 1 if the user has activities due and not yet submitted.
* '''Activities due''' -- This binary indicator is set to 1 if the user has activities due and not yet submitted.
Line 35: Line 35:


By categorizing each activity by potential cognitive depth and social breadth, we can anticipate what level of engagement is supported (and possibly expected) of the learner, even without a history of many learners’ actions in that activity instance. Note that higher levels along each axis include all lower levels, i.e. an activity that involves a student and all peers (social breadth 3) automatically includes levels 1 (student only) and 2 (student +1). In many cases, the specific level can only be determined by analyzing the parameter settings for the activity.
By categorizing each activity by potential cognitive depth and social breadth, we can anticipate what level of engagement is supported (and possibly expected) of the learner, even without a history of many learners’ actions in that activity instance. Note that higher levels along each axis include all lower levels, i.e. an activity that involves a student and all peers (social breadth 3) automatically includes levels 1 (student only) and 2 (student +1). In many cases, the specific level can only be determined by analyzing the parameter settings for the activity.


=== Cognitive depth ===
=== Cognitive depth ===
Line 49: Line 48:
This model begins by assigning a maximum potential value of cognitive depth to each activity module. For example, the Assignment module allows ''up to'' cognitive depth of 5.
This model begins by assigning a maximum potential value of cognitive depth to each activity module. For example, the Assignment module allows ''up to'' cognitive depth of 5.


Once the potential levels are assigned, each student enroled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 (viewing feedback) and the student has reached level 3, the student is participating at 100 percent of the possible level of cognitive depth and the value of the indicator is 1. If the student does not view the activity, they are at cognitive depth of 0, and the value of the indicator is -1. If the student has viewed the activity and submitted a response, but not viewed feedback, they are at cognitive depth level 2 (out of a potential depth in this activity of 3) so the indicator is calculated at .33.
Once the potential levels are assigned, each student enrolled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 (viewing feedback) and the student has reached level 3, the student is participating at 100 percent of the possible level of cognitive depth and the value of the indicator is 1. If the student does not view the activity, they are at cognitive depth of 0, and the value of the indicator is -1. If the student has viewed the activity and submitted a response, but not viewed feedback, they are at cognitive depth level 2 (out of a potential depth in this activity of 3) so the indicator is calculated at .33.


==== Course level cognitive depth indicators ====
==== Course level cognitive depth indicators ====
Line 101: Line 100:
This model begins by assigning a maximum potential value of social breadth to each activity module. For example, the Assignment module allows up to social breadth of 2. See below for more details of how these levels are assigned for core activity modules.
This model begins by assigning a maximum potential value of social breadth to each activity module. For example, the Assignment module allows up to social breadth of 2. See below for more details of how these levels are assigned for core activity modules.


Once the potential levels are assigned, each student enroled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 and the student has reached level 3, the student is participating at 100 percent of the possible level of social breadth.
Once the potential levels are assigned, each student enrolled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 and the student has reached level 3, the student is participating at 100 percent of the possible level of social breadth.


'''Note''': at present, Moodle only provides indicators up to Social Breadth level 2. All "[[Resources|resources]]" have a maximum social breadth of 1. [[Activities|Activities]] vary in their supported social breadth. This feature is meant to be expanded in future versions of Moodle.  
'''Note''': at present, Moodle only provides indicators up to Social Breadth level 2. All "[[Resources|resources]]" have a maximum social breadth of 1. [[Activities|Activities]] vary in their supported social breadth. This feature is meant to be expanded in future versions of Moodle.  
Line 134: Line 133:
* '''SCORM social'''
* '''SCORM social'''
* '''Workshop social'''
* '''Workshop social'''


[[es:Indicadores de analítica del aprendizaje]]
[[es:Indicadores de analítica del aprendizaje]]
[[de:Analytics-Indikatoren]]
[[de:Analytics-Indikatoren]]

Latest revision as of 14:43, 4 June 2021

Overview

Models can contain multiple indicators (predictors). Conceptually, indicators collect and aggregate values from a set of data and will combine with targets to make predictions using the selected machine learning algorithms. Indicators can be re-used between models, or can be created specifically for a new model. However, it is best to understand the target you are trying to predict before selecting indicators.

indicators.png

To be used in a model, indicators need to be centered and scaled. In Moodle learning analytics, indicators need to return values converted to a range between -1 and 1. Not all of this range needs to be used-- some indicators will make more sense if scaled from 0 to 1, with 0 indicating a missing value.

centering scaling indicators.png

How the data points are processed involves important assumptions that affect the indicators. In particular, indicators can be absolute, meaning that the value of the indicator stays the same no matter what other samples are in the context, or relative, meaning that the indicator compares the sample to others in the context.

Currently, Moodle does not filter the available indicators for a model according to the analyser (controlled by the target of the model). However, not all indicators are designed to work with all targets. For example, an indicator about a user will not perform well in a model about course design. In the future, Moodle may offer more guidance about how to select indicators that match the target.

General indicators

  • Course accessed after end date -- This binary indicator is set to 1 if the user has accessed the course after the end date.
  • Course accessed before start date -- This binary indicator is set to 1 if the user has accessed the course before the start date.
  • Any write action -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) anywhere on the site.
  • Any write action in the course -- This binary indicator is set to 1 if the user has completed a "write" action (saved content) in the enrolled course.
  • Read actions amount -- This indicator estimates the amount of content the user has accessed (read) on a scale of -1 to 1.
  • Activities due -- This binary indicator is set to 1 if the user has activities due and not yet submitted.
  • Completion tracking enabled -- This binary indicator is set to 1 if the course has completion tracking enabled.
  • There are no students -- This binary indicator is set to 1 if there are no students enroled in the course.
  • There are no teachers -- This binary indicator is set to 1 if there are no teachers enroled in the course.
  • User profile is completed -- This indicator returns a value ranging between -1 and 1 depending on how much the user has completed the user's profile.
  • User is tracking forums -- This binary indicator is set to 1 if the user has forum tracking turned on.

Community of Inquiry indicators

A set of indicators is provided that are based on the Community of Inquiry model of student engagement. These indicators are divided into two types: cognitive depth and social breadth. For each activity in Moodle core, one indicator of each type has been implemented. There are also two course-level "potential" indicators that estimate the potential cognitive depth and social breadth of the course as a whole.

Third-party activity plugins require support by their maintainers to add indicators of any type.

Potential indicator levels for selected activity modules

The potential for engagement via cognitive presence and social presence constitutes instructional design, which is one key element of teaching presence. This diagram shows the potential cognitive depth and social breadth of all core activities and select non-core activities:

coi model indicators.png

By categorizing each activity by potential cognitive depth and social breadth, we can anticipate what level of engagement is supported (and possibly expected) of the learner, even without a history of many learners’ actions in that activity instance. Note that higher levels along each axis include all lower levels, i.e. an activity that involves a student and all peers (social breadth 3) automatically includes levels 1 (student only) and 2 (student +1). In many cases, the specific level can only be determined by analyzing the parameter settings for the activity.

Cognitive depth

Cognitive depth is a measure of the construct "cognitive presence" within the Community of Inquiry theoretical framework. Cognitive presence is defined as “The extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication” (Garrison, Anderson & Archer, 2000, p 89). Cognitive presence has usually been determined in research by manual content analysis. In these indicators, we define this construct based on the type of activity offered to the student, and the extent to which the student demonstrates cognitive engagement in that activity. The level of depth ranges from 0 to 5, where 0 indicates that the learner has not even viewed the activity. The levels of potential cognitive depth are:

  1. The learner has viewed the activity details
  2. The learner has submitted content to the activity
  3. The learner has viewed feedback from an instructor or peer for the activity (including automated feedback)
  4. The learner has provided feedback to the instructor or a peer within the activity
  5. The learner has revised and/or resubmitted content to the activity

This model begins by assigning a maximum potential value of cognitive depth to each activity module. For example, the Assignment module allows up to cognitive depth of 5.

Once the potential levels are assigned, each student enrolled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 (viewing feedback) and the student has reached level 3, the student is participating at 100 percent of the possible level of cognitive depth and the value of the indicator is 1. If the student does not view the activity, they are at cognitive depth of 0, and the value of the indicator is -1. If the student has viewed the activity and submitted a response, but not viewed feedback, they are at cognitive depth level 2 (out of a potential depth in this activity of 3) so the indicator is calculated at .33.

Course level cognitive depth indicators

  • Course potential cognitive depth - calculated from all the activities in the course as a measure of potential engagement

Course enrolment level cognitive depth indicators

The following cognitive depth indicators have 1 level and report 0% (not viewed), or 100% (viewed).

  • Book cognitive
  • Folder cognitive
  • IMS pkg cognitive
  • Label cognitive (considered viewed if that portion of the course is viewed)
  • Page cognitive
  • File cognitive
  • URL cognitive

The following cognitive depth indicators have 2 levels and report 0% (not viewed), 50% (viewed) or 100% (interacted with at least one other participant).

  • Choice cognitive (if results are not viewable)
  • Database cognitive
  • Feedback cognitive
  • Glossary cognitive
  • Survey cognitive
  • Wiki cognitive

The following cognitive depth indicators have 3 levels and report 0% (not viewed), 33% (viewed), 67% (submitted) or 100% (submitted and viewed feedback).

  • Choice cognitive (if results are viewable)
  • LTI cognitive
  • SCORM cognitive

The following cognitive depth indicators have 4 levels and report 0% (not viewed), 25% (viewed), 50% (submitted), 75% (submitted and viewed feedback) or 100% (provided feedback to instructor or peer).

  • Chat cognitive
  • Forum cognitive

The following cognitive depth indicators have 4 levels and report 0% (not viewed), 20% (viewed), 40% (submitted), 60% (submitted and viewed feedback), 80% (provided feedback to instructor or peer), or 100% (revised and resubmitted).

  • Assignment cognitive
  • Lesson cognitive
  • Quiz cognitive
  • Workshop cognitive

Social breadth

Social breadth is a measure of the construct "social presence" within the Community of Inquiry theoretical framework. It is defined as “The ability of participants to identify with the group or course of study, communicate purposefully in a trusting environment, and develop personal and affective relationships progressively by way of projecting their individual personalities” (Garrison, 2009, p 352). In the past, social presence has usually been measured via post-course surveys and manual discourse analysis, though there have been increasing attempts to automate this process. This model implements social presence as "social breadth" by examining the breadth of opportunities the participant has to communicate with others. The level of breadth ranges from 0 to 5, where 0 indicates the learner has not interacted with anyone. The levels of potential social breadth are:

  1. The learner has not interacted with any other participant in this activity (e.g. they have read a page)
  2. The learner has interacted with at least one other participant (e.g. they have submitted an assignment or attempted a self-grading quiz providing feedback)
  3. The learner has interacted with multiple participants in this activity, e.g. posting to a discussion forum, wiki, database, etc.*
  4. The learner has interacted with participants in at least one "volley" of communications back and forth*
  5. The learner has interacted with people outside the class, e.g. in an authentic community of practice*

* Not yet implemented

This model begins by assigning a maximum potential value of social breadth to each activity module. For example, the Assignment module allows up to social breadth of 2. See below for more details of how these levels are assigned for core activity modules.

Once the potential levels are assigned, each student enrolled in a course is evaluated based on the proportion of the potential depth reached. For example, if an activity only supports up to level 3 and the student has reached level 3, the student is participating at 100 percent of the possible level of social breadth.

Note: at present, Moodle only provides indicators up to Social Breadth level 2. All "resources" have a maximum social breadth of 1. Activities vary in their supported social breadth. This feature is meant to be expanded in future versions of Moodle.

Course level social breadth indicators

  • Course potential social breadth - calculated from all the activities in the course

Enrolment level social breadth indicators

The following social breadth indicators have only one levels and report either 0% (not viewed) or 100% (viewed).

  • Book social
  • Database social
  • Folder social
  • Glossary social
  • IMS pkg social
  • Label social
  • Page social
  • File social
  • Survey social
  • URL social
  • Wiki social

The following social breadth indicators have two levels and report 0% (not viewed), 50% (viewed) or 100% (interacted with at least one other participant).

  • Assignment social
  • Chat social
  • Choice social
  • Feedback social
  • Forum social
  • Lesson social
  • LTI social
  • Quiz social
  • SCORM social
  • Workshop social
Retrieved from "https://docs.moodle.org/311/en/index.php?title=Learning_analytics_indicators&oldid=140577"
Powered by MediaWiki