ELIS Course results engine

What is the ELIS Adaptive Learning Results Engine?

The Results Engine brings advanced adaptive learning functionality to Moodle via ELIS. Moodle 2 enables adaptivity within a course, for example activities can be shown based on the results of a previous activity. The ELIS Results Engine enables the results of an activity or set of activities in a course to enroll a learner in a new class, a set of classes, and/or update their profile information. For example, the Results Engine could be used to act on the poor score a learner earns in a course, and based on that score enroll them in a remedial program. Conversely, it can be used to enroll a learner in advanced courses if they do well in a course. When used for competency management, http://en.wikipedia.org/wiki/Competency_management_system, the Results Engine can update user profiles with new skills and/or competencies based on assessment and/or courses taken. Progress against an organization's competency architecture, a href="http://en.wikipedia.org/wiki/Competency_architecture, can then be tracked via the User Class Completion Report.

Example of the Results Engine in Use:

There are several ways to use the Results Engine, below is one example from an Adaptive Learning program. In this example, learners take an initial assessment, then the results engine assigns them instructional material designed to fill gaps revealed by the initial assessment. This process is repeated as the learners move through the program.

Results Engine Profile Field Update:

The Results Engine can also update a learner's profile field, based on their score on an assessment, a set of assessments, or a their score in a course. For example, a learner could be given an assessment that measures their success at achieving state standards for their grade level. Based on the results of this assessment they could be enrolled in classes with content designed for their current achievement level. This enables credit to be granted based on competency. The results engine can also be used to help boost graduation rates by assigning learners alternative course material when they do poorly on an assignment, set of assignments, or a course.

In a corporate setting, the Results Engine can be used to perform gap analysis on a skill or set of skills, and employees automatically assigned coursework designed to cover their gaps. A single Moodle course could have multiple assessments that then acted on by the results engine. So this single course could have a set of different assessments to evaluate their competency in different important skills, and automatically assign them course work designed to address gaps. This leads to more efficient and effective training, as learners are not spending time on content they have already mastered, and can focus the time they can spend on training on content that they have not mastered.

For example, an institution offers internal Project Management training, and institutes a staff profile field that records four project management levels: None, Basic, Advanced, Master. Coursework and and assessments enable staff to have their Project Management expertise assessed. The Results Engine is used to update learners profile based on the results of their coursework and/or assessments. When a major project is proposed, the User Class Completion Report can then be used by a manager to search all staff who have a Master level of Project Management expertise to assess internal resource competency for managing the project.

A manager/administrator can setup the UCC Report to search for and display users with selected custom Profile field settings:

The report then displays all users with those values in their Profile. The report can be exported as CSV to facilitate contacting the users if required. In the example report below, the report is configured to show Project Management level, along with the city and email address of the staff with the Master level. Of course the report could also show other custom fields, for example Department, Manager, Manager contact information, etc.

As both the Results Engine and the UCC Report utilize ELIS Custom Fields that can be created and modified in the Custom Field screens, the system is highly customizable to match different competency/skill/learning level requirements.

Using the Adaptive Results Engine:

The ELIS Results Engine operates at the Program-Course Description level. To activate the results engine for a course description go to the Program Management block > Admin > Course Descriptions.

1. Select the Course Description link in the Program Management block.
2. If there are many courses on your site, you may want to use the Search function above the course list. In the example I have selected the letter P from the list of letters, which searched for and displayed course descriptions starting with the letter P.
   
3. Select the Results Engine icon for a course description to link to the Results Engine tab of the course.

1. The tab in a course description's page for accessing/editing the Results Engine.
2. Check this to activate the Results Engine for this ELIS CD.
3. This determines when the Results Engine will update a learner's record.

Setting up a Results Engine instance consists of setting up three main actions:

1. When the Results Engine is run is set by the Event trigger
2. What criteria the Results Engine acts on is set by the Criterion setting
3. What happens - currently there are three kinds of actions the Results Engine can take:

• The learner may be assigned to an ELIS track of classes - this is often used to move the learner up a level in their coursework
  
• The learner may be assigned to a single class - this is often used to assign remedial course work
• Update a learner's Profile Field - this is often used in Competency/Skills training - the learner's profile can be updated to show their competency level in a skill based on the results of the course, assessment, etc.
  

Currently, each Course Description in a given ELIS Program can have one results engine assigned to it.

Setting up the the Event Trigger

The event trigger determines when the results engine is run.

1. If this is is then the learner grade reaches a certain level. This can be the learner's course grade, or the grade on an ELIS Learning Objectives. ELIS Learning Objectives can be individual Moodle activities, or collections of activities in a Moodle gradebook category. For example, the results engine could be triggered simply by a learner's grade on a particular assessment, such as a competency exam or pre-test. Or the trigger could be the learner's grade on an assessment, a presentation, and attendance, if all of those grades are collected into a Moodle gradebook category. Or the trigger could be the learner's overall course grade.
2. This will run the Results Engine on a set number of days after the ELIS Class Instance start date. For example if the class is set to start on May 5, then the Results engine can be set to run X days after the start date.
3. If this is set, then the Results Engine will only run after an administrator activates it manually.

Setting up the Action:

This is where you determine what action happens when the Results Engine runs. There are three possible Actions:

1. Assign the learner to a track
2. Assign the leaner to a class
3. Update a profile field in the learner's profile.
   

Assigning a learner to a track as the result of a Class Instance/Learning Objective Score:

1. The Results Engine can be used to automatically assign a learner to a track of classes. For example, if a learner gets below 50% grade on a course or assessment, they can be assigned to a track of remedial classes.
2. You can edit the score ranges, and each score range can have a different result.
3. Click the Select track link to open up the track selection window.
4. You can search for a track and the click it to assign to a score range.

More blank score ranges can be be added by clicking the Add new score range button. If you just want to assign a track based on one or two score ranges, then remove the unused score ranges.

Assigning a learner to a new Class Instance as the result of a Class Instance/Learning Objective Score:

1. The Results Engine can be used to automatically assign a learner to a class instance. For example, if a learner gets below 50% grade on a course or assessment, they can be assigned to a remedial class.
2. You can edit the score ranges, and each score range can have a different result.
3. Click the Select class instance link to open up the track selection window.
4. You can search for a track and the click it to assign to a score range.

User Profile Field update.

This function will update the ELIS User Profile field with the settings specified the results. For example, a user who passes a Project Management course might have their User Profile updated with information indicating they have passed the course. Similarly, if a user who has passed coursework signifying that they are ready for promotion in a company, that information can be entered into their profile. In a K-20 setting, this function can be used to indicate learners who are at risk or who have earned certain skills.

The User Class Completion Report can be used to generate reports on these custom profile fields, for example to show all users who have achieved a certain skill level (Project Manager Level 1, for example), all users who are ready for Management, all users who are at risk, etc.

These fields can also be used for automatic Userset assignment if the ELIS fields are set to synchronize values back to Moodle.

Example of using the Results Engine to automatically update a learner's profile field:
For example, if a learner gets below 50% grade on a course or assessment - in this example an assessment of their Project Management skill level, a custom profile field for Project Management Level is updated to PM0, in this case a value denoting a low level of Project Management competency.

1. Set the assessment score range for the profile field update action
   
2. Choose from any custom ELIS profile field to update. In this example we are updating the values of the Project Management Level profile field with all score ranges, however you can have different score ranges update different profile fields, if that fit your requirements.
   
3. Choose from the available values. If the Profile field is of the Menu of Choices type, with a set range of values, you will see those values in the selection drop down.

Profile field type Menu of Choices:

When the type of Profile field being updated is a "Menu of Choices" type, a drop down menu showing the available choices is displayed.

Setting the value of an Open Text Profile field:

When the type of Profile field being updated is a text field, then you can enter any value you like into the field. In this example, the Ready for Promotion custom field is a text field.