Progress tracking
Moodle 2.0
Note: This page outlines ideas for the Progress tracking feature planned for Moodle 2.0. It's a specification under construction! If you have any comments or suggestions, please add them to the page comments.
Introduction
This feature will allow tracking of the progress of each student, displaying what has been achieved and what is still to be achieved.
There are two ways to do this: by course and by competency:
- Showing progress by course is pretty obvious and can work on any Moodle site: we just list the courses that have been completed and some courses that are still to be done. The completed courses can show the final grades and the date completed. Courses which are not yet completed can be shown with dates of when they start (if courses use a 'weekly' course format) or shown with the text "not yet completed". In Moodle 2.0, the course completion feature should also allow courses to be shown as either completed or not yet complete (see Course_completion).
- Showing progress using competencies requires the Outcomes features in Moodle 1.9 to be enabled and used across the whole site. When outcomes are enabled, we have a way of saying "these courses and these activities are intended to achieve these outcomes" and teachers can grade students against those outcomes. The progress report can then show a list of outcomes as past achievements and future goals for each user.
Definitions
Scale : A way to measure ability
eg : Terrible, Bad, Good, Excellent
Outcome : A statement about potential knowledge together combined with a scale
eg : “Can catch a ball” Scale: Terrible, Bad, Good, Excellent
Skill : An outcome and the actual grade a learner obtained
eg : Can catch a ball - “Bad”
Required grade : The minimum grade that is needed to deem a competency
eg : "Good"
Actual grade : The grade a learner obtained for an outcome
eg : "Bad"
Competency : A meaningful ability a learner acquires when achieving one or more skills.
Eg : An "admin" may define a competency as “Ability to play cricket” with the following skills and corresponding required grades. “Ability to catch a ball” – good “Ability to throw a ball” – good “Ability to swing a bat” – good. If a learner obtains good, bad, excellent respectively for the above three outcomes, then the learner fails to achieve an “Ability to play cricket” competency.
Organization role : A name for a role that has one or more required competencies.
Eg : Athlete - "Ability to play cricket", "Ability to swim", "Ability to ski"
Goals
- Allow "admins" to devise a learning plan for an individual user. This can be either in terms of "courses" or "competencies".
- Allow "admins" to develop a learning plan template. This can also be either in terms of "courses" or "competencies".
- Allow "admins" to assign a learning plan from a learning plan template to an individual a user or a group of users. This assignment can be made manually in one action (similar to role or group assignments), or automatically using methods such as external databases or parsing text files containing data.
- Allow "admins" to adjust an individual learning plan for a user, including a portion previously assigned from a learning plan template.
- Allow "admins" to assign a due date for completing each "course" or "competency".
- Allow "students" to see their own individual learning plan and their current progress along it (showing what is: done; in progress; or not done).
- Allow "admins" the ability to see different reports for this data and answer questions like "how many people fit this profile" or "what is everybody working on right now?" or "which people haven't met this competency"?
General architecture
Most of the scripts will be in a top-level subdirectory called "track". This directory will have a similar structure to the grade subdirectory, with plugin support for reports, import and export plugins.
Course-based progress tracking
- Each user can have courses linked to them (past, current and future).
- For each course, we look if the user has finished it or not.
- Finished courses are show as "Done", with final grade
- Unfinished but currently-enrolled courses are shown as "In progress"
- Other courses are shown as "Not done yet" and will be done in the future
Competency-based progress tracking
- Each user can have outcomes linked to them , with a required level of competency (grade) for that outcome
- For each outcome, we look at completed courses to see what grades they have
- These are aggregated in some admin-defined way (Mean? Maximum?)
- Outcomes with high grades are show as "Done", with final grade
- Other outcomes in currently-enrolled courses are shown as "In progress"
- Other outcomes are shown as "Not done yet" and will be done in the future. They can be used to suggest further courses by matching outcomes.
Learning plan
- Each user can have a set of either courses to be completed, outcomes to be completed, or a mixture of both. These comprise an individual user's learning plan.
- Due dates for course completions and outcome completions can be assigned in an individual user's learning plan.
- A comment can be added to a user's learning plan item when a course or outcome is manually added or removed from it.
Learning plan template
- Each template can contain a set of courses to be completed, outcomes to be completed, or a mixture of both.
- A course or outcome can be assigned to a learning plan template (using an interface similar to assigning a user to a role or group).
- A course or group of courses can be selected for a template from a list of all courses (by drilling-down category/sub-category) or from the results of a search (by course name or course description). Since outcomes are defined within a course, an outcome or a group of outcomes can be selected from a list of courses or search result.
Requirements
- We'll need to access and use the existing site-wide Outcomes Outcomes tables
- We'll need to use the tables that indicate completion status for courses (these are not yet in place) Course Completion
- We need to get aggregate information about the course outcomes and aggregate grades for each user from the gradebook API
Standards support
Moodle should support global standards in order to enable importing and exporting of competency information. IMS specifications will be used on this matter. IMS RDCEO (IMS Reusable Definition of Competency or Educational Objectives) will be used to capture the competencies/educational objectives and IMS LIP (IMS Learner Information Package) will be used to model the user profiles and their competencies.
IMS RDCEO
IMS Global Learning Consortium, Inc released IMS RDCEO for describing, referencing, and exchanging definitions of competencies, primarily in the context of online and distributed learning. Mostly the specification information are unstructured text and intended for human interpretation. This specification provides means to create common understandings of competencies that appear as a part of a learning plan. The IMS RDCEO also provides a way to store competency information in many languages at a time.
IMS RDCEO specification contains following elements:
- Identifier : Globally unique label that identifies the Competency or Learning objects.
- Title : A single short mandatory human readable text for competency.
- Description : Human readable unstructured description of competency and this is an optional element.
- Definition : An optional structured description that provides more meaning to the competency.
- Metadata : An optional meta-data used to denote authors information, creation date etc. It is recommended that this should conform to IEEE LOM standard.
Definition element contains following sub elements :
- Model source : An identifier of the model or structure upon which the definition is based. It is good to have a URI as model source.
- Statement : Description about single characteristic of a definition and a definition must contain one or more statements.
- Statement Id : An optional string that is local identifier for the statement within the model
- Statement Name : This is an optional sub elements used to label the statement.
- Statement text : An optional unstructured textual description of those aspects of the competency referred to by the statement Name.
- Statement token : An optional vocabulary token, along with an identifier of the source of the vocabulary.
IMS LIP
IMS LIP describes the characteristics of a learner, which are needed for recording and managing learning-related history, goals, and accomplishments, engaging a learner in a learning experience, and discovering learning opportunities for learners. Here, the learner information is a collection of information about a learner (individual or group learners) or a producer of learning content (creators, providers or vendors). In competency modeling, the IMS LIP can be used to merge the competencies in IMS RDCEO and the users.
IMS LIP Core datastructures:
- Identification: Biographic and demographic data relevant to learning;
- Goal: Learning, career and other objectives and aspirations;
- Qualifications, Certifications and Licenses (qcl): Qualifications, certifications and licenses granted by recognized authorities;
- Activity: Any learning-related activity in any state of completion. Could be self-reported. Includes formal and informal education, training, work experience, and military or civic service;
- Transcript: A record that is used to provide an institutionally-based summary of academic achievement. The structure of this record can take many forms;
- Interest: Information describing hobbies and recreational activities;
- Competency: Skills, knowledge, and abilities acquired in the cognitive, affective, and/or psychomotor domains;
- Affiliation: Membership of professional organizations, etc. Membership of groups is covered by the IMS Enterprise specification;
- Accessibility: General accessibility to the learner information as defined through language capabilities, disabilities, eligibilities and learning preferences including cognitive preferences (e.g. issues of learning style), physical preferences (e.g. a preference for large print), and technological preferences (e.g. a preference for a particular computer platform);
- Securitykey: The set of passwords and security keys assigned to the learner for transactions with learner information systems and services;
- Relationship: The set of relationships between the core components. The core structures do not have within them identifiers that link to the core structures. Instead all of these relationships are captured in a single core structure thereby making the links simpler to identify and manage.
None of the above data structures is mandatory and also the multiple occurrences of these data structures are allowed. Each of these data structures itself has various sub elements.
An example :
An outcome is defined as "ability to identify verbs in simple sentences" and the scale is Aways, Sometimes and Never. If it is also defined that, learner can gets the "ability to identify verbs in simple sentences" competency if obtain Sometimes grade. Assume that a Lerner called Jenney has deemed this competency. This information can be modeled as follows using IMS RDCEO and IMS LIP. Here the Pseudo-Structures of IMS RDCEO and IMS LIP are used. Note that the given structure is not complete.
IMS RDECEO Pseudo-Structure rdceo.identifier = rdceo434323 rdceo.title = Identify verbs in simple sentences rdceo.description = “Identify verbs in simple sentences” rdceo.definition.statement. statementid = 123 statementname = Always statementtext = Scale statementtoken = scl55435 rdceo.definition.statement. statementid = 124 statementname = Never statementtext = Scale statementtoken = scl55436 rdceo.definition.statement. statementid = 125 statementname = Sometimes statementtext = Scale statementtoken = scl55437
IMS LIP Pseudo-Structure lip.contenttype.referencial.sourceid.id = jenney_profile lip.identification.cotenttype.referential.indexid = id123 lip.identification.name.partname.typename=Jenny lip.competency cotenttype.referential.indexid = j123 exrefrecord = rdceo434323 description = Always
Tables
competency _required_grade
This table is used to map the outcome and required the grade to the corresponding competency. A competencyid can be mapped to more than one outcome and the required grade when competency comprises more than one outcome.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
outcomeid | int(10) | The id of the outcome being assigned to the required grade | |
competencyid | int(10) | The competency that the outcome belongs too | |
timecreated | int(10) | The time that the required grade was first assigned to outcome | |
timemodified | int(10) | The time that the required grade was last modified | |
usermodified | int(10) | The userid of the person who last modified the required grade |
competency_items
This table is used to describe the competencies in text form. As the example in the section above that defines Competency, this may have an entry like "Ability to play cricket". If so it also should have 3 corresponding entries “Ability to catch a ball” – good, “Ability to throw a ball” – good and “Ability to swing a bat” – good in competency _required_grade table.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
description | Text | Competency description | |
timecreated | int(10) | The time that a competency was first created | |
timemodified | int(10) | The time that a competency was last modified | |
usermodified | int(10) | The userid of the person who last modified the competency |
learning_plan_competency
This table is used to describe the competency based learning plan of a user. This table also states whether a user has achieved a competency or not.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
userid | int(10) | Userid of the user who is assigned to the competency based leaning plan | |
competencyid | int(10) | Id of the competency which belongs to the competency based learning plan | |
learningtemplateid | int(10) | Id of the template that the competency based plan belongs to | |
competencyachieved | Text | Describe whether a competency has been achieved by a user or not. | |
timeachieved | int(10) | The time when this has been achieved | |
timecreated | int(10) | The time that the competency based learning plan was first created | |
timemodified | int(10) | The time that the competency based learning plan was last modified | |
usermodified | int(10) | The userid of the person who last modified the competency based learning plan |
learning_plan_course
This table is used to describe the course based learning plan of a user. Initially a set of courses that the user has to sit for can be defined and whether it is completed or not is derived from course completion
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
userid | int(10) | Userid of the user who is assigned to the course based leaning plan | |
timecreated | int(10) | The time that the course based learning plan was first created | |
timemodified | int(10) | The time that the course based learning plan was last modified | |
usermodified | int(10) | Id of the course which belongs to the course based learning plan |
learning_plan_template
This table is used to describe a learning template, which describes the competencies that need to achieve a particular role, like Software Engineer. Therefore if this template is assigned to a user, then all the required competencies also will be mapped to the user. Admin does not have to assign the competencies one by one.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
title | Text | Title of the learning plan | |
Description | Text | Description of the Learning plan | |
timecreated | int(10) | The time when the learning template was first created | |
timemodified | int(10) | The time this learning template was last modified | |
usermodified | int(10) | The userid of the person who last modified the learning template |
Development plan
Progress tracking administration
Progress tracking can be administrated from course point of view and the competency point of view.
Course based tracking administration
There should be a link called Progress in the site admin area and it should contain the links to handle course-based progress tracking and the competency-based progress tracking. Currently, only the course based progress tracking is considered in this case. As given in the following diagram, Progress link should contain 4 sub menus to handle the course-based progress tracking.
- Create learning template should direct us to a page where we can create / edit templates, assign courses to the template and sequence the courses.
- Upload learning template should help us to create a template and assign courses using an external file.
- Assign users should help to assign users to a learning template
- Progress monitor should allow us to do learning based monitoring and the template based monitoring
Add Learning template
This screen should look like this :
Here the admin should be able to add a new template, edit an existing template and delete a template. This should be a ‘paged’ page. If there are 100s of templates that should be displayed as part by part in separate pages they should be listed as 1,2,3 as shown in the bottom of the above figure. When the admin adds a template, then the admin should be directed to a new page where the courses can be added and sequenced. Also if the admin clicks on the Edit link, the admin should get a page (similar to the sample given below) where the admin can sequence or add new courses to the existing learning template.
A page, which is similar to the following one, should be developed to facilitate the admin to add and sequence the courses.
In the available course area, only the short name of the courses that are about to start or that will start in the future should be listed. The admin should be able to filter the courses by clicking on the corresponding starting letter on the left. Also the admin should be able to search for a course by typing a keyword (can be the course full name, course short name...etc). Also there should be a facility provided in the available course pane so that when the admin double clicks on a course, a pop-up should be displayed with the important course information. If a course is not available in the list, then the admin should be able to add a new course by giving minimal information using the [New Course] link. The admin ought to shift the required courses from the left pane to the right pane using the Add selection button. Also the other two remove buttons should give the intended functionality. Among the selected courses which are on the right pane, the admin is able to sequence the courses. There are two ways this can be developed.
- By dragging the courses up and down using mouse pointer.
- As shown in the picture below, the courses can be moved up and down using arrow links.
Admins also should be able to create a template by importing the details from an external file. To achieve this as proposed before there should be a link added to the Site administration called upload learning template. That should give an interface like below, where user also should be able to select the CSV and the encoding before importing the file. The importing file should consist of following format: "Template_name, course1, course2, course3, etc." where course1, course2.. should be the short name of the courses and it should be properly ordered.
Assign a template to users
Once the learning templates are created then they should be assignable to users. An interface as given below should be provided to admin to do this assignment. According to the filter information, the users' names should be listed in the left hand side pane. Those users' names should be able to further filtered using the filter links provided in the left side of the Users pane. When the admin clicks on a Letter the names starting with that letter should be listed. Moreover the admin should be able to search for a user by giving a keyword (Here keyword can be First name, Last name, Username etc.) Also when the admin clicks on a User’s name a short description about that user should pop-up. After selecting the required users, the admin should be able to move those users to the selected user pane using Add to Selection button. The Remove from selection and Remove all button also should do the intended functions. Then there should be a drop down menu, in which all the available templates should be listed.
As given below, there should be an interface to filter out the students' names. This part of the interface should go in the NEW FILTER area of the above interface.
The filter should have following functionality and it should be a course-based filter. The filtration should be made based on the courses, whether a learner did / doing /didn’t do a course. There should be a drop down, where all “did”, “didn’t do”, “doing”, “incomplete” etc should be listed. And there should also be a textarea where the admin should be allowed to input courses. Here the courses’ short names should be listed and separated by commas. Once the admin adds a filter, it should go to the active filter block. Likewise the admin can have many filers. According to the selection that the admin does on this active filer block the user names should be displayed in the user pane.
Progress monitoring
Progress monitoring based on courses can be done in two ways :
- Learner based progress monitoring
- Learning template based progress monitoring
In the Learner profile page another tab called - Course based Progress - should be added. And it should show the learner's progress based on courses as shown in the given picture.
There is another interface like below that should be provided to monitor the learners' performance from courses point of view. According to this, when the admin selects the name of a learning template, all the courses corresponding to that template should be listed as follows with the status of completion. For each course there should be a [view] link by means of which the admin should be able to view the learners' information corresponds to that state.
Learner windows
This section describes how the learn side screen should be.
When a learner logins to the system, leaner will see the Progress Monitor block in the SITE.
Learner will get a similar window like below when she clicks on Find more link in the above picture.
By clicking on the Course based or Competency based links the learner can see their detail progress. These will be similar to the Grade reports. Also there will be a graphical representation of the progress will be visible in the bottom right hand bottom corner. In addition to these, an option can be added to suggest the learner that this course has this competency, you can follow this course, etc.
Note : In the above interface, the graph and the table are not real. The table may contain some more information and there may be some more options too.
Directory Structure
There will be two directories with the name Progress. One will be there in Moodle root and the other will be inside the Blocks folder. The Progress which is in the Blocks will contain the scripts that are needed to display the block to the learners. The progress directory, which is in Moodle root directory, will contain following scripts :
- Library files
- Scripts for admin feature development
- Scripts for the report generation for the learner.
Issues
- Does completing a course mean all the outcomes in a course have been met? Or should we treat them independently of the course grade? Perhaps the admin needs to decide this as a setting.
- How do we deal with the same outcome being used by more than one course? Should the admin specify an aggregation method for this?
- Rather than assigning these duties to the admin, they should be capabilities that could be assigned to a role.
- Could there be more ways of assigning students to a template? Perhaps from the same database that controls course enrollments, or a CSV file, or a field in their profile?