Competency Based Education

Revision as of 02:22, 7 August 2015 by Damyon Wiese (talk | contribs) (Competencies)

Jump to: navigation, search
Framework for competency-based education (CBE) in Moodle
Project state In Progress
Tracker issue MDL-49458
Discussion https://moodle.org/mod/forum/discuss.php?d=316151
Assignee User:Damyon Wiese


Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.


Framework for competency-based education (CBE) in Moodle

Summary

This specification describes our plan to improve support for outcomes-based education in Moodle core. There have been many other attempts to add such systems to Moodle but they all use different data structures and processes and are incompatible. Our aim here is to build basic central structures and APIs to store and manipulate this data, allowing the Moodle community to build innovative custom interfaces on top of it.

This design is based on input from many existing implementations of outcomes/competencies and learning plans including:

  • “Outcomes 2” in Joule
  • Totara
  • Referentiel Activity Module (Skills Repository)
  • ELIS

The goal is to implement a system that supports competency based education in a flexible way so that it can be used effectively by the many different target audiences.

The two main concepts are:

Competencies - a unit of learning, such as "Can make exceptional beer". Competencies can be assigned as goals of courses “This course will teach you to make exceptional beer”, and also to users: “Billy can make exceptional beer”. Competencies can also be hierarchical, so that the competency above might be marked proficient when some combination of sub-competencies has been marked proficient. For example : Can make exceptional beer ( Can choose good malt / Can choose good yeasts / Can brew beer in the good temperature / etc..)

Learning plans - a set of competencies that can be assigned to a student (or cohorts of students). Initially, they will all be marked "incomplete". As students achieve each competency it is marked complete, thereby showing their progress towards finishing all of them. Students achieve competencies either automatically (by successfully completing a course with that competency) or manually (by proving to someone that they are competent via portfolio items etc).

Competencies

A competency is a unit of learning. All competencies belong to a competency framework, and it is likely that a competency framework will be defined by an external body such as an academic standards board.

To create a competency, you first define the framework and then add the competency to it.

Competency frameworks are defined by “Framework Administrators” at the site level. To create and edit competency frameworks, go to “Site administration” -> “Competency Frameworks”.

Manage competency frameworks
Add new competency framework

Choose “Add new competency framework” and create a new competency framework.

The scale defines how a competency is marked as complete. Each competency framework can define which of the standard Moodle scales should be used when marking a competency. In addition, the “default” value from the scale (the one automatically set when a course is completed) can be configured, and the items from the scale that are deemed to be “proficient” can be defined.

A competency framework may be available to the entire site - or limited to a course category.

The taxonomy refers to the naming for each level in the framework. Up to 4 levels are supported, and the name for each level can be set from the following list: Domain, Competency, Behaviour, Indicator, Outcome, Level, Concept, Value, Practice, Skill, Proficiency

Configure scale

Now we need to add competencies to the framework. We can import these from a file, or create them from scratch, or import them from Moodle outcomes. When importing competencies, any existing competencies with a matching idnumber will be updated.

Import competencies

Competencies can be imported from the following formats:

 (TBD - start with import from CSV).

To see the competencies for a framework, click on the competency framework name.

Manage competencies

This is where we can add and edit competencies in a framework. To edit an existing competency, select it in the competencies tree browser and choose “Edit” from the action menu in the selected competency panel.

To add a new competency, first select the competency that should be the “parent” of this competency from the tree of competencies, and then click “Add Indicator” (the name comes from the taxonomy) from the selected competency panel. Tags can be used to categorise competencies for easy searching, e.g. year level, subject. Tags used for competencies are kept in a separate “tagcloud” (see https://tracker.moodle.org/browse/MDL-50851).

A competency framework can be duplicated, and there is an admin tool for migrating courses from one framework to another (as long as the idnumbers between the competencies in the frameworks can be matched).

If a competency has sub-competencies, you can configure the completion rule for the competency. The default completion rule is “Completed when all sub-competencies are complete”. For more complex rules - you can use a points based system to say when to complete or recommend a competency.

Add a competency

Competencies can be “related” to other competencies. The only effect of this is that the related competencies are listed in the summary of each competency, whenever it is displayed in Moodle. To add a “related” competency to an existing competency, select it from the competency framework and choose “Add related competency” from the action menu.

Relating competencies to each other

Learning plan templates

It would be cumbersome to individually add competencies to a large group of students, and it would be awkward if you made a mistake and had to change the competencies for all the students at a later date. To make this manageable, you can create a “Learning plan template” and assign it to many students. An example of a learning plan template could be: “Computer science, Undergraduate” which lists all the required competencies for a first year computer science student, or it could be more fine grained.

Templates are created by a ”Learning Plan Administrator”. Templates can be imported and exported via CSV. Learning plan templates can be created at a course category, or a site level.

Manage Learning Plan Templates

Once you have created a template, you can add competencies to it.

List competencies in a template
Add competency to a template

Adding competencies to a template will update all the learning plans based on that template.

User learning plans

A user learning plan is a collection of all the competencies they should achieve, optionally before a certain date.

Any user can create their own learning plan, if they have the capability on their own context.

A “Learning Plan Administrator” can create learning plans for other users. They can create the plans individually though another users profile page, or they can create learning plans in bulk from a learning plan template.

Users learning plans can be in one of 3 states, Draft, Active or Completed. Only someone with the capability tool/lp:planmanage (or tool/lp:planmanageown if it’s their own plan) capability can approve learning plans.

A plan can be manually set to completed, by someone with the capabilities listed above, or it will be set to completed when the due date for the plan expires.

Single users learning plans page
Bulk create user learning plans

Note that sites using enrolment plugins can create a cohort based on the enrolments of a set of courses using the admin tool “Course cohorts”. More detail required here.

Competencies can be added to a users learning plan, or to the template the plan was created from and they will automatically appear in the users learning plan. Competencies that exist in the template cannot be removed from the users learning plan. If a single student requires e.g. exemptions from certain competencies - their learning plan can be "unlinked" from the template. This plan will no longer show up in reports generated from the learning plan template - but can now be freely edited.

User learning plan competencies

Note, there can be multiple types of evidence for the same competency. E.g. 2 courses may provide evidence for the same competency and both courses were completed. In this case - both would show as evidence. There can also be multiple examples of "Evidence of prior learning" attached to each competency.

Each competency in the table is colour coded to indicate one of the following:

  • brown - competency not met, and no evidence or enrolled courses cover this competency
  • orange - the competency is not met, but there is evidence, or “in-progress” courses that cover this competency
  • green - the competency has been met

If you have the capability tool/lp:planaddevidence you can upload files and urls against a competency in your plan as evidence of prior learning. You can also request that a someone with the capability tool/lp:planmanage reviews the evidence and either rejects or accepts the evidence and changes the status for the competency in the plan.

Provide evidence
Link evidence

To view all the current requests for review, a new “Learning plans” block can be added to an “approvers” dashboard.

Dashboard Block

A User with the capability “tool/lp:planmanage” (at the context of another users plan) can review and approve competency approval requests. A competency approval request is a request for someone to review the evidence that a user has provided against a specific competency and make a decision whether the user has either met or not met the competency. The valid statuses for a review are:

  • Waiting for review
  • In review
  • Accepted
  • Rejected
Flow chart of review process

There can only be one “review” per competency for each user (if a review is rejected, and then a new request is made, the previous review will be changed from “Rejected” to “Waiting for review”).

Following the link in the Learning Plans dashboard block, the approver goes to a page that lists all the requests waiting to be approved, and from here they can approve or reject each request. They can also leave comments that will be visible to the student, or to any other reviewers.

To allow multiple authorisers to work together, when looking at a review an authoriser can click on a “Start review” button which will change the status of the request to “In review” and assign them as the reviewer for the request.

Whenever a comment is made against a review, or a review is accepted or rejected, the student will receive a message about the change.

When a completion rule triggers a “recommendation” - the evidence will be added to the competency, and the system will request a “review” of this competency.

TODO Define a new page for assigning roles relative to a user. In totara, there is a “positions” page which does this. This will also be useful for “Mentees”.

Review competency requests

If users have the capability to create their own draft learning plans, but not approve them themselves, someone will need to review all the draft learning plan requests and decide whether or not to make them active. This user can be anyone with “tool/lp:planmanage” capability (at the context of each user with a plan to approve) and they can view all the “Learning plans to approve” link in the Learning Plans dashboard block.

A learning plan approval request made by a user with a draft learning plan from their “Manage learning plans page”. Once a request has been made, the valid statuses for a learning plan approval request are:

  • Waiting for review
  • In review
  • Accepted
  • Rejected


Flow chart of review process

To allow multiple authorisers to work together, when looking at a learning plans approval request an authoriser can click on a “Start review” button which will change the status of the request to “In review” and assign them as the reviewer for the request.

Whenever a comment is made against a learning plan approval request, or a learning plan approval request is accepted or rejected, the student will receive a message about the change.

Review Learning Plan Approval Requests

Course Competencies

Competencies can also be linked to a course. This means that when a student completes a course (see course completion) - all of the linked competencies will be marked as “complete” or “recommended” in any of the students active learning plans and the course will be added as evidence against that competency. If the competency is already “met”, the course will still be added as evidence for that competency. If the competency is only set to “recommend” the course will be added as evidence and a review request will be created.

The competency will be given the default value from the competency scale. You can select multiple competencies and add them all at once.

Course Competencies

A teacher in a course, can manually mark the students against the course competencies. For each competency, the teacher will see all of the evidence linked to that competency for the student. They then can assign a value from the competency scale for the student, and optionally make a comment. A record of the teacher marking this competency will be added to the evidence of completion for this competency and the status of the competency will be updated in all active learning plans for the student.

TODO - see how the information from the coverage report can be shown when linking course competencies.

Grading competencies within a course

When manually marking a competency, it is recorded as evidence ie: "Teacher X manually specified this value in this course" - so if the same competency is linked to multiple course it can be marked in each, and this will show up as 2 pieces of evidence.

Reports

Students

For an active Learning Plan, students can see a report of the competencies that are covered/not covered by their current list of enrolled courses. For each competency, they should also be able to see which courses they could enrol in to meet that competency.

Student Learning Plan Competencies page shows progress, and course discovery

Students can see a list of their active learning plans on their dashboard, with a visual indication of progress (a progress bar). Each learning plan is a link to the learning plan report shown above.

Teachers

For teachers and course designers building a set of courses against a competency framework, they should be able to identify competencies that are over/underrepresented by a set of courses. To do this they access a learning plan template, and view the coverage report for that template. This competency coverage report initially shows the coverage from all courses that link to one of the competencies in the template, but the teacher can remove some courses from the selection and regenerate the report.

Coverage report

Teachers and course designers can also see some statistics about a learning plan template. From the learning plan template page, they choose “Statistics” from the action menu to see the number of active/completed learning plans using this template, and the average completion % for the learning plans.

Statistics for Learning Plan Template

Teachers can also see a report in a course that shows for each course competency, which users have completed or not completed it.

Breakdown report

User roles and capabilities

In this system, the actions a user can perform are governed by various capabilities, and are checked at one of several different context levels.

As a reference, the relevant context levels in Moodle are organised in a tree like structure and an example is shown below:

  • Site
    • User A
    • User B
    • Category A
      • Course A
      • Course B
    • Category B
      • Category C
        • Course C
        • Course D

Framework Administrators

Competency framework administrators are the central people responsible for creating and maintaining the system wide competency frameworks.

The actions a competency framework administrator can take are:

Create/Edit/Delete a competency framework

Requires “tool/lp:competencymanage” capability defined at the “Site” or "Course Category" context level.

Create/Edit/Delete a competency within a competency framework

Requires “tool/lp:competencymanage” capability defined at the “Site” or "Course Category" context level.

Mark a competency as related to another competency

Requires “tool/lp:competencymanage” capability defined at the “Site” or "Course Category" context level.

Learning Plan Administrators

Learning plan administrators are responsible for creating and maintaining learning plan templates relating to e.g. a specific faculty, set of courses, degree or certification.

The actions a learning plan administrator can take are:

Create/Edit/Delete a learning plan template

Requires “tool/lp:templatemanage” capability defined at the “Site” or "Course Category" context level.

Add and remove competencies from a learning plan template

Requires “tool/lp:templatemanage” capability defined at the “Site” or "Course Category" context level.

Assign a user a learning plan from a learning plan template

Requires “tool/lp:planmanage“ capability defined on the “User” context level.

View coverage report for a learning plan template

Requires “report/lpcoverage:view“ capability defined on the “Site” or "Course Category" context level.

View statistics report for a learning plan template

Requires “report/lpstatistics:view“ capability defined on the “Site” or "Course Category" context level.

View breakdown report for a learning plan template

Requires “report/lpbreakdown:view“ capability defined on the “Course” context level.

Approve draft learning plans

Requires “tool/lp:planapprove“ capability defined on the “User” context level.

Approve competency review requests

Requires “tool/lp:competencygrade“ capability defined on the “User” context level.

Teachers

Teachers create and teach courses, that provide evidence of competencies.

The actions a teacher can take are:

Assign competencies to a course

Requires “tool/lp:coursecompetencymanage” capability defined at the “Course” context level.

Manually grade competencies linked to a course

Requires “tool/lp:coursecompetencygrade” capability defined at the “Course” context level.

Students

Students are users that can have learning plans.

The actions a student can take are:

View their own learning plans

Requires “tool/lp:planviewown” capability defined at the “User” context level.

Create their own draft learning plan

Requires “tool/lp:plancreatedraft” capability defined at the “User” context level.

Request approval for their draft learning plan

Requires “tool/lp:plancreatedraft” capability defined at the “User” context level.

Provide evidence of prior learning for a competency

Requires “tool/lp:planaddevidence” capability defined at the “User” context level.

Request for someone to grade them for a specific competency

Requires “tool/lp:plancompetencyrequestreview” capability defined at the “User” context level.

Further work (out of scope for first release)

Add a rule to award badges when certain competencies are met

This is a simple addition to the existing rules that can be configured to award a badge.

Custom badge condition

Linking activities as evidence of competencies

Linking an activity to some course competencies

Teachers can also quickly find resources and activities that have not been mapped to any competencies. This is done on the competency mapping report which shows all resources and activities for the course, as well as the linked competencies for each. This report will also list any course competencies that are not linked to any activities or resources.

Alternative ways to “automatically complete” competencies

  • Trigger based on activity completion. Configured when linking competencies to a course.

Additional reports

E.g. https://moodle.org/mod/forum/discuss.php?d=316151#p1266980

Interaction with the existing Moodle “Outcomes”

Description of existing “Outcomes”

Outcomes define a set of system wide scales that can be used to grade activities. The value chosen from the scale is pushed to the gradebook for each activity and a custom report will show the average of outcomes across a course, or the entire site. Not all activities and resources support outcomes, only scorm, quiz, lti, lesson, glossary, data, chat, assign and forum.

Any activity that has an outcome attached receives a new column in the gradebook where that outcome is assessed for that activity.

Outcomes has some features that would need to be “removed” in order to make sense with competency frameworks (mainly pushing of grades to the gradebook) - this means we cannot build on outcomes, and maintain backwards compatibility. It is therefore recommended to provide:

  • A site wide setting to allow competencies - allow either/or but the default will be competencies with no outcomes.
  • A simple way to create a competency framework from an existing set of outcomes.
  • (optional - requires activity level competencies) A way to import existing outcomes grades as evidence for competencies.

Open questions for further feedback

  • Would you expect a teacher to be able to create their own competencies (limited to their own course)? Would competencies that are visible only to a single course (and potentially conflicting with competencies from another course) be really useful?
  • Do we need to allow competencies to be linked to individual questions?
  • Do we need to allow competencies to be linked to individual items on a rubric/marking guide?

Outstanding work for this spec

  • The examples should use consistent names for user types, competencies and courses. It would be easier to use a real example of competencies from a real framework
  • The terminology for child/parent relationships should consistently be:
    • Parent
    • Sub-competency
  • Technical spec (this doc is the requirements).