Competencies: Difference between revisions
(moved "example frameworks" toward the end of the spec) |
(editing goals) |
||
Line 12: | Line 12: | ||
==Goals== | ==Goals== | ||
# Measured amount of abstraction | # Measured amount of abstraction | ||
#* Allow "admins" to | #* Allow "admins" to organise competencies in a hierarchy, folksonomy (tags), or hybrid | ||
#* Allow "admins" to specify parent-child | #* Allow "admins" to specify parent-child relations between competencies and related competencies | ||
#* Allow "admins" to allow unlimited hierarchical levels | #* Allow "admins" to allow unlimited hierarchical levels | ||
#* Allow "admins" to describe | #* Allow "admins" to describe competencies on each depth level consistently with custom meta data fields and categories | ||
# Easy navigation | # Easy navigation | ||
#* Allow "admins" to view the competency structure (expanding/collapsing competencies, sorting/searching/filtering by competency meta data) | #* Allow "admins" to view the competency structure (expanding/collapsing competencies, sorting/searching/filtering by competency meta data) | ||
#* Allow "admins" to easily add new competencies and meta-data | #* Allow "admins" to easily add new competencies and meta-data | ||
# | # Connecting competencies and evidence items | ||
#* Allow "admins" to attach | #* Allow "admins" to attach allowed evidence (course, activity, etc) to a competency (from a competency view): | ||
#** One or more course grades | |||
#** One or more course | |||
#** One or more course completions (e.g., when completing course A, course B, and course C) | #** One or more course completions (e.g., when completing course A, course B, and course C) | ||
#** One or more activity outcomes, from one or more courses | |||
#** One or more activity completions (e.g., when completing activity x in course A and activity y in course B, and activity z in course C) | #** One or more activity completions (e.g., when completing activity x in course A and activity y in course B, and activity z in course C) | ||
#** | #* Allow "course creators" to attach competencies to evidence in a course they are assigned to: | ||
#* | #** Course grade | ||
#* | #** Completion the course | ||
# Specifying | #** Activity grade | ||
#* Allow "admins" to assign to a competency: | #** Completion of an activity in the course | ||
# Portability | |||
#* Allow "admins" to import competencies (using import plugins) into a Moodle install | |||
#* Allow "admins" to export competencies (using export plugins) from a Moodle install | |||
# Specifying expected proficiency levels | |||
#* Allow "admins" to assign to a competency evidence item: | |||
#** An expected proficiency level (i.e., competency scale value) | #** An expected proficiency level (i.e., competency scale value) | ||
#** Course completion criteria for a course | #** Course completion criteria for a course | ||
#** Activity completion criteria for an activity | #** Activity completion criteria for an activity | ||
# Aggregating measured proficiency level for each user | # Aggregating measured proficiency level for each user | ||
#* Allow "admins" to specify aggregation methods for each competency: | #* Allow "admins" to specify aggregation methods for each competency: |
Revision as of 02:02, 17 July 2009
Moodle 2.0
Note: This page outlines ideas for the Competencies feature planned for Moodle 2.0. It's a specification under construction! If you have any comments or suggestions, please add them to the page comments.
- PROJECT STATE: Proposal
- MAIN TRACKER ISSUE:
- DISCUSSION: Not yet announced on Moodle.org
- COMMENTS: Development_talk:Competencies
- AUTHOR: Jonathan Newman with help from Matt Clarkson and others
Overview
To support the range of different types of competency frameworks used by organisations, competencies in Moodle 2.0 needs to expand on the 1.9 outcomes architecture.
Goals
- Measured amount of abstraction
- Allow "admins" to organise competencies in a hierarchy, folksonomy (tags), or hybrid
- Allow "admins" to specify parent-child relations between competencies and related competencies
- Allow "admins" to allow unlimited hierarchical levels
- Allow "admins" to describe competencies on each depth level consistently with custom meta data fields and categories
- Easy navigation
- Allow "admins" to view the competency structure (expanding/collapsing competencies, sorting/searching/filtering by competency meta data)
- Allow "admins" to easily add new competencies and meta-data
- Connecting competencies and evidence items
- Allow "admins" to attach allowed evidence (course, activity, etc) to a competency (from a competency view):
- One or more course grades
- One or more course completions (e.g., when completing course A, course B, and course C)
- One or more activity outcomes, from one or more courses
- One or more activity completions (e.g., when completing activity x in course A and activity y in course B, and activity z in course C)
- Allow "course creators" to attach competencies to evidence in a course they are assigned to:
- Course grade
- Completion the course
- Activity grade
- Completion of an activity in the course
- Allow "admins" to attach allowed evidence (course, activity, etc) to a competency (from a competency view):
- Portability
- Allow "admins" to import competencies (using import plugins) into a Moodle install
- Allow "admins" to export competencies (using export plugins) from a Moodle install
- Specifying expected proficiency levels
- Allow "admins" to assign to a competency evidence item:
- An expected proficiency level (i.e., competency scale value)
- Course completion criteria for a course
- Activity completion criteria for an activity
- Allow "admins" to assign to a competency evidence item:
- Aggregating measured proficiency level for each user
- Allow "admins" to specify aggregation methods for each competency:
- All sub-competencies or evidence
- Any sub-competencies or evidence
- A specific number of sub-competencies or evidence
- A percentage of sub-competences or evidence
- Sum of weighted sub-competencies or evidence
- Average of weighted sub-competencies or evidence
- Reporting
- Allow "admins" to view competency status for all students across all courses.
- Allow "teachers" to view student competency status within his/her courses.
- Allow "students" to view their own competency status (within a course, or via their profile or My Moodle).
- Allow "admins" to specify aggregation methods for each competency:
- Specifying weights
- Allow "admins" to assign weight to each competency and competency evidence
- Scalability
- Store and retrieve data efficiently to allow the scalability to very large numbers of courses, users, competencies, evidence, etc
- Extensibility
- Use a plugin architecture for evidence types
General requirements
- We'll need to access and use the existing site-wide Outcomes Outcomes tables
- We'll need to use the tables that indicate completion status for courses (these are not yet in place) Course Completion
- We need to get aggregate information about the course outcomes and aggregate grades for each user from the gradebook API
Definitions
Scale: A way to measure a competency. Examples:
"Below average, Average, Above average" "Not competent, Competent"
Competency: A measurable knowledge, skill, ability, attitude or other learning-related characteristic which a student may have. Competencies can be organised into a hierarchy (i.e., category-based), folksonomy (tag-based), or a hybrid of both. An example organisation of competencies:
top-level competency: "Literacy" -> child competency: "Reading Comprehension" -> child competency: "Responding to Text" -> child competency: "Initial Understanding of Literary Text" -> child competency: "Demonstrate initial understanding of elements of literary texts (including text read aloud, reading independently" -> evidence: activity outcome - "Identifying characters in a story" -> evidence: activity outcome - "Retelling the beginning, middle, and end of a story" -> evidence: activity outcome - "Responding to simple questions about a book's content"
Competency evidence type: A form of evidence that can be accepted as evidence that a competency has been achieved. Examples:
Observation Activity grade Activity completion Activity outcome Course completion Course grade Course outcome Question results File (diploma)
Competency evidence: A competency can be measured by one or more competency evidence types. Evidence can be stated to include the evidence type, evidence date, and measured value. Examples:
"Observation - Teacher - Above average - 4:00pm May 28, 2009" "Observation - Spanish Language Proficiency Interview - 4:00pm May 28, 2009 - Rating 4 out of 5" "Quiz - 4:00pm May 28, 2009 - Grade 90% out of 100%" "Certificate - Public Speaking Institute - 4:00pm May 28, 2009 - Passed. "File - College Diploma - 4:00pm May 28, 2009 - Bachelor of Science Computer Science"
Expected proficiency level: The required rating (based on a scale) or grade for a competency. Examples:
"Average" "5" "50%" "Passed"
Measured proficiency level: A user's observed, scored, reported proficiency level for a competency.
"Above average" "8" "80%" "Passed"
Competency weighting: Child competencies can be assigned weights which can be aggregated. Examples:
"Communication" -> child "Written Communication - weight: 60/100" -> child "Oral Communication - weight: 40/100"
User interface
Individual learning plan
- Each user can have a set of either courses to be completed, competencies to be achieved, or a mixture of both. These comprise a user's individual learning plan (IDP).
- Each user's IDP will show:
- Completed courses
- Progress toward courses which are not yet complete. This would be similar to view of the course progress completion report for the user across the incomplete courses.
- Competencies achieved. This will include the competencies and meta data made visible to the user, as well as the evidence info (type, date, ratings and grades).
- Competencies not yet achieved. This will also include the competencies and meta data made available to the user, as well as the expected proficiency levels for each competency.
- Each user can see a list of all his/her courses and competencies as a "transcript" or "record of learning."
Data structures
competency
A competency can be part of a hierarchy and can have evidence attached to it.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
shortname | varchar(100) | The shortname for the competency | |
fullname | varchar(255) | The full name for the competency | |
description | text | The description of the competency | |
idnumber | int(10) | The id, code number or local identifier for the competency used for reporting | |
parentid | int(10) | The id of the parent competency | |
sortorder | int(10) | The order number of this competency (useful for ordering competencies for viewing/navigation/reporting) | |
depth | int(10) | The depth of the competency (beginning at 1) | |
path | varchar(255) | The path to the competency | |
aggregation | int(10) | A constant representation one of the competency aggregation methods (any, all, unit, fraction) | |
scaleid | int(10) | The scale used to measure the competency | |
proficiencyexpected | int(10) | The expected proficiency for the competency | |
timecreated | int(10) | The time when the competency was created | |
timemodified | int(10) | The time when the competency was modified | |
usermodified | int(10) | The userid of the person who last modified the competency |
competency_relations
For tracking relations between competencies.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
competencyid1 | int(10) | The id of a comptency | |
competencyid2 | int(10) | The id of a related competency |
competency_depth_info_field
Data for a competency depth level field.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
shortname | varchar(255) | Shortname for the field | |
fullname | text | Full name for the field | |
datatype | varchar(255) | The type of field data | |
description | text | Description of the field | |
categoryid | int(10) | The id for the field category | |
sortorder | int(10) | The sort order for the field within its category | |
hidden | int(1) | The field should be hidden | |
required | int(1) | The field required | |
defaultdata | text | Default text for the selected parameter | |
param1 | text | Parameter text | |
param2 | text | Parameter text | |
param3 | text | Parameter text | |
param4 | text | Parameter text | |
param5 | text | Parameter text |
competency_depth_info_category
A competency's depth level can have categories for its fields.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
name | varchar(255) | Shortname for the depth level category | |
sortorder | int(10) | The order number for this category |
competency_depth_info_data
Data for a depth level field.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
fieldid | int(10) | The id for the field | |
competencyid | int(10) | The id for the competency | |
data | text | The actual data for the competency field |
competency_evidence_items
A competency can be achieved by one or more forms of evidence.
Field | Type | Default | Info |
id | int(10) | Auto incrementing | |
competencyid | int(10) | The id for the competency that this evidence describes | |
itemtype | varchar(30) | The type of evidence 'observation', 'course/outcome', 'mod/outcome', 'course/completion', 'mod/completion', 'course/grade', 'mod/grade', 'question/category', 'file' | |
itemmodule | varchar(30) | The type of evidence 'observation', 'quiz', 'scorm', 'question' | |
iteminstance | int(10) | The id for the evidence instance (ie, grade_outcome id, course id, course_modules id, grade_item id, quiz_question_instance id, files id) | |
required | int(1) | The evidence is required | |
certification | int(1) | The evidence is for certification | |
expirationdays | int(10) | The evidence expires after this number of days | |
timecreated | int(10) | The time when this competency evidence was allowed | |
timemodified | int(10) | The time when this competency evidence was last changed | |
usermodified | int(10) | The user who last modified this evidence |
competency_evidence
Track historical and current competency information for each user.
Field | Type | Default | Info |
id | int(10) | auto-incrementing | |
userid | int(10) | The id of the user | |
competencyid | int(10) | The id of the competency | |
proficiencyexpected | int(10) | The measured/evaluated proficiency level for the competency | |
proficiencymeasured | int(10) | The measured/evaluated proficiency level for the user | |
timecreated | int(10) | The time when this competency evidence was started | |
timemodified | int(10) | The time when this competency evidence was last changed |
Plugin architecture
Competency evidence type plugins
Each type of evidence will need its own methods for interacting with the competency architecture and will need to interact via a base competency evidence API. Below are the item types.
- Observation
- Course completion
- Course grade
- Course outcome
- Activity completion
- Activity grade
- Activity outcome
- File
Report plugins
Similar to the gradebook, we need to allow development of report plugins for competencies.
Import/Export plugins
We need to be able to import and export data related to competencies.
Competency framework import/export A competency framework should be able to be exported using an appropriate format in either xml and csv file format. There are some standards on exchanging competencies including:
- IEEE P1484.20.1 Standard on Reusable Competency Definitions and the IEEE Draft Standard on Simple Reusable Competency Map
- ISO/IEC PDTR 24763 Conceptual Reference Model for Competencies and Related Objects
Competency evidence import/export
There are a variety of standards for exchanging competencies evidence including:
- IMS Reusable Definition of Competency or Educational Objectives Spec (RDCEO)
- IMS Learner Information Package Spec (LIP)
- HR-XML Competencies (Measurable Characteristics) Recommendation, 2007 April 15
IMS RDCEO and LIP recommend formats as shown in the examples below (more IMS RDCEO here and IMS LIP home). Special thanks to Sarveswaran Kengatharaiyer (GSOC 2008 student for the Moodle Progress Tracker project) who developed these examples.
rdceo.identifier = rdceo434323 rdceo.title = Identify verbs in simple sentences rdceo.description = “Identify verbs in simple sentences” rdceo.definition.statement. statementid = 123 statementname = Always statementtext = Scale statementtoken = scl55435 rdceo.definition.statement. statementid = 124 statementname = Never statementtext = Scale statementtoken = scl55436 rdceo.definition.statement. statementid = 125 statementname = Sometimes statementtext = Scale statementtoken = scl55437 lip.contenttype.referencial.sourceid.id = jenney_profile lip.identification.cotenttype.referential.indexid = id123 lip.identification.name.partname.typename=Jenny lip.competency cotenttype.referential.indexid = j123 exrefrecord = rdceo434323 description = Always
HR-XML recommends a format like shown below in an example which uses recursive competencies (more here).
<Competency name="Communication Skills"> <Competency name="Written Communication Skills"> <CompetencyEvidence name="WRITTENTEST1-A" dateOfIncident="1995-01-01" lastUsed="2000-01-01"> <NumericValue minValue="3" maxValue="5" description="SEP-equivalent Skill-Level Range">5</NumericValue> </CompetencyEvidence> <CompetencyWeight> <NumericValue minValue="0" maxValue="100">35</NumericValue> </CompetencyWeight> </Competency> <Competency name="Oral Communication Skills"> <CompetencyEvidence name="ManagerObservation" dateOfIncident="1996-01-01" lastUsed="2000-01-01"> <NumericValue minValue="1" maxValue="5" description="Company XYZ Skill Range">5</NumericValue> </CompetencyEvidence> <CompetencyWeight> <NumericValue minValue="0" maxValue="100">65</NumericValue> </CompetencyWeight> </Competency> </Competency>
Upgrade from 1.9
Below is a proposed mapping from the 1.9 outcomes architecture to the proposed competencies architecture.
grade_outcomes - site-wide outcomes
Site wide outcomes will need to be imported into the competency table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_outcomes | id | null | competency | id | null |
grade_outcomes | courseid | null | competency | courseid | null |
grade_outcomes | shortname | not null | competency | shortname | not null |
grade_outcomes | fullname | not null | competency | fullname | not null |
grade_outcomes | scaleid | not null | competency | scaleid | not null |
grade_outcomes | description | null or not null | competency | description | null or not null |
grade_outcomes | timecreated | not null | competency | timecreated | not null |
grade_outcomes | timemodified | not null | competency | timemodified | not null |
grade_outcomes | usermodified | not null | competency | usermodified | not null |
grade_outcomes - custom course outcomes
Custom course outcomes will also need to be imported into the competency table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_outcomes | id | null | competency | id | null |
grade_outcomes | courseid | not null (e.g., 2, 3) | competency | courseid | null |
grade_outcomes | shortname | not null | competency | shortname | not null |
grade_outcomes | fullname | not null | competency | fullname | not null |
grade_outcomes | scaleid | not null | competency | scaleid | not null |
grade_outcomes | description | null or not null | competency | scaleid | null or not null |
grade_outcomes | timecreated | not null | competency | timecreated | not null |
grade_outcomes | timemodified | not null | competency | timemodified | not null |
grade_outcomes | usermodified | not null | competency | usermodified | not null |
grade_outcomes_courses
Grade outcomes in a course become an evidence item (itemtype='course') for a competency and get moved to the competency_evidence_items table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_outcomes_courses | id | not null | competency_evidence_items | id? not crucial | not null |
grade_outcomes_courses | courseid | not null | competency_evidence_items | instanceid | not null |
grade_outcomes_courses | outcomeid | not null | competency_evidence_items | competencyid | not null |
grade_items - course outcomes
Course outcomes need to be entered as a competency evidence items (type="course") for a competency and get moved to the competency_evidence_items table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_items | id | not null | competency_evidence_items | id? not crucial | not null |
grade_items | itemname | not null | competency_evidence_items | name | not null |
grade_items | itemtype | course | competency_evidence_items | itemtype | course |
grade_items | itemmodule | null | competency_evidence_items | itemmodule | null |
grade_items | iteminstance | not null (1,2) | competency_evidence_items | iteminstance | not null (1,2) |
grade_items - activity outcomes
Grade outcomes in a course become a course 'type' allowed evidence for a competency and get moved to the competency_evidence_allowed table. We also need to use the value "course" for the evidence 'type' field.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_items | id | not null | competency_evidence | id? not crucial | not null |
grade_grades
In 1.9, course outcomes appear in the grade_items when they are linked to an activity. Only activity outcomes appear as graded in grade_grades. The activity outcomes need to appear as an entries in competency_evidence_items table and the grade in the competency_evidence_items table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_grades | id | not null | competency_evidence_items | id | not null |
grade_grades | itemid | not null | competency_evidence_items | itemid | not null |
grade_grades | userid | not null | competency_evidence_items | userid | not null |
grade_outcomes | scaleid | not null | competency_evidence_items | scaleid | not null |
grade_grades | finalgrade | not null | competency_evidence_items | scalevalueid | not null |
We also need to add an entry for the user in the competency_evidence table.
From Table | From Field | From Value | To Table | To Field | To Value |
---|---|---|---|---|---|
grade_grades | id | not null | competency_evidence | id | not null |
grade_grades | itemid | not null | competency_evidence | itemid | not null |
grade_grades | userid | not null | competency_evidence | userid | not null |
grade_outcomes | scaleid | not null | competency_evidence | scaleid | not null |
grade_grades | finalgrade | not null | competency_evidence | scalevalueid | not null |
Logging
All competency related changes should be logged in history tables.
Role capabilities and permissions
Description | Capability |
View competency competencies | moodle/competency:viewcompetencies |
View hidden competencies | moodle/competency:viewhiddencompetencies |
Manage competencies | moodle/competency:managecompetencies |
Example frameworks
There are an enormous number of example competency frameworks published online. I've randomly selected a comprehensive example Department of Education for the State of Vermont's Grade 1 Expectations for Literacy for Reading, and detailed here how it would fit into the proposed schema.
Ideas for the future
- Due dates for course completions and outcome completions can be assigned in an individual user's learning plan.
- A comment can be added to a user's learning plan item when a course or outcome is manually added or removed from it.
- Allowing competencies to be attached to an individual user (either by user profile field or by drag and drop).