Difference between revisions of "Competencies"

Jump to: navigation, search
(Goals: edited linking competencies and evidence items)
(Goals: edited various other sections)
Line 28: Line 28:
 
#* Allow "admins" to export competencies (using export plugins) from a Moodle install
 
#* Allow "admins" to export competencies (using export plugins) from a Moodle install
 
# Specifying expected proficiency levels
 
# Specifying expected proficiency levels
#* Allow "admins" to assign an expected proficiency level (i.e., competency scale value) to a competency evidence item
+
#* Allow "admins" to assign an expected proficiency level (i.e., competency scale value) to a competency and a competency evidence item
# Aggregating measured proficiency level for each user
+
# Aggregating measured proficiency
#* Allow "admins" to specify aggregation methods for each competency:
+
#* Allow "admins" to specify aggregation methods for each competency
#** All child competencies or evidence
 
#** Any child competencies or evidence
 
#** A specific number of child competencies or evidence
 
#** A percentage of child competences or evidence
 
#** Sum of weighted child competencies or evidence
 
#** Average of weighted child competencies or evidence
 
 
#* Reporting
 
#* Reporting
#** Allow "admins" to view competency results for all students across all courses.
+
#** Allow "admins" to view competency results for all students
#** Allow "teachers" to view student competency results within his/her courses.
+
#** Allow "teachers" to view student measured proficiencies related to evidence in his/her courses.
#** Allow "students" to view their own competency results (within a course, or via their profile or My Moodle).
+
#** Allow "students" to view their own competency results (via either their personal profile or My Moodle).
 
# Specifying weights
 
# Specifying weights
 
#* Allow "admins" to assign weight to a competency or a competency evidence item
 
#* Allow "admins" to assign weight to a competency or a competency evidence item

Revision as of 02:54, 18 August 2009

Moodle 2.0

Note: This page outlines ideas for the Competencies feature planned for Moodle 2.0. It's a specification under construction! If you have any comments or suggestions, please add them to the page comments.

Overview

To support the range of different types of competency frameworks used by organisations, competencies in Moodle 2.0 needs to expand on the 1.9 outcomes architecture.

Goals

  1. Measured amount of abstraction
    • Allow "admins" to organise competencies in a hierarchy
    • Allow "admins" to create unlimited hierarchical levels
    • Allow "admins" to map related competencies on a given depth level
    • Allow "admins" to define custom info fields on a given depth level
  2. Easy navigation
    • Allow "admins" to view the competency structure
      • Expanding/collapsing competencies
      • Sorting/searching/filtering by competency meta data
    • Allow "admins" to easily add new competencies and meta-data
  3. Linking competencies and evidence items
    • Allow "admins" to easily attach allowed evidence (course, activity, etc) to a competency (from a site-level competency view):
    • Allow "course creators" to attach competencies to evidence within a course:
  4. Portability
    • Allow "admins" to import competencies (using import plugins) into a Moodle install
    • Allow "admins" to export competencies (using export plugins) from a Moodle install
  5. Specifying expected proficiency levels
    • Allow "admins" to assign an expected proficiency level (i.e., competency scale value) to a competency and a competency evidence item
  6. Aggregating measured proficiency
    • Allow "admins" to specify aggregation methods for each competency
    • Reporting
      • Allow "admins" to view competency results for all students
      • Allow "teachers" to view student measured proficiencies related to evidence in his/her courses.
      • Allow "students" to view their own competency results (via either their personal profile or My Moodle).
  7. Specifying weights
    • Allow "admins" to assign weight to a competency or a competency evidence item
  8. Scalability
    • Store and retrieve data efficiently to allow the scalability to very large numbers of courses, users, competencies, evidence, etc
  9. Extensibility
    • Use a plugin architecture for evidence types

General requirements

  1. We'll need to access and use the existing site-wide Outcomes Outcomes tables
  2. We'll need to use the tables that indicate completion status for courses (these are not yet in place) Course Completion
  3. We need to get aggregate information about the course outcomes and aggregate grades for each user from the gradebook API

Definitions

Scale: A way to measure a competency using descriptors and/or numerical scale values.

Example scales using descriptors values:

"Below average, Average, Above average"
"Not competent, Competent"

Example scales using numerical values:

"1, 2, 3, 4, 5"
"0, 100"
"0.0, 0.5, 1.0"
"0, 500, 1000"

Example scales using descriptors and numerical values:

"Poor [1], Below average [2], Average [3], Above average [4], Excellent [5]"
"Not yet competent [0], Competent [100]"
"Not yet competent [0.0], Not yet performing [0.5], Competent [1.0]"

Competency: A measurable knowledge, skill, ability, attitude or other learning-related characteristic which a student may have. Competencies can be organised into a hierarchy (i.e., category-based), folksonomy (tag-based), or a hybrid of both.

Example hierarchy of competencies:

top-level competency: "Literacy"
    -> child competency: "Reading Comprehension"
        -> child competency: "Responding to Text"
            -> child competency: "Initial Understanding of Literary Text"
                -> child competency: "Demonstrate initial understanding of elements of literary texts (including text read aloud, reading independently"

Competency evidence type: A form of evidence that can be accepted as evidence that a competency has been achieved. Examples:

Activity grade
Activity completion
Activity outcome
Course completion
Course grade
Course outcome
File (diploma)

Competency evidence: A competency can be measured by one or more competency evidence types. Evidence can be stated to include the evidence type, evidence date, and measured value. Examples:

"Quiz - 4:00pm May 28, 2009 - Grade 90% out of 100%"
"Certificate - Public Speaking Institute - 4:00pm May 28, 2009 - Passed."
"File - College Diploma - 4:00pm May 28, 2009 - Bachelor of Science Computer Science"

Expected proficiency level: The expected (or required) rating (based on a scale descriptor or numerical value) for a competency. Examples:

"Average"
"5"
"50%"
"Passed"

Measured proficiency level: A user's observed, scored, reported proficiency level for a competency.

"Above average"
"8"
"80%"
"Passed"

Competency weighting: Child competencies can be assigned weights which can be used for aggregating either sums or averages of child competencies. Examples:

Example with percentage weights

-> top-level competency: "Communication"
    -> child competency: "Written Communication" : weight: 60%
    -> child competency: "Oral Communication" : weight: 40%

Example frameworks

Example 1: Competency hierarchy with activity outcomes as evidence

The example data below is based roughly on the Department of Education for the State of Vermont's Grade 1 Expectations for Literacy for Reading.

Depth level Depth name Competency

full name

Competency

evidence

Competency

aggregation method

Competency

weight

Competency

proficiency expected

0 "Curriculum" "Literacy" none all 20% 80%
1 "Type" "Reading" none all 20% 100%
2 "Standard" "Standard 1.1 Reading Strategies" none all 20% 80%
3 "Strategy" "Early Reading Skills and Strategies: Phonological Awareness" none all 100% 80%
4 "Code" "RK: 1 - Applies phonological knowledge and skills by..." activity outcome "Recognizing pairs of rhyming words and producing rhymes"

activity outcome "Blending and segmenting syllables and onset-rimes"

activity outcome "Isolating phonemes in single-syllable words (e.g., "tell me the first sound in 'mop' "; "tell me the last sound in 'mop,'" "tell me the middle sound in 'mop'")"

all 50% 80%
3 "Strategy" "Early Reading Skills and Strategies - Concepts of Print" none all 50% 80%
4 "Code" "RK: 2 - Demonstrates understanding of concepts of print during shared or individual reading by..." activity outcome: "Distinguishing between printed letters and words"

activity outcome: "Following text with finger-pointing (e.g., charts, simple books), demonstrating left-to-right and top-to-bottom directionality"

activity outcome: "Identifying the first and last parts of a word (beginning/end of the word)"

activity outcome: "Identifying key parts of a book: front and back, print, illustrations"

all 50% 80%

See here for how the data above fits into the proposed schema.

Settings

Site settings

Activity Completion provides an "Enable outcomes" setting (found in the Site Administration block's Advanced features / optional subsystems settings page). We need to keep outcomes, however we will need to add competencies.

Proposed changes:

  • Create a new setting named "Enable competencies" (the setting will allow competencies to be defined)

Data structures

Competencies data model

scale

Scales (introduced in Moodle 1.9) give a scale value a single attribute: text (i.e., the name of the scale value like "below standard," "meets standard," "above standard"). Scale values can be particularly useful if given attributes including: numerical equivalent, idnumber (code id), and a description. The new attributes will require a new table.

Proposed changes: Create a new table 'scale_values' that includes a foreign key to the 'scale' table, and migrate the 'scale' field values from the 'scale' table (as serialised text) to the 'scale_values' table (as separate records). A sort order field will be used to track ranking (starting with the lowest with a sortorder of 0).

Below is the proposed 'scale' table, and further below is the proposed 'scale_value' table.

Field Type Default Info
id int(10) Auto incrementing
courseid int(10) Id of the course, which the scale belongs to
userid int(10) Id of the user who last modified the scale
name varchar(255) Name of the scale value
description text Description of the scale value
timemodified int(10) Timestamp of when the scale was modified

scale_values

Field Type Default Info
id int(10) Auto incrementing
scaleid int(10) Id of the scale that the scale value describes
name varchar(255) Name of the scale value
idnumber varchar(100) Optional id, code number or local identifier for the scale value (used for reporting).
description text Description of the scale value
numeric number(8) Numerical value for the scale value
sortorder int(10) The order for the scale value
timemodified int(10) Timestamp of when the scale value was modified

competency

A competency can be part of a hierarchy and can have evidence attached to it.

Field Type Default Info
id int(10) Auto incrementing
shortname varchar(100) The shortname for the competency
fullname varchar(255) The full name for the competency
description text The description of the competency
idnumber int(10) The id, code number or local identifier for the competency used for reporting
parentid int(10) The id of the parent competency
sortorder int(10) The order number of this competency (useful for ordering competencies for viewing/navigation/reporting)
depthid int(10) The id of the depth level the competency belongs to
path varchar(255) The path to the competency
aggregationmethod int(10) A constant representation one of the competency aggregation methods (any, all, unit, fraction, sum of weighted, average of weighted)
scaleid int(10) The scale used to measure the competency
proficiencyexpected int(10) The expected proficiency for the competency, foreign key to the scale_value table's id
timecreated int(10) The time when the competency was created
timemodified int(10) The time when the competency was modified
usermodified int(10) The userid of the person who last modified the competency

competency_relations

For tracking relations between competencies.

Field Type Default Info
id int(10) Auto incrementing
description text Info about how competencies are related
competencyid1 int(10) The id of a competency
competencyid2 int(10) The id of a related competency

competency_depth

Data for a competency depth level.

Field Type Default Info
id int(10) Auto incrementing
shortname varchar(255) Shortname for the depth level
fullname varchar(255) Full name for the depth level
description text Description for the depth level
level int(2) The actual depth level, starting at 0

competency_depth_info_field

Data for a competency depth level field.

Field Type Default Info
id int(10) Auto incrementing
shortname varchar(255) Shortname for the field
fullname text Full name for the field
depthid int(10) The id of the competency depth for the field. Foreign key to the competency_depth table's id
datatype varchar(255) The type of field data
description text Description of the field
categoryid int(10) The id for the field category. Foreign key to the competency_depth_info_category table's id field
sortorder int(10) The sort order for the field within its category
hidden int(1) The field should be hidden
required int(1) The field required
defaultdata text Default text for the selected parameter
param1 text Parameter text
param2 text Parameter text
param3 text Parameter text
param4 text Parameter text
param5 text Parameter text

competency_depth_info_category

A competency's depth level can have categories for its fields.

Field Type Default Info
id int(10) Auto incrementing
name varchar(255) Shortname for the depth level category
sortorder int(10) The order number for this category

competency_depth_info_data

Data for a depth level field.

Field Type Default Info
id int(10) Auto incrementing
fieldid int(10) The id for the field
competencyid int(10) The id for the competency
data text The actual data for the competency field

competency_evidence_items

A competency can be achieved by one or more forms of evidence.

Field Type Default Info
id int(10) Auto incrementing
competencyid int(10) The id for the competency that this evidence describes
itemtype varchar(30) The type of evidence 'course/outcome', 'mod/outcome', 'course/completion', 'mod/completion', 'course/grade', 'mod/grade', 'file'
itemmodule varchar(30) The type of evidence 'quiz', 'scorm', etc
iteminstance int(10) The id for the evidence instance (ie, grade_outcome id, course id, course_modules id, grade_item id, files id)
weight num(10) The weight applied to this evidence item (for aggregation)
timecreated int(10) The time when this competency evidence was allowed
timemodified int(10) The time when this competency evidence was last changed
usermodified int(10) The user who last modified this evidence

competency_evidence

Track historical and current competency information for each user.

Field Type Default Info
id int(10) auto-incrementing
userid int(10) The id of the user
competencyid int(10) The id of the competency
proficiencymeasured int(10) The measured/evaluated proficiency level for the user
timecreated int(10) The time when this competency evidence was started
timemodified int(10) The time when this competency evidence was last changed

Plugin architecture

Competency evidence type plugins

Each type of evidence will need its own methods for interacting with the competency architecture and will need to interact via a base competency evidence API. Below are the item types.

  • Course completion
  • Course grade
  • Course outcome
  • Activity completion
  • Activity grade
  • Activity outcome
  • File

Report plugins

Similar to the gradebook, we need to allow development of report plugins for competencies.

Import/Export plugins

We need to be able to import and export data related to competencies.

Competency framework import/export A competency framework should be able to be exported using an appropriate format in either xml and csv file format. There are some standards on exchanging competencies including:

  • IEEE P1484.20.1 Standard on Reusable Competency Definitions and the IEEE Draft Standard on Simple Reusable Competency Map
  • ISO/IEC PDTR 24763 Conceptual Reference Model for Competencies and Related Objects

Competency evidence import/export

There are a variety of standards for exchanging competencies evidence including:

  • IMS Reusable Definition of Competency or Educational Objectives Spec (RDCEO)
  • IMS Learner Information Package Spec (LIP)
  • HR-XML Competencies (Measurable Characteristics) Recommendation, 2007 April 15

IMS RDCEO and LIP recommend formats as shown in the examples below (more IMS RDCEO here and IMS LIP home). Special thanks to Sarveswaran Kengatharaiyer (GSOC 2008 student for the Moodle Progress Tracker project) who developed these examples.

rdceo.identifier = rdceo434323
rdceo.title = Identify verbs in simple sentences
rdceo.description = “Identify verbs in simple sentences”
rdceo.definition.statement.
   statementid = 123
   statementname = Always
   statementtext = Scale
   statementtoken = scl55435
rdceo.definition.statement.
   statementid = 124
   statementname = Never
   statementtext = Scale
   statementtoken = scl55436
rdceo.definition.statement.
   statementid = 125
   statementname = Sometimes
   statementtext = Scale
   statementtoken = scl55437

lip.contenttype.referencial.sourceid.id = jenney_profile
lip.identification.cotenttype.referential.indexid = id123
lip.identification.name.partname.typename=Jenny 
lip.competency
   cotenttype.referential.indexid = j123
   exrefrecord = rdceo434323
   description = Always

HR-XML recommends a format like shown below in an example which uses recursive competencies (more here).

<Competency name="Communication Skills">
      <Competency name="Written Communication Skills">
            <CompetencyEvidence name="WRITTENTEST1-A" dateOfIncident="1995-01-01" lastUsed="2000-01-01">
                   <NumericValue minValue="3" maxValue="5" description="SEP-equivalent Skill-Level Range">5</NumericValue>
            </CompetencyEvidence>
            <CompetencyWeight>
                   <NumericValue minValue="0" maxValue="100">35</NumericValue>
            </CompetencyWeight>
      </Competency>
      <Competency name="Oral Communication Skills">
            <CompetencyEvidence name="ManagerObservation" dateOfIncident="1996-01-01" lastUsed="2000-01-01">
                   <NumericValue minValue="1" maxValue="5" description="Company XYZ Skill Range">5</NumericValue>
            </CompetencyEvidence>
            <CompetencyWeight>
                   <NumericValue minValue="0" maxValue="100">65</NumericValue>
            </CompetencyWeight>
      </Competency>
</Competency>


Logging

All competency related changes should be logged in history tables.

Role capabilities and permissions

Description Capability
View competency competencies moodle/competency:viewcompetencies
View hidden competencies moodle/competency:viewhiddencompetencies
Manage competencies moodle/competency:managecompetencies

Ideas for the future

  1. Due dates for course completions and outcome completions can be assigned in an individual user's learning plan.
  2. A comment can be added to a user's learning plan item when a course or outcome is manually added or removed from it.
  3. Allowing competencies to be attached to an individual user (either by user profile field or by drag and drop).
  4. Allow competencies to be tagged (in a folksonomy) using Moodle's tag/lib.