Advanced grading methods

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Advanced grading methods
Project state	Planning
Tracker issue	TODO
Discussion	TODO
Assignee	David Mudrak

Moodle 2.2

Project goals

• Long-term goal: To be able to use advanced grading methods (like rubrics and other multi-criteria assessment forms) to assess all reasonable places (like assignment submissions, forum posts, wiki pages, database records etc.), not just the submissions in the workshop module.
• Short-term: To allow using rubric as a grading tool for the assignment module and for the manual grade item in the gradebook.

Project scope

This project does NOT change the gradebook internals or its API for obtaining and storing grades. It has also nothing to do with other gradebook issues like aggregating hidden grades and friends.

This project aims to provide a clear API to

• embed a selected grading form (rubric, scale, single grade with decimal points) into the caller's page in both editable and read-only modes
• process the submitted data
• obtain the calculated grade if it is available
• obtain the raw filled data for post-processing (eg to calculate grade for peer-assessment in the workshop) in a unified format

Moodle core (internally implemented as a new part of the gradebook subsystem) will provide UI to edit assessment forms at standalone pages. The caller (client) like the assignment module or the grader report should be able to easily embed the selected form in its own page. For example:

• the quick grading mode in the assignment module should be able to embed multiple instances of grading forms in the table with all submissions and process them in once
• the forum module should be able to load (preferably via AJAX) the grading form for any post on demand
• the workshop module should be able to display the grading form under the student's submission for all peer-reviewers and then be able to compare them to calculate the grade for assessment

Things that must be decided and designed carefully:

• forms sharing - the situation is similar to what question bank tries to deal with. But the concept of storing forms in various contexts seems to be suboptimal to me
• export/import - how to easily publish forms definitions (XML + embedded files) and re-use them (maybe this is a solution for the sharing issue?)
• backup/restore - grading forms must be part of the gradebook structures, like grade categories and grade items are (XML structures similar to export/import)

Relevant tracker issues

• MDL-795 - 'workshop-like' teacher assessment methods (rubrics, accumulative, etc) for assignments
• MDL-2255 - Rubrics for all Activities
• CONTRIB-2058 - Ability to edit/delete rubrics (Assignment Rubric activity module)

Implementation plan

• New core plugin type called "Grading method" to be added into get_plugin_types() as 'grademethod' with the location in /grade/method/*
• All grading methods to be implemented as plugins - 'Rubric', 'Multi-scale' (aka Accumulative grading strategy in the Workshop) etc. Current grading tools could be re-implemented as 'Single scale' and 'Decimal points' plugins
• The mechanism of embedding the forms into pages in Moodle to follow the patterns already used in Files API, Comments API or Ratings API (using component, context, gradearea and itemid).
• Ability to either use the default renderer of the form of even define own renderer for it. This would allow the caller (client) to have complete control of how the grading form is embedded into the page.

Database schema

ER diagram of new tables

Basic data flow

1. Teacher goes to the course gradebook and clicks "Define new grading form". As the grading method she selects "Rubric". She fills the name and the description of the grading form. New record is inserted into {grade_forms} and the rubric editor opens.
2. For each new row of the rubric grid, a new record is inserted into {gradeform_rubric_criteria}.
3. For each new column of the rubric grid column in the given row, a new record is inserted into {gradeform_rubric_levels}.
4. Teacher goes to the Assignment module and modifies its setting to use the newly created rubric as the default grading method. TODO: where is this information going to be stored?
5. Teacher goes thru the list of submissions. After clicking "Grade" for a particular submission, new record is created (or existing one updated) in {grade_forms_instances} (component = "mod_assignment", gradearea = "submission", itemid = submission id, userid = $USER->id, timecreated = now).
6. The information about how the teacher actually fill the rubric is saved into {gradeform_rubric_fulfilments} - that is what level was chosen for each criterion. Optional comments can be stored, too.
7. When the teacher hits "Finish", the rubric subplugin calculates the final grade and provides it to the Assignment module.
8. The assignment module pushes the calculated grade to the gradebook.

Note that the rubric does not actually store the calculated grade. It just behaves as a tool that is able to calculate the value from how the form is fulfilled. Once the value is calculated, the Assignment module works with it as if it was selected via current <select> field.

UI mock-ups

Forum thread with embedded rubric

See also

• The description of rubric at Wikipedia
• Kathy Schrock's Guide for Educators - Examples of Assessment Rubrics
• LMS Instructure and especially their way of defining rubrics
• Rubrics Manual for JOULE by Moodle Rooms
• Gradebook_2.x_architecture to come in 2.3