Advanced grading methods

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Advanced grading methods
Project state	Planning
Tracker issue	TODO
Discussion	TODO
Assignee	David Mudrak

Moodle 2.2

Project goals

• Long-term goal: To be able to use advanced grading methods (like rubrics and other multi-criteria assessment forms) to assess all reasonable places (like assignment submissions, forum posts, wiki pages, database records etc.), not just the submissions in the workshop module.
• Short-term goal: To allow using rubric as a grading tool for the assignment module.

Project scope

This project does NOT change the gradebook internals or its API for obtaining and storing grades. It has also nothing to do with other gradebook issues like aggregating hidden grades and friends.

This project aims to:

• provide a way to define grading forms
• allow activity modules to use defined forms to calculate grades

Things that must be decided and designed carefully:

• export/import - how to easily publish forms definitions (XML + embedded files) and re-use them
• backup/restore - grading forms must be part of the gradebook structures, like grade categories and grade items are (XML structures similar to export/import)

Relevant tracker issues

• MDL-795 - 'workshop-like' teacher assessment methods (rubrics, accumulative, etc) for assignments
• MDL-2255 - Rubrics for all Activities
• CONTRIB-2058 - Ability to edit/delete rubrics (Assignment Rubric activity module)

Use cases

Using rubric in the assignment

• When editing the Assignment settings, there is a new field in the "Grade" section (see callout 1 in the UI mockup below). The teacher can define whether the grade (value or scale) will be put directly by the marker (as it is now) or some advanced grading form will be used. She sets the grade for this Assignment to a value 20 and decides to use advanced grading strategy to calculate that grade. She picks "Rubric" from the list of available form types.

UI mockups of adding rubric into the Assignment

• At the assignment page, the Settings side block contains new node to define the rubric (callout 2). The teacher clicks at "Rubric editor" link.
• The editor allows the teacher to design the rubric from scratch, to import it from XML file or use some existing rubric at the site as a template to start off from. She prepares a rubric consisting of three criteria, each of them having four levels with scores 0, 1, 2 and 3. Hence the total score obtained from the rubric will be something from 0 to 12. When the teacher is happy with the preview, goes back to the assignment.
• The list of submissions does not offer the Quick grading mode now. The "Grade" link next to each submission leads to a standalone page /grade/form/grading.php where the grading happens.

Screen with the online assignment submission being graded with a rubric

• The screen displays the graded submission and the defined rubric. The teacher fills the rubric and clicks "Save and close". She is taken back to the list of submissions at /mod/assignment/submissions.php.
• There is a grade now for that submission displayed, as if the teacher chose it manually. Note the score from the rubric is converted to the selected grade for the assignment. So for example the maximum score 12 leads to the grade 20, score 3 would give grade 5 etc (comparable with how the total score for questions used in a quiz is converted to a grade in the course). The same would apply if the grade for the submission would be set to use a scale.

Implementation plan

• New core plugin type called "Grading method" to be added into get_plugin_types() as 'grademethod' with the location in /grade/method/*
• All grading methods to be implemented as plugins - 'Rubric', 'Multi-scale' (aka Accumulative grading strategy in the Workshop) etc. Current grading tools could be re-implemented as 'Single scale' and 'Decimal points' plugins
• The mechanism of embedding the forms into pages in Moodle to follow the patterns already used in Files API, Comments API or Ratings API (using component, context, gradearea and itemid).
• Ability to either use the default renderer of the form of even define own renderer for it. This would allow the caller (client) to have complete control of how the grading form is embedded into the page.

Database schema

ER diagram of new tables

Basic data flow

1. Teacher goes to the course gradebook and clicks "Define new grading form". As the grading method she selects "Rubric". She fills the name and the description of the grading form. New record is inserted into {grade_forms} and the rubric editor opens.
2. For each new row of the rubric grid, a new record is inserted into {gradeform_rubric_criteria}.
3. For each new column of the rubric grid column in the given row, a new record is inserted into {gradeform_rubric_levels}.
4. Teacher goes to the Assignment module and modifies its setting to use the newly created rubric as the default grading method. TODO: where is this information going to be stored?
5. Teacher goes thru the list of submissions. After clicking "Grade" for a particular submission, new record is created (or existing one updated) in {grade_forms_instances} (component = "mod_assignment", gradearea = "submission", itemid = submission id, userid = $USER->id, timecreated = now).
6. The information about how the teacher actually fill the rubric is saved into {gradeform_rubric_fulfilments} - that is what level was chosen for each criterion. Optional comments can be stored, too.
7. When the teacher hits "Finish", the rubric subplugin calculates the final grade and provides it to the Assignment module.
8. The assignment module pushes the calculated grade to the gradebook.

Note that the rubric does not actually store the calculated grade. It just behaves as a tool that is able to calculate the value from how the form is fulfilled. Once the value is calculated, the Assignment module works with it as if it was selected via current <select> field.

See also

• The description of rubric at Wikipedia
• Kathy Schrock's Guide for Educators - Examples of Assessment Rubrics
• RubiStar is a free tool to help teachers create quality rubrics
• LMS Instructure and especially their way of defining rubrics
• Rubrics Manual for JOULE by Moodle Rooms
• Gradebook_2.x_architecture to come in 2.3