Development:Grades
There is an ongoing discussion about this spec here
Executive Summary
The gradebook mechanisms must be rebuilt to:
- Improve performance and scalability - All grades from throughout the system will be pushed to a central system of tables. This means reports based on grades can be generated much faster, and the gradebook has ultimate control over the content.
- Improve flexibility - All aspects of the new gradebook will use simple plugin structures, namely: exports, imports and displays/reports. It is expected that the community will be very active in producing special-purpose reports analysing the basic grade data in new ways, for example, or writing plugins to transfer grades to student information systems.
- Allow rubrics for outcomes (aka standards,competencies,goals) - As well as numerical grades, each grading item can consist of a number of scores made on a rubric against a standard outcome statement. These can be automatically converted to a numerical grade if desired or just shown as is.
- Implement an event-oriented architecture - Such an architecture allows all parts of Moodle to interact with the gradebook, but particularly gradebook plugins can react instantly to grading events throughout the system.
- Allow arbitrary columns and derived columns - Arbitrary columns of data can be added (either manually or via import). Columns can also be automatically filled based on formulas.
Glossary
Here are some terms used in the gradebook, both in the development and the user interface. Using these terms in discussions about the gradebook will help to reduce confusion.
Term | Definition |
Activity | An instance of an activity module (e.g. a single quiz, assignment etc...) |
Calculation | A formula used to calculate grades, based (optionally) on other grade items. Not the same as Calculated question types. |
Category | A set of Grade Items. A Category also has its own aggregated Grade which is calculated from its Grade Items. There is no limit to the level of nesting of Categories (a Category may belong to another Category). However, each Grade Item may belong to only one Category. |
Course completion | The concept |
Event | Events are a new feature of Moodle 1.9, of which the Gradebook will take full advantage. Modules that have grades to publish will define Events for which the Gradebook will listen. When an Event matches an Event Handler set up in the Gradebook, an Event Reaction will be triggered, performing any steps required (such as aggregating multiple grades, storing the grades etc...). Modules can also have their Event Handlers set up to respond to Events sent by the Gradebook (such as updating their internal grades). |
Grade | A Grade is a single assessment. It may be a number or an item on a scale (possibly tied to an Outcome). |
Gradebook | A central location in Moodle where students' Grades are stored and displayed. Teachers can keep track of their students' progress and organise which set of Grades their students will be able to see. Students see their own Grades. |
Grade Item | A "column" of Grades. It can be created from a specific Activity or other module, calculated from other Grade Items, or entered manually. |
Grade Locks | See linked section of this page |
History | The gradebook has its own type of log, which keeps a History of all changes made to grades. |
Outcome | Outcomes are specific descriptions of what a person is expected to be able to do or understand at the completion of an activity or course. An activity might have more than one outcome, and each may have a grade against it (usually on a scale). Other terms for Outcomes are Competencies and Goals. See some Examples. |
Scale | A scale is a set of responses from which the teacher can choose one. eg A,B,C,D,E,F |
Database structures
grade_items
This table keeps information about gradeable items (ie columns). If an activity (eg an assignment or quiz) has multiple grade_items associated with it (eg several outcomes or numerical grades), then there will be a corresponding multiple number of rows in this table.
idnumber is a unique tag identifying the grade_item, useful for identifying data in exports and for referring to the grade_item in calculations. Usually it will be the same as the idnumber in course_modules but if one is not specified then it's automatically set as itemmodule.iteminstance-itemnumber (eg forum3).
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
courseid | int(10) | The course this item is part of | |
categoryid | int(10) | (optional) the category group this item belongs to | |
itemname | varchar(255) | The name of this item (pushed in by the module) | |
itemtype | varchar(30) | 'mod', 'blocks', 'import', 'calculated', 'category' etc | |
itemmodule | varchar(30) | 'forum', 'quiz', 'csv', etc | |
iteminstance | int(10) | id of the item module | |
itemnumber | int(10) | Can be used to distinguish multiple grades for an activity | |
iteminfo | text | Info and notes about this item XXX | |
idnumber | varchar(255) | Arbitrary idnumber provided by the module responsible (must be defined and unique) | |
gradetype | int(4) | 0 = none, 1 = value, 2 = scale, 3 = text | |
grademax | float(11,10) | What is the maximum allowable grade? | |
grademin | float(11,10) | What is the minimum allowable grade? | |
scaleid | int(10) | If this grade is based on a scale, which one is it? | |
outcomeid | int(10) | If this grade is related to an outcome, which one is it? | |
gradepass | float(11,10) | What grade is needed to pass? grademin <= gradepass <= grademax | |
multfactor | float(11,10) | Multiply all grades by this | |
plusfactor | float(11,10) | Add this to all grades | |
aggregationcoef | float(11,10) | Weight applied to all grades in this grade item during aggregation with other grade items. | |
sortorder | int(10) | Sorting order of the columns | |
hidden | int(10) | 1 is hidden, 1 is hide always, > 1 is a date to hide until (prevents viewing of all user grades) | |
locked | int(10) | 0 is not locked, > 0 is a date when was item locked (no final grade or grade_item updates possible) | |
locktime | int(10) | 0 no auto locking, > 0 is a date to lock grade item and final grades after automatically | |
deleted | int(10) | 1 means the associated module instance has been deleted | |
needsupdate | int(10) | If this flag is set, then the whole column will be recalculated | |
timecreated | int(10) | The first time this grade_item was created | |
timemodified | int(10) | The last time this grade_item was modified |
grade_categories
This table keeps information about categories, used for grouping items. An associated grade_item will be maintained for each category to store the aggregate data.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
courseid | int(10) | The course this grade category is part of | |
parent | int(10) | Parent grade_category (hierarchical) | |
depth | int(10) | How deep is this category from the highest level (0,1,2)? Used for table display purposes. | |
path | varchar(255) | Shows the path as /1/2/3 (like course_categories) | |
fullname | varchar(255) | The name of this grade category | |
aggregation | int(10) | A constant pointing to one of the predefined aggregation strategies (none, mean,median,sum, etc) | |
keephigh | int(10) | Keep only the X highest items | |
droplow | int(10) | Drop the X lowest items | |
timecreated | int(10) | The first time this grade_category was created | |
timemodified | int(10) | The last time this grade_category was modified |
grade_calculations
This table describes the calculated grade_items in more details.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
itemid | int(10) | The grade_item this relates to | |
calculation | text | Formula describing how to derive this grade from other items, referring to them using [idnumber] ... eg something like:sin(square([XXXXX])) + [YYYY] | |
timecreated | int(10) | the time this calculation was first created | |
timemodified | int(10) | the time this calculation was last modified | |
usermodified | int(10) | the userid of the person who last modified this calculation |
grade_grades
This table keeps individual grades for each user and each item. The raw grade is exactly as imported or submitted by modules. The rawgrademax/min and rawscaleid are stored here to record the values at the time the grade was stored, because teachers might change this for an activity! All the results are normalised/resampled/calculated for the finalgrade, which is relative to the max/min/scaleid values stored in the grade_item. The finalgrade field is effectively a cache and values are rebuilt whenever raw values or the grade_item changes.
Note that the finalgrade for a scale-based item may be non-integer! It needs to be rounded on display.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
itemid | int(10) | The item this grade belongs to | |
userid | int(10) | The user who this grade is for | |
rawgrade | float(11,10) | The raw grade that came into the system | |
rawgrademax | float(11,10) | The maximum allowable grade when this was created | |
rawgrademin | float(11,10) | The minimum allowable grade when this was created | |
rawscaleid | int(10) | If this grade is based on a scale, which one was it? | |
usermodified | int(10) | the userid of the person who last modified the raw grade value | |
finalgrade | float(11,10) | The final grade (cached) after all calculations are made | |
hidden | int(10) | 0 is not hidden, 1 is hide always, > 1 is a date to hide until | |
locked | int(10) | 0 is not locked, > 0 when was the grade locked | |
locktime | int(10) | 0 is never, > 0 is a date to lock the final grade after automatically | |
exported | int(10) | 0 is not exported, > 0 is the last exported date | |
timecreated | int(10) | the time this grade was first created | |
timemodified | int(10) | the time this grade was last modified |
grade_grades_text
This table keeps additional textual information about each individual grade, whether it be automatically generated from the module or entered manually by the teacher. It's here separate from the all-numeric grade_grades for database efficiency reasons.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
gradesid | int(10) | The exact grade in grade_grades this corresponds to | |
information | text | Further information like forum rating distribution 4/5/7/0/1 | |
informationformat | int(10) | Text format for information | |
feedback | text | Manual feedback from the teacher. Could be a code like 'mi'. | |
feedbackformat | int(10) | Text format for feedback | |
timecreated | int(10) | the time these entry was first created | |
timemodified | int(10) | the time this entry was last updated | |
usermodified | int(10) | the userid of the person who last modified this entry |
grade_outcomes
This table describes the outcomes used in the system. An outcome is a statement tied to a rubric scale from low to high, such as “Not met, Borderline, Met” (stored as 0,1 or 2). For more info about these see Development:Outcomes.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
courseid | int(10) | Mostly these are defined site wide ie NULL | |
shortname | varchar(255) | The short name or code for this outcome statement | |
fullname | text | The full description of the outcome (usually 1 sentence) | |
scaleid | int(10) | The recommended scale for this outcome. | |
timecreated | int(10) | the time this outcome was first created | |
timemodified | int(10) | the time this outcome was last updated | |
usermodified | int(10) | the userid of the person who last modified this outcome |
grade_history
This table keeps track of raw grade changes in grade_grades. Using this it should be possible to reconstruct the grades at any point in time in the past, or to audit grade changes over time. It should be quicker to use this table for that, rather than storing this information in the main Moodle log.
Note we use itemid and userid as a key rather than grade_grade id, just in case one of the things we want to log here is the deletion of a grade entirely.
Field | Type | Default | Info |
id | int(10) | autoincrementing | |
itemid | int(10) | The grade_item the grade is from | |
userid | int(10) | The user that the grade belongs to | |
oldgrade | float | NULL | The original grade before the change |
newgrade | float | NULL | The new grade after the change |
note | text | NULL | An optional note about why this change was made |
howmodified | varchar(255) | manual | What caused the modification? manual/module/import/... |
usermodified | int(10) | The user id of the person who made the change | |
timemodified | int(10) | The exact time this change was made |
Overview of module communication
Modules will often store internal copies of grades for calculations or for showing grades to students etc, but these are purely internal. The gradebook system will never access these directly.
A new feature in Moodle 1.9 is a simple event handler. The new gradebook will be the first feature in Moodle to take advantage of events, and all communication between the gradebook and the modules will take place via event messages. See Development:Events for full details.
There are two ways that data is transferred between the module and the gradebook:
- 1. Modules post grade events
- Whenever a new grade is created, the module should post a grade_updated event so that the gradebook can pick it up. As long as that column in the gradebook is not “locked” then the new grades will be entered (overwriting any existing ones).
- 2. Gradebook posts grade events
- Whenever a grade is altered in the gradebook (for example, a teacher may want to change an assignment grade), then an grade_updated_external event is posted so that the module can pick it up. If the module implements this “reverse” function then the internal grade in the module can be changed.
Backward compatibility with Moodle 1.8 and earlier
- For backward compatibility with old third-party modules, we will add a new function called grade_grab_legacy_grades() to admin/cron.php to search all mod/xxx/lib.php files for functions named xxx_grades().
- These legacy functions will be called to extract all the grades for all the activities in each course. Once the data is extracted, the grade_updated event is triggered as usual and copy/upgrade the data in the gradebook tables.
- A similar function will be added to upgraded modules to trigger a “mass copy” of all the existing grades into the new gradebook. This will be useful during the upgrade from 1.8 as well as being available in the normal gradebook interface to “refresh” a column from primary data.
Core API functions
Even though most of the communication will take place via events, there are a few core functions in lib/gradelib.php that will be useful for modules:
grade_get_items()
eg grade_get_items($courseid, $itemtype=NULL, $itemmodule=NULL, $iteminstance=NULL, $itemname=NULL, $itemnumber=NULL, $idnumber=NULL)
For extracting all the information about what grading items are attached to something. For example, an assignment may want to retrieve all the grade_items for itself, and get three outcome scales in return. This will affect the grading interface.
grade_create_item()
Create a new grade_item. This function would be called when a module is created or during upgrade.
grade_create_category()
eg grade_create_category(fullname, array of items, aggregation, etc ...) For a given set of items, create a category to group them together (if one doesn't exist yet). Modules may want to do this when they are created. However, the ultimate control is in the gradebook interface itself.
grade_is_locked()
eg grade_is_locked($itemtype, $itemmodule, $iteminstance, $itemnumber=NULL, $userid=NULL)
This function will tell a module whether a grade (or grade_item if $userid is not given) is currently locked or not. This is a combination of the actual settings in the grade tables and a check on moodle/course:editgradeswhenlocked. If it's locked to the current user then the module can print a nice message or prevent editing in the module. If the grade_item is locked, this setting overrides any grades under that grade_item. However, if it is unlocked AND a $userid is passed to the function, the state of the specific grade will be returned. (info)
grade_upgrade_final_grades()
grade_upgrade_final_grades($courseid=NULL, $gradeitemid=NULL)
Updates all grade_grades->finalgrade records for each grade_item matching the given attributes. The search is further restricted, so that only grade_items that have needs_update == true or that use calculation are retrieved and used for the update. The function returns the number of grade_items updated (NOT the same as the number of grades_grades updated!).
Dealing with multiple grades
Modules will often produce several grade items. This may be several attempts, say, like the quiz module, or more commonly, ratings based on several outcomes. It is up to the module whether these grades are aggregated BEFORE sending to the gradebook as a single number, or are sent in raw form to the gradebook.
Pre-aggregration makes the most sense for a quiz, say, which already implements algorithms to calculate a single number for each student. In such a case there will only be one grade item from a particular quiz.
For anything that uses outcomes with grades against multiple rubrics (such as an assignment), it would make the most sense to just pass all the results through to the gradebook as individual grade items.
If the gradebook receives multiple grade items from a module, then they are automatically grouped together in a unique grade category (with the same name as the module instance). This is simply done by checking/creating the category whenever a second item from the same instance is being added. All the grading items will by default appear separate in the gradebook GUI, but can be easily “combined” by choosing an aggregation algorithm from a list (such as “Sum, Mean, Median, etc ...), plus “Keep X high, Drop X Low” together with scaling/shifting via the Multiplication and Plus factors etc.
For more info about multiple grade items see Development:Outcomes.
Calculated grade items
A calculated grade item can operate on any arbitrary other items. It is not connected to any particular module.
The final calculated values are re-calculated and stored in the grade_grades table whenever any of the values in the source items is changed. To avoid doing this for every grade_post event, we just set the needsupdate field on grade_items every time grades are added or changed.
At the next display time or export time (ie whenever grades_grades->finalgrade is accessed) we do this:
If any current grade_items are flagged as needsupdate then # for each grade_item with the flag, update all grade_grades->final for them and remove the grade_item flag # for each grade_item with a calculation, update their grade_grades->finalgrade
The format for the calculations may as well use the same format as Excel and Openoffice, as that is what most people are probably used to.
eg: MEAN(quizstart1, quizend1) + assignment1 + 20.0
Adjustment of grades
Grade_item contains optional rules for adjusting the raw grade before it is cached into a final grade. These rules are processed BEFORE the calculation discussed above, because they may include reprocessing the grade's place within a different scale or grade range.
Adjustment to a new range
A raw grade may have been initially calculated within a module inside a range of 0 to 100 (grademin - grademax). However, the grade_item may be given a different grademin and grademax. For example, it could be 30 to 70. If you take a raw value of 30 on the original range (i.e. 30%), after adjusting this score for the new range, you should obtain 42, not 30 (30 being the equivalent of 0 on the previous range). This calculation takes place before the multfactor and plusfactor are applied, and before the calculation formula is applied.
Adjustment to a new scale
A raw grade may also have a scale value, which always starts at 0 (for the first item in the scale), and has a maximum set by the number of items in the scale. Because this value is always an integer, the above calculation is performed in addition to a rounding up to the nearest integer. Also, multfactor and plusfactor are both ignored, because using them makes no sense in the context of a scale value.
Displaying the grades to ordinary participants
The module takes responsibility for displaying grades within the module (to a student, say). Normally they would use internal tables to do so. However, there is a grade_get_grades() function available for modules to query the core gradebook for grades (if required).
For full display of grades in a whole course say, the student uses the same link as teachers use to access the gradebook. However, due to their different permissions they will only have access to specific reports. By default this is the “singleuser” report which only shows their own grades and has very few configuration options.
Locked grades
Both whole columns and individual grades can be locked in the gradebook, via the "locked" field. Teachers may want to do this to prevent further changes from the modules, or from other teachers. When a grade is locked, any grading events that might affect that grade are ignored. When the graded is unlocked, an "unlock" event is triggered, which allows a module to handle it by sending the latest new grades back.
In the main GUIs the lock toggling will be achieved by clicking on a little padlock icon beside each entry or column.
Manually modified grades
Grades can be manually modified (overriden) in the gradebook. When this is done to grade_items derived from a module's grades, and the new value is different from the original one, then the field will automatically be given a locked status (it can of course be unlocked immediately if one wishes). This is to prevent new grading events from changing the manually entered values unexpectedly.
Clean up after old activities
By default columns (grade items) will remain in the gradebook even when activities are deleted. A manual 'clean-up' button can delete old columns if required.
When you click the 'Clean up' button, it shows you a list of columns that should be deleted, with a checkbox next to each one (on by default, but you can turn it off if you want to keep that column still) and then another button click to really delete the marked columns.
Otherwise there will be a site option to do this automatically whenever activities are deleted.
Logging
As well as datestamps and userids in the tables, all grading events will be logged in case an audit needs to be made.
Security Issues
For security an option to force SSL for the gradebook might be good.
Overall grade
Having an overall grade handy allows us to know exactly when a course is "finished". Exactly how do we specify the "overall" grade? It's a special case, and might not always be just the sum.
The best solution is probably to have a special calculated column (grade_item) for every course that can not be deleted. It's made special because the itemtype is 'total' (one per course). The calculation formula should be updated automatically whenever grade_items are changed in that course to be the "sum" of all the grade_items in the course.
However, if the calculation is edited manually then we should lock the calculation formula. We could do this by setting the iteminfo field to something. Clearing the calculation to an empty string could force it back to an automatic calculation formula.
Report plugins
All the main interface of the gradebook will be implemented as report plugins. Each plugin is fully responsible for defining the interface between the header and footer. They can even define their own capabilities and extra tables if the core tables are not enough, as they'll have a full /grade/report/xxxx/db directory.
Each report will need to define one capability to allow people to see that report, so that admins have control over who can see what reports. For example, the student interface can be a totally separate report plugin.
This allows for the widest flexibility and safety in how grades are presented.
Default teacher interface
This interface will be what teachers see by default, and will subsume everything the current interface (in Moodle 1.8) does.
Some snippets of functionality:
Overall it's a grid, with student names down one side and grade items along the top.
Teachers can type “straight into” the grid using AJAX or fallback to forms. No popup menus for values.
Columns will be able to be collapsed together by grouping them into categories. Grades for categories can be calculated via various means.
“Eye-cons” on the columns and checkboxes by every grade (this bit possibly controlled with a switch) allow hiding by category, by column, by individual grade.
Textual notes can be added to each grade for more info. These show up to students as well. Later on we can support customisable shorthand codes to make data entry quick (eg type 'ab' for absent, or 'nge' for not good enough).
A groups menu allows the teacher to switch between showing EACH of the groups they have access to, or ALL the groups they have access to.
All grade items will link to modulepath/grade.php?id=44 which will work out what the current person should be allowed to see and either redirect them to the correct page or just show them immediately. This copes with situations like the quiz, say, where we want editing teachers to go to the detailed reports there while students just see their own grade or whatever the quiz is set to show.
User preference to SWITCH between showing raw grades, percentage grades, or both, or grade letters (A/B/C etc).
Settings for grade letters not only define the transformation from percentage to grades, but also the transformation from letters to grades (in case the teacher edits some of the letter grades).
Categories are shown above the headings for each column. Clicking for more info on a category will just show the category with a summary column showing total/average for just that category (PLUS the summary column for the whole course).
All columns should be sortable up/down.
At the bottom of each column is a row with the mean course score. If in groups mode, then add ANOTHER row with just the group mean. Add the number of grades used in brackets. eg 56% (11). When the report is paged, these means are still for the whole course/group (not the page!)
Default student interface
This interface will be what students see by default:
Some snippets of functionality:
Invert the grid to show one item per row, with the total/average at the bottom.
Use second/third columns to show categories.
Include ranking score in another column.
Include class mean scores in a final column for comparison.
No editing functionality.
Export plugins
The API for these is extremely simple. Each export plugin should occupy a directory under /grade/export/xyz and needs to provide only an index.php file as a the primary interface. This file can accept parameters that produce a subset of the grades:
- courseid
- groupid
- categoryid
- itemid
- userid
The index.php can choose to simply dump something (eg an Excel spreadsheet) and return, or it can show an interface for further options and selections. Ideally even if it dumps something it should also show some feedback about what was exported to make it easier to double check what was sent, but this is up to the plugin to implement.
Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin (eg by checking access permissions). If this function returns false then the export method won't appear in the main gradebook interfaces.
The export plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an export plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.
Import plugins
Each import plugin should occupy a directory under /grade/import/xyz and needs to provide only an index.php file as a the primary interface. This file just accepts a 'courseid' parameter.
The index.php will show an interface for further options and selections.
Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin. If this function returns false then the import method won't appear in the main gradebook interfaces.
The import plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an import plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.
The entire import must be treated as one operation. If it fails, then no records should be imported and the database is unchanged.
Some sample import plugins are:
Import from CSV
Accepts an upload of (or URL to) a CSV file. Multiple options describe how to process the file, which columns to add etc. These should be sticky, and retained via user preferences to make future imports easier.
Import from XML
Accepts an upload of (or URL to) an XML file with this kind of format (from OU).
<results batch="[someuniqueimportnumber]"> <result> <state>['new' or 'regrade']</state> <assignment>[idnumber]</assignmentid> <student>[studentid]</student> <score>[score]</score> </result> <result> <state>['new' or 'regrade']</state> <assignment>[idnumber]</assignmentid> <student>[studentid]</student> <score>[score]</score> </result> [...] </results>
Capabilities and Permissions
- moodle/grade:view - view your own grades
- moodle/grade:viewall - view grades of other users
- moodle/grade:viewhidden - see grades that are marked as hidden for the owner
- moodle/grade:hide - be able to hide/unhide cells, items or categories
- moodle/grade:lock - be able to lock cells, items or categories
- moodle/grade:unlock - be able to unlock cells, items or categories
- moodle/grade:manage - manage grade items and categories in gradebook (create, edit, lock, hide, delete, etc.)
- moodle/grade:import - import grades
- moodle/grade:export - export grades
- moodle/grade:override - edit or override individual grades
- gradereport/grader:view - can view the grader report
- gradeimport/csv:view - can view/use the csv import plugin
- gradeexport/csv:view - can view/use the csv export plugin
- moodle:site/accessallgroups
Interface Demo
The mockup is no longer applicable. Instead you can play with the live demo at the following site:
Gradebook live demo
Please base your future feedback on that interface.
Development Tasks and Tracking
See [https://tracker.moodle.org/browse/MDL-9137 MDL-9137] for the full details.
See also
- Gradebook Development ideas forum discussion
- Using Moodle New gradebook for Moodle forum discussion
- Development:Events