Development:Grades

>>Still under construction<<

There is an ongoing discussion about this spec here

Executive Summary

The gradebook mechanisms must be rebuilt to:

1. Improve performance and scalability - All grades from throughout the system will be pushed to a central system of tables. This means reports based on grades can be generated much faster, and the gradebook has ultimate control over the content.
2. Improve flexibility - All aspects of the new gradebook will use simple plugin structures, namely: exports, imports and displays/reports. It is expected that the community will be very active in producing special-purpose reports analysing the basic grade data in new ways, for example, or writing plugins to transfer grades to student information systems.
3. Allow rubrics for outcomes (aka standards,competencies,goals) - As well as numerical grades, each grading item can consist of a number of scores made on a rubric against a standard outcome statement. These can be automatically converted to a numerical grade if desired or just shown as is.
4. Implement an event-oriented architecture - Such an architecture allows all parts of Moodle to interact with the gradebook, but particularly gradebook plugins can react instantly to grading events throughout the system.
5. Allow arbitrary columns and derived columns - Arbitrary columns of data can be added (either manually or via import). Columns can also be automatically filled based on formulas.

Glossary

Here are some terms used in the gradebook, both in the development and the user interface. Using these terms in discussions about the gradebook will help to reduce confusion.

Term	Definition
Activity	An instance of an activity module (e.g. a single quiz, assignment etc...)
Calculation	A formula used to calculate grades, based (optionally) on other grade items. Not the same as Calculated question types.
Category	A set of Grade Items. A Category also has its own aggregated Grade which is calculated from its Grade Items. There is no limit to the level of nesting of Categories (a Category may belong to another Category). However, each Grade Item may belong to only one Category.
Course completion	The concept
Event	Events are a new feature of Moodle 1.9, of which the Gradebook will take full advantage. Modules that have grades to publish will define Events for which the Gradebook will listen. When an Event matches an Event Handler set up in the Gradebook, an Event Reaction will be triggered, performing any steps required (such as aggregating multiple grades, storing the grades etc...). Modules can also have their Event Handlers set up to respond to Events sent by the Gradebook (such as updating their internal grades).
Grade	A Grade is a single assessment. It may be a number or an item on a scale (possibly tied to an Outcome).
Gradebook	A central location in Moodle where students' Grades are stored and displayed. Teachers can keep track of their students' progress and organise which set of Grades their students will be able to see. Students see their own Grades.
Grade Item	A "column" of Grades. It can be created from a specific Activity or other module, calculated from other Grade Items, or entered manually.
Grade Locks	See linked section of this page
History	The gradebook has its own type of log, which keeps a History of all changes made to grades.
Outcome	Outcomes are specific descriptions of what a person is expected to be able to do or understand at the completion of an activity or course. An activity might have more than one outcome, and each may have a grade against it (usually on a scale). Other terms for Outcomes are Competencies and Goals. See some Examples.
Scale	A scale is a set of responses from which the teacher can choose one. eg A,B,C,D,E,F

Database structures

grade_items

This table keeps information about gradeable items (ie columns). If an activity (eg an assignment or quiz) has multiple grade_items associated with it (eg several outcomes or numerical grades), then there will be a corresponding multiple number of rows in this table.

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)		The course this item is part of
categoryid	int(10)	NULL	(optional) the category group this item belongs to
itemname	varchar(255)		The name of this item (pushed in by the module)
itemtype	varchar(30)		'mod', 'blocks', 'import', 'calculated', 'category' etc
itemmodule	varchar(30)		'forum', 'quiz', 'csv', etc
iteminstance	int(10)		id of the item module
itemnumber	int(10)	NULL	Can be used to distinguish multiple grades for an activity
iteminfo	text		Info and notes about this item XXX
idnumber	varchar(255)	NULL	Arbitrary idnumber provided by the module responsible
gradetype	int(4)	0	0 = value, 1 = scale, 2 = text
grademax	float(11,10)	100	What is the maximum allowable grade?
grademin	float(11,10)	0	What is the minimum allowable grade?
scaleid	int(10)	NULL	If this grade is based on a scale, which one is it?
outcomeid	int(10)	NULL	If this grade is related to an outcome, which one is it?
gradepass	float(11,10)	0	What grade is needed to pass? grademin < gradepass <= grademax
multfactor	float(11,10)	1.0	Multiply all grades by this
plusfactor	float(11,10)	0.0	Add this to all grades
sortorder	int(10)	0	Sorting order of the columns
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until (prevents viewing)
locked	int(10)	0	1 is locked, > 1 is a date to lock after (prevents update)
needsupdate	int(10)	0	If this flag is set, then the whole column will be recalculated
timecreated	int(10)		The first time this grade_item was created
timemodified	int(10)		The last time this grade_item was modified

grade_categories

This table keeps information about categories, used for grouping items. An associated grade_item will be maintained for each category to store the aggregate data.

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)		The course this grade category is part of
categoryid	int(10)	NULL	Categories can be hierarchical
fullname	varchar(255)		The name of this grade category
aggregation	int(10)	0	A constant pointing to one of the predefined aggregation strategies (none, mean,median,sum, etc)
keephigh	int(10)	0	Keep only the X highest items
droplow	int(10)	0	Drop the X lowest items
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until
timecreated	int(10)		The first time this grade_category was created
timemodified	int(10)		The last time this grade_category was modified

grade_calculations

This table describes the calculated grade_items in more details.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The grade_item this relates to
calculation	text	NULL	Formula describing how to derive this grade from other items, referring to them using [idnumber] ... eg something like:sin(square([XXXXX])) + [YYYY]
timecreated	int(10)		the time this calculation was first created
timemodified	int(10)		the time this calculation was last modified
usermodified	int(10)	NULL	the userid of the person who last modified this calculation

grade_grades_raw

This table keeps individual grades for each user and each item, exactly as imported or submitted by modules. The grademax/min and scaleid are stored here to record the values at the time the grade was stored, because teachers might change this for an activity! All the results are normalised/resampled for the grade_grades_final table.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The item this grade belongs to
userid	int(10)		The user who this grade is for
gradevalue	float(11,10)	NULL	If the grade is a float value (or has been converted to one)
gradescale	int(10)	NULL	If the grade is a scale value
grademax	float(11,10)	100	The maximum allowable grade when this was created
grademin	float(11,10)	0	The maximum allowable grade when this was created
scaleid	int(10)	NULL	If this grade is based on a scale, which one was it?
timecreated	int(10)		the time this grade was first created
timemodified	int(10)		the time this grade was last modified
usermodified	int(10)	NULL	the userid of the person who last modified this grade

grade_grades_final

This table keeps individual grades for each user and each item/category– they have undergone all scaling and other calculations, and are ready for display. This table is effectively a cache and values are rebuilt whenever source values change. The gradevalue or gradescale values are all normalised to the max/min or scaleid as defined in the grade_item table.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The associated grade_item these grades belong to
userid	int(10)		The user who this grade is for
gradevalue	float(11,10)	NULL	If the grade is a float value (or has been converted to one)
gradescale	int(10)	NULL	If the grade is a scale value
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until
exported	int(10)	0	0 is not exported, > 1 is the last exported date
timecreated	int(10)		the time this grade was first created
timemodified	int(10)		the time this grade was last modified
usermodified	int(10)	NULL	the userid of the person who last modified this grade

grade_grades_text

This table keeps additional textual information about each individual grade, whether it be automatically generated from the module or entered manually by the teacher. It's here separate from the all-numeric grade_grades for database efficiency reasons.

Field	Type	Default	Info
id	int(10)		autoincrementing
gradesid	int(10)		The exact grade in grade_grades_raw this corresponds to
information	text	NULL	Further information like forum rating distribution 4/5/7/0/1
feedback	text	NULL	Manual feedback from the teacher. Could be a code like 'mi'.
timecreated	int(10)		the time these entry was first created
timemodified	int(10)		the time this entry was last updated
usermodified	int(10)	NULL	the userid of the person who last modified this entry

grade_outcomes

This table describes the outcomes used in the system. An outcome is a statement tied to a rubric scale from low to high, such as “Not met, Borderline, Met” (stored as 0,1 or 2)

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)	NULL	Mostly these are defined site wide ie NULL
shortname	varchar(255)		The short name or code for this outcome statement
fullname	text		The full description of the outcome (usually 1 sentence)
scaleid	int(10)		The recommended scale for this outcome.
timecreated	int(10)		the time this outcome was first created
timemodified	int(10)		the time this outcome was last updated
usermodified	int(10)	NULL	the userid of the person who last modified this outcome

grade_history

This table keeps track of grade changes. Using this it should be possible to reconstruct the grades at any point in time in the past, or to audit grade changes over time. It should be quicker to use this table for that, rather than storing this information in the main Moodle log.

Note we use itemid and userid as a key rather than grade_grade id, just in case one of the things we want to log here is the deletion of a grade entirely.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The grade_item the grade is from
userid	int(10)		The user that the grade belongs to
oldgrade	float	NULL	The original grade before the change
newgrade	float	NULL	The new grade after the change
note	text	NULL	An optional note about why this change was made
howmodified	varchar(255)	manual	What caused the modification? manual/module/import/...
usermodified	int(10)	NULL	The user id of the person who made the change
timemodified	int(10)		The exact time this change was made

Overview of module communication

Modules will often store internal copies of grades for calculations or for showing grades to students etc, but these are purely internal. The gradebook system will never access these directly.

A new feature in Moodle 1.9 is a simple event handler. The new gradebook will be the first feature in Moodle to take advantage of events, and all communication between the gradebook and the modules will take place via event messages. See Development:Events for full details.

There are two ways that data is transferred between the module and the gradebook:

1. Modules post grade events

Whenever a new grade is created, the module should post a grading event so that the gradebook can pick it up. As long as that column in the gradebook is not “locked” then the new grades will be entered (overwriting any existing ones).

2. Gradebook posts grade events

Whenever a grade is altered in the gradebook (for example, a teacher may want to change an assignment grade), then an event is posted so that the module can pick it up. If the module implements this “reverse” function then the internal grade in the module can be changed.

Backward compatibility with Moodle 1.8 and earlier

For backward compatibility with old third-party modules, we will add a new function called grades_grab_grades() to admin/cron.php to search all mod/xxx/lib.php files for functions named xxx_grades().

These legacy functions will be called to extract all the grades for all the activities in each course. Once the data is extracted, the moodle_event_post() function can be called to initiate an event as usual and copy/upgrade the data in the gradebook tables.

A similar function will be added to new modules to trigger a “mass copy” of all the existing grades into the new gradebook. This will be useful during the upgrade from 1.8 as well as being available in the normal gradebook interface to “refresh” a column from primary data.

Core API functions

Even though most of the communication will take place via events, there are a few core functions in lib/gradelib.php that will be useful for modules:

grade_get_items()

eg grade_get_items($courseid, $itemname=NULL, $itemtype=NULL, $itemmodule=NULL, $iteminstance=NULL, $itemnumber=NULL, $idnumber=NULL)

For extracting all the information about what grading items are attached to something. For example, an assignment may want to retrieve all the grade_items for itself, and get three outcome scales in return. This will affect the grading interface.

grade_create_item()

To create a new grade_item in case it doesn't exist. This function would be called when a module is created or updated, for example, to ensure grade_item entries exist. It's not essential though - if grades are being added later and a matching grade_item doesn't exist yet, the gradebook will create them on the fly.

grade_create_category()

eg grade_create_category(fullname, array of items, aggregation, etc ...) For a given set of items, create a category to group them together (if one doesn't exist yet). Modules may want to do this when they are created. However, the ultimate control is in the gradebook interface itself.

grade_is_locked($itemtype, $itemmodule, $iteminstance, $userid=NULL)

This function will tell a module whether a grade (or grade_item if $userid is not given) is currently locked or not. This is a combination of the actual settings in the grade tables and a check on moodle/course:editgradeswhenlocked. If it's locked to the current use then the module can print a nice message or prevent editing in the module. (info)

Dealing with multiple grades

Modules will often produce several grade items. This may be several attempts, say, like the quiz module, or more commonly, ratings based on several outcomes. It is up to the module whether these grades are aggregated BEFORE sending to the gradebook as a single number, or are sent in raw form to the gradebook.

Pre-aggregration makes the most sense for a quiz, say, which already implements algorithms to calculate a single number for each student. In such a case there will only be one grade item from a particular quiz.

For anything that uses outcomes with grades against multiple rubrics (such as an assignment), it would make the most sense to just pass all the results through to the gradebook as individual grade items.

If the gradebook receives multiple grade items from a module, then they are automatically grouped together in a unique grade category (with the same name as the module instance). This is simply done by checking/creating the category whenever a second item from the same instance is being added. All the grading items will by default appear separate in the gradebook GUI, but can be easily “combined” by choosing an aggregation algorithm from a list (such as “Sum, Mean, Median, etc ...), plus “Keep X high, Drop X Low” together with scaling/shifting via the Multiplication and Plus factors etc.

Calculated grade items

A calculated grade item can operate on any arbitrary other items. It is not connected to any particular module.

The calculated values are re-calculated and stored in the grade_grades_final table whenever any of the values in the source items is changed. To avoid doing this for every grade_post event, we just set the “needsupdate” field on the relevant calculated grade item columns, then recalculate those columns at the next display time and reset the flag.

Displaying the grades to ordinary participants

The module takes responsibility for displaying grades within the module (to a student, say). Normally they would use internal tables to do so. However, there is a grade_get_grades() function available for modules to query the core gradebook for grades (if required).

For full display of grades in a whole course say, the student uses the same link as teachers use to access the gradebook. However, due to their different permissions they will only have access to specific reports. By default this is the “singleuser” report which only shows their own grades and has very few configuration options.

Locked grades

Both whole columns and individual grades can be locked in the gradebook, via the "locked" field. Teachers may want to do this to prevent further changes from the modules, or from other teachers. When a grade is locked, any grading events that might affect that grade are ignored. When the graded is unlocked, an "unlock" event is triggered, which allows a module to handle it by sending the latest new grades back.

In the main GUIs the lock toggling will be achieved by clicking on a little padlock icon beside each entry or column.

Manually modified grades

Grades can be manually modified (overriden) in the gradebook. When this is done to grade_items derived from a module's grades, and the new value is different from the original one, then the field will automatically be given a locked status (it can of course be unlocked immediately if one wishes). This is to prevent new grading events from changing the manually entered values unexpectedly.

Clean up after old activities

By default columns (grade items) will remain in the gradebook even when activities are deleted. A manual 'clean-up' button can delete old columns if required.

When you click the 'Clean up' button, it shows you a list of columns that should be deleted, with a checkbox next to each one (on by default, but you can turn it off if you want to keep that column still) and then another button click to really delete the marked columns.

Otherwise there will be a site option to do this automatically whenever activities are deleted.

Logging

As well as datestamps and userids in the tables, all grading events will be logged in case an audit needs to be made.

Security Issues

For security an option to force SSL for the gradebook might be good.

Overall grade

Having an overall grade handy allows us to know exactly when a course is "finished". Exactly how do we specify the "overall" grade? It's a special case, and might not always be just the sum.

The best solution is probably to have a special calculated column (grade_item) for every course that can not be deleted. It's made special because the itemtype is 'total' (one per course). The calculation formula should be updated automatically whenever grade_items are changed in that course to be the "sum" of all the grade_items in the course.

However, if the calculation is edited manually then we should lock the calculation formula. We could do this by setting the iteminfo field to something. Clearing the calculation to an empty string could force it back to an automatic calculation formula.

Metacourses

All grades in sub-courses should also be viewable from a metacourse.

Report plugins

All the main interface of the gradebook will be implemented as report plugins. Each plugin is fully responsible for defining the interface between the header and footer. They can even define their own capabilities and extra tables if the core tables are not enough, as they'll have a full /grade/report/xxxx/db directory.

Each report will need to define one capability to allow people to see that report, so that admins have control over who can see what reports. For example, the student interface can be a totally separate report plugin.

This allows for the widest flexibility and safety in how grades are presented.

Default teacher interface

This interface will be what teachers see by default, and will subsume everything the current interface (in Moodle 1.8) does.

Some snippets of functionality:

Overall it's a grid, with student names down one side and grade items along the top.

Teachers can type “straight into” the grid using AJAX or fallback to forms. No popup menus for values.

Columns will be able to be collapsed together by grouping them into categories. Grades for categories can be calculated via various means.

“Eye-cons” on the columns and checkboxes by every grade (this bit possibly controlled with a switch) allow hiding by category, by column, by individual grade.

Textual notes can be added to each grade for more info. These show up to students as well. Later on we can support customisable shorthand codes to make data entry quick (eg type 'ab' for absent, or 'nge' for not good enough).

A groups menu allows the teacher to switch between showing EACH of the groups they have access to, or ALL the groups they have access to.

All grade items will link to modulepath/grade.php?id=44 which will work out what the current person should be allowed to see and either redirect them to the correct page or just show them immediately. This copes with situations like the quiz, say, where we want editing teachers to go to the detailed reports there while students just see their own grade or whatever the quiz is set to show.

User preference to SWITCH between showing raw grades, percentage grades, or both, or grade letters (A/B/C etc).

Settings for grade letters not only define the transformation from percentage to grades, but also the transformation from letters to grades (in case the teacher edits some of the letter grades).

Categories are shown above the headings for each column. Clicking for more info on a category will just show the category with a summary column showing total/average for just that category (PLUS the summary column for the whole course).

All columns should be sortable up/down.

At the bottom of each column is a row with the mean course score. If in groups mode, then add ANOTHER row with just the group mean. Add the number of grades used in brackets. eg 56% (11). When the report is paged, these means are still for the whole course/group (not the page!)

Default student interface

This interface will be what students see by default:

Some snippets of functionality:

Invert the grid to show one item per row, with the total/average at the bottom.

Use second/third columns to show categories.

Include ranking score in another column.

Include class mean scores in a final column for comparison.

No editing functionality.

Export plugins

The API for these is extremely simple. Each export plugin should occupy a directory under /grade/export/xyz and needs to provide only an index.php file as a the primary interface. This file can accept parameters that produce a subset of the grades:

• courseid
• groupid
• categoryid
• itemid
• userid

The index.php can choose to simply dump something (eg an Excel spreadsheet) and return, or it can show an interface for further options and selections. Ideally even if it dumps something it should also show some feedback about what was exported to make it easier to double check what was sent, but this is up to the plugin to implement.

Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin (eg by checking access permissions). If this function returns false then the export method won't appear in the main gradebook interfaces.

The export plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an export plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.

Import plugins

Each import plugin should occupy a directory under /grade/import/xyz and needs to provide only an index.php file as a the primary interface. This file just accepts a 'courseid' parameter.

The index.php will show an interface for further options and selections.

Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin. If this function returns false then the import method won't appear in the main gradebook interfaces.

The import plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an import plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.

The entire import must be treated as one operation. If it fails, then no records should be imported and the database is unchanged.

Some sample import plugins are:

Import from CSV

Accepts an upload of (or URL to) a CSV file. Multiple options describe how to process the file, which columns to add etc. These should be sticky, and retained via user preferences to make future imports easier.

Import from XML

Accepts an upload of (or URL to) an XML file with this kind of format (from OU).

<results batch="[someuniqueimportnumber]">
    <result>
        <state>['new' or 'regrade']</state>
            <assignment>[idnumber]</assignmentid>
            <student>[studentid]</student>
            <score>[score]</score>
        </result>
        <result>
            <state>['new' or 'regrade']</state>
            <assignment>[idnumber]</assignmentid>
            <student>[studentid]</student>
            <score>[score]</score>
        </result>
        [...]
</results>

Capabilities and Permissions

• moodle/course:viewcoursegrades

• moodle/course:downloadcoursegrades

• moodle/user:viewusergrades

• moodle/grade:viewhidden

• moodle/grade:editlocked

• moodle:site/accessallgroups

Interface Mockups

Nicolas is working on a mock-up interface that you can play with:

   Mock Gradebook

It's changing rapidly in response to your feedback.

Rough Development Timeline

1. Develop Event API
   1. Core tables
   2. events.php to load/maintain handlers
   3. Core API functions lib/eventslib.php and unit tests
2. Develop Core Gradebook
   1. Core tables
   2. Core API functions lib/gradelib.php and unit tests
   3. Core handler for grade changes
   4. Define plugin directory structure
   5. Cron job for support of legacy grade functions in modules
3. Core Moodle improvements
   1. Support for all grades-based plugins for db, cron, access.php, events.php etc
4. Develop Core activity modules
   1. Triggers for all major activity modules
   2. Handlers for all major activity modules
5. Reports
   1. Standard report for teachers
   2. Standard report for students
6. Exports
   1. Standard exports (like 1.8)
   2. XML export (OU format)
7. Imports
   1. CSV format

See also

• Gradebook Development ideas forum discussion
• Using Moodle New gradebook for Moodle forum discussion
• Development:Events