Grades

ALL OF THIS IS VERY DRAFT AND UNDER CONSTRUCTION!!!!!

Executive Summary

The gradebook mechanisms must be rebuilt to:

1. Improve performance and scalability - All grades from throughout the system will be pushed to a central system of tables. This means reports based on grades can be generated much faster, and the gradebook has ultimate control over the content.
2. Improve flexibility - All aspects of the new gradebook will use simple plugin structures, namely: exports, imports and displays/reports. It is expected that the community will be very active in producing special-purpose reports analysing the basic grade data in new ways, for example, or writing plugins to transfer grades to student information systems.
3. Allow rubrics for outcomes (aka standards,competencies,goals) - As well as numerical grades, each grading item can consist of a number of scores made on a rubric against a standard outcome statement. These can be automatically converted to a numerical grade if desired or just shown as is.
4. Implement an event-oriented architecture - Such an architecture allows all parts of Moodle to interact with the gradebook, but particularly gradebook plugins can react instantly to grading events throughout the system.
5. Allow arbitrary columns and derived columns - Arbitrary columns of data can be added (either manually or via import). Columns can also be automatically filled based on formulas.

Database structures

grade_items

This table keeps information about gradeable items (ie columns). If an activity (eg an assignment or quiz) has multiple grade_items associated with it (eg several outcomes or numerical grades), then there will be a corresponding multiple number of rows in this table.

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)		The course this item is part of
categoryid	int(10)	NULL	(optional) the category group this item belongs to
itemname	varchar(255)		The name of this item (pushed in by the module)
itemtype	varchar(30)		'mod', 'blocks', 'import', 'calculated' etc
itemmodule	varchar(30)		'forum', 'quiz', 'csv', etc
iteminstance	int(10)		id of the item module
itemnumber	int(10)	NULL	Can be used to distinguish multiple grades for an activity
iteminfo	text		Info and notes about this item XXX
idnumber	varchar(255)	NULL	Arbitrary idnumber provided by the module responsible
gradetype	int(4)	0	0 = value, 1 = scale, 2 = text
grademax	float(11,10)	100	What is the maximum allowable grade?
grademin	float(11,10)	0	What is the minimum allowable grade?
scaleid	int(10)	NULL	If this grade is based on a scale, which one is it?
outcomeid	int(10)	NULL	If this grade is related to an outcome, which one is it?
multfactor	float(11,10)	1.0	Multiply all grades by this
plusfactor	float(11,10)	0.0	Add this to all grades
sortorder	int(10)	0	Sorting order of the columns
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until (prevents viewing)
locked	int(10)	0	1 is locked, > 1 is a date to lock until (prevents update)
needsupdate	int(10)	0	If this flag is set, then the whole column will be recalculated
timecreated
timemodified

grade_categories

This table keeps information about categories, used for grouping items.

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)		The course this grade category is part of
categoryid	int(10)	NULL	Categories can be hierarchical
fullname	varchar(255)		The name of this grade category
aggregation	int(10)	0	A constant pointing to one of the predefined aggregation strategies (none, mean,median,sum, etc)
keephigh	int(10)	0	Keep only the X highest items
droplow	int(10)	0	Drop the X lowest items
multfactor	float(11,10)	1.0	multiply total grade by this
plusfactor	float(11,10)	0.0	add this to total grade
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until

grade_calculations

This table describes the calculated grade_items in more details.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The grade_item this relates to
calculation	text	NULL	Formula describing how to derive this grade from other items, referring to them using [idnumber] ... eg something like:sin(square([XXXXX])) + [YYYY]

grade_grades

This table keeps individual grades for each user and each item, exactly as imported or submitted by modules. The grademax/min and scaleid are stored here to record the values at the time the grade was stored, because teachers might change this for an activity! All the results are normalised/resampled for the grade_grades_final table.

Field	Type	Default	Info
id	int(10)		autoincrementing
itemid	int(10)		The item this grade belongs to
userid	int(10)		The user who this grade is for
gradevalue	float(11,10)	NULL	If the grade is a float value (or has been converted to one)
gradescale	int(10)	NULL	If the grade is a scale value
grademax	float(11,10)	100	The maximum allowable grade when this was created
grademin	float(11,10)	0	The maximum allowable grade when this was created
scaleid	int(10)	NULL	If this grade is based on a scale, which one was it?
timecreated	int(10)		the time this grade was first created
timemodified	int(10)		the time this grade was last modified

grade_grades_final

This table keeps individual grades for each user and each item/category– they have undergone all scaling and other calculations, and are ready for display. This table is effectively a cache and values are rebuilt whenever source values change. The gradevalue or gradescale values are all normalised to the max/min or scaleid as defined in the grade_item table.

Field	Type	Default	Info
id	int(10)		autoincrementing
columnid	int(10)		itemid or categoryid
columntype	enum		'category' or 'item'
userid	int(10)		The user who this grade is for
gradevalue	float(11,10)	NULL	If the grade is a float value (or has been converted to one)
gradescale	int(10)	NULL	If the grade is a scale value
hidden	int(10)	0	1 is hidden, > 1 is a date to hide until
exported	int(10)	0	0 is not exported, > 1 is the last exported date
timecreated	int(10)		the time this grade was first created
timemodified	int(10)		the time this grade was last modified

grade_grades_text

This table keeps additional textual information about each individual grade, whether it be automatically generated from the module or entered manually by the teacher. It's here separate from the all-numeric grade_grades for database efficiency reasons.

Field	Type	Default	Info
id	int(10)		autoincrementing
gradesid	int(10)		The exact grade in grade_grades this corresponds to
information	text	NULL	Further information like forum rating distribution 4/5/7/0/1
feedback	text	NULL	Manual feedback from the teacher. Could be a code like 'mi'.

grade_outcomes

This table describes the outcomes used in the system. An outcome is a statement tied to a rubric scale from low to high, such as “Not met, Borderline, Met” (stored as 0,1 or 2)

Field	Type	Default	Info
id	int(10)		autoincrementing
courseid	int(10)	NULL	Mostly these are defined site wide ie NULL
shortname	varchar(255)		The short name or code for this outcome statement
fullname	text		The full description of the outcome (usually 1 sentence)
scaleid	int(10)		The recommended scale for this outcome.

Overview of module communication

Modules will often store internal copies of grades for calculations or for showing grades to students etc, but these are purely internal. The gradebook system will never access these directly.

A new feature in Moodle 1.9 is a simple event handler. The new gradebook will be the first feature in Moodle to take advantage of events, and all communication between the gradebook and the modules will take place via event messages. See Events for full details.

There are two ways that data is transferred between the module and the gradebook:

1. Modules post grade eventsWhenever a new grade is created, the module should post a grading event so that the gradebook can pick it up. As long as that column in the gradebook is not “locked” then the new grades will be entered (overwriting any existing ones).
2. Gradebook posts grade events Whenever a grade is altered in the gradebook (for example, a teacher may want to change an assignment grade), then an event is posted so that the module can pick it up. If the module implements this “reverse” function then the internal grade in the module can be changed.

Backward compatibility with Moodle 1.8 and earlier

For backward compatibility with old third-party modules, we will add a new function called grades_grab_grades() to admin/cron.php to search all mod/xxx/lib.php files for functions named xxx_grades().

These legacy functions will be called to extract all the grades for all the activities in each course. Once the data is extracted, the moodle_event_post() function can be called to initiate an event as usual and copy/upgrade the data in the gradebook tables.

A similar function will be added to new modules to trigger a “mass copy” of all the existing grades into the new gradebook. This will be useful during the upgrade from 1.8 as well as being available in the normal gradebook interface to “refresh” a column from primary data.

Dealing with multiple grades

Modules will often produce several grade items. This may be several attempts, say, like the quiz module, or more commonly, ratings based on several outcomes. It is up to the module whether these grades are aggregated BEFORE sending to the gradebook as a single number, or are sent in raw form to the gradebook.

Pre-aggregration makes the most sense for a quiz, say, which already implements algorithms to calculate a single number for each student. In such a case there will only be one grade item from a particular quiz.

For anything that uses outcomes with grades against multiple rubrics (such as an assignment), it would make the most sense to just pass all the results through to the gradebook as individual grade items.

If the gradebook receives multiple grade items from a module, then they are automatically grouped together in a unique grade category (with the same name as the module instance). This is simply done by checking/creating the category whenever a second item from the same instance is being added. All the grading items will by default appear separate in the gradebook GUI, but can be easily “combined” by choosing an aggregation algorithm from a list (such as “Sum, Mean, Median, etc ...), plus “Keep X high, Drop X Low” together with scaling/shifting via the Multiplication and Plus factors etc.

Calculated grade items

A calculated grade item can operate on any arbitrary other items. It is not connected to any particular module.

The calculated values are re-calculated and stored in the grade_grades_final table whenever any of the values in the source items is changed. To avoid doing this for every grade_post event, we just set the “needsupdate” field on the relevant calculated grade item columns, and recalculate those at the next display time.

Displaying the grades to ordinary participants

The module takes responsibility for displaying grades within the module (to a student, say). Normally they would use internal tables to do so. However, there is a grade_get_grades() function available for modules to query the core gradebook for grades (if required).

For full display of grades in a whole course say, the student uses the same link as teachers use to access the gradebook. However, due to their different permissions they will only have access to specific reports. By default this is the “singleuser” report which only shows their own grades and has very few configuration options.

Report plugins

All the main interface of the gradebook will be implemented as report plugins. Each plugin is fully responsible for defining the interface between the header and footer. They can even define their own capabilities and extra tables if the core tables are not enough, as they'll have a full /grade/report/xxxx/db directory.

Each report will need to define one capability to allow people to see that report, so that admins have control over who can see what reports. For example, the student interface can be a totally separate report plugin.

This allows for the widest flexibility and safety in how grades are presented.

Default report interface

This interface will be what teachers see by default, and will subsume everything the current interface (in Moodle 1.8) does.

Some snippets of functionality:

Overall it's a grid, with student names down one side and grade items along the top.

Teachers can type “straight into” the form using AJAX or fallback to forms. No popup menus for values.

Columns will be able to be collapsed together by grouping them into categories. Grades for categories can be calculated via various means.

“Eye-cons” on the columns and checkboxes by every grade (this bit possibly controlled with a switch) allow hiding by category, by column, by individual grade.

Textual notes can be added to each grade for more info. These show up to students as well. Later on we can support customisable shorthand codes to make data entry quick (eg type 'ab' for absent, or 'nge' for not good enough).

Export plugins

Coming soon ...

Import plugins

Coming soon ...

Capabilities and Permissions

moodle/course:viewcoursegrades

moodle/user:viewusergrades

moodle/grade:viewhidden

moodle/grade:editlocked

moodle:site/accessallgroups

Interface Mockups

Coming soon ...

OLD STUFF BELOW

This page is to create a specification for the next generation of Moodle gradebook. If you edit it, please try and keep modifications under the correct headings.

Centralising data

Currently (in all versions up to 1.6) Moodle uses a "pull" model for grade collection. That is, at the time a user requests a view of the gradebook, each activity module is polled for grades and either returns grades or it doesn't. These are then massaged into the gradebook view.

Problems with this are bad performance and less flexibility.

In the next version, the modules must "push" their grades to the central gradebook at the time grades are changed. For example, if a student completes a quiz, the quiz module should call a central function such as grade_store() with courseid, userid, moduleid, column name, grade type and grade. This data gets stored in a central table (like grades).

Modules can store grades internally as well if they want to (for efficiency and reports etc). All storing of grades centrally should take place via the API function: modules must not manipulate the grades table directly.

The API should allow for locking grade items. This should allow modules to lock the grade from being edited via the gradebook, and allow the gradebook to lock modules from changing grades. This will be useful so that the instructor can set the gradebook to "locked" for an item and then say the quiz module cannot change the grade for a student if they take the quiz again.

An option in the gradebook should allow grades to be retained even if the activity that generated it has been deleted. If this is turned off, then when displaying the columns, a clean-up function should check the course format, and remove columns belonging to obsolete activities.

For backward compatibility, a Moodle cron trigger should cause the gradebook to periodically look in activity modules for an old-style xxxx_grades() function, then call that function to extract and cache grades locally.

Grade status should be easily discernable at a glance in the gradebook possible grade item status include: incomplete, excluded, complete (not graded), graded. This will help with creating conditional activities, and the features required by conditional activities should be considered.

There should be a history of grades and how they were changed. There should be a clear history of how a grade was changed information should include: when it was changed, who (or what) changed it, comments.

Display API

Modular display plug-in sytem. Will allow alternate viewing methods of grade information. eg. Alternate display methods, statistics, or Graphing plug-ins.

Import/Export API

A great need is to connect grades to external systems, such as Student Information Systems.

A modular plugin system will be added that can export and import grades to and from external systems. The plugin system should allow for grades to be imported or exported at the users request or via cron. The plugins will have access to necessary data and will be responsible for choosing the correct data and proper location of information. Consider if XML with XSLT is a viable solution.

To facilitate this as well as other aspects of grading there will be "gradebook only" grades. That is, there will be grades that appear only in the gradebook, they will not show up through other aspects of the course eg. in sections, or as assignments.

Other ideas

Grades should be calculated as an ongoing grade and an overall grade. That is, the ongoing grade should calculate a grade for only those grade items that have been attempted or completed, and the overall grade should calculate a grade for everything in the gradebook. This is really for informational purposes, but gives a student a sense of "where they are now" in a course.

Existing functionality of the gradebook will remain; weighting, categories, curving/scaling, extra credit, drop the 'X' lowest.

Existing category functionality will be extended to allow nesting and active/inactive status. This will allow for greater flexibility of calculations.

Grading scheme or plug-in interface This will allow for greater flexibility in calculating a final grade for a student. Things to consider will be statistics based grade scales eg. "+/- X Standard deviations", must pass with X or better in category Y or no grade. This will also allow for alternatives to the USA High School/College format so that the gradebook scheme is appropriate for other countries and grading methodologies. Consider how this may overlap with support for manual or ad-hoc calculation columns that are not connected to any activity table "calculated columns".

Better implementation of scales This may require work to the scales engine. But there should be better gradebook support for scales. There are three scenarios for the gradebook with scales. 1. Entirely points (numeric grades) 2. Mixed Scales and Points 3. Entirely Scales. These three scenarios should be appropriately handled.

Security Issues There need to be sitewide security options. These will include at least the following; grades must be delivered over SSL (https), turn off grade download features so that grades cannot be downloaded, and maybe encryption of downloaded grades?

Roles Implement proper roles for the gradebook, maybe create a new role "Grader" that will only see and create individual assignments at a time and not aggregate grade data, and will not be able to change any other information.

Metacourses The option to have the gradebook collect grades from several sub-courses e.g. for most 1 or 2 year long courses in schools which simply don't fit into a single moodle course conveniently.

Deadlines Currently, its not easy to see whether work has been handed in on time just by looking at the gradebook. A simple colour coding scheme for the backgrounds such as green=on time, red=late would be great. Tooltips with data on the time it was submitted would help. A way to quickly message all those who have not done a certain piece of work would also be useful.

Marking There is no way currently to glance at the gradebook and see who has done work which has not been graded yet. maybe a different colour background, or a special symbol to show that work has been submitted. It would also be excellent to be able to grade work directly from the gradebook e.g. by clicking on the box with this symbol or colour and being taken straight to the grading dialogue for that students piece of work. This would make mopping up late pieces of work many times easier and would replace some of the functionality of the excellent marking block for 1.6.

See also

• Using Moodle New gradebook for Moodle forum discussion