Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Grades: Difference between revisions

From MoodleDocs
Line 127: Line 127:
|varchar(255)  
|varchar(255)  
|  
|  
|Arbitrary idnumber provided by the module responsible  (must be defined)
|Arbitrary idnumber provided by the module responsible  (must be defined and unique)
|-
|-
|gradetype  
|gradetype  

Revision as of 07:22, 5 June 2007

>>Still under construction<<

There is an ongoing discussion about this spec here

Executive Summary

The gradebook mechanisms must be rebuilt to:

  1. Improve performance and scalability - All grades from throughout the system will be pushed to a central system of tables. This means reports based on grades can be generated much faster, and the gradebook has ultimate control over the content.
  2. Improve flexibility - All aspects of the new gradebook will use simple plugin structures, namely: exports, imports and displays/reports. It is expected that the community will be very active in producing special-purpose reports analysing the basic grade data in new ways, for example, or writing plugins to transfer grades to student information systems.
  3. Allow rubrics for outcomes (aka standards,competencies,goals) - As well as numerical grades, each grading item can consist of a number of scores made on a rubric against a standard outcome statement. These can be automatically converted to a numerical grade if desired or just shown as is.
  4. Implement an event-oriented architecture - Such an architecture allows all parts of Moodle to interact with the gradebook, but particularly gradebook plugins can react instantly to grading events throughout the system.
  5. Allow arbitrary columns and derived columns - Arbitrary columns of data can be added (either manually or via import). Columns can also be automatically filled based on formulas.

Glossary

Here are some terms used in the gradebook, both in the development and the user interface. Using these terms in discussions about the gradebook will help to reduce confusion.

Term Definition
Activity An instance of an activity module (e.g. a single quiz, assignment etc...)
Calculation A formula used to calculate grades, based (optionally) on other grade items. Not the same as Calculated question types.
Category A set of Grade Items. A Category also has its own aggregated Grade which is calculated from its Grade Items. There is no limit to the level of nesting of Categories (a Category may belong to another Category). However, each Grade Item may belong to only one Category.
Course completion The concept
Event Events are a new feature of Moodle 1.9, of which the Gradebook will take full advantage. Modules that have grades to publish will define Events for which the Gradebook will listen. When an Event matches an Event Handler set up in the Gradebook, an Event Reaction will be triggered, performing any steps required (such as aggregating multiple grades, storing the grades etc...). Modules can also have their Event Handlers set up to respond to Events sent by the Gradebook (such as updating their internal grades).
Grade A Grade is a single assessment. It may be a number or an item on a scale (possibly tied to an Outcome).
Gradebook A central location in Moodle where students' Grades are stored and displayed. Teachers can keep track of their students' progress and organise which set of Grades their students will be able to see. Students see their own Grades.
Grade Item A "column" of Grades. It can be created from a specific Activity or other module, calculated from other Grade Items, or entered manually.
Grade Locks See linked section of this page
History The gradebook has its own type of log, which keeps a History of all changes made to grades.
Outcome Outcomes are specific descriptions of what a person is expected to be able to do or understand at the completion of an activity or course. An activity might have more than one outcome, and each may have a grade against it (usually on a scale). Other terms for Outcomes are Competencies and Goals. See some Examples.
Scale A scale is a set of responses from which the teacher can choose one. eg A,B,C,D,E,F

Database structures

grade_items

This table keeps information about gradeable items (ie columns). If an activity (eg an assignment or quiz) has multiple grade_items associated with it (eg several outcomes or numerical grades), then there will be a corresponding multiple number of rows in this table.

idnumber is a unique tag identifying the grade_item, useful for identifying data in exports and for referring to the grade_item in calculations. Usually it will be the same as the idnumber in course_modules but if one is not specified then it's automatically set as itemmodule.iteminstance-itemnumber (eg forum3).


Field Type Default Info
id int(10) autoincrementing
courseid int(10) The course this item is part of
categoryid int(10)
NULL
(optional) the category group this item belongs to
itemname varchar(255) The name of this item (pushed in by the module)
itemtype varchar(30) 'mod', 'blocks', 'import', 'calculated', 'category' etc
itemmodule varchar(30) 'forum', 'quiz', 'csv', etc
iteminstance int(10) id of the item module
itemnumber int(10)
NULL
Can be used to distinguish multiple grades for an activity
iteminfo text Info and notes about this item XXX
idnumber varchar(255) Arbitrary idnumber provided by the module responsible (must be defined and unique)
gradetype int(4)
0
0 = value, 1 = scale, 2 = text
grademax float(11,10)
100
What is the maximum allowable grade?
grademin float(11,10)
0
What is the minimum allowable grade?
scaleid int(10)
NULL
If this grade is based on a scale, which one is it?
outcomeid int(10)
NULL
If this grade is related to an outcome, which one is it?
gradepass float(11,10)
0
What grade is needed to pass? grademin <= gradepass <= grademax
multfactor float(11,10)
1.0
Multiply all grades by this
plusfactor float(11,10)
0.0
Add this to all grades
sortorder int(10)
0
Sorting order of the columns
hidden int(10)
0
1 is hidden, > 1 is a date to hide until (prevents viewing)
locked int(10)
0
1 is locked, > 1 is a date to lock after (prevents update)
deleted int(10)
0
1 means the associated module instance has been deleted
needsupdate int(10)
0
If this flag is set, then the whole column will be recalculated
timecreated int(10) The first time this grade_item was created
timemodified int(10) The last time this grade_item was modified

grade_categories

This table keeps information about categories, used for grouping items. An associated grade_item will be maintained for each category to store the aggregate data.

Field Type Default Info
id int(10) autoincrementing
courseid int(10) The course this grade category is part of
parent int(10)
NULL
Parent grade_category (hierarchical)
depth int(10)
0
How deep is this category from the highest level (0,1,2)? Used for table display purposes.
path varchar(255) Shows the path as /1/2/3 (like course_categories)
fullname varchar(255) The name of this grade category
aggregation int(10)
0
A constant pointing to one of the predefined aggregation strategies (none, mean,median,sum, etc)
keephigh int(10)
0
Keep only the X highest items
droplow int(10)
0
Drop the X lowest items
timecreated int(10) The first time this grade_category was created
timemodified int(10) The last time this grade_category was modified

grade_calculations

This table describes the calculated grade_items in more details.

Field Type Default Info
id int(10) autoincrementing
itemid int(10) The grade_item this relates to
calculation text
NULL
Formula describing how to derive this grade from other items, referring to them using [idnumber] ... eg something like:sin(square([XXXXX])) + [YYYY]
timecreated int(10) the time this calculation was first created
timemodified int(10) the time this calculation was last modified
usermodified int(10)
NULL
the userid of the person who last modified this calculation

grade_grades_raw

This table keeps individual grades for each user and each item, exactly as imported or submitted by modules. The grademax/min and scaleid are stored here to record the values at the time the grade was stored, because teachers might change this for an activity! All the results are normalised/resampled for the grade_grades_final table.

Field Type Default Info
id int(10) autoincrementing
itemid int(10) The item this grade belongs to
userid int(10) The user who this grade is for
gradevalue float(11,10)
NULL
If the grade is a float value (or has been converted to one)
grademax float(11,10)
100
The maximum allowable grade when this was created
grademin float(11,10)
0
The minimum allowable grade when this was created
scaleid int(10)
NULL
If this grade is based on a scale, which one was it?
timecreated int(10) the time this grade was first created
timemodified int(10) the time this grade was last modified
usermodified int(10) NULL the userid of the person who last modified this grade

grade_grades_final

This table keeps individual grades for each user and each item/category– they have undergone all scaling and other calculations, and are ready for display. This table is effectively a cache and values are rebuilt whenever source values change. The gradevalue values are all normalised to the max/min or scaleid as defined in the grade_item table. Note that the gradevalue for a scale-based item may be non-integer! It needs to be rounded on display.

Field Type Default Info
id int(10) autoincrementing
itemid int(10) The associated grade_item these grades belong to
userid int(10) The user who this grade is for
gradevalue float(11,10)
NULL
If the grade is a float value (or has been converted to one)
hidden int(10)
0
1 is hidden, > 1 is a date to hide until
locked int(10)
0
1 is locked, > 1 is a date to lock after
exported int(10)
0
0 is not exported, > 1 is the last exported date
timecreated int(10) the time this grade was first created
timemodified int(10) the time this grade was last modified
usermodified int(10) NULL the userid of the person who last modified this grade

grade_grades_text

This table keeps additional textual information about each individual grade, whether it be automatically generated from the module or entered manually by the teacher. It's here separate from the all-numeric grade_grades for database efficiency reasons.

Field Type Default Info
id int(10) autoincrementing
gradesid int(10) The exact grade in grade_grades_raw this corresponds to
information text
NULL
Further information like forum rating distribution 4/5/7/0/1
informationformat int(10)
0
Text format for information
feedback text
NULL
Manual feedback from the teacher. Could be a code like 'mi'.
feedbackformat int(10)
0
Text format for feedback
timecreated int(10) the time these entry was first created
timemodified int(10) the time this entry was last updated
usermodified int(10)
NULL
the userid of the person who last modified this entry

grade_outcomes

This table describes the outcomes used in the system. An outcome is a statement tied to a rubric scale from low to high, such as “Not met, Borderline, Met” (stored as 0,1 or 2)

Field Type Default Info
id int(10) autoincrementing
courseid int(10)
NULL
Mostly these are defined site wide ie NULL
shortname varchar(255) The short name or code for this outcome statement
fullname text The full description of the outcome (usually 1 sentence)
scaleid int(10) The recommended scale for this outcome.
timecreated int(10) the time this outcome was first created
timemodified int(10) the time this outcome was last updated
usermodified int(10)
NULL
the userid of the person who last modified this outcome

grade_history

This table keeps track of changes to grade_grades_raw. Using this it should be possible to reconstruct the grades at any point in time in the past, or to audit grade changes over time. It should be quicker to use this table for that, rather than storing this information in the main Moodle log.

Note we use itemid and userid as a key rather than grade_grade id, just in case one of the things we want to log here is the deletion of a grade entirely.

Field Type Default Info
id int(10) autoincrementing
itemid int(10) The grade_item the grade is from
userid int(10) The user that the grade belongs to
oldgrade float NULL The original grade before the change
newgrade float NULL The new grade after the change
note text NULL An optional note about why this change was made
howmodified varchar(255) manual What caused the modification? manual/module/import/...
usermodified int(10)
NULL
The user id of the person who made the change
timemodified int(10) The exact time this change was made

Overview of module communication

Modules will often store internal copies of grades for calculations or for showing grades to students etc, but these are purely internal. The gradebook system will never access these directly.

A new feature in Moodle 1.9 is a simple event handler. The new gradebook will be the first feature in Moodle to take advantage of events, and all communication between the gradebook and the modules will take place via event messages. See Events for full details.

There are two ways that data is transferred between the module and the gradebook:

1. Modules post grade events
Whenever a new grade is created, the module should post a grade_updated event so that the gradebook can pick it up. As long as that column in the gradebook is not “locked” then the new grades will be entered (overwriting any existing ones).
2. Gradebook posts grade events
Whenever a grade is altered in the gradebook (for example, a teacher may want to change an assignment grade), then an grade_updated_external event is posted so that the module can pick it up. If the module implements this “reverse” function then the internal grade in the module can be changed.

Backward compatibility with Moodle 1.8 and earlier

For backward compatibility with old third-party modules, we will add a new function called grade_grab_legacy_grades() to admin/cron.php to search all mod/xxx/lib.php files for functions named xxx_grades().
These legacy functions will be called to extract all the grades for all the activities in each course. Once the data is extracted, the grade_updated event is triggered as usual and copy/upgrade the data in the gradebook tables.
A similar function will be added to upgraded modules to trigger a “mass copy” of all the existing grades into the new gradebook. This will be useful during the upgrade from 1.8 as well as being available in the normal gradebook interface to “refresh” a column from primary data.

Core API functions

Even though most of the communication will take place via events, there are a few core functions in lib/gradelib.php that will be useful for modules:

grade_get_items()

eg grade_get_items($courseid, $itemtype=NULL, $itemmodule=NULL, $iteminstance=NULL, $itemname=NULL, $itemnumber=NULL, $idnumber=NULL)

For extracting all the information about what grading items are attached to something. For example, an assignment may want to retrieve all the grade_items for itself, and get three outcome scales in return. This will affect the grading interface.

grade_create_item()

Create a new grade_item. This function would be called when a module is created or during upgrade.

grade_create_category()

eg grade_create_category(fullname, array of items, aggregation, etc ...) For a given set of items, create a category to group them together (if one doesn't exist yet). Modules may want to do this when they are created. However, the ultimate control is in the gradebook interface itself.

grade_is_locked()

eg grade_is_locked($itemtype, $itemmodule, $iteminstance, $itemnumber=NULL, $userid=NULL)

This function will tell a module whether a grade (or grade_item if $userid is not given) is currently locked or not. This is a combination of the actual settings in the grade tables and a check on moodle/course:editgradeswhenlocked. If it's locked to the current user then the module can print a nice message or prevent editing in the module. If the grade_item is locked, this setting overrides any grades under that grade_item. However, if it is unlocked AND a $userid is passed to the function, the state of the specific grade will be returned. (info)

grade_upgrade_final_grades()

grade_upgrade_final_grades($courseid=NULL, $gradeitemid=NULL)

Updates all grade_grades_final records for each grade_item matching the given attributes. The search is further restricted, so that only grade_items that have needs_update == true or that use calculation are retrieved and used for the update. The function returns the number of grade_items updated (NOT the same as the number of grades_final updated!).

Dealing with multiple grades

Modules will often produce several grade items. This may be several attempts, say, like the quiz module, or more commonly, ratings based on several outcomes. It is up to the module whether these grades are aggregated BEFORE sending to the gradebook as a single number, or are sent in raw form to the gradebook.

Pre-aggregration makes the most sense for a quiz, say, which already implements algorithms to calculate a single number for each student. In such a case there will only be one grade item from a particular quiz.

For anything that uses outcomes with grades against multiple rubrics (such as an assignment), it would make the most sense to just pass all the results through to the gradebook as individual grade items.

If the gradebook receives multiple grade items from a module, then they are automatically grouped together in a unique grade category (with the same name as the module instance). This is simply done by checking/creating the category whenever a second item from the same instance is being added. All the grading items will by default appear separate in the gradebook GUI, but can be easily “combined” by choosing an aggregation algorithm from a list (such as “Sum, Mean, Median, etc ...), plus “Keep X high, Drop X Low” together with scaling/shifting via the Multiplication and Plus factors etc.

Calculated grade items

A calculated grade item can operate on any arbitrary other items. It is not connected to any particular module.

The calculated values are re-calculated and stored in the grade_grades_final table whenever any of the values in the source items is changed. To avoid doing this for every grade_post event, we just set the needsupdate field on grade_items every time grades are added or changed.

At the next display time or export time (ie whenever grades_grades_final is accessed) we do this:

If any current grade_items are flagged as needsupdate then 
# for each grade_item with the flag, update all grade_grades_final for them and remove the grade_item flag
# for each grade_item with a calculation, update their grade_grades_final

The format for the calculations may as well use the same format as Excel and Openoffice, as that is what most people are probably used to.

eg:  MEAN(quizstart1, quizend1) + assignment1 + 20.0

Adjustment of grades

Grade_item contains optional rules for adjusting the raw grade before it is cached into a final grade. These rules are processed BEFORE the calculation discussed above, because they may include reprocessing the grade's place within a different scale or grade range.

Adjustment to a new range

A raw grade may have been initially calculated within a module inside a range of 0 to 100 (grademin - grademax). However, the grade_item may be given a different grademin and grademax. For example, it could be 30 to 70. If you take a raw value of 30 on the original range (i.e. 30%), after adjusting this score for the new range, you should obtain 42, not 30 (30 being the equivalent of 0 on the previous range). This calculation takes place before the multfactor and plusfactor are applied, and before the calculation formula is applied.

Adjustment to a new scale

A raw grade may also have a scale value, which always starts at 0 (for the first item in the scale), and has a maximum set by the number of items in the scale. Because this value is always an integer, the above calculation is performed in addition to a rounding up to the nearest integer. Also, multfactor and plusfactor are both ignored, because using them makes no sense in the context of a scale value.

Displaying the grades to ordinary participants

The module takes responsibility for displaying grades within the module (to a student, say). Normally they would use internal tables to do so. However, there is a grade_get_grades() function available for modules to query the core gradebook for grades (if required).

For full display of grades in a whole course say, the student uses the same link as teachers use to access the gradebook. However, due to their different permissions they will only have access to specific reports. By default this is the “singleuser” report which only shows their own grades and has very few configuration options.

Locked grades

Both whole columns and individual grades can be locked in the gradebook, via the "locked" field. Teachers may want to do this to prevent further changes from the modules, or from other teachers. When a grade is locked, any grading events that might affect that grade are ignored. When the graded is unlocked, an "unlock" event is triggered, which allows a module to handle it by sending the latest new grades back.

In the main GUIs the lock toggling will be achieved by clicking on a little padlock icon beside each entry or column.

Manually modified grades

Grades can be manually modified (overriden) in the gradebook. When this is done to grade_items derived from a module's grades, and the new value is different from the original one, then the field will automatically be given a locked status (it can of course be unlocked immediately if one wishes). This is to prevent new grading events from changing the manually entered values unexpectedly.

Clean up after old activities

By default columns (grade items) will remain in the gradebook even when activities are deleted. A manual 'clean-up' button can delete old columns if required.

When you click the 'Clean up' button, it shows you a list of columns that should be deleted, with a checkbox next to each one (on by default, but you can turn it off if you want to keep that column still) and then another button click to really delete the marked columns.

Otherwise there will be a site option to do this automatically whenever activities are deleted.

Logging

As well as datestamps and userids in the tables, all grading events will be logged in case an audit needs to be made.

Security Issues

For security an option to force SSL for the gradebook might be good.

Overall grade

Having an overall grade handy allows us to know exactly when a course is "finished". Exactly how do we specify the "overall" grade? It's a special case, and might not always be just the sum.

The best solution is probably to have a special calculated column (grade_item) for every course that can not be deleted. It's made special because the itemtype is 'total' (one per course). The calculation formula should be updated automatically whenever grade_items are changed in that course to be the "sum" of all the grade_items in the course.

However, if the calculation is edited manually then we should lock the calculation formula. We could do this by setting the iteminfo field to something. Clearing the calculation to an empty string could force it back to an automatic calculation formula.


Report plugins

All the main interface of the gradebook will be implemented as report plugins. Each plugin is fully responsible for defining the interface between the header and footer. They can even define their own capabilities and extra tables if the core tables are not enough, as they'll have a full /grade/report/xxxx/db directory.

Each report will need to define one capability to allow people to see that report, so that admins have control over who can see what reports. For example, the student interface can be a totally separate report plugin.

This allows for the widest flexibility and safety in how grades are presented.

Default teacher interface

This interface will be what teachers see by default, and will subsume everything the current interface (in Moodle 1.8) does.

Some snippets of functionality:

Overall it's a grid, with student names down one side and grade items along the top.

Teachers can type “straight into” the grid using AJAX or fallback to forms. No popup menus for values.

Columns will be able to be collapsed together by grouping them into categories. Grades for categories can be calculated via various means.

“Eye-cons” on the columns and checkboxes by every grade (this bit possibly controlled with a switch) allow hiding by category, by column, by individual grade.

Textual notes can be added to each grade for more info. These show up to students as well. Later on we can support customisable shorthand codes to make data entry quick (eg type 'ab' for absent, or 'nge' for not good enough).

A groups menu allows the teacher to switch between showing EACH of the groups they have access to, or ALL the groups they have access to.

All grade items will link to modulepath/grade.php?id=44 which will work out what the current person should be allowed to see and either redirect them to the correct page or just show them immediately. This copes with situations like the quiz, say, where we want editing teachers to go to the detailed reports there while students just see their own grade or whatever the quiz is set to show.

User preference to SWITCH between showing raw grades, percentage grades, or both, or grade letters (A/B/C etc).

Settings for grade letters not only define the transformation from percentage to grades, but also the transformation from letters to grades (in case the teacher edits some of the letter grades).

Categories are shown above the headings for each column. Clicking for more info on a category will just show the category with a summary column showing total/average for just that category (PLUS the summary column for the whole course).

All columns should be sortable up/down.

At the bottom of each column is a row with the mean course score. If in groups mode, then add ANOTHER row with just the group mean. Add the number of grades used in brackets. eg 56% (11). When the report is paged, these means are still for the whole course/group (not the page!)

Default student interface

This interface will be what students see by default:

Some snippets of functionality:

Invert the grid to show one item per row, with the total/average at the bottom.

Use second/third columns to show categories.

Include ranking score in another column.

Include class mean scores in a final column for comparison.

No editing functionality.

Export plugins

The API for these is extremely simple. Each export plugin should occupy a directory under /grade/export/xyz and needs to provide only an index.php file as a the primary interface. This file can accept parameters that produce a subset of the grades:

  • courseid
  • groupid
  • categoryid
  • itemid
  • userid

The index.php can choose to simply dump something (eg an Excel spreadsheet) and return, or it can show an interface for further options and selections. Ideally even if it dumps something it should also show some feedback about what was exported to make it easier to double check what was sent, but this is up to the plugin to implement.

Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin (eg by checking access permissions). If this function returns false then the export method won't appear in the main gradebook interfaces.

The export plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an export plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.

Import plugins

Each import plugin should occupy a directory under /grade/import/xyz and needs to provide only an index.php file as a the primary interface. This file just accepts a 'courseid' parameter.

The index.php will show an interface for further options and selections.

Further if the plugin contains a lib.php and an xyz_visible() function then the gradebook can check this function to make sure the current user can see/use the plugin. If this function returns false then the import method won't appear in the main gradebook interfaces.

The import plugins are a "full" Moodle plugin, so it can have tables, cron functions, event handlers, capabilities etc. For example, an import plugin might not even have a GUI (ie xyz_visible() is always false and/or index.php doesn't exist) - it might operate solely off cron or event triggers.

The entire import must be treated as one operation. If it fails, then no records should be imported and the database is unchanged.

Some sample import plugins are:

Import from CSV

Accepts an upload of (or URL to) a CSV file. Multiple options describe how to process the file, which columns to add etc. These should be sticky, and retained via user preferences to make future imports easier.

Import from XML

Accepts an upload of (or URL to) an XML file with this kind of format (from OU).

<results batch="[someuniqueimportnumber]">
    <result>
        <state>['new' or 'regrade']</state>
            <assignment>[idnumber]</assignmentid>
            <student>[studentid]</student>
            <score>[score]</score>
        </result>
        <result>
            <state>['new' or 'regrade']</state>
            <assignment>[idnumber]</assignmentid>
            <student>[studentid]</student>
            <score>[score]</score>
        </result>
        [...]
</results>

Capabilities and Permissions

  • moodle/course:viewowngrades - view your own grades in a course
  • moodle/course:viewallgrades - view all grades in a course
  • moodle/course:downloadallgrades - download all grades from a course
  • moodle/user:viewusergrades - see all grades from a user
  • moodle/grade:hide - be able to hide cells
  • moodle/grade:lock - be able to lock cells
  • moodle/grade:viewhidden - see grades that are hidden (eg from students)
  • moodle/grade:editlocked -
  • moodle:site/accessallgroups

Interface Mockups

Nicolas is working on a mock-up interface that you can play with:

   Mock Gradebook

It's changing rapidly in response to your feedback.

Development Tasks and Tracking

See [https://tracker.moodle.org/browse/MDL-9137 MDL-9137] for the full details.

See also