Question engine

Moodle has a powerful question engine with a modular structure to allow question type plug-ins. The question engine is responsible for rendering the questions and for processing student responses. It is used by the quiz module and it is planned that in future it will be used by the Lesson and other modules.

Historically the question engine started as a part of the quiz module. Only since Moodle 1.6 is it a separate core component of Moodle that can be used by any other Moodle component or module. During this restructuring the code was moved from mod/quiz/ to question/ and the tables and functions were renamed. Wherever the old table or function name contained 'quiz_' the new one will contain 'question_'

Terminology

When talking about the question engine there are certain terms that can cause confusion because they can be used with different meanings. In Moodle we have adopted a certain terminology that will be explained below.

Questions

A question is the set of definitions (question name, question text, possible answers, grading rules, feedback, etc.) that constitute a reusable assessment item. So it includes much more than what one would in everyday language call a question. In the terminology of the QTI specification a 'question' is more appropriately called an assessment item or just 'item' for short.

There are different types of questions, like for example multiple-choice questions or numerical questions. These are referred to as question types in Moodle.

Since version 1.5 Moodle is able to handle so-called Adaptive questions, also known as 'adaptive items' in QTI speak. These are questions that interact with the student by going through several states depending on the student responses. For example a complicated mathematical question that is answered incorrectly, but is likely to be incorrect because of a common mistake, could provide the user with a hint towards this mistake, apply a penalty and allow a second attempt at this question. Quizzes can be run in 'adaptive mode', in which case Moodle provides buttons to mark each question individually.

Answers

In Moodle the term 'answer' is used exclusively for the teacher-defined answers of a question. It is easy to get confused between these teacher-defined answers and the answers that the students actually give. We have therefore adopted the convention to refer to the student-supplied answers as 'responses' and to reserve the term 'answers' to apply to teacher-defined answers. In question types that rely on teacher-supplied answers these are used in the grading process by comparing them with the student responses. Of course not all question types use teacher-defined answers but use some more intelligent way to process the student responses.

Perhaps one should also stress that 'answer' is not always used in the sense of 'correct answer'. For example every choice in a multiple-choice question is referred to as an answer. Other systems use the term 'distractor' for wrong answers.

In Moodle we always use the term 'responses' to refer to the students' responses to a question. This term is always used in plural, although for some questiontypes there is only one possible response.

There is unfortunately, for historical reasons, one exception to the above rule: The question_states table has a field 'answer' whose purpose it actually is to hold the student's responses.

Attempts

In Moodle the term 'attempt' is used in the sense of "Attempt at a quiz" (or another activity involving questions). Depending on the quiz settings, a student may be allowed several attempts at a quiz. An attempt is finished when the student clicks on the corresponding button on the attempt page. Students do not have to complete an attempt in one visit. They can navigate away from the quiz page and return later to continue the same attempt.

Each module that uses the question engine should hold its own data for the attempts in its own tables. When the module calls the question engine functions it is often expected to pass an attempt object.

Within one and the same quiz attempt a student may make several attempts at answering a particular question, at least if the questiontype allows it and the quiz is set up in adaptive mode. These will always be referred to as 'question sessions' or sometimes 'attempts at a question, never just as 'attempts'.

Sessions, States, Events

When a new attempt is started, a new session is started for each question. So in a sense a session is for a question what an attempt is for a whole quiz. A question session lasts no longer than an attempt and for each question there can only by one session within one attempt.

Moodle allows the student to interact with a question repeatedly within one session and each such interaction leads to a new state. The first state is created when the session is created. A new state is then created when a student saves, validates or submits an answer or .... The student's responses and, if appropriate, the results of response processing (grading) are stored in the new state that gets created.

The type of event that led to the creation of a particular state is saved along with the state. The types of events currently used are:

open: A new session has just been created and this is the opening state. Usually it doesn't hold student responses yet (except where a quiz attempt is based on a previous attempt because the 'attemptonlast' option is set).
save: The student has clicked the save button.
validate: The student has asked for his responses to be validated. This means it is checked that they are valid responses. In the case of mathematical questions which requires the input of a mathematical expression in some linear format the question type may want to display the validated result back to the student in typeset form. Similar things may apply to other subject-specific question types. If a student response is found to be invalid the student is told so but no penalty is applied. The invalid response is stored with the state.
grade: The student has pressed the submit button. The grade is calculated and stored with the attempt.
duplicategrade: The student has pressed the submit button but the response to this question has not actually changed. This happens a lot in quizzes with several questions on one page where the student may have changed the responses for one question only. I believe that states created by this type of event are not stored in the database.
close: The last state in a session which is now closed. Currently a session closes only when the attempt closes, either because the student requests it or because the timelimit elapses.

There are now plans to introduce another event type

submit: The student has submitted his responses for grading but grading has not yet taken place. This will be used by teacher-marked question types like the essay questions for example.

Code documentation

The code is documented according to PHPdocumentor conventions. The explanations here in the wiki are meant to complement this.

Inline comments should be used liberally in the code. The following conventions make it easier to search for comments with special meaning:

use TODO in comments about things that need to be done
use ??? in comments that are questions about the code

Code is organised into packages

Package questionbank - code that relates to the questionbank system
- Subpackage questiontypes - code that relates to the question types, including the question type base class.
- Subpackage importexport - code that relates to importing and exporting questions.

Generally, the base classes default_questiontype and qformat_default should contain most of the documentation, since all question types should follow the same interface. There is no value in repeating the same information on all the various subclasses. The subclasses should concentrate on describing any new methods they have that do not come from the base class.

See the PHPdocumentor manual for more advice on writing good comments.

Moodle PHPdocs can be seen here: http://phpdocs.moodle.org/.

API

The library lib/questionlib.php contains all functions that need to be available to any module wanting to use questions (this is new in Moodle 1.6, in Moodle 1.5 this was part of mod/quiz/locallib.php). Loading this library instantiates all questiontype classes by loading the questiontype.php files

A description of the API still needs to be written. Also lib/questionlib.php should be cleaned up a bit to distinguish between the API functions and the helper functions.

Organisation

The default questiontype class is defined in question/type/questiontype.php (in Moodle 1.5 this was still in mod/quiz/locallib.php). The individual questiontypes extend this class in their own questiontype.php file. For documentation of the questiontype classes one should often look at the documentation of the default question type because much of the documentation that is in the default class is not repeated in the other questiontype classes.

While questiontypes are realized as classes, the question engine is not written in a truly object-oriented way. Instead it follows the Moodle model of using objects mostly only as alternatives to arrays to hold database records. So none of the question, attempt, and state objects that play a central role in the module have any methods. Only the questiontype objects have methods. Strangely enough the quiz module instantiates one object of each questiontype class at the start and then reuses their methods for the different questions. If one is used to the Moodle way of programming then this is easy enough to handle.

Objects and data structures

Key to understanding how the question engine works is to understand how the different kinds of object work together. The most important ones are:

Questions
Attempts
States

Questions are data created by the teacher. Attempts and States are data created by the student when interacting with a quiz.

The Moodle quiz module allows students to make several attempts at a quiz. Data about such an attempt is stored in an attempt object. The Moodle question engine was optimized to deal with such attempts and therefore every module wanting to use the question engine also has to implement the attempt object and pass it to many of the question engine functions. The attempt object holds for example information about how the quiz was randomized for this attempt and the ordering of the questions and answers.

Moodle allows students to interact repeatedly with a single question. So for example the student might initially just save an answer, later mark it, then correct it if it was marked incorrect. When the student first views a question within a particular attempt a question session and the first question state is created. Each time the student interacts with the question a new question state is created. So states are indexed by user id, attempt id and question id.

Database structure

All this data needs to be kept in Moodle's database. How this is achieved is explained on a separate page about the Quiz database structure, which also contains a useful diagram.

As is customary in Moodle, most runtime objects simply represent the data from a particular database record. So for example a $quiz object has fields corresponding to all the fields in the quiz table. In some cases the objects have some additional fields that are added at runtime. This is particularly the case for $question and $state objects. These additional fields are also described on the page about the Quiz database structure. Many functions that are used to process these objects make use of the additional fields and it is therefore necessary to use the correct functions for creating these objects.

Runtime objects

Some objects used by the quiz module are purely runtime object and do not correspond to a database table. The structure of these objects is explained in detail on a separate page about the Quiz runtime objects.

The main script of the quiz module is attempt.php which will have to deal with all these objects. Studying the explanation of attempt.php is therefore a good way to start to study the quiz module code.

Response storage

The student's responses to a question are stored in

$state->responses

. Questiontypes are completely free to implement the storage mechanism of their responses (and other state information) the way they want. Still, the standard questiontypes all follow a similar model. The default storage model and the questiontype specific variations are explained below. The flexibility for the questiontypes to choose their response storage mechanism freely and to convert from the storage model to the runtime model is provided by a set of three functions, which allow to initialise the runtime

$state->responses

field, to convert from the runtime to the storage model and vice versa:

create_session_and_responses()

Initializes the $state object, in particular the

$state->responses

field

restore_session_and_responses()

Loads the question type specific session data from the database into the

$state

object, in particular it loads the responses that have been saved for the given

$state

into the

$state->responses

field.

save_session_and_responses()

Saves the question type specific session data from the $state object to the database. In particular, for most questiontypes, it saves the responses from the

$state->responses

to the database.

The generic quiz module code saves the contents form the

<nowiki>$state->responses['']</nowiki>

field to the answer field in the quiz_states table and also automatically restores the contents of this field to

<nowiki>$state->responses['']</nowiki>

. This means that any questiontype, which only expects a single value as its response can skip the implementation of the three methods described above. All questiontypes that have multiple value responses need to implement these methods. The default questiontypes handle this problem by serializing/de-serializing the responses to/from the answer field in the quiz_states table. However, it is also possible (and may be better practice) to extend the quiz_states table with a questiontype specific table, i.e. take the id of the quiz_states record as a foreign key in the questiontype specific table. Because the value of

<nowiki>$state->responses['']</nowiki>

is set to the value of the answer field, questiontypes that serialize their response need to overwrite (in

save_session_and_responses()

) whatever value the generic code set this field to with their serialized value (usually achieved with a simple set_field). In the method

restore_session_and_responses()

the serialized value can be read from

<nowiki>$state->responses['']</nowiki>

because this is where the value from answer field of the quiz_states table has been moved. Care needs to be taken that this array value is then unset or the whole array overwritten, so that the array does not accidentally contain a value with the empty string index.

Response processing

The runtime model for responses dictates the structure of the $state->responses array. Starting with the names of the form elements this section goes through the relevant processing steps and thus attempts to clarify why the keys of the $state->responses array can differ for different questiontypes; even more, it explains how the array keys are chosen and set.

Although it may initially seem strange to start with the naming convention of the form fields, the reason for this will become clear later on. The controls (i.e. the form fields) of a question get printed by the method

print_question_formulation_and_controls()

. The convention only dictates that the name of the control element(s) must begin with the value of

$question->name_prefix

. The

$question->name_prefix

is a string starting with "resp" followed by the question id and an underscore, e.g.

resp56_

. In the default case, when there is only a single control element (this includes the case of a list of equally named radio buttons), no postfix is appended to the name prefix. For questiontypes that allow or require multiple form elements, an arbitrary string can be appended to the name prefix to form the name of these form elements. The postfix must not include any relational data (i.e. ids of records in the quiz_answers table), because this can lead to problems with regrading of versioned questions. After the printing of the question the server only sees it again when it is submitted. So the submitted data will contain several values indexed by strings starting with

respXX_

. Upon submission, the function

quiz_process_responses()

is called, which assigns the submitted responses to the state of the question with id XX, using the postfix (i.e. everything after the underscore) as array keys. In the default case with only one control element the name only consists of the name prefix. This explains why the default index of the

$state->responses

array is the empty string. The value of each array element is obviously the value that was submitted by the form, basically a raw response. The function

quiz_process_responses()

in turn calls the questiontype specific method

grade_responses()

to assign a grade to the submitted responses and

compare_responses()

to determine whether the response was identical to the previous submission and to avoid regrading the same responses repeatedly. These questiontype specific functions need to be aware of the expected keys of the

$state->responses

array. Finally, the methods

restore_session_and_responses()

and

save_session_and_responses()

also need to know the questiontype specific layout of the

$state->responses array

and restore or save the information, e.g. by converting from or to the data representation.

Question types

Making a new question type
- import/export
Question types
Code in CVS
Cross-referenced Code

The quiz module is itself modular and allows question type plug-ins. For each question type there should be a page, accessible via the menu at the right, which provides at least the details about

Database tables
Response storage
Question options object
State options object

Question type code flow

It is hoped that Moodlers will contribute a lot of non-core question types in the future. For this, see Question type plugin how to.

Grades

The handling of grades is a bit complicated because there are so many different grades around that get rescaled and combined in various ways. This section should summarize how this is done and why.

The following grade fields are being used:

$question->defaultgrade

This is the default value for the maximum grade for this question. This is set up when the teacher creates the question and it is stored in an int(10) field in the quiz_questions table. However when the question is actually used in a particular quiz the teacher can overrule this default and this is stored in:

$question->maxgrade

This is the maximum grade that the teacher has assigned to this question in the context of the current quiz. This is by default equal to $questions->defaultgrade but the teacher can change this when editing the quiz. In the database it is stored in an int(10) field in the quiz_question_instances table.

$question->penalty

$state->raw_grade
$state->grade
$state->penalty
$state->sumpenalty

$attempt->sumgrades

The maximum grades set by the teacher, $question->defaultgrade and $question->maxgrade, are integers. All student-obtained grades are in principle floating point numbers. For historical reasons they are stored in the database as varchar(10) fields. Care has to be taken when writing to the database to make sure all grades are correctly rounded and squeezed into a string of no more than 10 characters, otherwise the writing to database will fail, see bug 4220.

The final outcome of the calculation of the grade for a user at a particular quiz is stored in the 'grade' field of the quiz_grades table. This field has type double.

Penalty mechanism

What it is for

When the quiz is run in adaptive mode the student can interact with a question repeatedly. So in particular the student can try again when he gets a wrong answer. Clearly the final mark for the question must reflect the fact that the student did not get it right originally. Therefore a penalty is subtracted from the final mark.

How the penalty is determined

First of all penalties are relevant only if a quiz is run in adaptive mode. Only in this case can a student have a second attempt and therefore only in this mode can there be any occasion to subtract a penalty.

Even in adaptive mode the penalty mechanism is only used when it is selected in the quiz options. If "Apply penalties" is set to "No" then the final mark for the question is the mark for the last graded response.

Each question has a 'penalty' field (which should really be called 'penaltyfactor') which is a number between 0 and 1. The penalty for a wrong response is calculated as the product ($quiz->penalty * $quiz->grade), i.e., as the product of the penaltyfactor with the maximum achievable grade for the question. This product is stored in $state->penalty. So $quiz->penalty is the fraction of the maximum grade that is subtracted as a penalty for each wrong response.

The $quiz->penalty field has a default value of 0.1, both in the database and in mod/quiz/defaults.php. This default can of course be overwritten by the admin on the quiz configuration page. This admin-selected default is (as usual for admin defaults) stored in $CFG->quiz_penalty. The teacher can choose a different penalty factor for each individual question when adding or editing a question. (In Moodle 1.8 the default value of 0.1 is actually hardcoded at line 87 of moodle/question/type/edit_question_form.php)

Now if a student makes repeated wrong attempts (or partially correct attempts) the penalties for all these attempts are added up in $state->sumpenalties. The mark for the question is then calculated as the mark for the last graded response minus the sum of the penalties.

One curious fact about $state->sumpenalties is that, for efficiency reasons, it is not stored in the quiz_states table but instead in the 'sumpenalty' field of the quiz_newest_states table. That way it only has to be stored once per attempt rather than once per response.

Where it is done in the code

The function quiz_apply_penalty_and_timelimit() subtracts the penalty in $state->sumpenalty from the raw grade in $state->raw_grade to obtain $state->grade for the response. However it is ensured that the grade of a new attempt at the question never falls below the previously achieved grade. This function also increases $state->sumpenalty by the amount in $state->penalty. The assumption is that $state->penalty has just been set appropriately by the code calling this function, e.g., quiz_process_responses.

Files belonging to questions

This talks about the situation in relation to the Moodle 2.0 file API.

File areas

There are various file areas associated with each question.

For example there is a file are 'questiontext', belonging to the 'question' core component, which stores any images or other embedded files used by the question text. There are also 'generalfeedback' and 'question_answer' file areas.

There may also be some file areas specific to each question type. For example the 'qtype_multichoice' component has a 'correctfeedback' file area (among others).

All these file areas belong to the context that is the context of $question->category.

Serving files and access checks

This is a bit complicated because, although the files belong to the question system or a particular question type, then whether the user should be able to see a particular file (for example an image in the general feedback) is influenced by whichever part of Moodle is using the question (for example the quiz module or question preview).

The easiest way to explain is probably to show the call-stack for determining whether a particular image is displayed, then you can go and read the code.

For the request .../pluginfile.php/123/question/questiontext/234/345/image.png for an impage in the questiontext of a question in a quiz attempt.

(here, 123 is the contextid, 234 if the attemptid (corresponding to question_attempts.id) and 345 is the questionid.)

pluginfile.php
question_pluginfile() in lib/questionlib.php
quiz_question_pluginfile() in mod/quiz/lib.php
quiz_attempt->check_file_access() in mod/quiz/attemptlib.php
question->check_file_access() in lib/questionlib.php
qtype_whatever->check_file_access() in question/type/whatever/quetsiontype.php

If, instead, the question was appearing in the question preview pop-up window, the call stack would be

pluginfile.php
question_pluginfile() in lib/questionlib.php
question_preview_question_pluginfile() in question/previewlib.php
question->check_file_access() in lib/questionlib.php
qtype_whatever->check_file_access() in question/type/whatever/quetsiontype.php

(the different bit is highlighted in bold).

If, instead, the image url was .../pluginfile.php/123/qtype_multichoice/correctfeedback/234/345/image.png, the call stack would be

pluginfile.php
qtype_multichoice_pluginfile() in question/type/multichoice/lib.php
question_pluginfile() in lib/questionlib.php
quiz_question_pluginfile() in mod/quiz/lib.php
quiz_attempt->check_file_access() in mod/quiz/attemptlib.php
question->check_file_access() in lib/questionlib.php
qtype_multichoice->check_file_access() in question/type/multichoice/quetsiontype.php

Roughly speaking, qtype_whatever->check_file_access() takes the same $question, $state and $options objects as print_question_formulation_and_controls, so, for example, you can do the same permissions check when trying to access a file in the feedback as was done when deciding whether to display the feedback.

The quiz_question_pluginfile or question_preview_question_pluginfile functions are responsible for getting the right $question, $state, $options based on the $attemptid and $questionid in the URL. They also do some basic access checks.

All the other functions are really just about dispatching the request to the right place to be handled.

Question engine upgrade helper 2.1

Site administration > Question engine upgrade helper has a menu to assist quiz conversion. It contains links for:

List quizzes still to upgrade
List already upgrade quizzes than can be reset
Extract test case
Config cron

Documentation