Question Engine 2:Rationale

From MoodleDocs
Jump to: navigation, search

Contents


The Question Engine is the part of the question code in Moodle that deals with what happens when a question is attempted by a Student, or later reviewed by the Teacher or the student themselves or included in reports. Roughly speaking this is half the question code, the other part being the Question Bank, which allows people to create, edit, organise, import, export and share questions. The Question Engine is the part that has to perform when you have a large class of students taking a quiz simultaneously.

An irrelevant but pretty picture of Sydney, taken from the hotel room where most of this was written.
This proposal is a by-product of more than three years working on the Moodle quiz and question code. During that time I have fixed hundreds of bugs and feature requests. Some of the bugs with the Question Engine have requried careful fixing in order to avoid regressions, and there are other issues that are hard to do anything about at the moment.

As well as bug fixes, I have also overseen major developments in the quiz/question system. For example Jamie's changes to the Question Bank, Ollie's new quiz editing interface, and Mahmoud's new quiz navigation that I added to Core Moodle while refactoring the quiz attempt code.

Over the three years, I have also followed the Moodle quiz forum and seen what it is that people struggle with, complain about, want to change (and also what they like!). For example, having coached some people when writing new question types I realise that creating a new questions type is more difficult than it needs to be.

The changes proposed here fix parts of the question engine code that I have been dissatisfied with since about the first time I looked at it. There are several reasons why I have not addressed them before now.

  1. To fix them properly, we must change how the data is stored in some of the key question tables. In some sites, these tables contain millions of rows, and the processing required on update is non-trivial and is different for different question types. This is not something to undertake lightly.
  2. It took me about two and a half years to work out this solution. Some parts of this proposal where clear from the start, but the exact notion of question interaction models, and how they would relate to question types, only crystallised recently in my mind. Because of 1. I did not want to change anything until I was sure what I was proposing was right, not just in general, but in all the details.
  3. The mundane reason that there have been plenty of other things to work on, and I have not had time to address this until now.
  4. These are not critical problems. There have been no major changes to the question engine since the 1.6 release. Clearly there are not fundamental flaws with the current system. It is just not as good as it could be.

As this is a major change. My estimate is that doing what I propose here is a couple of month's solid work.

Related tracker issues

One way to see why things need to change is this list of tracker issues which these changes should fix.

Correctness issues

Key Votes Watchers Summary
MDL-3030 31 21 attempts should be auto-closed
MDL-3936 3 2 Essay Question used in score before grading
MDL-9303 9 8 General Feedback not shown for essay questions on review screen
MDL-9327 9 10 Text in Essay type questions is not saved in second or more attempts when the teacher has not graded previous attempts
MDL-13289 0 1 Essay question with adaptive mode not show feedback
MDL-17681 0 0 Manually assigned grades cannot easily by re-scaled when the marks for a question is changed

Robustness issues

Key Votes Watchers Summary
MDL-11852 0 1 Quiz does not score properly
MDL-12344 0 0 Student takes quiz, upon completion of quiz receives the following
MDL-12665 3 3 quiz attempts sometimes show zero despite correct answers
MDL-13073 0 1 Regrading (really) messes up grades
MDL-13246 0 2 Quiz does not save score. Shows 0/20
MDL-14873 0 2 quiz answers submit freezes (occationally)
MDL-15305 1 1 Quiz doesn't work after submit all and finish button is selected - get a "No quiz with id=0" or "No attempt with id=0" error message
MDL-16451 0 1 Quiz scores 0 with secure window; Grades do not transfer
MDL-16490 1 2 timed quiz generates bad 2nd attempt
MDL-16663 0 1 errors and missing work

(Many of these are intermittent and only affect some people. Sometimes bugs like this turn out to be the result of a corrupt table in MySQL. Still, we really should not be getting problems like this with a system that is sometimes relied on as much as the quiz is.)

Richness issues

Key Votes Watchers Summary
MDL-1647 14 13 Allowing negative marks for questions
MDL-4860 0 0 Questiontype giving hints
MDL-12821 0 1 Essay quiz question redundant submit button
MDL-17894 1 1 Support certainty/Confidence based marking
MDL-17895 0 1 Allow question types to validate responses before grading
MDL-17896 1 1 Question types like Opaque should not require nasty hacks in question_process_responses

Issues I don't propose to fix with this change

... but which would become easier to fix as a result:

Key Votes Watchers Summary
MDL-3452 0 0 Use saved answers when marking late submissions
MDL-4309 3 7 Ability for late quizzes (with penalty)
MDL-11047 9 4 Quiz limit to one question per page with no access to previous or next question
MDL-15596 0 0 Allow a question to be used more than once in an attempt

See also

The next section, How the question engine currently works, contains further remarks about why change is needed.

Personal tools
User docs (English)