Development:States of a quiz attempt

This page relates to the problem of the following scenario:

1. Student starts an attempt at a timed quiz.
2. Student logs out and goes away.
3. Quiz time limit expires.
4. Student (or teacher) comes back. They now cannot continue (or review, or complete) the quiz attempt. They are stuck.

That's not what actually happens. Actually on 4 student see a button to continue his attempt. He press it and goes to the attempt, which is instantly submitted, and he get returned to quiz intro. He could then start a new attempt ... if there is no time delay between attempts. If there is he is stuck.--Oleg Sychev 19:55, 24 November 2009 (UTC)

Different people have different desires for how this is handled. The aim of this page is to collect together all the things that different people might want, so we can try to design a solution that makes everyone happy.

Currently this page is just a random dump of things Tim remembered, please feel free to add to, or edit it.

What currently happens

The states a quiz attempt can be in

• {before the attempt is started}. Not sure if this really exists. Before the attempt has been started, it does not exist!
• During the quiz attempt (Quiz attempt open). This is distinguished by $attempt->timefinish being 0.
• After the attempt is finished. This is distinguished by $attempt->timefinish being non-zero.

Behaviour

• A user can have at most one quiz attempt open at any time.
• If the user is attempting the quiz when the time expires, and if they have JavaScript on, then the quiz is automatically submitted for them.
• If the student is not attempting the quiz when the time expires, then their attempt is not submitted, and neither the student nor the teacher can submit it.

 Actually the attempt will be graded 0, but individual questions may be regraded if page was saved. 
 Teachers could use it to calculate attempt grade manually, but they really don't like it.--Oleg Sychev 19:55, 24 November 2009 (UTC)

• Time can expire either because of $quiz->timelimit, or $quiz->timeclose.
• There is a capability mod/quiz:ingoretimelimit for accessibility reasons. This causes $quiz->timelimit, but not $quiz->timeclose, to be ignored.
• After the attempt is submitted, studnets may, or may not be able to review, depending on the quiz review settings and whether $quiz->timeclose has passed.
• In addition to the only one open attempt rule, there may also be a delay between the end of one quiz attempt and the start of the next ($quiz->delay1/2).

What we would like to happen

The states we need to distinguish

• {before the attempt is started}
• Quiz attempt open, time has not expired.
• Quiz attempt open, time has expired.
• Quiz attempt submitted, some questions need manual grading.
• Quiz attempt submitted, all questions graded.

 Actually these states makes quiz time delay more complicated. 
 The question is what state it should use as an end of previous attempt? 
 It should be submission for quizzes without time limit.
 But for quizzes with time limit it should be min(time expiration,  submission) otherwise we still may get 
 problem with time delay mentioned above.--Oleg Sychev 19:55, 24 November 2009 (UTC)

Might we also want/need to distinguish the states 'results not visible yet', and 'results released to student', for a submitted quiz. These two are orthogonal to whether any questions need manual grading, so that actually leads to four sub-states under submitted.

Behaviour

• Teacher should have the option to submit an attempt when the student forgot.
• Student should have the option to submit an attempt late (but not answer any more questions) when they forgot. Probably the best way to do this is that after time has expired, if the student is still allowed to Submit all and finish, they can only go to the attempt summary page (in Moodle 2.0) and click the submit button.
• Some people like the current behaviour: the student must submit on time, and if they don't they don't get a grade, and it is tough luck. Therefore, if students are allowed to submit the quiz late, there should be a time limit for how much extra time they have in which to do so (which can be anything from 0 to infinity).
• There should be an option which, at the moment when time expires on an attempt, send an email to the student reminding them that they have not submitted it, and explaining to them how to do so, and how long in which they have to do so.
• Alternatively, the quiz may have an option set to automatically submit any quiz attempt the moment time has expired.
• May want an option where, if the student is allowed to submit late, we do process any updated answers, but a penalty is applied to their score.
• May want an option for a teacher to un-submit a quiz attempt, for those situations where a student has (claims to have) clicked Submit all and Finish too early, by mistake.
• Teachers can delete any quiz attempt (open or submitted) at any time (this has been implemented for a long time)
• Giving teacher the ability to decide, that particular student in particular attempt could bypass time delay (without deleting previous attempt) will resolve a great deal of the problems. (this is implemented in Moodle 2.0)

Other ideas

• Some people may want to count un-submitted attempts if quiz has a limit on a number of attempts, other may want to exclude them.

This is not really possible, given the assumption the quiz has that there is at most one open attempt, which can either be submitted or deleted, and once it is submitted, it counts. I also don't really see the point of this. If you are being that flexible, why have a limit on the number of attempts at all?

• May want an option for a teacher to extend time limit for particular attempt (if, for example, computer hangs during the attempt, or the student press submit near the finish and wait more than 60 seconds)

This is, as opposed to extending it for all attempts for a particular student. This would be a lot of work to implement, and only occasionally useful. For now, I think it is sufficient to have the option to change the time limit for one student. You can always extend it until they have submitted that one attempt, and then change it back.

• (I don't know whether it really possible, but it's a very good option to have) Save current attempt state with responses in a browser window if submitting request on server failed. Sometimes server go down during important attempts (exams and so no), that's quite bad for timed quizzes. At least we could send AJAX request to the server before submitting to verify it still working (should work in couple with teacher-defined time limit extension mentioned above to prevent abuse).

This is not possible.

• ... please add more ideas here ...

Changes to the code

Note that this builds on top of Moodle 2.0. If you don't know what things like the summary page are, or how the quiz reports work in the latest version, this may not make any sense.

• New capability mod/quiz:submitlateattempt - can apply to students, and/or teachers, but with slightly different meaning. (Teacher here is probably defined by mod/quiz:grade, or mod/quiz:viewreports. Or perhaps it is clearly to have separate capabilities mod/quiz:submitmylateattempt and mod/quiz:submitanylateattempt.) Or perhaps rather than a capability, a quiz setting saying how much extra time students have after their attempt becomes overdue, to go in and submit it.

• If a student has mod/quiz:submitlateattempt, and/or if they are still within the extra time allowed, then they can get at overdue quiz attempts, but they can only go to the summary page in order to click the Submit all and finish button.

• For teachers with mod/quiz:submitlateattempt, there will be a new button in the quiz reports (next to delete attempts and regrade attempts) to submit any overdue quiz attempts that have been selected using the check-boxes.

• New database column quiz_attempt.state, to make the current state of each attempt clear. (Not sure if this is really necessary.)

• Code on cron, or when the attempt is accessed, to update the attempt state if, for example, time has expired since it was last changed.

• A quiz option, and code on cron, to automatically submit overdue quiz attempts if that is what the teacher wants.

• A quiz option, and code on cron, to email a student when an attempt becomes overdue, telling them how long the still have to submit it, and how to do so.

• Possibly new column quiz_attempt.finishedby, to record the userid of the user who submitted the attempt, so we know if the teacher has submitted a later attempt for the student. Could be NULL, if late attempts are submitted automatically.

• Code in the question engine to correctly process incomplete attempts when an overdue attempt is submitted.

See also

Discussion

• Please discuss this in this Quiz forum thread.

Relevant tracker issues

• MDL-3030 Late quiz attempts should be closed automatically
• MDL-3452 Use saved answers when marking late submissions
• MDL-4309 Ability for late quizzes (with penalty)
• MDL-20956 Student can't start new attempt if a quiz has timle limit between attempts and time limit for a quiz, and he doesn't submit previous attempt closing browser window instead
• ... please add other relevant tracker issues here.

See also

• Goals of an online assessment system
• Closing unfinished attempts - possible solution for 1.9

Template:CategoryDeveloper