Plans for adaptive mode

This page is currently work in progress.

At the OU we are planning to do some work on adaptive mode of the quiz, because we think that there are some usability problems with how it currently works, and because it does not quite meet our needs. This page outlines those plans so we can discuss them.

We have been thinking about the on-and off for quite a long time, and have recently come up with this definite proposal. We being three members of OU staff Tim Hunt (leading developer and Moodle quiz maintainer), Phil Butcher (eAssessment business process leader - that means the person who consults with academic staff to gather requirements) and Mahmoud Kassaei (developer, who will be doing most of the work on implementing this).

Please discuss this proposal in this quiz forum thread.

The main usability problem

To demonstrate the big usability problem:

A. In an adaptive quiz, you enter the wrong answer to a question, the click submit. This is what you see:

B. Then you think again, and enter the right answer (but don't click submit yet). This is what you see:

C. Then you click save, and go away, perhaps you want to check a resource in the course that explains the underlying theory. Then you come back. Now it looks subtly different:

D. Finally when you do click submit you get this:

We believe that what you see in B. and C. is confusing, and students should never see a question in this state.

So how do we avoid this?

The problem is that we only have a single way to display questions during the attempt in adaptive mode, which is "showing feedback to any previous response, and allowing the student to enter their next response". (Then, whether in adaptive or non-adaptive mode, there is the reviewing the quiz after the attempt mode.)

We propose to change and have two separate states during an adaptive attempt. First "letting the student enter their next response, and then "showing feedback for the previous response".

Something like this:

What do we lose by our proposed approach?

The one thing this loses is that when they student is entering their second try, they cannot see the feedback from their first try. Since the whole point of feedback is to help the student correct their own mistake, and in the process learn something, this at first appears to be a big loss.

The other side of the argument is that our way forces the student to read and absorb the feedback before clicking next. Also, the student may work out their next answer (perhaps making notes on paper) while the feedback is still visible, before clicking try again.

Other changes we propose

If you looked closely at the above state diagram, you will have seen these already.