Development:Usability testing in August 2008/Executive summary
Return to: Quiz UI redesign | Quiz Usability portal
The final usability tests of the project were done on Wednesday and Thursday in the University of Kuopio, Finland. We got plenty of results based on which to work on, and we seem to have gotten to a good start at making Moodle Quiz the best thing there is for all your quiz... umh, examining, umh, all sorts of asking-questions-from-students needs. Based on the results, I gathered a roadmap-style document for guidelines about which direction to take the Quiz module's navigation: [Where to go next with Quiz UI?].
Seriously, the main purpose of this test was to find out how well the Quiz UI redesign works – those results are in a separate document: Results related to Quiz UI redesign. However, as the editing UI seemed relatively stable already, I decided to expand the test a bit to cover more of the surrounding Quiz app (see test setting). I did not at all test grading, for example, but the tests revealed much about the general outlines of Quiz UI.
The principal messages from the usability tests were as follows.
Novices want to start simple, to find advanced functionality only when they need it.
This is the progressive disclosure we have now introduced in the Quiz editing UI, and which the question editing and quiz settings (“update this quiz”) screens are screaming to have. There are many challenges here still to be solved.
Teachers still have problems understanding what the actual result of their work will look like.
Things like the effects of different grading fields or when is feedback visible are not communicated in the UI clearly enough. [The single question preview screens are misleading], [in quiz preview teachers can not be quite sure they see exactly what students see (since the tabs are there, how do I know what else is different from student view?)], and [role switching can have fatal consequences] (locking quiz down) teachers are not aware of.
How to approach these issues? Take the elements of a student's quiz experience: paging bar, different buttons for controlling how to proceed with the quiz, time limit display, etc. Document each of those pieces, and communicate to the user clearly how to determine which of them are displayed. Of course, I am just scratching the surface here, since this really wasn't what I was designing or testing for this summer, but as reported also by the numerous forum posts about the subject lately, people have many different views about how, for example, quiz navigation should look like.
Understanding the different grading options is hard, so you better make them obvious.
The document Where to go next with Quiz UI? deals with the current navigational issues Quiz suffers.
Iron out the obvious mistakes.
Related issues in the previous iteration's testing
- If you can easily help the lives of users without any cost, do so. Strictly make all in-content links use standard colours and underlining (bright colour for unvisited, darker on visited links where it makes sense). See Jakob Nielsen: Guidelines for Visualizing Links . Using only color to denote anything in a UI is a bad idea, since users have problems noticing it for various reasons. 'Nuff said. Solution: . In menus underlining is not necessary. See issues
- If the teacher is creating a question, they will assume the first thing asked is the actual question and not some abstract “text to identify the question from in the editing UI”. In case someone actually uses question name for something real, make it an option (and put it after the actual question text), but don't impose it on users. See issues
- Teachers do not understand the various question types. We need to help teachers make an informed choice even the first time they are searching for a question, since unless they happen to notice the help button, all they can do is guess and get frustrated since many of the different question names suggest something else than they are or just don't mean anything to the uninitiated. This is quite nicely solved in Questionmark Perception by providing a description of the question selected type in the pane next to the question type list. Doing this in Quiz would require adding a new question be a modal dialog where there would be room for this kind of a control. See issues
