Quiz UI redesign scenarios - conclusions

From MoodleDocs

Back to Quiz_UI_redesign#Scenarios

Scenarios work conclusions (executive summary)

The scenarios work I conducted during April and May resulted in a fair amount of high-level information about creating quizzes: it provided valuable insight into how teachers see the usage of exams/tests/quizzes (from now on, I will use the word 'quiz').

The scenarios have also been taken into account in the new user interface to the extent I saw applicable. However, most of it turned out to be higher-level understanding that would only be applicable after the functional requirement specs of the software were first changed; an excellent example of this is the various needs of different levels of metadata various users showed.

The status bar

The status bar is one important addition, which was supported by the scenarios interviews, that I actually made. It turned out that when a teacher creates exams/quizzes, it is for many a task that requires rather intense concentration. On the other hand, it also became clear that today's teaching world does not make it very easy for teachers to isolate long periods of concentrating just on quiz/exam creating.

Thus, the UI should support the teacher's memory: when a teacher gets interrupted, it should be as easy as possible to continue where they left off. That is, it needs to be visible at a glance what the exam's/quiz's current status is, and showing at least whether the exam is open and if not, when it will be (and possibly, when it will close) is the bare minimum to reach for this. Ideas about what else might be wise to show in the status display, are welcomed.

What is interesting about the OOo-prototype tests is that although the instructions gave the users the idea that they need to create the questions into a category first and then add that category as a random question into an exam, all of them successfully created a random question with the "create random question" button.

Considerations for future development

There are plenty further conclusions to be made through the analysis of the scenarios already discovered and the ones that are yet to be found -- namely, creating quizzes with the focus of not so much testing but just helping students to practice, is something that we did not get very deep with yet.

The process of creating quiz content starts way before the teacher actually adds/writes the final questions in the actual quiz.

Since quizzes are usually part of a wider context (provided by a course of some sort), the first sketches of questions may be thought of by the teacher while they design the course or even before they have a concrete course.

  • Conclusion: The UI could support a much more direct, brainstorming-style workflow for the entry of questions:
    • Create quiz
    • Select question type
    • Click 'create question' -> the question box appears in the quiz with a note saying that necessary details need to be filled in before the question can appear in the actual quiz
    • Type question content directly in a text box inside the question box, right in the quiz editing screen
  • Issues: each question type would need an editing view of their own, to be quick enough this would require an AJAX backend in addition to the non-javascript implementation, an additional mode in the UI brings complications

It is not easy to predict when changes to exams need to be made.

In addition to the scenarios interviews I did this spring, this has been seen as a constant issue with the Electronic Exam System with which I worked previously.

  • Conclusion: Exams need to be editable at any time, even after attempts have been made. If this is technically difficult, the teacher must be offered options right in the editing view, to get an editable copy of the quiz. If it means somehow archiving the existing attempts, or detaching them from the current instance of the quiz, then the implications must be made clear to the teacher.

Question and Quiz metadata are an important part of the process.

Quiz could support having also related metadata saved with questions and quizzes:

For example, how well a particular quiz/question has worked in terms of being understood by students or in terms of more general student performance is important to the teacher so that they can develop (and distribute) their questions. This could be stored in the metadata of the question (while teacher is grading a quiz) and then when the same teacher is using the quesition in a new quiz (possibly months after grading the previous instance) they can see their notes about which question did not work and ideas about how they could possibly be enhanced).

More info: Mack/Metadata Ida/Metadata

During both creating quizzes and grading, being capable of comparing existing answers is central to the work.

Existing answers can also be considered metadata of a question: they affect the decisions about how to modify questions or which questions to pick to future quizzes.