Quiz UI redesign prototype presumptions

From MoodleDocs

Olli Savolainen

Back to Quiz_UI_redesign

This page is related to the discussion in the forums: http://moodle.org/mod/forum/discuss.php?d=92797 .

Quiz UI redesign prototype presumptions / Frequently Asked Questions

The basic idea behind the new Quiz editing UI: Focus on the content being edited.

The quiz module of Moodle 2.0 brings one main change to the screen for editing quizzes, compared to Moodle 1.9: quiz-centric editing. That is, the focus of the screen is now balanced towards managing the content of the quiz, both visually and functionally. The workflow of creating questions directly to quizzes (stored in question bank only for future use) is now supported in addition to the existing workflow of first creating questions to the question bank and adding them to a quiz from there. The question bank window is hidden by default to new users. Question bank related functionality is communicated as secondary, although it can still be used to serve the primary focus of editing the quiz.

In the following, I have gathered many refutations to the new UI I have confronted during its design, and tried to respond to them with the ideas behind the design decisions.

The users who don't use the quiz creating UI

The improvement will mainly benefit those users who actually use the web UI for creating and managing quiz content. There is also a group of users who generate their questions in a format they can import into quiz, and while clarifying the UI and making it communicate its data structures better will also aid those users, they are not the main focus of this work. The argument from this user group that is not targeted in this project presents itself as follows:

No need: the additional support for an alternate use case is not needed, since there is no problem with the approach Moodle 1.9 takes to quiz editing. The added controls for editing quiz in new approach make the UI cluttered. The removed options which do not directly serve quiz editing need to be brought back for convenience

Response: For beginners, moving from pen and paper exams to making online quizzes is easier if they can concentrate on the quiz as the main object to manipulate. It is a familiar concept and the users need not initially understand the relationship between the question bank and the quiz, map the understanding to that relationship to the UI and then try to navigate it. The initial push for this UI redesign project came from the issues beginners were having with the 1.9 style UI (leading to refusing making quizzes altogether), which provided no guidance in itself as to how to get started making quizzes.

But users really should learn to use the question bank

“Experience design is more about the kind of experience users actually have than about controlling the experience you try to give them.” - Robin Good (via InspireUX)

Arguments against the new design:

  1. It is misleading to teach users a way to use quiz that is not sustainable i.e. questions should be stored in central question banks, neatly organized. Just adding questions directly to quizzes is irresponsible since in the long term, questions become disorganized.
  2. Since the final understanding that ALL teachers need to ultimately land in is using the question bank primarily, then we should direct everybody on the right path from the start, teaching them to first use the question bank and only then, quiz.

Corresponding responses:

  1. In the case of longer-term quiz usage with a large amount of questions, using the question bank indeed is favourable. However, Moodle is not in the position to dictate to users what they need: There are use cases where such scalability is not necessary. And since questions are stored in the question bank no matter which workflow is used, there is no problem taking extensive question bank functionality in use when needed (progressive disclosure).
  2. Some users are not in the position to have extensive support services. User interfaces should be learnable on their own, without documentation if possible. While evaluating Moodle and trying to determine whether or not to take into use, it is very beneficial if the user interface provides guidance to users, letting them explore it on their own, learning features in the order they are actually needed - not enforcing question bank usage before it is really necessary.

See also the following: Quiz_UI_redesign_prototype_presumptions#Why_can.27t_we_just_make_smaller_fixes_to_the_current_ui.3F


"Question bank is a familiar concept from pen&paper, too!"

It has been argued the question bank is a concept used with pen and paper exams.

At the time I just replied that I still think there are several use cases - some that just do quizzes and treat quizzes as the main unit, and some that use question banks. Of course the latter is the more advanced model and more practical in the long run.

Now I realized that this comes to progressive disclosure again. Think of a situation where you are just testing Moodle Quiz as a beginner. Even if you already have a question bank say, in a Word document.

  1. the user might still want to keep their question bank separate from Moodle and only concentrate on the quiz at first
  2. if the user has not yet committed him/herself on Moodle, he might not want to invest time in moving the entire question bank.

"The random question adding functionality in the new UI is confusing."

For random questions, the logic goes as follows: even though the user might not understand the logic of the functionality initially, there is guidance: users can get far by just keeping the defaults and clicking on, effectively learning the logic of the UI, engaging in a learning-as-we-go manner with it, without being overwhelmed with too much choice in the process.

That said, random questions are more advanced functionality than adding single questions. The goal of UI design can be seen as making simple things simple and complicated things only as hard as they need to be.

There is also a lot that can still be enhanced about the UI. The add random question dialog that is intended to provide guidance seems to potentially confuse experienced users (there is really no data about this). It does not currently provide means to add a random question from an existing category, which corresponds to the Moodle 1.9 model of creating random questions from existing categories. In usability testing (hae viite), having both options (creating new category and using an existing one) got novices, whom the dialog is intended for in the first place, completely confused.

Experienced users can still use the familiar functionality that was there in Moodle 1.9 to use questions from existing categories, and novices can relatively effortlessly learn to at least create random questions in a pseudo-quiz-centric manner.

Without actually having any data, I suspect that the current Moodle 2.0 UI (April 2009) might be somewhat problematic for those users who are aiming to be "advanced users", that is , harnessing the full potential of question bank and quiz, but have no prior experience and do not know that there is the separate, less visible control for adding random questions, in the question bank toolbar window (on the left of the quiz edit screen).

It would probably be good in the long run to add the option to use an existing category into the dialog. To do this in a way which does not distract the dialog's primary purpose required further design effort and usability testing.

  • This would probably be a good time to explore introducing brief instructional label in the dialog as well, to explain the idea of a random question very briefly.
  • Starting Spring 2010 I will most likely be available to do this and other similar work for Moodle.

Adding questions to a quiz or to the question bank?

It has been argued that adding questions directly into quizzes gives a misleading impression about the underlying actual model, since the question is really added into the question bank and only artificially into the quiz. My response to this is that you can see the underlying it either way: as the question added to quiz and also to the question bank as a convenience, or the other way round. Neither perception is incorrect.

Why can't we just make smaller fixes to the current ui?

Secondly: The problems addressed by the proposition cannot be solved simply by making small enhancements to the current UI, one by one.

Thus, a new UI needed to be designed: The current conceptual model of the Quiz UI assumes a process, where the user first adds questions to the question bank, and from there on to the Quiz, either as single questions or categories as random questions. This is in wild contradiction with the conceptual model some (expecially novice) teachers have, with experience from traditional exams. The only way you to get less committed novice users to create quizzes, is by making the UI structure communicate using their concepts, their language, and their process. You cannot (gradually or not) add the natural conceptual model to an UI that already uses a different model altogether, since this way you will confuse all users who try to use either of those models.

An example is in place. As an attempt to let a novice user add questions directly to a quiz, you could add an "Add this question directly to the quiz" tick box to all the screens for creating a question. However, this would not help the novice users, since according to their model, "OF COURSE the question goes into the quiz, where in the world else would it go?" Alas, that tick box actually would take the UI further away from the user's conceptual model, since there would be yet another option which does not make sense to them.

Novice users see the quiz itself as the fundamental unit, since that is how exam making works in the real world. They need not bother with the idea that questions also exist in another dimension, the question bank. The best we can do for these users is allowing them to just create the quiz and ideally to give meaningful category names for their questions. They do not yet need to have a concrete understanding of the fact that the questions in the categories are somewhat separate entities from the ones in their quiz. Also, allowing them to create random questions directly into the quiz is feasible by creating the necessary category - transparently to the user - when creating the random question.

Categories or hierarchies are not difficult concepts in themselves. What takes time to learn are the relationships between categories and questions and random questions and quizzes, and so on, and then mapping all that understanding to the UI (which itself currently does not clearly communicate those underlying relationships). The user who just wants to create a quiz, does not need to and cannot be forced to understand all this, just to make an one-time quiz. (Of course, this does not stop us from encouraging users to organize their questions in meaningful categories, but that’s as far as we can go.)

However, in Quiz, the eventual full model is that which also includes also categories etc., in addition to just the Quiz content. For use cases that involve exporting importing questions or keeping them organized, users will need to understand the full set of concepts. Still, we cannot shove the full set of concepts, foreign to the novices, down their throats by force. The solution is progressive disclosure (http://www.useit.com/alertbox/progressive-disclosure.html), which we can have to a degree even if the new interface is a separate of the full UI at first: users first use the UI that mostly speaks to them with only the concepts they already know (with the exception of random question) but manipulates the same data structures as actual Quiz. In this UI users may gain "conceptual handles" (such as a category), which they can use if they choose to learn the concepts required when using the more advanced UI.

The problems and the solutions

We can also approach the old UI in with a problem based approach:

Problem: The main quiz editing screen in Moodle 1.9 (Edit -> Quiz) has three separate functionalities (category content management, quiz content management, quiz reordering/organizing), confusing users since the UI does not communicate what users are supposed to use it for.

Solution: Separate the functionality so that the user can concentrate on one task at a time. Do not overload the user's cognitive space with unnecessary information for a given task. Category content management already has its own screen and thus the functionality can be removed from the Quiz content management screen. Reordering/organizing the content of the quiz is a central use case to managing a quiz, but since switching from and to is the reordering already a mode (triggered by a checkbox) in the old UI, we make that mode explicit by creating a separate, specially designed tab for managing the order and paging of the quiz.


Problem: In the main quiz editing screen, users are often unaware of the actual content of questions in the quiz or in the categories since they are not visible. They only see the question names. Alas, users are often unable to make the decisions they are in the screen for. It is not obvious for a novice, how to find out which question is where in the quiz.

Solution: Quiz content (the actual questions) is central to the task of creating or managing a quiz and this information needs to be readily available for that task. As we have above freed screen estate by moving unrelated functionality into screens of their own, we can fully concentrate on the quiz content in the main quiz editing screen. Also the content in the categories can be expanded at user request, for example using AJAX.