Quiz UI redesign - prototype testing background
Back to Quiz_UI_redesign
I conducted the usability tests somewhat like the way I was taught to by my teachers on the usability courses of the dept. of Computer Sciences in the University of Tampere. The tests were conducted in Finnish, which is the native language of the test subjects, as well as my native language.
With some of the subjects I failed to remember to mark the subjects name on the page I wrote the issues down on; alas, with some of the issues, I was not sure who it was related to. I considered it not worth it to listen to the audio recordings just for this (at least not at this point); the most important is knowing whether something has been confirmed an issue, not whose issue it was!
As an afterthought, I was probably a bit too impatient to fix issues that arose between tests, so that I could have just some sort of an idea about whether or not my fix would help. This, naturally, defeats actually getting some sort of an idea about how well a specific idea works, since the number of iterations for a specific UI element that was changed in the middle might not have gotten too many test subjects. Nevertheless, the broad idea of the UI was very much the same throughout *all* the tests.
The test was conducted on an IBM Thinkpad T41 laptop from 2005, and it served the purpose well. OpenOffice.org was run on Ubuntu Linux 8.04 (Gnome desktop, which was not visible during the test, though). The keyboard was covered for the duration of the test and subjects were told to just speak out loud what they would do if they would normally use the keyboard. The test subjects had the choice to use either a mouse or the mousepad of the laptop.
During the tests, the speech was recorded with an mp3 player with a mic sensitive enough, though so far I have opted not to go through those recordings. The user's onscreen actions were not recorded since I could not find a decent screen recorder for Ubuntu. I took notes during the tests, on which these observations are mainly based.
Checklist: mention to the test subject
Roughly what I mentioned to the students at the start of each test; in the test, I had a checklist which contained keywords for these. Inspired by Don't make me think. (Of course in reality, all of this was in Finnish.)
- We are developing Moodle's quiz tool; we need to find out how real users work while in interaction with our application.
- We are testing the application, not you. We hope that if anything puzzles or annoys you during this test, you say it out loud. You do not have to be afraid of hurting anybody's feelings here.
- This is probably the only place in the world where you don't have to be afraid of making mistakes, since that is what we are indeed looking for: we want to find about where users might have problems with the application.
- In usability tests a technique called thinking out loud is used. This is very simple, but it varies between different people how easy it is for them. It means simply that while doing the tasks you are going to receive, you say out loud whatever pops into your mind, whatever you are wondering about or perhaps if something is difficult, or whatever.
- If you have any questions during the test, please do not hesitate to ask. It may be that I cannot reply during the test, but we will try to remember everything until the end of the test, when we can talk through all the questions you may have unanswered.
- I have a couple of forms for you to fill out before you can continue, please fill them out - they are just to provide some background information for us.
- Thanks - before we start, I want to remind you of thinking out loud. But don't worry -- if you should stay silent for too long during the test, I will remind you.
After the test:
- Now that we still ahve the last screen here, please give us feedback: do you see anything on the screen that you think is extra, or the meaning of which you don't understand?
- Finally, please fill out this feedback form