Quiz UI redesign - what makes a scenarios interview?
Back to Quiz_UI_redesign
The idea and the content of the scenarios interviews I did
A.k.a. Quiz UI scenario gathering work in Summer 2008 - what, why and how?
After I had received the Kesäkoodi  funding for the project , it quickly started to dawn to me that if I want to take designing the new UI seriously, what it takes is really getting to know the users - teachers, IT support, and even administrators. I used all the contacts I had - namely, my old job at the university of Tampere, the ITPEDA mailing list and Moodle-rinki, a community of Finnish Moodle users.
I studied a bit about what the methods to get learn about the users are. First, I considered doing contextual inquiries, but that would have required a lot more time with each user than the one hour I had managed to agree with some teachers. So, a rather informal interview it was. Next, I am going to introduce here the different parts of what I did, so you can do so too if you want to. Everything here has been freely translated from Finnish so that you can get an idea.
- Consent for recording audio (1 A4 sheet)
- Background information form (1 A4 sheet):
- Frequency of website/browser usage
- Do you experience with which learning environments?
- How often do you use a learning environment as a teacher
- Question types you use: multiple choice; essay; other, what
- Categorisation basis of question categories
- Do you use electronic exams, with which software?
- If you use websites, which kinds of services do you mostly use?
- Question list
"We are developing the Moodle Quiz module. The goal is to understand how actual users/teachers do their work, especially related to exams. The setting here, so to speak, is that you are the master and I am an apprentice - that is, I will mostly just ask questions, listen and take notes. I apologize in advance that I might seem very concentrated on my notes at some point, since I am not a very accustomed interviewer. "
Actual interview questions
The idea is not to follow this question list strictly, but to use it as a rough guideline of the questions to find out about. When a teacher's thought wanders, we let it do so and may ask previously unthought of questions related to what a teacher has said, if it seems related.
- Please describe the character of your work.
- What kinds of responsibilities do you have?
- What forms of teaching do you use; group work, lectures, online teaching, ...?
- Apart from reality, if you could imagine a tool that would optimally serve your way of making exams/quizzes, what would the experience of using it be like? What tasks related to making quizzes/exams could a machine help you with or do for you?
- What is usually the context of evaluating students' learning?
- Within that context, how do you end up with the conclusion that your course requires an exam/quiz?
- Curriculum, course?
- What do you think about exams as a way to evaluate students' learning?
- What tools or accessories do you use while making exams/quizzes, physical or software?
- Please describe how you proceed from that to making an exam/quiz. What are the different stages?
- If you have made electronic exams, what do you do, how does the process differ from making pen and paper exams/quizzes?
- Chronologically (on a time scale), at which stage of a course do you think of the exam/quiz content for the first time?
- How long a time is there between making the quiz/exam, actually having the students do the exam, and grading it? How often do students take your quizzes/exams, what determines this?
- When do you create the actual, exam/quiz which you give to students to do?
- What is the situation like when you create an exam? Where do you do the exam physically? Are there distractions or can you tolerate distractions?
- What do you use as material or tools when creating an exam?
- Do you have to copy some information to many places?
- Do you cooperate with other parties when making exams or otherwise about exams? What are the ways/medias in which you communicate with them?
- What determines the structure of an exam?
- Do you use old exams as material?
- Is the order of questions predefined and determined already before you create the actual exam, or do you reorder questions when they are already in the exam?
- What is problematic about the current tools for making quizzes/exams?
- What is good about current tools for making quizzes/exams?
- When you create an exam, where there are questions that are randomly selected from a larger selection of questions, to each exam attempt, what are the stages to doing this?
If they have used Moodle Quiz:
- Do you use names of questions? How, for what? What could you imagine them being used for?