Development:Quiz UI redesign - what makes a scenarios interview?: Difference between revisions
(4 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
Back to [[Development:Quiz_UI_redesign]] | Back to [[Development:Quiz_UI_redesign]] | ||
== | == The idea and the content of the scenarios interviews I did == | ||
'''A.k.a. Quiz UI scenario gathering work in Summer 2008 - what, why and how?''' | '''A.k.a. Quiz UI scenario gathering work in Summer 2008 - what, why and how?''' | ||
After I had received the Kesäkoodi [http://www.coss.fi/web/coss/developers/summercode/2008] funding for the [http://www.pilpi.net/software/moodle_quiz_ui/] | After I had received the Kesäkoodi [http://www.coss.fi/web/coss/developers/summercode/2008] funding for the project [http://www.pilpi.net/software/moodle_quiz_ui/], it quickly started to dawn to me that if I want to take designing the new UI seriously, what it takes is really getting to know the users - teachers, IT support, and even administrators. I used all the contacts I had - namely, my old job at the university of Tampere, the ITPEDA mailing list and Moodle-rinki, a community of Finnish Moodle users. | ||
I studied a bit about what the methods to get learn about the users are. First, I considered doing contextual inquiries, but that would have required a lot more time with each user than the one hour I had managed to agree with some teachers. So a rather informal interview it was. Next, I am going to introduce here the different parts of what I did, so you can do so too if you want to. | I studied a bit about what the methods to get learn about the users are. First, I considered doing contextual inquiries, but that would have required a lot more time with each user than the one hour I had managed to agree with some teachers. So, a rather informal interview it was. Next, I am going to introduce here the different parts of what I did, so you can do so too if you want to. Everything here has been freely translated from Finnish so that you can get an idea. | ||
=== Background material === | === Background material === | ||
* Forms | * Forms | ||
** Consent for recording audio | ** Consent for recording audio (1 A4 sheet) | ||
** Background information form | ** Background information form (1 A4 sheet): | ||
*** Frequency of website/browser usage | |||
*** Do you experience with which learning environments? | |||
*** How often do you use a learning environment as a teacher | |||
*** Question types you use: multiple choice; essay; other, what | |||
*** Categorisation basis of question categories | |||
*** Do you use electronic exams, with which software? | |||
*** If you use websites, which kinds of services do you mostly use? | |||
* Question list | * Question list | ||
=== Preparatory talk === | |||
"We are developing the Moodle Quiz module. The goal is to understand how actual users/teachers do their work, especially related to exams. The setting here, so to speak, is that you are the master and I am an apprentice - that is, I will mostly just ask questions, listen and take notes. I apologize in advance that I might seem very concentrated on my notes at some point, since I am not a very accustomed interviewer. " | |||
=== Actual interview questions === | |||
The idea is not to follow this question list strictly, but to use it as a rough guideline of the questions to find out about. When a teacher's thought wanders, we let it do so and may ask previously unthought of questions related to what a teacher has said, if it seems related. | |||
==== Background ==== | |||
* Please describe the character of your work. | |||
** What kinds of responsibilities do you have? | |||
** What forms of teaching do you use; group work, lectures, online teaching, ...? | |||
* Apart from reality, if you could imagine a tool that would optimally serve your way of making exams/quizzes, what would the experience of using it be like? What tasks related to making quizzes/exams could a machine help you with or do for you? | |||
* What is usually the context of evaluating students' learning? | |||
* Within that context, how do you end up with the conclusion that your course requires an exam/quiz? | |||
** Curriculum, course? | |||
* What do you think about exams as a way to evaluate students' learning? | |||
==== Exam making ==== | |||
Process | |||
* What tools or accessories do you use while making exams/quizzes, physical or software? | |||
* Please describe how you proceed from that to making an exam/quiz. What are the different stages? | |||
* If you have made electronic exams, what do you do, how does the process differ from making pen and paper exams/quizzes? | |||
* Chronologically (on a time scale), at which stage of a course do you think of the exam/quiz content for the first time? | |||
** How long a time is there between making the quiz/exam, actually having the students do the exam, and grading it? How often do students take your quizzes/exams, what determines this? | |||
** When do you create the actual, exam/quiz which you give to students to do? | |||
Environment/situation, content | |||
* What is the situation like when you create an exam? Where do you do the exam physically? Are there distractions or can you tolerate distractions? | |||
* What do you use as material or tools when creating an exam? | |||
* Do you have to copy some information to many places? | |||
* Do you cooperate with other parties when making exams or otherwise about exams? What are the ways/medias in which you communicate with them? | |||
** What determines the structure of an exam? | |||
** Do you use old exams as material? | |||
* Is the order of questions predefined and determined already before you create the actual exam, or do you reorder questions when they are already in the exam? | |||
Tools | |||
* What is problematic about the current tools for making quizzes/exams? | |||
* What is good about current tools for making quizzes/exams? | |||
* When you create an exam, where there are questions that are randomly selected from a larger selection of questions, to each exam attempt, what are the stages to doing this? | |||
If they have used Moodle Quiz: | |||
* Do you use names of questions? How, for what? What could you imagine them being used for? |
Latest revision as of 18:55, 24 June 2008
Back to Development:Quiz_UI_redesign
The idea and the content of the scenarios interviews I did
A.k.a. Quiz UI scenario gathering work in Summer 2008 - what, why and how?
After I had received the Kesäkoodi [1] funding for the project [2], it quickly started to dawn to me that if I want to take designing the new UI seriously, what it takes is really getting to know the users - teachers, IT support, and even administrators. I used all the contacts I had - namely, my old job at the university of Tampere, the ITPEDA mailing list and Moodle-rinki, a community of Finnish Moodle users.
I studied a bit about what the methods to get learn about the users are. First, I considered doing contextual inquiries, but that would have required a lot more time with each user than the one hour I had managed to agree with some teachers. So, a rather informal interview it was. Next, I am going to introduce here the different parts of what I did, so you can do so too if you want to. Everything here has been freely translated from Finnish so that you can get an idea.
Background material
- Forms
- Consent for recording audio (1 A4 sheet)
- Background information form (1 A4 sheet):
- Frequency of website/browser usage
- Do you experience with which learning environments?
- How often do you use a learning environment as a teacher
- Question types you use: multiple choice; essay; other, what
- Categorisation basis of question categories
- Do you use electronic exams, with which software?
- If you use websites, which kinds of services do you mostly use?
- Question list
Preparatory talk
"We are developing the Moodle Quiz module. The goal is to understand how actual users/teachers do their work, especially related to exams. The setting here, so to speak, is that you are the master and I am an apprentice - that is, I will mostly just ask questions, listen and take notes. I apologize in advance that I might seem very concentrated on my notes at some point, since I am not a very accustomed interviewer. "
Actual interview questions
The idea is not to follow this question list strictly, but to use it as a rough guideline of the questions to find out about. When a teacher's thought wanders, we let it do so and may ask previously unthought of questions related to what a teacher has said, if it seems related.
Background
- Please describe the character of your work.
- What kinds of responsibilities do you have?
- What forms of teaching do you use; group work, lectures, online teaching, ...?
- Apart from reality, if you could imagine a tool that would optimally serve your way of making exams/quizzes, what would the experience of using it be like? What tasks related to making quizzes/exams could a machine help you with or do for you?
- What is usually the context of evaluating students' learning?
- Within that context, how do you end up with the conclusion that your course requires an exam/quiz?
- Curriculum, course?
- What do you think about exams as a way to evaluate students' learning?
Exam making
Process
- What tools or accessories do you use while making exams/quizzes, physical or software?
- Please describe how you proceed from that to making an exam/quiz. What are the different stages?
- If you have made electronic exams, what do you do, how does the process differ from making pen and paper exams/quizzes?
- Chronologically (on a time scale), at which stage of a course do you think of the exam/quiz content for the first time?
- How long a time is there between making the quiz/exam, actually having the students do the exam, and grading it? How often do students take your quizzes/exams, what determines this?
- When do you create the actual, exam/quiz which you give to students to do?
Environment/situation, content
- What is the situation like when you create an exam? Where do you do the exam physically? Are there distractions or can you tolerate distractions?
- What do you use as material or tools when creating an exam?
- Do you have to copy some information to many places?
- Do you cooperate with other parties when making exams or otherwise about exams? What are the ways/medias in which you communicate with them?
- What determines the structure of an exam?
- Do you use old exams as material?
- Is the order of questions predefined and determined already before you create the actual exam, or do you reorder questions when they are already in the exam?
Tools
- What is problematic about the current tools for making quizzes/exams?
- What is good about current tools for making quizzes/exams?
- When you create an exam, where there are questions that are randomly selected from a larger selection of questions, to each exam attempt, what are the stages to doing this?
If they have used Moodle Quiz:
- Do you use names of questions? How, for what? What could you imagine them being used for?