Student projects/Usability issues/T4: Difference between revisions
No edit summary |
|||
| Line 146: | Line 146: | ||
|- | |- | ||
|} | |} | ||
==== Preparation of the scenarios, tasks, questions and materials that will be presented to the user in the usability test==== | ==== Preparation of the scenarios, tasks, questions and materials that will be presented to the user in the usability test==== | ||
==== Test set-up preparation of the usability test ==== | ==== Test set-up preparation of the usability test ==== | ||
Revision as of 00:28, 14 July 2008
Usability problems found in the Fluid project
At Research of Moodle usability problems mentioned in the Fluid project one can find more information about this issue.
Usability problems found in the tracker
In order to classify the importance of the different usability problems found in the tracker, they are put in a ranking. The solution found of this issues can be found at this wiki page
Moreover, after having heard the needs of several people being with Moodle as teachers and administrators, two main issues should be improved:
Usability problems found when talking with Moodle users
Properly order of selections when using the tab key of the keyboard
This functionality would help both experience and new users. Usability problems have been found mainly in two different situations:
- When you press the tab key it goes from one item to the help button, which is not really usable because the help button is not frequently used.
- When you edit a text and you press a tab in order to go to the text field and write something, the tab goes first to all the writing options (bold, italics, etc.) making it not usable for the user that wants usually to get directly to the text field.
Gradebook
The gradebook perhaps is the biggest Moodle usability issue although students are not afected. The gradebook, with so many options, makes difficult for the teachers to find the issue.
For instance, there is a problem when choosing and action or when selectiong category items, the aggregation or editing category homework. So some usability testing should be done in the gradebook in order to improve the usability.
Some quick grading tutorials have been writen in order to help teachers and some complaints and some grading questions have been posted to the Moodle Forum. It must also be considered that Fluid project analyzes the grading of Moodle 1.8.2 and it is not the same as Moodle version 1.9, so it is necessary to do some usability tests in order to improve grading usability.
Next the test plan of the gradebook usability test will be described:
Objectives of the gradebook usability test
The profile of the users who will participate of the usability test are the following:
| #ID | User | Profile | Objectives |
|---|---|---|---|
| 1 | Administrator and teacher | He is really used to the administration interfaces. He has been using Moodle as an administrator and teacher for several years | Test de functionalities, the improvements he has thought while using the grading system and some comments he could have heard from other teachers |
| 2 | Prospective teacher familiar with new technologies | He is computer scientist so he is used to new applications. He will use Moodle as a student but he has never used Moodle in a teacher role. | Usability issues that a new teacher familiar with Moodle user view and new applications can find. |
| 3 | Teacher not familiar with new technologies | She is not used to new applications. She has never used Moodle neither as a student nor a teacher. | Usability issues that a new teacher neither familiar with Moodle nor new applications can find. |
The objective of the developper is to modify the grading design with both quantitative and qualitative information of the usability test.
Definition of the tasks that will be evaluated in the usability test
All tasks will be done to all the participants of the usability test.
T1: Items weighted by their point values
Description:
- In a sample course of three students, create three assignments titled "10 point assignment", "20 point assignment", and "50 point midterm".
- Assign scores to each of the assignments (simple weighted mean of grades)
- Verify that the course grade is as you desire.
- Change various scores and confirm that the gradebook is computing the course grade in the way you want. If not, you might want to make additional changes to your gradebook setup.
T2: Items all given equal weight
Description: In some courses, the number of points(or marks) given to an item is arbitrary and is not a factor in deciding its weight in computing the course grade. In this case, each assignment is given equal weight, so a 10 point essay might be worth just as much as a 20 point essay. With this grading strategy, one would first convert each item to a percentage value and then average the percentages in computing the course total.
T3: Course grade weighted by category
Description: In many courses, the teacher may want to assign different elements of performance (such as daily work and tests) to defined percentages in determining a course grade. Moodle calls this Aggregation method of computing course grades Weighted mean of grades because the course grade is computed by weights that the teacher assigns to categories (or items).
Definition of the variables that will be tested in the usability test
The following variables will be analyzed:
- Verbal protocols in order to extract qualitative information.
- Corporal expressions in order to extract qualitative information.
- User expectatives: In order to have a qualitative analysis specially of the search and visualization of the information.
- Reaction time: the time the user spend knowing how to do an activity, to orientate, etc.
- Navegation patterns: This variable includes:
- Knowledge of where is the start page
- Knowledge of the content of the pages
- Knowledge of how to find clues or more information of an item.
- Knowledge of what buttons or resources are not used
- Knowledge of the reason that this buttons or resources are not used
- Clicks and "traza"
- Knowledge of his/her movement all over the page
- Knowledge of how he / she goes to the next page
- Quantitative errors (success / problems)
- Time spent performing the task
| ID | Parameter | Variable | Quantitative analysis | Qualitative analysis |
|---|---|---|---|---|
| 1 | Time to do a task | Seconds | Average and standard desviation of all users | See if there are tasks where the user spends a considerable amount of time |
| 2 | Time to know how to do a task | Seconds | Average and standard desviation of all users | See if there are tasks where the user spends a considerable amount of time |
| 3 | Percentage of sucess / fail attempts | Percentage | Average and standard desviation of all users | See if there are tasks with a high percentage of failure. |
| 4 | Questions after the interview | Grading scale from 1 to 5 | Average and standard desviation of all users | Get the suggermients of the users |
| 5 | Pressed links | Number of clicks to perform a task | Average and standard desviation of all users | Analyze tasks with a great number of clicks |
| 6 | Information about help | # of times that help is used | Average and standard desviation of times that help is needed by all users. | See what tasks / users need more help |
