Student projects/Usability issues/T4

Usability problems found in the Fluid project

At Research of Moodle usability problems mentioned in the Fluid project one can find more information about this issue.

Usability problems found in the tracker

In order to classify the importance of the different usability problems found in the tracker, they are put in a ranking. The solution found of this issues can be found at this wiki page

Moreover, after having heard the needs of several people being with Moodle as teachers and administrators, two main issues should be improved:

Usability problems found when talking with Moodle users

Properly order of selections when using the tab key of the keyboard

This functionality would help both experience and new users. Usability problems have been found mainly in two different situations:

• When you press the tab key it goes from one item to the help button, which is not really usable because the help button is not frequently used.
• When you edit a text and you press a tab in order to go to the text field and write something, the tab goes first to all the writing options (bold, italics, etc.) making it not usable for the user that wants usually to get directly to the text field.

Gradebook

The gradebook perhaps is the biggest Moodle usability issue although students are not afected. The gradebook, with so many options, makes difficult for the teachers to find the issue.

For instance, there is a problem when choosing and action or when selectiong category items, the aggregation or editing category homework. So some usability testing should be done in the gradebook in order to improve the usability.

Some quick grading tutorials have been writen in order to help teachers and some complaints and some grading questions have been posted to the Moodle Forum. It must also be considered that Fluid project analyzes the grading of Moodle 1.8.2 and it is not the same as Moodle version 1.9, so it is necessary to do some usability tests in order to improve grading usability.

Next the test plan of the gradebook usability test will be described:

Objectives of the gradebook usability test

The profile of the users who will participate of the usability test are the following:

Objectives of each participant of the usability test

#ID	User	Profile	Objectives
1	Administrator and teacher	He is really used to the administration interfaces. He has been using Moodle as an administrator and teacher for several years	Test de functionalities, the improvements he has thought while using the grading system and some comments he could have heard from other teachers
2	Prospective teacher familiar with new technologies	He is computer scientist so he is used to new applications. He will use Moodle as a student but he has never used Moodle in a teacher role.	Usability issues that a new teacher familiar with Moodle user view and new applications can find.
3	Teacher not familiar with new technologies	She is not used to new applications. She has never used Moodle neither as a student nor a teacher.	Usability issues that a new teacher neither familiar with Moodle nor new applications can find.

The objective of the developper is to modify the grading design with both quantitative and qualitative information of the usability test.

Definition of the tasks that will be evaluated in the usability test

All tasks will be done to all the participants of the usability test.

T1: Items weighted by their point values

Description:

• In a sample course of three students, create three assignments titled "10 point assignment", "20 point assignment", and "50 point midterm".
• Assign scores to each of the assignments (simple weighted mean of grades)
• Verify that the course grade is as you desire.
• Change various scores and confirm that the gradebook is computing the course grade in the way you want. If not, you might want to make additional changes to your gradebook setup.

T2: Items all given equal weight

Description: In some courses, the number of points(or marks) given to an item is arbitrary and is not a factor in deciding its weight in computing the course grade. In this case, each assignment is given equal weight, so a 10 point essay might be worth just as much as a 20 point essay. With this grading strategy, one would first convert each item to a percentage value and then average the percentages in computing the course total.

T3: Course grade weighted by category

Description: In many courses, the teacher may want to assign different elements of performance (such as daily work and tests) to defined percentages in determining a course grade. Moodle calls this Aggregation method of computing course grades Weighted mean of grades because the course grade is computed by weights that the teacher assigns to categories (or items).

Definition of the variables that will be tested in the usability test

The following variables will be analyzed:

1. Verbal protocols in order to extract qualitative information.
2. Corporal expressions in order to extract qualitative information.
3. User expectatives: In order to have a qualitative analysis specially of the search and visualization of the information.
4. Reaction time: the time the user spend knowing how to do an activity, to orientate, etc.
5. Navegation patterns: This variable includes:

• Knowledge of where is the start page
• Knowledge of the content of the pages
• Knowledge of how to find clues or more information of an item.
• Knowledge of what buttons or resources are not used
• Knowledge of the reason that this buttons or resources are not used
• Clicks and "traza"
• Knowledge of his/her movement all over the page
• Knowledge of how he / she goes to the next page

1. Quantitative errors (success / problems)
2. Time spent performing the task

Metrics to measure this variables

ID	Parameter	Variable	Quantitative analysis	Qualitative analysis
1	Time to do a task	Seconds	Average and variance of all users	See if there are tasks where the user spends a considerable amount of time
2	Time to know how to do a task	Seconds	Average and variance of all users	See if there are tasks where the user spends a considerable amount of time
3	Percentage of sucess / fail attempts	Percentage	Average and variance of all users	See if there are tasks with a high percentage of failure.
4	Questions after the interview	Grading scale from 1 to 5	Average and variance of all users	Get the suggermients of the users
5	Pressed links	Number of clicks to perform a task	Average and variance of all users	Analyze tasks with a great number of clicks
6	Information about help	# of times that help is used	Average and variance of times that help is needed by all users.	See what tasks / users need more help

a