Note: You are currently viewing documentation for Moodle 2.4. Up-to-date documentation for the latest stable version of Moodle may be available here: Quiz reports.

Quiz reports: Difference between revisions

From MoodleDocs
m (Formulas and typos)
m (→‎Item analysis: typo corrections)
Line 31: Line 31:
==Item analysis==
==Item analysis==


This table presents processed quiz data in a way suitable for anayzing and judging the performance of each question for the function of assessment. The statistical parameters used are calculated as explained by classical test theory (ref. 1)
This table presents processed quiz data in a way suitable for analysing and judging the performance of each question for the function of assessment. The statistical parameters used are calculated as explained by classical test theory.


* '''Facility Index (% Correct)'''
* '''Facility Index (% Correct)'''
Line 37: Line 37:
This is a measure of how easy or difficult is a question for quiz-takers. It is calculated as:  
This is a measure of how easy or difficult is a question for quiz-takers. It is calculated as:  
  FI = (Xaverage) / Xmax  
  FI = (Xaverage) / Xmax  
where Xaverage is the mean credit obtained by all users attempting the item, and Xmax is the maximum credit achievable for that item. If questions can be distributed dicotomically into correct / uncorrect categories, this parameter coincides with the percentage of users that answer the question correctly.  
where Xaverage is the mean credit obtained by all users attempting the item, and Xmax is the maximum credit achievable for that item. If questions can be distributed dicotomically into correct / incorrect categories, this parameter coincides with the percentage of users that answer the question correctly.  


* '''Standard Deviation (SD)'''
* '''Standard Deviation (SD)'''
Line 45: Line 45:
* '''Discrimination Index (DI)'''
* '''Discrimination Index (DI)'''


This provides a rough indicator of the performance of each item to separate proficient vs. less-proficient users. This parameter is calculated by first dividing learners into thirds based on the overall score in the quiz. Then the average score at the analyzed item is calculated for the groups of top and bottom performers, and the average scored substracted. The matematical expression is:  
This provides a rough indicator of the performance of each item to separate proficient vs. less-proficient users. This parameter is calculated by first dividing learners into thirds based on the overall score in the quiz. Then the average score at the analysed item is calculated for the groups of top and bottom performers, and the average scored subtracted. The mathematical expression is:  
  DI = (Xtop - Xbottom)/ N  
  DI = (Xtop - Xbottom)/ N  
where Xtop is the sum of the fractional credit (achieved/maximum) obtained at this item by the 1/3 of users having tha highest grades in the whole quiz (i.e. number of correct responses in this group), and Xbottom is the analog sum for users with the lower 1/3 grades for the whole quiz.
where Xtop is the sum of the fractional credit (achieved/maximum) obtained at this item by the 1/3 of users having the highest grades in the whole quiz (i.e. number of correct responses in this group), and Xbottom is the analog sum for users with the lower 1/3 grades for the whole quiz.


This parameter can take values between +1 and -1. If the index goes below 0.0 it means that more of the weaker learners got the item right than the stronger learners. Such items should be discarded as worthless. In fact, they reduce the accuracy of the overall score for the quiz.  
This parameter can take values between +1 and -1. If the index goes below 0.0 it means that more of the weaker learners got the item right than the stronger learners. Such items should be discarded as worthless. In fact, they reduce the accuracy of the overall score for the quiz.  
Line 64: Line 64:


* '''Quiz effectiveness'''
* '''Quiz effectiveness'''
Both DC and DI can be used as powerful methods of evaluating the effectiveness of the quiz when assessing differentiation of learners. The advantage of using Discrimination Coefficient as opposed to Discrimitation Index is that the former uses information from the whole population of learners, not just the extreme upper and lower thirds. Thus, this parameter may be more sensitive to detect item performance.
Both DC and DI can be used as powerful methods of evaluating the effectiveness of the quiz when assessing differentiation of learners. The advantage of using Discrimination Coefficient as opposed to Discrimination Index is that the former uses information from the whole population of learners, not just the extreme upper and lower thirds. Thus, this parameter may be more sensitive to detect item performance.


==See also==
==See also==

Revision as of 19:10, 2 March 2007

Template:Quizzes

The quiz Results tab includes several sub tabs, one for each report. How many tabs you see depends on which report plugins your Moodle administrator has installed. There will be at least the following:

Overview

This contains the list of quiz attempts arranged in four columns:

  1. First name / Surname
  2. Started on - that contains the information about the exact time the test was started
  3. Time taken - the amount of time it took a given student to do the test
  4. Grade/x - the number of points students scored; 'x' is the maximum number of points students could score

The default view lists only the students who attempted the test. You can, however, change the display settings checking either of the two boxes (followed by clicking Go):

  1. Show students with no attempts - the list will include all the course students no matter if they did the test or not
  2. Show mark details - this extends the list with as many columns as there are questions in the test; each column is headed by 'n' (where 'n' stands for the question number)

With the Select all / Deselect all options you can check / uncheck all the names in the list, and, with selected, delete.

To sort the results by two columns, first click on the column heading you want to be the second key, and then click on the column heading you want to be the primary key.

Regrade

That tab will recalculate the quiz grades. This may become necessary if you have changed one of the questions or the grade possible for the quiz or a question.

Manual grading

Template:Moodle 1.6 Manual grading has so far been implemented for essay questions only.

Item analysis

This table presents processed quiz data in a way suitable for analysing and judging the performance of each question for the function of assessment. The statistical parameters used are calculated as explained by classical test theory.

  • Facility Index (% Correct)

This is a measure of how easy or difficult is a question for quiz-takers. It is calculated as:

FI = (Xaverage) / Xmax 

where Xaverage is the mean credit obtained by all users attempting the item, and Xmax is the maximum credit achievable for that item. If questions can be distributed dicotomically into correct / incorrect categories, this parameter coincides with the percentage of users that answer the question correctly.

  • Standard Deviation (SD)

This parameter measures the spread of answers in the response population. If all users answers the same, then SD=0. SD is calculated as the statistical standard deviation for the sample of fractional scores (achieved/maximum) at each particular question.

  • Discrimination Index (DI)

This provides a rough indicator of the performance of each item to separate proficient vs. less-proficient users. This parameter is calculated by first dividing learners into thirds based on the overall score in the quiz. Then the average score at the analysed item is calculated for the groups of top and bottom performers, and the average scored subtracted. The mathematical expression is:

DI = (Xtop - Xbottom)/ N 

where Xtop is the sum of the fractional credit (achieved/maximum) obtained at this item by the 1/3 of users having the highest grades in the whole quiz (i.e. number of correct responses in this group), and Xbottom is the analog sum for users with the lower 1/3 grades for the whole quiz.

This parameter can take values between +1 and -1. If the index goes below 0.0 it means that more of the weaker learners got the item right than the stronger learners. Such items should be discarded as worthless. In fact, they reduce the accuracy of the overall score for the quiz.

  • Discrimination Coefficient (DC)

This is another measure of the separating power of the item to distinguish proficient from weak learners. The discrimination coefficient is a correlation coefficient between scores at the item and at the whole quiz. Here it is calculated as:

DC = Sum(xy)/ (N * sx * sy) 

where Sum(xy) is the sum of the products of deviations for item scores and overall quiz scores, N is the number of responses given to this question, sx is the standard deviation of fractional scores for this question and, sy is the standard deviation of scores at the quiz as a whole.

Again, this parameter can take values between +1 and -1. Positive values indicate items that do discriminate proficient learners, whereas negative indices mark items that are answered best by those with lowest grades. Items with negative DC are answered incorrectly by the seasoned learners and thus they are actually a penalty against the most proficient learners. Those items should be avoided.

  • Quiz effectiveness

Both DC and DI can be used as powerful methods of evaluating the effectiveness of the quiz when assessing differentiation of learners. The advantage of using Discrimination Coefficient as opposed to Discrimination Index is that the former uses information from the whole population of learners, not just the extreme upper and lower thirds. Thus, this parameter may be more sensitive to detect item performance.

See also