Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Quiz statistics calculations: Difference between revisions

From MoodleDocs
No edit summary
 
No edit summary
Line 19: Line 19:
===Incomplete attempts===
===Incomplete attempts===


There is an issue about what you do when not all students have answered all questions.
There is an issue about what you do when not all students have answered all questions. Depending on how you handle these missing items, you distort the statistics in different ways.


There are basically two reasons why a student may not have answered a particular question:
* they may have chosen to omit it, or
* they may have run out of time, if the test is timed. In this case, omitted questions tend to be towards the end of the test.


Available approaches for handling this are:
* treat omitted items as having a score of 0.
* exclude any attempt with missing scores from the analysis.
* something more complicated.
''(Need to decide what to do here)''.


==Notation used in the calculations==
==Notation used in the calculations==

Revision as of 15:36, 4 December 2007

General issues

Quizzes that allow multiple attempts

For quizzes that allow multiple attempts, by default the report should only include data from the first attempt by each student. Subsequent attempts probably do not satisfy the assumptions that underlie item analysis. However, there should be an option 'Include data from all attempts', which should have a disclaimer that this may be statistically invalid either near it on screen, or possibly in the help file. (For small data sets, it may be better to include all data.)

Using the first attempt also avoids problems caused by each attempt builds on last.

Within the analysis, when multiple attempts per student are included, each attempt is treadted as an independant attempt.

Adaptive mode

Adaptive mode does not pose a problem. We just assume that each item in the test returns a score, and these scores are added up to get the test score. That is, we use the item score including penalties in the calculation of the statistics.

Certainty based marking

Similarly, should CBM, and/or negative scoring for multiple choice questions be implemented, we just use the final item score in the calculations, making sure that the formulae are not assuming that the minimum item score is zero.

Incomplete attempts

There is an issue about what you do when not all students have answered all questions. Depending on how you handle these missing items, you distort the statistics in different ways.

There are basically two reasons why a student may not have answered a particular question:

  • they may have chosen to omit it, or
  • they may have run out of time, if the test is timed. In this case, omitted questions tend to be towards the end of the test.

Available approaches for handling this are:

  • treat omitted items as having a score of 0.
  • exclude any attempt with missing scores from the analysis.
  • something more complicated.

(Need to decide what to do here).

Notation used in the calculations

Item statistics

Test statistics

Detailed item information