Difference between revisions of "Quiz statistics calculations in practice"

Jump to: navigation, search
(Skewness and Kurtosis: k2 k3 k4)
 
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{Moodle 2.0}}
 +
{{Quiz item analysis calculations}}
 +
 +
==Purpose of this page==
 +
 
Here I'm going to document what the code actually does. Documenting it here principally because I can't use TEX notation in php comments, well I guess I can but only real TEX geeks will be able to read it :-)
 
Here I'm going to document what the code actually does. Documenting it here principally because I can't use TEX notation in php comments, well I guess I can but only real TEX geeks will be able to read it :-)
  
==Notation used in the calculations==
+
==Code==
  
If you want to see things nicely formatted, you will have to copy them to a Moodle with a working TeX filter, or copy and paste the following magic into your browser's Address bar and hit enter (or get the [http://www.mathtran.org/wiki/index.php/MathTran_bookmarklet MathTran bookmarklet):''
+
The calculations below are all done in quiz/report/statistics/report.php The code refers to links to this doc in php comments.
javascript:(function(){function searchWithinNode(node,re){var pos,imgnnode,middlebit,endbitskip=0;if(node.nodeType==3){pos=node.data.search(re);if(pos>=0){middlebit=node.splitText(pos);endbit=middlebit.splitText(RegExp.lastMatch.length);imgnode=document.createElement(%22img%22);imgnode.src=%22http://www.mathtran.org/cgi-bin/mathtran?tex=%22 + encodeURI(middlebit.data).replace(%22+%22,%22%252B%22);middlebit.parentNode.replaceChild(imgnode,middlebit);}}else if(node.nodeType==1&& node.childNodes){for (var child=0; child < node.childNodes.length; ++child){searchWithinNode(node.childNodes[child], re);}}}searchWithinNode(document.body, /\$\$(.*?)\$\$/);})();
 
  
 
==Calculating MEAN of grades for all attempts by students==
 
==Calculating MEAN of grades for all attempts by students==
  
$$\displaystyle = \bar{T} = \frac{1}{S} \sum_{s \in S} T_s$$
+
<math>\displaystyle = \bar{T} = \frac{1}{S} \sum_{s \in S} T_s</math>
  
 
We calculate the MEAN of first and all attempts by :
 
We calculate the MEAN of first and all attempts by :
Line 14: Line 18:
 
* fetching two totals and counts of all grades for attempts from the db the total grades for first attempts and the rest of the attempts.
 
* fetching two totals and counts of all grades for attempts from the db the total grades for first attempts and the rest of the attempts.
 
* then to get the mean of first attempts :
 
* then to get the mean of first attempts :
** we divide the total grade for all first attempts.
+
** we divide the total grade for all first attempts by the number of all first attempts.
 
* for the mean for all attempts :
 
* for the mean for all attempts :
** we add the total of all first attempts to the total for the rest of the attempts (not the first attempt) and divide by the total count of first attempts plus the total count of the rest of the attempts.
+
** we add the total of all first attempts to the total for the rest of the attempts and divide by the total count of first attempts plus the total count of the rest of the attempts.
 +
 
 +
Before display I divide by quiz.sumgrades and times by a hundred to express this as a percentage of total possible grade.
  
 
Depending on whether we are calculating the rest of the statistics for all attempts or for just the first attempts we select the appropriate mean to use in the rest of the calculations and save a empty string or a string of sql that selects first attempts only.
 
Depending on whether we are calculating the rest of the statistics for all attempts or for just the first attempts we select the appropriate mean to use in the rest of the calculations and save a empty string or a string of sql that selects first attempts only.
Line 26: Line 32:
 
We can get sql to do the following calculations :
 
We can get sql to do the following calculations :
  
power2 = $$\sum_{s \in S} (T_s - \bar{T})^2$$
+
power2 <math>\displaystyle = \sum_{s \in S} (T_s - \bar{T})^2</math>
  
power3 = $$\sum_{s \in S} (T_s - \bar{T})^3$$
+
power3 <math>\displaystyle = \sum_{s \in S} (T_s - \bar{T})^3</math>
  
power4 = $$\sum_{s \in S} (T_s - \bar{T})^4$$
+
power4 <math>\displaystyle = \sum_{s \in S} (T_s - \bar{T})^4</math>
  
 
===Standard Deviation===
 
===Standard Deviation===
  
Test standard deviation $$\displaystyle = SD = \sqrt{V(t)} = \sqrt{\frac{1}{S - 1} \sum_{s \in S} (T_s - \bar{T})^2}$$
+
Test standard deviation <math>\displaystyle = SD = \sqrt{V(t)} = \sqrt{\frac{1}{S - 1} \sum_{s \in S} (T_s - \bar{T})^2}</math>
 +
 
 +
<math>= \sqrt{\frac{power2}{S-1}}</math>
  
$$= \sqrt{\frac{power2}{S-1}$$
+
Before display I divide by quiz.sumgrades and times by a hundred to express this as a percentage of total possible grade.
  
 
===Skewness and Kurtosis===
 
===Skewness and Kurtosis===
Line 42: Line 50:
 
So then :
 
So then :
  
$$\displaystyle m_2 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^2 = \frac{power2}{S}$$
+
<math>\displaystyle m_2 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^2 = \frac{power2}{S}</math>
 +
 
 +
<math>\displaystyle m_3 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^3 = \frac{power3}{S}</math>
 +
 
 +
<math>\displaystyle m_4 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^4 = \frac{power4}{S}</math>
 +
 
 +
 
 +
<math>\displaystyle k_2 = \frac{S}{S - 1} m_2 = \frac{S m_2}{S - 1}</math>
 +
 
 +
<math>\displaystyle k_3 = \frac{S^2}{(S - 1)(S - 2)} m_3 = \frac{S^2 m_3}{(S - 1)(S - 2)}</math>
 +
 
 +
<math>\displaystyle k_4 = \frac{S^3}{(S - 1)(S - 2)(S - 3)} \left((S + 1)m_4 - 3(S - 1)m_2^2\right)</math>
 +
 
 +
Then Skewness <math>\displaystyle = \frac{k_3}{k_2^{2/3}}</math>
 +
 
 +
and :
 +
 
 +
Kurtosis <math>\displaystyle = \frac{k_4}{k_2^2}</math>
 +
 
 +
==CIC, ER and SE==
 +
 
 +
First we fetch the average grade for each position :
 +
 
 +
<math>\displaystyle \bar{x}_p = \frac{1}{S} \sum_{s \in S} x_p(s)</math>.
 +
 
 +
Then :
 +
 
 +
<math>\displaystyle CIC = 100 \frac{P}{P - 1} \left(1 - \frac{1}{V(T)}\sum_{p \in P} V(x_p) \right)</math>
 +
 
 +
and :
 +
 
 +
<math>\displaystyle V(T) = k_2 = \frac{S}{S - 1} m_2</math>
 +
 
 +
and :
 +
 
 +
<math>\displaystyle V(x_p) = \frac{1}{S - 1} \sum_{s \in S} (x_p(s) - \bar{x}_p)^2</math>
 +
 
 +
So :
 +
 
 +
<math>\displaystyle CIC = 100 \frac{P}{P - 1} \left(1 - \frac{1}{k_2}\sum_{p \in P} \frac{1}{S - 1} \sum_{s \in S} (x_p(s) - \bar{x}_p)^2 \right)= 100 \frac{P}{P - 1} \left(1 - \frac{1}{k_2(S - 1)} \sum_{p \in P} \sum_{s \in S} (x_p(s) - \bar{x}_p)^2 \right) </math>
 +
 
 +
We fetch :
 +
 
 +
<math>\displaystyle \sum_{s \in S} (x_p(s) - \bar{x}_p)^2 </math>
 +
 
 +
From the db for all questions and then sum them all so we have :
 +
 
 +
<math>\displaystyle sum =\sum_{p \in P} \left( \sum_{s \in S} (x_p(s) - \bar{x}_p)^2 \right)</math>
 +
 
 +
Then we do :
  
$$\displaystyle m_3 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^3 = \frac{power3}{S}$$
+
<math>sumofvarianceforallpositions = sum/(S - 1)</math>
  
$$\displaystyle m_4 = \frac{1}{S} \sum_{s \in S} (T_s - \bar{T})^4 = \frac{power4}{S}$$
+
Then :
  
 +
<math>\displaystyle CIC = 100 \frac{P}{P - 1} \left(1 - \frac{sumofvarianceforallpositions}{k_2} \right) </math>
  
Then Skewness $$\displaystyle = \frac{k_3}{k_2^{2/3}}$$
+
And :
  
$$\displaystyle k_2 = \frac{S}{S - 1} m_2 = \frac{S m_2}{S - 1}$$
+
<math>\displaystyle ER = 100 \sqrt{1 - \frac{CIC}{100}}</math>
  
$$\displaystyle k_3 = \frac{S^2}{(S - 1)(S - 2)} m_3 = \frac{S^2 m_3}{(S - 1)(S - 2)}$$
+
And :
  
$$\displaystyle k_4 = \frac{S^3}{(S - 1)(S - 2)(S - 3)} \left((S + 1)m_4 - 3(S - 1)m_2^2\right)$$
+
<math>\displaystyle SE = \frac{ER}{100}SD</math>
  
$$= \frac{S^3 ((S + 1)m_4 - 3(S - 1)m_2^2)}{(S - 1)(S - 2)(S - 3)}$$
+
Since SE is a grade we divide it by the maximum possible grade and times by 100 to get a percentage of the max possible grade for the whole quiz.

Latest revision as of 08:29, 26 March 2014

Moodle 2.0


Purpose of this page

Here I'm going to document what the code actually does. Documenting it here principally because I can't use TEX notation in php comments, well I guess I can but only real TEX geeks will be able to read it :-)

Code

The calculations below are all done in quiz/report/statistics/report.php The code refers to links to this doc in php comments.

Calculating MEAN of grades for all attempts by students

We calculate the MEAN of first and all attempts by :

  • fetching two totals and counts of all grades for attempts from the db the total grades for first attempts and the rest of the attempts.
  • then to get the mean of first attempts :
    • we divide the total grade for all first attempts by the number of all first attempts.
  • for the mean for all attempts :
    • we add the total of all first attempts to the total for the rest of the attempts and divide by the total count of first attempts plus the total count of the rest of the attempts.

Before display I divide by quiz.sumgrades and times by a hundred to express this as a percentage of total possible grade.

Depending on whether we are calculating the rest of the statistics for all attempts or for just the first attempts we select the appropriate mean to use in the rest of the calculations and save a empty string or a string of sql that selects first attempts only.

Calculating Standard Deviation, Skewness and Kurtosis of grades for all attempts by students

In order not to have to load potentially large datasets into memory we get the DB to do the bulk of the work doing these calculations.

We can get sql to do the following calculations :

power2

power3

power4

Standard Deviation

Test standard deviation

Before display I divide by quiz.sumgrades and times by a hundred to express this as a percentage of total possible grade.

Skewness and Kurtosis

So then :


Then Skewness

and :

Kurtosis

CIC, ER and SE

First we fetch the average grade for each position :

.

Then :

and :

and :

So :

We fetch :

From the db for all questions and then sum them all so we have :

Then we do :

Then :

And :

And :

Since SE is a grade we divide it by the maximum possible grade and times by 100 to get a percentage of the max possible grade for the whole quiz.