Question Engine 2:Numerical tolerances
Analysis of the various tolerances for numerical and calculated question type answer (MDL-31837)
This page is currently built and NOT completed Pierre Pichet 22:59, 14 August 2012 (WST)
In grading a numerical response, the (student) numerical value is compared to the answer numerical value.
This comparison allow a tolerance that is associated with the answer and that can be expressed in various ways.
For calculated question answers the tolerance can be of 3 different types: relative, nominal or geometric.
Limits of real numbers in PHP
The following is from http://www.php.net/manual/en/language.types.float.php
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).
Warning Floating point precision
Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error propagation must be considered when several operations are compounded.
Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....
So never trust floating number results to the last digit, and do not compare floating point numbers directly for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.
Since the internal representation is in base 2, 0 and 1 will have the same exponent so the precision ( i.e. init_get('precision') should have the same value.?
....
1,9 version
function get_tolerance_interval(&$answer) {
// No tolerance
if (empty($answer->tolerance)) {
$answer->tolerance = 0;
}
// Calculate the interval of correct responses (min/max)
if (!isset($answer->tolerancetype)) {
$answer->tolerancetype = 2; // nominal
}
// We need to add a tiny fraction depending on the set precision to make the
// comparison work correctly. Otherwise seemingly equal values can yield
// false. (fixes bug #3225)
$tolerance = (float)$answer->tolerance + ("1.0e-".ini_get('precision'));
switch ($answer->tolerancetype) {
case '1': case 'relative':
/// Recalculate the tolerance and fall through
/// to the nominal case:
$tolerance = $answer->answer * $tolerance;
// Do not fall through to the nominal case because the tiny fraction is a factor of the answer
$tolerance = abs($tolerance); // important - otherwise min and max are swapped
$max = $answer->answer + $tolerance;
$min = $answer->answer - $tolerance;
break;
case '2': case 'nominal':
$tolerance = abs($tolerance); // important - otherwise min and max are swapped
// $answer->tolerance 0 or something else
if ((float)$answer->tolerance == 0.0 && abs((float)$answer->answer) <= $tolerance ){
$tolerance = (float) ("1.0e-".ini_get('precision')) * abs((float)$answer->answer) ; //tiny fraction
} else if ((float)$answer->tolerance != 0.0 && abs((float)$answer->tolerance) < abs((float)$answer->answer) && abs((float)$answer->answer) <= $tolerance){
$tolerance = (1+("1.0e-".ini_get('precision')) )* abs((float) $answer->tolerance) ;//tiny fraction
}
$max = $answer->answer + $tolerance;
$min = $answer->answer - $tolerance;
break;
case '3': case 'geometric':
$quotient = 1 + abs($tolerance);
$max = $answer->answer * $quotient;
$min = $answer->answer / $quotient;
break;
default:
error("Unknown tolerance type $answer->tolerancetype");
}
$answer->min = $min;
$answer->max = $max;
return true;
}
2,0 actual code
class qtype_numerical_answer extends question_answer {
/** @var float allowable margin of error. */
public $tolerance;
/** @var integer|string see {@link get_tolerance_interval()} for the meaning of this value. */
public $tolerancetype = 2;
public function __construct($id, $answer, $fraction, $feedback, $feedbackformat, $tolerance) {
parent::__construct($id, $answer, $fraction, $feedback, $feedbackformat);
$this->tolerance = abs($tolerance);
}
public function get_tolerance_interval() {
if ($this->answer === '*') {
throw new coding_exception('Cannot work out tolerance interval for answer *.');
}
// We need to add a tiny fraction depending on the set precision to make
// the comparison work correctly, otherwise seemingly equal values can
// yield false. See MDL-3225.
$tolerance = (float) $this->tolerance + pow(10, -1 * ini_get('precision'));
switch ($this->tolerancetype) {
case 1: case 'relative':
$range = abs($this->answer) * $tolerance;
return array($this->answer - $range, $this->answer + $range);
case 2: case 'nominal':
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision')) *
max(1, abs($this->answer));
return array($this->answer - $tolerance, $this->answer + $tolerance);
case 3: case 'geometric':
$quotient = 1 + abs($tolerance);
return array($this->answer / $quotient, $this->answer * $quotient);
default:
throw new coding_exception('Unknown tolerance type ' . $this->tolerancetype);
}
}
public function within_tolerance($value) {
if ($this->answer === '*') {
return true;
}
list($min, $max) = $this->get_tolerance_interval();
return $min <= $value && $value <= $max;
}
}
2,0 vs 1,9 differences
case relative
The math treatment is equivalent as
$this->tolerance = abs($tolerance);
case geometric
The math treatment is equivalent as
$this->tolerance = abs($tolerance);
case nominal
The use of
pow(10, -1 * ini_get('precision')) * max(1, abs($this->answer));
The 'tiny fraction' part of the tolerance is not related to the answer when the answer is less than 1.
We should get back to the 1,9 equivalent code first by removing the max function
case 2: case 'nominal':
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision')) * abs($this->answer);
return array($this->answer - $tolerance, $this->answer + $tolerance);
The 1,9 code has a special treatment of the answer == 0 .
Should we keep the code ?
Answer is 0
What is the precision of 0 ? It cannot be 0 , it only can be the precision of the digital part of the number i.e. pow(10, -1 * ini_get('precision')). As a mather of fact the exponent part of 0 is the same one as 1 so its precision is the same. We should not forget that the "turning point" (positive to negative) of real number exponent value is 1 not 0.
So to get the "tiny number" we should multiply the pow(10, -1 * ini_get('precision')) by $this->answer except when $this->answer == 0 when it should be 1.
Tolerance is 0
The 0 value should be used as is, the php should only retain the tiny part as a final result;
Tolerance < pow(10, -1 * ini_get('precision'))
The tolerance i.e 1e-26 should become the number that control the process and define the "tiny number". So
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision'))* $this->tolerance ;
Such a case could result when the 0 answer comes from something like 0,3333333e-24-1/3e24 i.e.
The addition process of
if ($this->answer === 0.0){
if($this->tolerance === 0.0){
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision'))* $this->tolerance ;
} else if( $this->tolerance < pow(10, -1 * ini_get('precision')){ // the 0
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision'))* $this->tolerance ;
}else {
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision')) ;
}
} else {
$tolerance = $this->tolerance + pow(10, -1 * ini_get('precision')) * abs($this->answer);
}
If the answer is a smaller number than the pow(10, -1 * ini_get('precision')) i.e. 1e-24, then the tiny fraction calculation
OUPS !!! ....
...
