Note: You are currently viewing documentation for Moodle 3.3. Up-to-date documentation for the latest stable version of Moodle is probably available here: Formulas: Answers and marking.

Formulas: Answers and marking

From MoodleDocs


For a subquestion to become valid, you must give a mark and define an answer for it. Also, grading criteria must be specified in order to check the correctness of a student answer.

Answer type

This question supports four answer types. Each type will accept a particular set of numbers, operators, functions and possibly algebraic variables. Depending on the purpose of the quiz, some or all of these answer types may be used.

Answer type Description
Number You can type in the standard scientific E notation such as: 3.14, 6.626e-34.
Numeric You can type in numbers and arithmetic operation + - * / ^ and ( ) and the constant π, such as 5+1/2, 2^9, 3pi.
Numerical formula You can type in everything of numeric plus a set of single variable functions sin(), cos(), tan(), asin(), acos(), atan(), exp(), log10(), ln(), sqrt(), abs(), ceil(), floor(), such as sin(pi/12), 10 ln(2).
Algebraic formula You can type in every numerical formula and any algebraic variables.

Notes:

  • Students will also need to know these rules in order to input the answers correctly.
  • The possible inputs have the following relation: Number ⊆ Numeric ⊆ Numerical formula ⊆ Algebraic formula.
  • The answer requires a list of strings for Algebraic formula and a list of numbers for the other answer types.
  • ^ in the algebraic formula means "power", not "exclusive or".
  • Juxtaposition between numbers or symbols mean multiplication.
  • The format check in the quiz interface shows a warning sign when the format is wrong for the answer type. It does not give any information about the correctness of the answer.
  • All symbols are treated as algebraic variable in the answer type of algebraic formula. Hence, you may need to hint students what symbols should be used in the question.

Correct answer

Depending on the answer type, the answer options will accept an expression that evaluates to either a list of numbers or a list of strings for Algebraic formula. The size of the list will determine how many input boxes for the part. If only one answer is required, you can specify a number or a string instead of a one element list.

For the answer type of Number, Numeric and Numerical formula, a list of numbers or a single number is required. Suppose the variables are defined, each line below is a possible answer:

pi()
[sin(pi()/2), cos(pi()/2)]
[ans[0], ans[1], ans[2]]
ans

For the answer type of Algebraic formula, a list of strings or a single string is required. Suppose the variables are defined, each line below is a possible answer:

"exp(-a x)"
" a x^2 + b y^2"
["a sin(x)", "b cos(x)"]

Note that all algebraic variables must be defined in order to be usable in the answer. For the answers above to work, you need to define the following variables:

a = 2;
b = 3;
x = {-100:100:1};
y = {1:100:1};

Grading criterion

A grading criterion is required to determine the correctness of the student answer. It requires an expression evaluated to a number whose 0 value means false and 1 value means true. Typically, the expression is either the absolute error or the relative error with a tolerance level.

For a question with only one answer, the absolute error is simply the different between the correct answer and the student answer. Hence if the correct answer is 3.2 and the student answer is 3.1, then the absolute error is |3.2-3.1| = 0.1 You may want to limit the range of correct answers, say to 0.05. In this case, you should select Absolute error < 0.05. The relative error is defined as the absolute error divided by the absolute value of correct answer. See _err and _relerr in Grading variables below for more details.

Grading variables

Most of the time, the absolute error or the relative error satisfy the grading criterion. However, sometimes there is a need for other grading criteria.

The scope of Grading variables contains all local variables and the student answers. You can define your own grading criterion with the student answers. The information related to the student answers and correct answers is stored in a set of special variables that start with an underscore as shown below:

Variable name Description
_0, _1, _2, ... The student answers. The first answer is _0 corresponding to the answer box {_0} in the part, etc.
_a The list of target answers, as defined in the answer field.
_r The list of student answers with the same size as _a. The 0th element is the same as _0, etc.
_d The list of differences between each element, where _d = diff(_a,_r);. See the Appendix for the function details.
_err The absolute error, using the Euclidean norm |a-r|, i.e. _err = sqrt(sum(map("*",_d,_d)));.
_relerr The relative error, obtained by dividing the absolute error by the norm of the correct answerd |a-r|/|a| , i.e. _relerr = _err/sqrt(sum(map("*",_a,_a)));.

Notes:

  • The corresponding input boxes of _0, _1, ... can be specified as {_0}, {_1}, ... in the part's text.
  • _relerr is not defined for algebraic answers! So _err should be used instead.
  • For non-algebraic answers, the student answer is rescaled towards the unit of the target answer. For example, if the target answer is 2 m and the student answer is 199 cm, then _a is [2] , _r is [1.99] and _0 is 1.99 . It has no effect if no unit is used.
  • When there are more than on answer, for example if _a = [100,100]; and == _r = [101,102];, then _relerr = sqrt(1*1+2*2)/sqrt(100*100+100*100) ≒ 0.0158 In this sense, the answer defined in the answer field is only a target answer because it may not be related directly to the correctness of the answer.

Manual grading criteria

Other than true or false, the grading criterion can be any number between 0 (all incorrevt) and 1 (all correct). The value 1 means that the student gets the full mark for this par see Gng scheme). A fractional value represents the partial correctness of the student answer. Note that values less than 0 are treated as 0 and that values greater than 1 are treated as 1. The following examples illustrate cases where a manual grading criterion is required.

With site themes Clean and More, the field for manual Grading Criteria is obtained by checking a box on the left hand side of the field. With the Boost theme, the check box does not display. This bug has been reported by Bernat Martinez a while ago, but has not yet been fixed.

As a teacher, you can get at the manual Grading criteria by changing the theme of the site to Clean or More, but if you do not have access to the site theme, hang on, this bug should be fixed soon!

Multiple correct answers

The possibility for a student to give any correct answer among several correct answers can be allowed by establishing manually the required Grading criteria.

Example

Give a number x that is a multiple of 7 and satisfies 40 < x < 50.

General
  Question name!       Multiples of 7
Main question
  Question text!       Give a number x that is a multiple of 7 and satisfies 40 < x < 50:
Part 1
  Part's mark*         1
  Answer type          Number
  Answer*              1
  Grading criteria*    _0 == 42 || _0 == 49
  Part's text          {_0}

In Part 1, Answer is specified as 1 (or any other number) to indicate to the system that just one answer is expected. Grading criteria _0 == 42 || _0 == 49 means that the answer is correct if it is equal to 42 or equal to 49.

The Formulas question should look like this:

Formules201801212219.png

Play it   Login info  (Open in new tab: Ctrl+Shift+Click)

Multiple criteria

The above example uses only a fixed number, so how can you grade an answer with random variations? Suppose that you now ask questions with a variable range for each question a < x< a + 10. To determine the correctness, you need to check two criteria, type the following in the Grading Variables and Grading Criteria box respectively:

criterion1 = _0 % 7 == 0;  # whether the remainder is 0. Note that true is 1 and false is 0.
criterion2 = a < _0 && _0 < a+10;   # whether the response is in the desired range.
criterion1 && criterion2

Mark for different accuracies

Suppose you want to give different marks for the accuracy of the response, say full mark for 1% absolute error and half mark for 1% to 5% absolute error. The following criteria can be used:

case1 = _err < 0.01;
case2 = _err < 0.05;
max (case1, 0.5*case2)

Trial mark sequence

This option only apply to the adaptive mode of moodle quiz.

In the adaptive mode, students are allowed to submit answer to a particular question again and again. This field defines the mark sequence that a student can get for each resubmission. Note that it actually alter the default behavior of the adaptive mode.

The input is a list of numbers separated by comma. Each number represents a fraction of the maximum mark that a student can get in the first, second, third, etc. submission. Hence, if this field has value 1, the student can try it once only.

If the sequence is ended with a comma, infinite resubmission is allow. In this case, the mark that students can get in the following unlisted trial is decreased uniformly with the value equal to the difference of last two value. Note that the minimum mark is always zero (See grading scheme).

Trial mark sequence Maximum mark for each trial Description 1 100% Only one trial is allowed 1, 0.7, 0.3 100%, 70%, 30% Three trials are allowed 1, 0.7, 0.3, 0 100%, 70%, 30%, 0% Four trials are allowed, but the last trial has no mark 1, 0, 0, 0 100%, 0%, 0%, 0% Four trials are allowed, but only the first trial has non-zero mark 1, 100%, 100%, 100%, ... Infinite trials. The difference is 0, which is repeated 1, .9, 100%, 90%, 80%, 70%, 60%, ... Infinite trials. Difference between 1st and 2nd trial is 10%, which is repeated 1, .6, 100%, 60%, 20%, 0%, 0%, ... Infinite trials. Difference between 1st and 2nd trial is 40%, which is repeated 1, .5, .3, 100%, 50%, 30%, 10%, 0%, 0%, ... Infinite trials. Difference between 2nd and 3rd trial is 20%, which is repeated

Grading scheme

The following is the grading formula used to grade a particular subquestion:

Symbol Description c Correctness. It takes value between 0 and 1. Boolean false is treated as 0 and true is treated as 1. Other values may be possible if manual condition is used (see Grading criteria) u Deduction for wrong unit (see Unit system). In the formula, it always takes value 0 if the unit is correct under Conversion rules m Default mark of the subquestion. r_n Maximum mark fraction of the n-th submission, for adaptive mode only (See Trial mark sequence) f The computed final mark. For non-adaptive mode:

f = m*c*(1-u) For adaptive mode (see moodle documentation):

f = max(r_n*m*c*(1-u)) The maximum is taken over all submissions. From the above formula, even though a student get a low mark in the first attempt, it is still possible for them to get a higher mark in the following attempt.

Appendix

Core function diff()

This function depends on context variables in addition to the function parameters. It is also the core function to compare the students' response and model answer (see Grading variables). If input X and Y are a list of string, then X[i] and Y[i] are treated as algebraic formulas and all variables in them must be defined before the location of evaluation. For example,

a = 3;
x = {1:100};
d = diff(["a x"],["a x^2"]);

Please note that the actual algebraic formulas should be "3 x" and "3 x^2" in the above case. In the above evaluation, the d will take a finite value but not close to zero because the algebraic formula are different. In general, any evaluation failure between two algebraic formula will result in a infinite value INF, so that the expression, say, sum(d) < 0.01 will always be false.

Idea of grading algebraic answer

The evaluation will take place at N randomly selected points defined in all algebraic variables. The result of diff("f(x,y)","f(x,y)") will be the root mean square difference at all evaluation points (1/N)Σi(fi-gi)2, which will converge when N tends to infinity. The N is 100 by default if it is not specified.


Back to top of page

← FORMULAS QUESTION TYPE

In other languages

Español
Français