Formulas question documentation

< Formulas question type

Introduction

This question type aims to be generic so that various type of non-trivial questions can be created easily. The variable system and answer boxes allow great flexibility to define question and simplify the job of making complex questions. In this documentation, various parts of this question type are described, including the formatting, variables, grading, units and trial mark sequence.

For creating a simple Formulas question, see the Tutorial.

Placeholders

You can insert a part's text and an answer box at a given point using placeholders. There are two kinds of placeholder: the part's text placeholder and the answer box placeholder.

Part's text placeholders

This kind of placeholder allows you to insert a part's text at a particular point in the main question. The placeholder name is a string starting with # and followed by any characters of [A-Z a-z 0-9] , such as #1, #a, #a1, #1a or #SubQuestion01.

At the point of insertion in the main question text, the placeholder name must be enclosed by curly braces { }. In the parts, the placeholder name is specified in the field called Placeholder name.

Example

Use part's text placeholders in the main question to insert the part's text of three parts.

General
  Question name*       A formulas question with three parts and part's text placeholders
Main question
  Question text*       With part's text placeholders
                       Two subquestions on the addition:
                       {#1a}
                       {#1b}
                       One subquestion on the subtraction:
                       {#2}
Part 1
  Part's mark*         1
  Answer type          Number
  Answer*              7
  Grading criteria*    Absolute error == 0
  Placehoder name      #1a
  Part's text          What is 3 + 4?
Part 2
  Part's mark*         1
  Answer type          Number
  Answer*              9
  Grading criteria*    Absolute error == 0
  Placehoder name      #1b
  Part's text          What is 4 + 5?
Part 3
  Part's mark*         1
  Answer type          Number
  Answer*              3
  Grading criteria*    Absolute error == 0
  Placehoder name      #2
  Part's text          What is 9 − 6?

The formulas question should look like this:

You could obtain the same question as above with the following, but clearly it is more convenient in general to use placeholders:

General
  Question name*       A formulas question with three parts and no part's text placeholders
Main question
  Question text*       Without part's text placeholders
                       Note: In general, it is more convenient to use placeholders as is done above.
Part 1
  Part's mark*         1
  Answer type          Number
  Answer*              7
  Grading criteria*    Absolute error == 0
  Placehoder name
  Part's text          Two subquestions on the addition:
                       What is 3 + 4?
Part 2
  Part's mark*         1
  Answer type          Number
  Answer*              9
  Grading criteria*    Absolute error == 0
  Placehoder name
  Part's text          What is 4 + 5?
Part 3
  Part's mark*         1
  Answer type          Number
  Answer*              3
  Grading criteria*    Absolute error == 0
  Placehoder name
  Part's text          One subquestion on the subtraction:
                       What is 9 − 6?

Note 

Although the designation of the part's text placeholders is quite flexible, a simple designation such as #1, #2, #3, etc. is often sufficient.

Answer box placeholders

This kind of placeholder provides a simple way to arrange the answer boxes in the part's text. It is particularly useful when the subquestion requires more than one answer. This flexibility allows you to arrange the answer boxes in the form of a matrix, coordinates, embedding into the question text or any other way you want.

There are two answer box types: the quantity box and the unit box. The placeholder names are _0 for the first quantity box, _1 for the second quantity box, etc, and _u for the unit box. The number of placeholder names that can be used is the same as the number of elements entered in the answer field, plus one if the unit is specified.

Example

A subquestion that has two elements in the answer field and a common unit:

Main question
  Question text*       A formulas question with answer box placeholders
Part 1
  Part's mark*         1
  Answer type          Number
  Answer*              [100, 200]
  Grading criteria*    Absolute error == 0
  Unit                 mm
  Part's text          What are the coordinates of point A?
                       (x,y) = ({_0}, {_1}) {_u}

The formulas question should look like this, where the correct answers are 100 in the first box, 200 in the second box and mm in the third box:

Each answer box placeholder can only be used once in a part. For example, none of the following will be accepted:

  Part's text          {_0}{_0}          (You cannot use {_0} twice)

  Part's text          {_0}{_1}{_1}      (You cannot use {_1} twice)

  Part's text          {_0}{_u}{_1}{_u}  (You cannot use {_u} twice)

  Part's text          {_3}{_3}          (You cannot use {_3} twice)

The placeholder names do not have to appear in any particular order such as the increasing order {_0}{_1}{_2}...{_u}. The order of the quantity boxes and the unit box correspond to the order of the placeholder names in the part's text. For example, the following:

Part 1
 Answer*              [0, 100, 200, 300]
 Unit                 mm
 Part's text          {_2}{_0}{_3}{_u}{_1}

gives

Neither do the answers have to be in increasing order and we would get exactly the same as above with the following:

 Answer*              [200, 0, 300, 100]
 Unit                 mm
 Part's text          {_0}{_1}{_2}{_u}{_3}

By default, all missing placeholder are automatically appended at the end. That is {_0}{_u} for one answer subquestion, and {_0}{_1}{_u} for two answers subquestion.

The box {_0} corresponds to the special variable _0 in the Grading variables.

Quantity and unit boxes side by side or merged

When the quantity placeholder {_0} and the unit placeholder {_u} are directly side by side, such as {_0}{_u}, the quantity box and the unit box merge into a longer single answer box in which both quantity and unit must be entered.

When the quantity placeholder and the unit placeholder are not directly side by side, i.e. when they are separated by a space or any other string of characters, the answer boxes do not merge and hence they are displayed side by side.

Example

Answer with both quantity and unit:

Main question
  Question text*       What is 2 mm + 3 mm?
Part 1
  Part's mark*         1
  Answer type          Number
  Answer*              5
  Grading criteria*    Absolute error == 0
  Unit                 mm
  Part's text          {_0} {_u}  Separate placeholders give separate quantity and unit boxes.
Part 2
  Part's mark*         1
  Answer type          Number
  Answer*              5
  Grading criteria*    Absolute error == 0
  Unit                 mm
  Part's text          {_0}{_u}  Attached placeholders give a merged quantity and unit box.

The formulas question should look like this:

Example

System of variables

Variables can be used to substitute question texts, define the answers and specify grading criteria. One of the main purpose of the system of variables is to simplify the task to make variation of the questions. Since the variables are generated at the beginning of a quiz, the quiz will be halted if any errors occur. Hence, in order to minimize the error, the system is designed to only have deterministic variables type, constant length lists and no branching.

Scope of variables

There are four types of variables (random, global, local and grading) which you can define and manipulate in the following fields:

Field	Scope of application
Random variables	The place to define the variation of variables for the whole questions.
Global variables	The scope includes all (instantiated) random variables.
Local variables	The scope includes all global variables. Note: Each subquestion has its own local variables scope.
Grading variables	The scope includes all local variables, plus special variables (e.g. _0, _1 ) depending on students' response.

Furthermore, you can use the four types of variables in the following fields:

Field	Scope of application
Main text	All global variables can be used in the substitution
Subquestion text	All local variables for the subquestion can be used in the substitution
Answer	All local variables for the subquestion can be used in the expression
Grading criteria	All grading variables can be used in the expression

Names of variables

A variable name is a string of alphanumeric characters (a-z, A-Z, 0-9 and _) that cannot start with a number or the underscore. Examples of valid variable names are:

x, y1, z_1, foo_bar, myFirstVariable

Variable names are case-sensitive so that, for example, the following are four different variables:

foobar, FooBar, fooBar, FOOBAR

Types of variables

Each variable will be assigned implicitly with a type. A type is either number, string or algebraic variable, plus list of numbers or list of strings. List is defined as the elements enclosed by [ and ]. An element in list can be referred using the syntax A[b], for example [4,5,6][0] gives the first number, that is, 4. These types are listed below:

Type	Description
Number	A number, for example: 1.2e-3
String	Characters enclosed by two double quotes, for example: "Hello"
List of number	Numbers enclosed by [ and ] , for example: [4,5,6] , The equivalent short hand syntax are [4:7] or [4:7:1.]
List of string	Strings enclosed by [ and ] , for example: ["A", "B", "C"]
Algebraic variable	It is simply a set of numbers (see syntax) defined in the non-random variable scope, for example: {1:100}

Random variables

During the quiz creation, each random variable will be assigned by ONE value defined by the expression. Hence, each student can have their own set of values for the quiz attempt.

A random variable has different syntax than other variables and it is only defined in the field Random variables. These variables can be defined by assigning a set of elements or a shuffling of a list. The probability of selecting each element is equal so that each element has equal chance to be drawn. There are three main types of expressions:

Set of elements

A set of elements is elements enclosed by { and }. The element can be either a number, a string, a number list or a string list. For example, the variable below is a set of list of numbers, and the probability for element [2,3] is 1/5:

F = {[0,0], [1,1], [2,4], [3,9], [4,16]};

Set of numbers

A set of numbers is numbers enclosed by { and }. You can also specify a range in the format of {start:stop:interval}, where the numbers satisfying (start + n*interval) < stop, n = 0,1,2,3,..., will be generated. If the interval is not specified, value 1 will be used. Note that the stop value is not included by definition, but may be included for non-integers due to numerical errors. The set of numbers defined below demonstrate different range syntaxes. The probabilities of the elements in each set are 1/8, 1/10 and 1/20 respectively.

A = {1,2,3,4,5,6,7,8};

B = {0:1:0.1};

C = {1:10, 10:100:10, 100, 200};

Shuffled list

A list can be passed to the shuffle function. In the example below, the instantiated variable S with take one of the permutation of the input list, say [4,3,2,5] or [2,5,4,2] The probability of each list is therefore 1/4! = 1/24.

S = shuffle([2,3,4,5]);

Variable assignments

Variable assignments allow you to define and manipulate variables. It can be defined in the option field of global variables, local variables and grading variables.

Component	Definition
Expression	Any combination of numbers and variables together with operators and functions listed in the Appendix. Typically, it is just a simple mathematical formula evaluated to a number. The variables used in the expression must be defined before.
Assignment	Used to assign the evaluated result of an expression to a variable, in the form of 'name = expression ;'
For loop	Allows a simple iteration in the form of 'for(element:list)'. All elements in the list are iterated.

Examples of assignments are shown below:

Example	Comments
a = 1;	# text after "#" and the end of line are treated as comment
b = exp(3.333);
d = round(b, 1);	# the function round() can round the number to the desired decimal
e = 1 + sin(2) + pow(a,2);
A = [1,2,3];
x = A[0];
y = A[a];
z = A[2];
w = A[0] + A[1] + A[2];
m = max(x, y);
distance = sqrt(x*x + y*y + z*z);
theta = atan2(y, x);
smaller = x < y;	# smaller is 1
con = (x < y) + (y < z);	# con is 2
B = fill(3,0);	# B is now [0,0,0]
C = map("sqrt",fill(3,16));	# square root of [16,16,16], so C is now [4,4,4]
s = 0;
for (i:A) s = s + i;	# s is 6 after the loop
for (i:[0:3]) {     B[i] = sum(map("+",A,fill(3,i)));    }	# B will be [6,9,12]
p = pick(a+9,"","A","B");	# pick() always choose the first element if index out of range
u = {-3,-2,-1,1:100};	# u and v define algebraic variables, the numbers are the points for evaluation
v = {-100:100:1};

Notes:

• Important! Index out of range cannot be checked by the validation during question save, so you have to check it yourself. Otherwise, the quiz initialization for some students may fail. Use pick() as a safe variant if applicable.
• Logical true is treated as 1 and false is 0, so the variable con above gets the value 2.
• There is no branching, however, you may use the ternary operator (condition) ? (true) : (false) for numbers or the pick() function for general case.
• The available functions are listed in the Appendix. There are many additional functions for the numeric list operation, and a few for the list of strings.

Using variable in text

It is simple to substitute the variables in the text, you only need to enclose the corresponding variables with { }. Each text field has a scope of variables. All variables x of either number or string in the scope of the text can be used to replace the corresponding placeholder {x} in the text.

It is also possible to evaluate an expression directly in the text by adding an equal sign at the beginning of the bracket such as {=x/1000}. It is easier to use if the named variables are not required. However, no error check is done unless the question is being instantiated in the quiz. An example is the rescaling of meters to kilometesr below:

What is the speed of the rocket if it travels with distance {=x/1000} km in {t} s?

Answer and grading criteria

For a subquestion to become valid, you must give a mark and define an answer for it. Also, grading criteria must be specified in order to check the correctness of a student answer.

Answer type

This question supports four answer types. Each type will accept a particular set of numbers, operators, functions and possibly algebraic variables. Depending on the purpose of the quiz, some or all of these answer types may be used.

Answer type	Description
Number	You can type in the standard scientific E notation such as: 3.14, 6.626e-34.
Numeric	You can type in numbers and arithmetic operation + - * / ^ and ( ) and the constant pi such as: 5+1/2, 2^9, 3pi.
Numerical formula	You can type in everything of numeric plus a set of single variable functions sin(), cos(), tan(), asin(), acos(), atan(), exp(), log10(), ln(), sqrt(), abs(), ceil(), floor() such as: sin(pi/12), 10 ln(2).
Algebraic formula	You can type in every numerical formula and any algebraic variables.

Notes:

• Students will also need to know these rules in order to input the answers correctly.
• The possible inputs have the following relation: Number ⊆Š‚ Numeric ⊆Š‚ Numerical formula ⊆Š‚ Algebraic formula.
• The answer requires a list of strings for Algebraic formula and a list of numbers for the other answer types.
• "^" in the algebraic formula means "power", not "exclusive or".
• Juxtaposition between numbers or symbols mean multiplication.
• The format check in the quiz interface shows a warning sign when the format is wrong for the answer type. It does not give any information about the correctness of the answer.
• All symbols are treated as algebraic variable in the answer type of algebraic formula. Hence, you may need to hint students what symbols should be used in the question.

Correct answer

Depending on the answer type, the answer options will accept an expression that evaluates to either a list of numbers or a list of strings for Algebraic formula. The size of the list will determine how many input boxes for the part. If only one answer is required, you can specify a number or a string instead of a one element list.

For the answer type of Number, Numeric and Numerical formula, a list of numbers or a single number is required. Suppose the variables are defined, each line below is a possible answer:

pi()
[sin(pi()/2), cos(pi()/2)]
[ans[0], ans[1], ans[2]]
ans

For the answer type of Algebraic formula, a list of strings or a single string is required. Suppose the variables are defined, each line below is a possible answer:

"exp(-a x)"
" a x^2 + b y^2"
["a sin(x)", "b cos(x)"]

Note that all algebraic variables must be defined in order to be usable in the answer. For the answers above to work, you need to define the following variables:

a = 2;
b = 3;
x = {-100:100:1};
y = {1:100:1};

Grading criterion

A grading criterion is required to determine the correctness of the student answer. It requires an expression evaluated to a number whose 0 value means false and 1 value means true. Typically, the expression is either the absolute error or the relative error with a tolerance level.

For a question with only one answer, the absolute error is simply the different between the correct answer and the student answer. Hence if the correct answer is 3.2 and the student answer is 3.1, then the absolute error is |3.2-3.1| = 0.1 You may want to limit the range of correct answers, say to 0.05. In this case, you should select 'Absolute error < 0.05'. The relative error is defined as the absolute error divided by the absolute value of correct answer. See _err and _relerr in Grading variables below for more details.

Grading variables

Most of the time, the absolute error or the relative error satisfy the grading criterion. However, sometimes there is a need for other grading criteria.

The scope of Grading variables contains all local variables and the student answers. You can define your own grading criterion with the student answers. The information related to the student answers and correct answers is stored in a set of special variables that start with an underscore as shown below:

Variable name	Description
_0, _1, _2, ...	The student answers. The first answer is _0 corresponding to the answer box {_0} in the part, etc.
_a	The list of target answers, as defined in the answer field.
_r	The list of student answers with the same size as _a. The 0th element is the same as _0 , etc.
_d	The list of differences between each element, where _d = diff(_a,_r);. See the Appendix for the function details.
_err	The absolute error, using the Euclidean norm |a-r|, i.e. _err = sqrt(sum(map("*",_d,_d)));.
_relerr	The relative error, obtained by dividing the absolute error by the norm of the correct answerd |a-r|/|a| , i.e. _relerr = _err/sqrt(sum(map("*",_a,_a)));.

Notes:

• The corresponding input boxes of _0, _1, ... can be specified as {_0}, {_1}, ... in the part's text.
• _relerr is not defined for algebraic answers! So _err should be used instead.
• For non-algebraic answers, the student answer is rescaled towards the unit of the target answer. For example, if the target answer is 2 m and the student answer is 199 cm, then _a is [2] , _r is [1.99] and _0 is 1.99 . It has no effect if no unit is used.
• When there are more than on answer, for example if _a = [100,100]; and == _r = [101,102];, then _relerr = sqrt(1*1+2*2)/sqrt(100*100+100*100) ≒ 0.0158 In this sense, the answer defined in the answer field is only a target answer because it may not be related directly to the correctness of the answer.

Manual grading criteria

Other than true or false, the grading criterion can be any number between 0 (all incorrevt) and 1 (all correct). The value 1 means that the student gets the full mark for this par see Gng scheme). A fractional value represents the partial correctness of the student answer. Note that values less than 0 are treated as 0 and that values greater than 1 are treated as 1. The following examples illustrate cases where a manual grading criterion is required:

Multiple correct answers

Suppose that you asked for a number x that is a multiple of 7 and 40 < x < 50 . How can you achieve this? As mentioned before, the variable _0 will store the student answer, say 42 . The following formula can check whether the answer is correct:

_0 == 42 || _0 == 49

Multiple criteria

The above example uses only a fixed number, so how can you grade an answer with random variations? Suppose that you now ask questions with a variable range for each question a < x< a + 10. To determine the correctness, you need to check two criteria, type the following in the Grading Variables and Grading Criteria box respectively:

criterion1 = _0 % 7 == 0;  # whether the remainder is 0. Note that true is 1 and false is 0.
criterion2 = a < _0 && _0 < a+10;   # whether the response is in the desired range.
criterion1 && criterion2

Mark for different accuracies

Suppose you want to give different marks for the accuracy of the response, say full mark for 1% absolute error and half mark for 1% to 5% absolute error. The following criteria can be used:

case1 = _err < 0.01;
case2 = _err < 0.05;
max (case1, 0.5*case2)

Trial mark sequence

This option only apply to the adaptive mode of moodle quiz.

In the adaptive mode, students are allowed to submit answer to a particular question again and again. This field defines the mark sequence that a student can get for each resubmission. Note that it actually alter the default behavior of the adaptive mode.

The input is a list of numbers separated by comma. Each number represents a fraction of the maximum mark that a student can get in the first, second, third, etc. submission. Hence, if this field has value 1, the student can try it once only.

If the sequence is ended with a comma, infinite resubmission is allow. In this case, the mark that students can get in the following unlisted trial is decreased uniformly with the value equal to the difference of last two value. Note that the minimum mark is always zero (See grading scheme).

Trial mark sequence Maximum mark for each trial Description 1 100% Only one trial is allowed 1, 0.7, 0.3 100%, 70%, 30% Three trials are allowed 1, 0.7, 0.3, 0 100%, 70%, 30%, 0% Four trials are allowed, but the last trial has no mark 1, 0, 0, 0 100%, 0%, 0%, 0% Four trials are allowed, but only the first trial has non-zero mark 1, 100%, 100%, 100%, ... Infinite trials. The difference is 0, which is repeated 1, .9, 100%, 90%, 80%, 70%, 60%, ... Infinite trials. Difference between 1st and 2nd trial is 10%, which is repeated 1, .6, 100%, 60%, 20%, 0%, 0%, ... Infinite trials. Difference between 1st and 2nd trial is 40%, which is repeated 1, .5, .3, 100%, 50%, 30%, 10%, 0%, 0%, ... Infinite trials. Difference between 2nd and 3rd trial is 20%, which is repeated

Grading scheme

The following is the grading formula used to grade a particular subquestion:

Symbol Description c Correctness. It takes value between 0 and 1. Boolean false is treated as 0 and true is treated as 1. Other values may be possible if manual condition is used (see Grading criteria) u Deduction for wrong unit (see Unit system). In the formula, it always takes value 0 if the unit is correct under Conversion rules m Default mark of the subquestion. r_n Maximum mark fraction of the n-th submission, for adaptive mode only (See Trial mark sequence) f The computed final mark. For non-adaptive mode:

f = m*c*(1-u) For adaptive mode (see moodle documentation):

f = max(r_n*m*c*(1-u)) The maximum is taken over all submissions. From the above formula, even though a student get a low mark in the first attempt, it is still possible for them to get a higher mark in the following attempt.

Appendix

List of additional functions for non-number type variables

function Examples Description concat(X1,X2,...) concat([1],[2,3],[4])

1. [1,2,3,4]

concat(["A"],["B"])

1. ["A","B"] return a list by concatenating X1, X2, ... together

diff(X,Y) diff(X,Y,N) diff([1,1,1],[1,4,9])

1. [0,3,8]

diff(["x*x"],["x^2"])

1. [0]

diff(["x"],["x^2"],20)

1. A list with number >= 0 *** return a list of number D of difference between the list X and Y elementwise.

If X and Y are a list of number, then D[i] will be abs(X[i]-Y[i]) If X and Y are a list of string, then D[i] will be the difference between the algebraic formula X[i] and Y[i] fill(N, value) fill(3,0)

1. [0,0,0] return a list with size N and all elements filled by value

len(X) len([0:10])

1. 10 return the length of the list X

inv(X) inv([0,3,1,2])

1. [0,2,3,1] return the inverse permutation of X, which is the same as R in for(i:[0:N]) R[X[i]]=i with size N of X.

It has the property that X==inv(inv(X)) and A==sublist(sublist(A,X),inv(X)) join(s,A,B...) join(" ",["Hello","world"])

1. "Hello world"

join("",1,"+",2,["="])

1. "1+2=" return a string that join all elements together and separated by string s .

The A, B, ... can be any number or string or list. map(uop,X) map(bop,X,Y) map("abs",[-1,2,-3])

1. [1,2,3]

map("+",[1,2,3],5)

1. [6,7,8]

map("<",[3,4],[1,7])

1. [0,1] return a list by applying the operator/function elemenwise to all elements in X.

The uop or bop are the string of unary or binary operators (or functions) for numbers respectively. map(uop,X) has the meaning as R in for(i:[0:N]) R[i]=op(X[i]); map(bop,X,Y) has the meaning as R in for(i:[0:N]) R[i]=op(X[i],Y[i]) for function, or for(i:[0:N]) R[i]=X[i] bop Y[i] for operator. Either X or Y can be a number and it will automatically convert to the same size of the other list. pick(i,X) pick(i,x0,x1,...) pick(1<2,["A","B"])

1. "B"

pick(10,"","A","B")

1. ""

pick(1,[1,2],[3,4])

1. [3,4] A safe variant of list subscript. It will pick the i-th element from the list X or the first element if index out of range.

pick(i,x0,x1,...) will pick the i-th element in the [x0,x1,...] , or x0 if index out of range. sort(X) sort(X,Y) sort(["B","C","A"])

1. ["A","B","C"]

sort(["B","C","A"],[0,2,1])

1. ["B","A","C"] return the sorted list of X in ascending order.

sort(X,Y) will sort the list X,Y together by sorting Y in ascending order and return the sorted list X. str(x) str(1.1)

1. "1.1" convert the number x into a string

sublist(X,Y) sublist([7,8,9],[1,1,2])

1. [8,8,9]

sublist([7,8,9],[2,1,0])

1. [9,8,7] return a sublist R of X using list Y as index, where R[i] = X[Y[i]] .

If Y is a permutation, say Y=shuffle([0:5]), then X will be permuted depending on Y sum(X) sum([1,2,3])

1. 6 return the sum of all element in the list of number X

• • • This function depends on context variables in addition to the function parameters. It is also the core function to compare the students' response and model answer (see Grading variables). If input X and Y are a list of string, then X[i] and Y[i] are treated as algebraic formula and all variables in them must be defined before the location of evaluation. For example,

a = 3; x = {1:100}; d = diff(["a x"],["a x^2"]); Please note that the actual algebraic formulas should be "3 x" and "3 x^2" in the above case. In the above evaluation, the d will take a finite value but not close to zero because the algebraic formula are different. In general, any evaluation failure between two algebraic formula will result in a infinite value INF, so that the expression, say, sum(d) < 0.01 will always be false.

The evaluation will take place at N randomly selected points defined in all algebraic variables. The result will be the root mean square difference at all evaluation points (1/N)Σi(Xi-Yi)2, which will converge when N tends to infinity. The N is 100 by default if it is not specified.

< Formulas question type