question/type/correctwriting

Goal

This question type aims to automatically report of student mistakes in a shortanswer question, when student is learning programming language or in simple cases of natural language learning, when student must write correct string (statement, sentence), and we should give him information about skipping some words or placing words in incorrect order. This also could be useful, when we don't give actual information about what words (or some kinds of symbols) was moved or absent in answer, but their meaning, thus forcing student to know what does this part of symbol means. This allows for some training without direct teacher supervision.

How input is splitted

Teacher should enter one or several correct answers (with possible feedback describing particular answer). These answers are tokenized (this process will be called scanning later) to break them down to smallest meaningful parts (tokens or lexems): words, numbers, punctuation marks, operators etc. These parts depend on the language used.

The question will later analyze and print errors in the placement of these tokens. But when we are telling student, that function name (or a subject) in his answer is misplaced (or absent), we often don't want to disclose to the student information about exact word for that function name or subject. If student will see the word he misplaced, he could just start trying to move it. If the student will see grammatical role of this word (e.g. "function name" or "subject" misplaced), that will stimulate him to think in the grammatical categories (i.e. what is a subject in my response and where it should be?). So the teacher asked to enter grammatical descriptions for each token in the correct answer. That is done as a two-step process.

In answer form, you can enter answer and supply a specific feedback for it:

The answer form changes, to allow you to enter a description for each token (for the answers above hint grader border). You can skip descriptions for some token, just hitting enter, making an empty line in text field, then the student will the literal representation of the token instead:

Than press "Save changes" button again.

Question type settings

When creating new CorrectWriting question type you will see the form with following settings:

The single most important setting therre is an answers language. It defines a way, in which you answer will be breaked down to lexemes (tokens). Other settings are much less important and you may not need to worry about them first. Most of them are used to fine tune question grading and are advanced.

You can see the following parameters:

 ; Lexical error threshold : a treshold, which will be used when comparing tokens from student response to a teacher answer will be used to match with errors with Levenstein distance. If amount of errors when comparing two tokens is lesser than product of this treshold and length of teacher token - than tokens are the same with errors. Since search for misspellings is not implementend this parameter is unused and hidden in form.
 ; Penalty for lexical error : a penalty, that will be substracted from grade, when found one misspelling. Since search for misspellings is not implementend this parameter is unused and hidden in form.

Penalty for missing token

a penalty, that will be substracted from grade for each absent token in student response.

Penalty for extra token

a penalty, that will be substracted from grade for each odd token in student response.

Penalty for misplaced token

a penalty, that will be substracted from grade for each misplaced token in student response, that is placed in incorrect order, which is taken from teacher's answer.

Minimum grade for answer to find and display mistakes

sometimes, teacher may point out for student some hints for especially bad answers, and their comments must be viewed, when student's and teacher's answers are the same. Teacher can define an answer with grade, lower than that border for this answer and point out some bad answer with custom error message

Maximum percent of mistakes in students response

when student writes response with lots of mistakes, we can reject his answer in favor of matching with another teacher's answer, when count of mistakes is bigger than product of this parameter and count of parts in answer.

Language of answer

a language, which will be used, when analizing some answer and student response. Currently supported languages are english language and C programming language.

Hinting settings

allows to enable particular hints for multi-stage behaviours and setting penalties for their use. See next sections for details about hints.

Hinting

The CorrectWriting question type uses hinting behaviours and is able to do special hints - for now in adaptive behaviour - for a penalty in the grade (may be set to 0). You could enable them setting penalty for the hint below 1.

What is hint

tells student a token text instead of description. I.e. The subject is "cat". Used for misplaced token and absent token mistakes. For absent token mistake the penalty is multiplied to the absent hint penalty factor, since it discloses exact text of the word student omitted.

Where is text hint

shows a message where a token should be placed, based on you answer, using descriptions where possible. I.e. The subject should be placed between definite article and verb. Used for misplaced token and absent token mistakes.

Where is picture hint

shows an image where a token should be placed, based on you answer, using descriptions where possible. You can see example below, highlighted by a red border.

Example of grading a student response

Please consider the following example, for question, partially described above:

You see the examples of student mistakes. You can toggle various options of quiz, hiding some picture of mistakes or hiding some mistakes sentences for your own purpose.

Installing CorrectWriting question type

To install this question type, you need formal languages block and question type itself. Also you need a poas question type and two hinting behaviours adaptive with hints and no penalties version of adaptive.

Copy all of them in respective places in your Moodle installation folder, overwriting if need to and go to Site administration in web-interface, proceeding with the installation and you're done. Alternatively you could download an entire set on developer's site, which could be unpacked in the root directory of Moodle and place everything to the place it belongs.

Analyzers

Typo analysis

Typo analysis allows you to correct typos in student's response. Typo analysis is off by default. It could find out several mistakes:

• typo - based on standard Damerau-Levenshtein algorithm, allowing editing operators
  • insert character
  • delete character
  • replace character
  • transposition - two ajanced characters changing places
• missing separator - no separator between two tokens, merging them in one
• extra separator - separator (usually a space) inside a token, splitting it into two

Typo analysis can find out mistakes even if token is misplaced, however in case of extra separator the resulting two tokens should be ajanced.

Mistakes images:

• insert character

• delete character

• replace character

• transposition - two ajanced characters changing places

• missing separator - no separator between two tokens, merging them in one

• extra separator - separator (usually a space) inside a token, splitting it into two

composite example:

TODO - typo analysis settings, once we get them correct in the question editing form.

Analysis of enumerations

Sometime correct answer can contain enumeration - a group of elements (with possible separators), that can be placed in any order. In natural languages, enumerations elements are often separated by comma, "and" or "or". Entering all possible variants of correct answer with enumeration is boring for teacher and prone to errors.

Analysis of enumerations detects enumerations in correct answer and try to change order of their elements to better match particular student's response.

For now enumeration analysis work only for C++ language, where they are autodetected based on languaged features.

Enumeration in English (and other natural languages) can not be precisely detected automatically. The interface to enter them is under construction.

C++ language enumerations

Elements of the enumerations shown in italic in these examples.

structure or union definition

fields can be declared in any order: struct MyNiceStructure { int FirstField; long Padding; char SmallPart; } DefaultValue;

enumeration definition

values can be declared in any order: enum Types { Int, Float = 32; Char };

definition of same type variables

variables can be declared in any order: int number, * count; numbers[10];

sequence of mod operators

operands excluding first can be written in any order: first % second % last;

sequence of division operators

operands excluding first can be written in any order: first /second / last;

sequence of subtraction operators

operands excluding first can be written in any order: first - second - last;

sequence of plus operators

operands can be written in any order: first * second % last;

sequence of multiply operators

operands can be written in any order: first * second * last;

sequence of assign operators

operands excluding last can be written in any order: first = second = last;

sequence of binary and operators

operands can be written in any order: first & second & last;

sequence of binary or operators

operands can be written in any order: first | second | last;

sequence of binary xor operators

operands can be written in any order: first ^ second ^ last;

sequence of logical equal operators

operands can be written in any order: first == second == last;

sequence of logical not equal and operators

operands can be written in any order: first != second != last;

sequence of logical or operators

operands can be written in any order: first || second || last;

sequence of logical and operators

operands can be written in any order: first && second && last;

 TODO - currently supported C++ language constructions with examples

Token sequence analysis

Token sequence analysis was unique feature CorrectWriting question from it's first release. It analyze sequence of tokens and find out such mistakes as:

• misplaced token
• missing token
• extraneous token

Token sequence analysis is active by default. You can turn it off if token sequence doesn not matter for you language or you answer (i.e. the student's response containing required tokens in any order will be correct).

Examples for token sequence analysis

Please, consider the following examples.

Function header in C language

Student must write simple function header for function, that is described naturally: void function(int abc, char def), with following descriptions of parts:

void

type of returned value

function

function name

(

bracket (or opening bracket for function arguments)

int

first argument type

abc

first argument name

,

argument list separator

char

second argument type

def

second argument name

)

bracket (or closing bracket for function arguments)

If student submits the following answer, like: function(abc, def) void , question type will produce the following output:

type of returned value is misplaced

first argument type is missing

second argument type is missing

English language sentence:

Student must write simple sentence, while learning foreign language: The cat ate the mouse, with following descriptions of parts

The

definite article

cat

subject

ate

verb

the

definite article

mouse

complement

.

sentence ending point

If student submits the following answer, like: The cat eat the mouse, question type will produce the following output:

"eat" should not be in response

verb is missing

sententence ending point is missing.

Examples

Please, consider the following examples.

Function header in C language

Student must write simple function header for function, that is described naturally: void function(int abc, char def), with following descriptions of parts:

void

type of returned value

function

function name

(

bracket (or opening bracket for function arguments)

int

first argument type

abc

first argument name

,

argument list separator

char

second argument type

def

second argument name

)

bracket (or closing bracket for function arguments)

If student submits the following answer, like: function(abc, def) void , question type will produce the following output:

type of returned value is misplaced

first argument type is missing

second argument type is missing

English language sentence:

Student must write simple sentence, while learning foreign language: The cat ate the mouse, with following descriptions of parts

The

definite article

cat

subject

ate

verb

the

definite article

mouse

complement

.

sentence ending point

If student submits the following answer, like: The cat eat the mouse, question type will produce the following output:

"eat" should not be in response

verb is missing

sententence ending point is missing.