CorrectWriting Auto-feedback question implementation

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Goal

This question type goal was to simplify report of student mistakes in a shortanswer question, when student is learning programming language or in simple cases of natural language learning, when student must write correct expression, and we should give him information about skipping some words or placing words in incorrect order. This also could be useful, when we don't give actual information about what words (or some kinds of symbols) was moved or absent in answer, but their meaning, thus forcing student to know what does this part of symbol means.

How input is splitted

Teacher input one or several correct answers (with possible feedback describing particular answer). These answers are tokenised (this process will be called scanning later) to break them down to smallest meaningful parts (lexems): words, numbers, punctuation marks, operators etc. These parts depend on the language used.

The question will later analyze and print errors in the placement of these lexems. But when we are telling student, that function name (or a subject) in his answer is misplaced )or absent), we often don't want to disclose to the student information about exact word for that function name or subject. If student will see the word he misplaced, he could just start trying to move it. If the student will see grammatical role of this word (e.g. "function name" or "subject" misplaced), that will stimulate him to think in the grammatical categories (i.e. what is a subject in my answer and where it should be). So the teacher asked to enter grammatical descriptions for each token in the correct answer. That is done as a two-step process.

In answer form, you can enter answer and supply a specific feedback for it:

Than hit "Save changes". The answer form changes, for entering a descriptions for each token. You can skip descriptions for some token, just hitting enter, making an empty line in text field:

Than press "Save changes" button again.

Question type settings

When creating new CorrectWriting question type you will see the form with following settings:

The single most important setting therre is an answers language. It defines a way, in which you answer will be breaked down to lexemes. Other settings are much less important and you may not need to worry about them first. Most of them are used to fine tune question grading and are advanced.

You can see the following parameters:

Lexical error threshold

a treshold, which will be used when comparing tokens from student response to a teacher answer will be used to match with errors with Levenstein distance. If amount of errors when comparing two tokens is lesser than product of this treshold and length of teacher token - than tokens are the same with errors. Since search for misspellings is not implementend this parameter is unused.

Penalty for lexical error

a penalty, that will be substracted from grade, when found one misspelling. Since search for misspellings is not implementend this parameter is unused.

Penalty for absent lexeme

a penalty, that will be substracted from grade for each absent token in student response.

Penalty for odd lexeme

a penalty, that will be substracted from grade for each odd token in student response.

Penalty for moving lexeme

a penalty, that will be substracted from grade for each misplaced token in student response, that is placed in incorrect order, which is taken from teacher's answer.

Minimum grade for answer with non-exact matching

sometimes, teacher may point out for student some hints for especially bad answers, and their comments must be viewed, when student's and teacher's answers are the same. Teacher can define an answer with grade, lower than that border for this answer and point out some bad answer with custom error message

Maximum percent of mistakes in student answer for non-exact matching

when student writes response with lots of mistakes, we can reject his answer in favor of matching with another teacher's answer, when count of mistakes is bigger than product of this parameter and count of parts in answer.

Language of answer

a language, which will be used, when analizing some answer and student response. Currently supported languages, is english language and C programming language.

Example of grading a student response

Please consider the following example, for question, partially described above:

You see the examples of student mistakes. You can toggle various options of quiz, hiding some picture of mistakes or hiding some mistakes sentences for your own purpose.

Installing CorrectWriting question type

To install this question type, you need formal languages block and question type. Also you need a poas question type.

Just copy all of them in your Moodle installation folder, overwriting if need to and go to Site administration in web-interface, proceeding with the installation and you're done.

Future examples

Please, consider the following tasks.

Function header in C language

Student must write simple function header for function, that is described naturally: void function(int abc, char def), with following descriptions of parts:

void

type of returned value

function

function name

(

bracket (or opening bracket for function arguments)

int

first argument type

abc

first argument name

,

argument list separator

char

second argument type

def

second argument name

)

bracket (or closing bracket for function arguments)

If student submits the following answer, like: void function(abc, def), question type will produce the following output:

first argument type was skipped

second argument type was skipped

English language sentence:

Student must write simple sentence, while learning foreign language: The cat ate the mouse, with following descriptions of parts

The

definite article

cat

subject

ate

verb

the

definite article

mouse

complement

.

sentence ending point

If student submits the following answer, like: The cat eat the mouse, question type will produce the following output:

"eat" is odd in response

verb is absent in response

sententence ending point is absent in response.