Correct writing question type

From MoodleDocs


Correct writing question


Authors:

1. Idea, string analysis method, general architecture, hints implementation - Oleg Sychev 2. Question type code, sequence analyzer, picture generation - Dmitry Mamontov

Thanks going to:

  • Maria Birukova - for help with icon

Description

When you teach the syntax of a new language - either a human or a programming one - you often finding youself teaching to write correct words (symbols etc) in a correct order. This question will help you with it.

It analyzes student's response on words (well, actuall tokens or lexeme) level, find the closest possible match with you answers and shows messages about mistakes to the students. Currently supported mistakes are: - misplaced word; - missing word; - extraneous word.

In order to teach student to think grammaticaly, you must not reveal the actual mistaken words, but their grammatical roles when possible (i.e. except extraneous word). Consider student thinking about such messages: - "cat" misplaced OR subject misplaced - "int" missing OR variable type missing So for each word (lexeme) of you answer you must enter a grammatical description to be showed to the student.


Installation

To work CorrectWriting question type needs some additional components. They all need to be installed in order for question to work. You could download all them in one archive (from GoogleCode) or separately.

If you downloaded one archive, unpacking it you should found "question" and "blocks" folders. Copy them in the main directory of Moodle installation (the one containing config.php) - it will install everything in the correct places.

If you downloaded all parts separately (for example because Moodle plugins Directory don't allow them to be downloaded together), you should get 3 archives. Place "correctwriting" and "poasquestion" folders in the question/type folder of Moodle installation. "formal_langs" should be placed in the blocks folder.

After having files in place login as administrator and go to the notifications page.


Bugs, suggestions and giving back

Please report bugs in GoogleCode tracker (set Component-WritingCompetently label) - this is preferable way - or Moodle Tracker. Google Code tracker could be found there: http://code.google.com/p/oasychev-moodle-plugins/issues/list You also may post a suggestions of how to develop question type in the future, but please accompany them with information of who are you, how you are using question type and how you intend to use enchancements you suggest pedagogically.

If you like the project, please consider helping it too. That doesn't necessary means money or tough programming.

You could publish a paper or thesis about using this question type and send me reference I could quote to improve rating of this project. If you could publish a scientific paper on this question and it's use with me as co-author this would help much more, please inform me immediately if you choose to do so.

When writing about using this question, you may wish to cite following sources: - Oleg Sychev, Dmitry Mamontov. Determining token sequence mistakes in responses to questions with open text answer. Arxiv e-print. URL: http://arxiv.org/abs/1301.2466

You could also help by testing - either manual or unit-testing. Default installation instructions for plugins in category Question types Make sure you have all the required versions. Download and unpack the module. Place the folder (eg "myquestiontype") in the "question/type" subdirectory. Visit http://yoursite.com/admin to finish the installation

Old Documentation

Goal

This question type aims to automatically report of student mistakes in a shortanswer question, when student is learning programming language or in simple cases of natural language learning, when student must write correct string (statement, sentence), and we should give him information about skipping some words or placing words in incorrect order. This also could be useful, when we don't give actual information about what words (or some kinds of symbols) was moved or absent in answer, but their meaning, thus forcing student to know what does this part of symbol means. This allows for some training without direct teacher supervision.

How input is splitted

Teacher should enter one or several correct answers (with possible feedback describing particular answer). These answers are tokenized (this process will be called scanning later) to break them down to smallest meaningful parts (tokens or lexems): words, numbers, punctuation marks, operators etc. These parts depend on the language used.

The question will later analyze and print errors in the placement of these tokens. But when we are telling student, that function name (or a subject) in his answer is misplaced (or absent), we often don't want to disclose to the student information about exact word for that function name or subject. If student will see the word he misplaced, he could just start trying to move it. If the student will see grammatical role of this word (e.g. "function name" or "subject" misplaced), that will stimulate him to think in the grammatical categories (i.e. what is a subject in my response and where it should be?). So the teacher asked to enter grammatical descriptions for each token in the correct answer. That is done as a two-step process.

In answer form, you can enter answer and supply a specific feedback for it:

CorrectWritingAnswerStep1.PNG

Than hit "Save changes". The answer form changes, to allow you to enter a description for each token. You can skip descriptions for some token, just hitting enter, making an empty line in text field, then the student will the literal representation of the token instead:

CorrectWritingAnswerStep2.PNG

Than press "Save changes" button again.

Question type settings

When creating new CorrectWriting question type you will see the form with following settings:

CorrectWritingEditForm.PNG

The single most important setting therre is an answers language. It defines a way, in which you answer will be breaked down to lexemes (tokens). Other settings are much less important and you may not need to worry about them first. Most of them are used to fine tune question grading and are advanced.

You can see the following parameters:

 ; Lexical error threshold : a treshold, which will be used when comparing tokens from student response to a teacher answer will be used to match with errors with Levenstein distance. If amount of errors when comparing two tokens is lesser than product of this treshold and length of teacher token - than tokens are the same with errors. Since search for misspellings is not implementend this parameter is unused and hidden in form.
 ; Penalty for lexical error : a penalty, that will be substracted from grade, when found one misspelling. Since search for misspellings is not implementend this parameter is unused and hidden in form.
Penalty for missing token
a penalty, that will be substracted from grade for each absent token in student response.
Penalty for extra token
a penalty, that will be substracted from grade for each odd token in student response.
Penalty for misplaced token
a penalty, that will be substracted from grade for each misplaced token in student response, that is placed in incorrect order, which is taken from teacher's answer.
Minimum grade for answer to find and display mistakes
sometimes, teacher may point out for student some hints for especially bad answers, and their comments must be viewed, when student's and teacher's answers are the same. Teacher can define an answer with grade, lower than that border for this answer and point out some bad answer with custom error message
Maximum percent of mistakes in students response
when student writes response with lots of mistakes, we can reject his answer in favor of matching with another teacher's answer, when count of mistakes is bigger than product of this parameter and count of parts in answer.
Language of answer
a language, which will be used, when analizing some answer and student response. Currently supported languages are english language and C programming language.
Hinting settings
allows to enable particular hints for multi-stage behaviours and setting penalties for their use. See next sections for details about hints.

Hinting

The CorrectWriting question type uses hinting behaviours and is able to do special hints - for now in adaptive behaviour - for a penalty in the grade (may be set to 0). You could enable them setting penalty for the hint below 1.

What is hint
tells student a token text instead of description. I.e. The subject is "cat". Used for misplaced token and absent token mistakes. For absent token mistake the penalty is multiplied to the absent hint penalty factor, since it discloses exact text of the word student omitted.
Where is text hint
shows a message where a token should be placed, based on you answer, using descriptions where possible. I.e. The subject should be placed between definite article and verb. Used for misplaced token and absent token mistakes.
Where is picture hint
shows an image where a token should be placed, based on you answer, using descriptions where possible. You can see example below, highlighted by a red border.

CorrectWritingImageHint.PNG

Example of grading a student response

Please consider the following example, for question, partially described above:

CorrectWritingStudentAnswer.PNG

You see the examples of student mistakes. You can toggle various options of quiz, hiding some picture of mistakes or hiding some mistakes sentences for your own purpose.

Installing CorrectWriting question type

To install this question type, you need formal languages block and question type. Also you need a poas question type.

Just copy all of them in your Moodle installation folder, overwriting if need to and go to Site administration in web-interface, proceeding with the installation and you're done.


Examples

Please, consider the following examples.

Function header in C language

Student must write simple function header for function, that is described naturally: void function(int abc, char def), with following descriptions of parts:

void
type of returned value
function
function name
(
bracket (or opening bracket for function arguments)
int
first argument type
abc
first argument name
,
argument list separator
char
second argument type
def
second argument name
)
bracket (or closing bracket for function arguments)

If student submits the following answer, like: function(abc, def) void , question type will produce the following output:

type of returned value is misplaced
first argument type is missing
second argument type is missing

English language sentence:

Student must write simple sentence, while learning foreign language: The cat ate the mouse, with following descriptions of parts

The
definite article
cat
subject
ate
verb
the
definite article
mouse
complement
.
sentence ending point

If student submits the following answer, like: The cat eat the mouse, question type will produce the following output:

"eat" should not be in response
verb is missing
sententence ending point is missing.