Auto-feedback shortanswer question

Goal

Develop a question type, that was similar to shortanswer (except that no initial wildcard support is planned, thought it can be added later) which will be able to analyze student answer and automatically generate feedback messages describing errors, which could be useful to the student without revealing correct answer (if necessary).

Such question is very useful in adaptive mode (and in non-adaptive mode too for courses with minimal teacher involvement) whenever you try to teach some language (eithern natural or artificial). If someone knows about works or papers that trying to do similar things please let me know there or on page comments.

Question still will be based (at least from the beginning) on entering a number of correct answers.

Question will be able to handle correct answer with several identical words, but that is discouraged (unless absolutely necessary) beacuse it will lower accuracy of feedback.

External specification draft

Creating a question

Teacher input one or several correct answers (with possible feedback describing particular answer). These answers are tokenised (this process will be called scanning later) to break them down to smaller parts (lexems): words, numbers, separators, operators etc.

Since I couldn't start this work from writing universal scanner similar to the flex etc, it would be good to have pluginable scanners for the first time (e.g. natural language scanner, C++ scanner etc).

Teacher than asked for a names for all lexems, that could be shown to the user without revealing correct answer. Examples of tokenised answer with named lexems:

Function header in C language: void function(int abc, char def)

• void - type of returned value
• function - function name
• ( - bracket (or opening bracket for function arguments)
• int - type of the first argument
• abc - name of the first argument
• , - argument list separator
• char - type of the second argument
• def - name of the second argument
• ) - bracket (or closing bracket for function arguments)

Could anyone write a natural language example please? I don't know well grammatical categories of English language in English to do so.

There will be probably some settings to determine how a grade should be calculated in case of errors, but they could be defined later.

Types of errors to report

1. misspelled word (number, another complex lexem)
2. missing separator
3. invalid separator
4. misplaced word (number, etc)
5. missing word (number, etc)
6. unwanted word (number, separator, etc)