Development:Shortanswer question development

Following discussion about regular expressions question types in Moodle Tim Hunt suggested a development of short answer question that allows the teacher to use a number of rules with different types (matches string, not matches string, contains the words, matches regexp an so on). The idea is good, though it needs further discussion.

Abilities

Initial list of types for rules

Feel free to place suggestions there:

• Exact match string
• Partial match string (can be implemented via next option)
• Contains the all/one/several substrings
• Exact match regular expression
• Partial match regular expression

Logical operators

Many teachers would want to combine several conditions in one answer condition using logical operators (for example contains some words and not contains another). We definitely should allow NOT and AND operators. The OR can be omitted as several blanks effectively work as OR (this would simplify interface greatly).

Interface

The actual problem is to get clever interface there, so the question creation would not be too cumbersome in simple cases.

Considering using logical operation to group complex expressions in one answer I think that using simple formal language (as in embedded question now) is better alternative than having many fields for each graded subasnwers.

You must type much in shortanswer questions anyway, and the less you'll get distracted by mouse (i.e. dropdowns) the better. Wiki is an example, that people easily adopts simple pseudocodes. We may get some buttons to simplify editing for newbies later (as in MediaWiki now), if someone will help me with it thought.

Considering using shortanswer question by other Moodle question types we must take pains to have it work as before if new functionality isn't used. So an 'answer' field will remain, but it will used mainly to enter the correct answer in user-readable form (and backward compatibility).The 'rules' textarea field will be added to describe rules for matching questions with types mentioned above using simple pseudocode. Then there wil be those options:

• both answer and rules are empty - validation error;
• answer filled, rules empty - compatibility mode, works just as now;
• answer empty, rules filled - rules used to test response;
• answer filled, rules filled - rules used to test response, answer used to display the correct answer (there must be at least one such answer with 100% grade in question in order to display the correct answer); there is possibility of validation of the answer against rules
  • validation of the answer that use wildcards will be somewhat problematic, the only way we can validate it is assuming wilcard to be literally *, and validate this option only (but removing any escapes before *).
  • I would happily drop using wildcard syntax in answer field at least in non-compatibility mode to allow better validation, but it may cause some inconvenience to the teachers, who use wildcards but didn't know about regexes for now (thought current syntax cause inconvenience for the people, that doesn't know what wilcard (and escaping) is and whant to have * in their answers, and this may be more common situation). In non-compatibility mode answer field changes it's meaning and wildcards are no longer welcome there; to avoid any troubles they can be left to themselves in compatibility mode.

Well, all this fuss about several modes and different working fields will probably be quite confusing to the users. An alternative is to always use rules field, but, for the sake of compatibility, upgrade all questions to the new form, and give a call to create/edit question compatibility mode, so that questions in old format will be automatically translated to the new format. The only possible problem there may be with close question, if it try to reconstruct the question syntax from database fields, this needs further investignation.

Syntax

The syntax of pseudocode should be simple, both for entering and parsing, and allows shorthands for quick typing (maybe these shorthands should automatically be changed on saving question). For example full negation should be 'not' and shorthand just '!'

Grammar of the pseudocode:

• <rule>::=<opt_negation><description><text>
• <opt_negation>::=not|! //for negative rules
• <description>::=(cont|contains)<unit><quant><separator>|match<unit>
• <quant>::=all|one|A|1|{<start>,<finish>}

where

• <text> is an actual text of the rule
• contains means partial matching, match means exact matching
• <unit> is a subplugin name (i.e. string, wildcardstring, word, regexp etc)
• <separator> is a symbol, that will be used as separator of different elements in text (words, regexes - if we need to buil rules that contains one of the several words)
• <start> and <finish> are integers

Examples:

• not contains word A:red:green:blue
• match regex [a-zA-z]+
• contains strings{2,3},a,b,c,d

Compatibility issues

Singular shortanswer questions compatibility

Well, with compatibility proposed above there would be no problems at all there.

Embedded answers (close), numerical, calculated compatibility issues

Pierre, you could actually help me there, as you knows embedded answers question type well.

Oleg take a look at the code using the 
http://xref.moodle.org/nav.html?index.html
You will see that calculated depends on numerical that depends on short answer.
Cloze create numerical and short answer.
You will then conclude that you cannot easily alter the short answer question type.
The solution is to create a specific variant of short answer.
Test it on contrib and propose it as a plug-in.
History will tell us if your new extended short answer will render the short answer obsolete...
Pierre Pichet 12:51, 13 November 2008 (CST)

Well, the challenge seems to be: add a whole load of a new functionality to a question, that is heavily used by another parts of the Moodle, while keeping things simple (both for the user and the programmer) and as compatible as practical. I like this challenge.

Numerical question inherited shortanswer, but it's overloads all functions that we need to replace (see Implementation below), so adding new option to shortanswer doesn't mean we must add it to numerical too. The only overloaded function that calls parent is get_correct_responses, which (fortunately) can work just as before in shortanswer. So there would be no problems there.

Calculated question use numerical only to display itself, but we need not to change the way shortanswer question display itself at all, so there would be no problems there too.

Close question creates shortanswer and numerical questions. In compatibility mode all is OK, except that shortanswer question-creating/updating function should take care about new option been unset (than set it as empty) - we may make simple modification to close to keep new field set too. I'm afraid that new functionality would not be immediately accessible to the close question type mainly because close must find his form of syntax for it (but you could do that later if you want, Pierre, there would be no problems from shortanswer). So these problems are easily handled too.

So, unless I miss something important, I think these issues can be easily handled. If anyone knows another Moodle areas that depends on shortanswer, please tell me about them.

Backup, import, export compatibility issues

This is a part of Moodle I'm unfamiliar too (thought any experienced developer probably knows that pretty well - so I'm need a help from community there). I need to introduce a new field in the table for subquestions of shortanswer.

There is no problem to just add new field handling in the backup/restore functions, but how to retain compatibility with old backups? Moodle must handle this somehow, but where can I see how exactly this is done?

We must add a new field at least to the Moodle XML export/import format (and maybe to any other formats that can be reimported into Moodle after export too). The question is how to handle old exports, that was made before upgrade.

Normal exports should not experience a problem there (especially if user fill all answer fields), as many external system has not this new functionality. If it supported somewhere, the matter must be handled by it's maintainers. There is no problems for imports, as the question will be in compatibility mode if new field isn't setted,

Implementation

The database - there is a need to add new field to question_answers table, a text field 'rules'. It must probably be an empty string by default, to retain compatibility with another questions that use this table. If plan 1 for test_response function will be accepted (see below), than new field in db must probably have some more generic name, any question may need something more than an answer, grade and feedback. Text field is a good choice for any additional information field.

Question editing form must get a new field to enter this, and more complicated validation too - if both answer (without wildcard) and rules is filled we can test an answer against the rules.

Functions that must be reworked in question_shortanswer_qtype class:

• save_question_options - to add a new field to the subquestions, it must be explicitely set to empty string if unset (so any caller that doesn't know about new functionality won't be upset).
• test_response and check_response - to make new functionality work
  • there is a bug in shortanswer compare_responses function now (see MDL-17247); it need to be fixed and dependence of check_response on compare_responses removed to make all functions work as intended; I can do it if Tim is busy. check_response should call test_response, not compare_responses
  • the most obnoxious thing is an interface of test_responce, which is based on the assumption, that answer is enought to test response (which leave us without rules). There are actually two ways to deal with this (this decision is for Tim probably):
    1. change asnwer to full subquestion object - this will cause breakage for 3d-party modules, but it is sort of expected thing on major version change (this development probably must be targeted to 2.0 anyway) and will be very easy to fix - but leave us very broad possibilities for future development
    2. append rules to an answer (separating them with line break) before the call - this would be some sort of workaround, but we still need to modify all calls (and ensure that close doesn't create answers with line breaks), and our options for future will be still narrowed be test_response interface

There are also two functions that need modification in backup/restore libs: question_backup_answers and question_restore_answers, so they can handle a new field.

New function, that'll test response against the rules must, probably, be static function of question_shortanswer_qtype class so validator can use it without duplicating code. test_response will internally call this function.