Development:Shortanswer question development

Following discussion about regular expressions question types in Moodle Tim Hunt suggested a development of short answer question that allows the teacher to use a number of rules with different types (matches string, not matches string, contains the words, matches regexp an so on). The idea is good, though it needs further discussion.

Abilities

Initial list of types for rules

Feel free to place suggestions there:

• Exact match string
• Partial match string (can be implemented via next option)
• Contains the all/one/several substrings
• Exact match regular expression
• Partial match regular expression

Logical operators

Many teachers would want to combine several conditions in one answer condition using logical operators (for example contains some words and not contains another). We definitely should allow NOT and AND operators. The OR can be omitted as several blanks effectively work as OR (this would simplify interface greatly).

Interface

The actual problem is to get clever interface there, so the question creation would not be too cumbersome in simple cases.

Considering using logical operation to group complex expressions in one answer I think that using simple formal language (as in embedded question now) is better alternative than having many fields for each graded subasnwers.--Oleg Sychev 10:02, 12 November 2008 (CST)

You must type much in shortanswer questions anyway, and the less you'll get distracted by mouse (i.e. dropdowns) the better. Wiki is an example, that people easily adopts simple pseudocodes. We may get some buttons to simplify editing for newbies later (as in MediaWiki now), of someone will help me with it thought.

Right now I think about two possibilities:

1. An 'answer' field will remain, but now it is used mainly to enter the correct answer in user-readable form (and backward compatibility), it may be empty; if it isnt't empty and rules is empty it will be matching against students response just as it is now, however there are better options. The 'rules' textarea field will be added to describe rules for matching questions with types mentioned above using simple pseudocode. If the both rules and answer field are filled, the answer must match with rules. That'll costs us a new field in table, and (probably) some problems with backups and import/export compatibility, which is an area in Moodle where I'm not competent enought - but I think it'll be quite easy to any experieced Moodle developer to help me there.
2. We may just replace 'answer' field with textarea, leaving first string in it for an answer (as it is now and for display to the user) and other for rules. However this'll create some strange things with an empty first strings, where it may be easy to make an error (thought validator will find it).

I would like you opinion about what path to take.

Syntax

The place for syntax proposals.

Compatibility issues

Singular shortanswer questions compatibility

Most existing questions must work as before, and the rest must be automatically upgraded, it's clear enough. Well, with interface proposed above there would be no problems at all there.

I think that a text analysis shortanswer question type should be a new question type because any additional syntax than the * character will interfere potentially with already created shortanswer questions.

Moodle need to be upward compatible.

We could even create more than one text analysis shortanswer question type in relation to the code or computer code (regexp, basic ) used to analyse the student responses.

Let the user choose the code ( or text analysis shortanswer question type) that he knows best.

I.e. regexp is the kind of code created to limit the typing in those days where code was enter on a teletype.

Efficient for a computer but not the kind of computer I have in my head...

I cannot use it without a verification program. Pierre Pichet 11:50, 12 November 2008 (CST)

Pierre, I hate incompatibily a lot as you can know already. I thought of kind of system, that will be compatible with most of existing questions (and the rest can be automatically upgraded). I'm not fond of having a lot of question types - even with standard questiontypes adding question dropdown is somewhat crowded. Also, there is nothing exceptional in wanting to use several 'codes' in one question (one to catch right answers, another to catch typical error for example - Joseph Rezeau actually tring to do this with his regexes).

Dropping OR allows us create a very simple interface. We replace text field with textarea, every string in it is a rule (fortunately shortanswer doesn't allows line breaks in answers), the rules in one answer are connected with AND. Each rule starts with short (or full - it may replace the short form automaticaly) form of the rule's type (that can be preceeded by 'NOT' or ! for negation), that ends with delimiter, followed by it's parameters.

The sort of interface that allows user write something like this:

{contains:all:}red:green:blue
{!contains:1/}purple/white/black
{not regexp:part}[0-9]

which will means the string which contains all of the words red,green and blue; doesn't contains purple, white or black and numbers too (defined using regexp).

We can keep compatibility in two ways:

1. expect simple string to be of "Exact Match with wildcards" type (that is how it work's now) and start any rule with some rare special character - as { above - (the character must be escaped if the string still starts with it thought - that is if we can't find a character that absolutely can't be a part of an answer now except line break). This will be more familiar to the user (which can still use old syntax), but may leads to mistakes by omission escaping in rare cases (thought validator can easily catch them most of the time).
2. automaticaly prepend every string with type of the rule for "Exact Match with wildcards" during upgrade. This may caught user by surprise first, but then he'll knows the syntax and won't mistakes just because he didn't know about it.

I consider also about dropping wildcard syntax using regex instead (it's not hard to automatically replace strings with wildcards with regexes). It was useful without regexes, but with them it is only a strange additional option. --Oleg Sychev 18:57, 12 November 2008 (CST)

You involve yourself in Moodle with all the energy of your age and I don't want 
to loose the possibilities this open for Moodle.

However Moodle is used by many important schools or universities to surely more 
than 1 million students (40 000 at my own university) and quiz (and questions) are 
involved in the grading process.

For schools the grades are like money for banks, everything should be done to maintain their value.
Short answer has been used in very different situations and we cannot change easily the way it works.
 
There are current limitations to short answer response analysis that can be improved.
They are more important than the simplification of the actual interface that seems to bother you.
I encourage you to work on response analysis more than just setting a list of equivalent response 
in a single line or a text area.
Pierre Pichet 19:38, 12 November 2008 (CST)

Well, Pierre, actually I'm sort of resting there :-), not doing any complicated work. You probably must state you concerns more clear as I can't understand them. The proposed changes don't hurt anything - in almost every situation (say, any answer except starting with { sign, which can be changed) it will work just as before, and the rest will be automatically upgraded. The only possible problem would be the teachers trying to enter new questions starting with { after upgrade and doesn't know that it now must be escaped. But (apart from clearly states it in instructions on question editing page) most of these situations will be handled by validator. I also can't quite understand what you mean by "working more on response analysis" and "setting a list of equivalent options" too. Could you be more clear on the matter?

I am a teacher from the somewhat large University and I know what is the support of big questions bank, don't waste you energy there.--Oleg Sychev 20:20, 12 November 2008 (CST)

Pierre, see interface section now. Is you content?

Embedded answers (close), numerical, calculated compatibility issues

Pierre, you could actually help me there, as you knows embedded answers question type well.

Oleg take a look at the code using the 
http://xref.moodle.org/nav.html?index.html
You will see that calculated depends on numerical that depends on short answer.
Cloze create numerical and short answer.
You will then conclude that you cannot easily alter the short answer question type.
The solution is to create a specific variant of short answer.
Test it on contrib and propose it as a plug-in.
History will tell us if your new extended short answer will render the short answer obsolete...
Pierre Pichet 12:51, 13 November 2008 (CST)

Pierre - I look on numerical question and found nothing that'll create compatibility issues if we choose first alternative in the interface section (do you read it?). The numerical question overloads all functions that are intented to alteration (and calls parent function only once at all, but we need not to change it). Just a new field which opens new possibilies when filled, but leave things to the way they were if blank rarely hurts anything.

This is not my personal itch. The idea for this change was proposed by Tim. I can help him in developing shortanswer question to more generic means, but I have neither intention nor resources to create and support another similar generic question type - for an additional type I'm content with our preg question very well.

Now, when we discuss things on more specific basis, can I delete our discussion above? It pretty overloads the page and hardly contains anything useful related to the current state of things. --Oleg Sychev 14:33, 13 November 2008 (CST)

First of all could you please describe me connection between close and shortanswer questions:

1. Which parts of code from shortanswer directory close question actually uses?
2. Are subquestion descriptions reconstructed when editing question, or they are stored in DB just as the user enters them?
3. How shortanswer question parser handles line breaks in answers (either simple line breaks, or

   that HTMLArea generates on pressing enter)?

4. How shortanswer question parser handles *(wildcard) and : (answer's separator) if they must be a part of an answer?

Right now I can't see an easy way to get access from close question to the new functionality of shortanswer. You may implement something about this later thought.

Implementation

The options should be implemented as subclasses of abstract class to make adding new options easier. Does we need to have them as actual plugins (with versions, language packs, installation and so on) or more simple subplugins?

Does we need to store all options (with logical operators) with given subanswer as a whole, or create a new table with one entry per options? Right now I'm against new tables. They add many problems with moving questions to another category and so on. --Oleg Sychev 10:02, 12 November 2008 (CST)