Talk:Question Engine 2:Design

Jump to: navigation, search

Some thoughts

Some ideas from reading this proposal:

1. question_states table store state as an Integer. However, some complex qtypes (like our planned training) may want to have custom sub-states (especially for Incomplete state). They can be stored in question_responses but that is somewhat inconsistent. What do you think about adding a field for substate in questions_states or making state field a float, so substates can be after comma while states before?

2. question_responses table structure is generic and can be used by many other modules (like assignment). I think that it will may be good to have a library of functions that work with such tables.

3. Also I wonder is naming convention (!, _) a good way to resolve conflicts in the names in question_responses table, or an additional field for the source of data (i.e. qtype, interaction model etc) would be better. The later is definitely more scalable IMHO.

--Oleg Sychev 12:48, 3 October 2009 (UTC)

Thank you Oleg. Good questions.
1. The purpose of the state field in the the question_states table is for communication. Communication between particular question types/interaction modules, and whatever is using the question engine. That is why I tried to have the smallest possible number of states. The less the outside world knows, the more freedom the question engine has.
So, I expect particular questions will do different things with the complete and incomplete states. For example a maths question that needed the student to type a complex syntax might use the Incomplete state for both the initial situation where the answer is blank, but also if the student types an answer that cannot be validated. However, all the quiz needs to know (for the navigation and the summary page in 2.0) is to tell the student that they have not finished working on that question.
Similarly, in the new Interactive mode, you might image in needs a 'Try again' state, but actually that is another Incomplete state.
So, actually, fewer states are more flexible.
2. That structure is indeed generic. In fact Moodle already has similar other structures, for example in the config and config_plugins tables. The they key point about this instance of this table is that it is linked to the question states, and the way it will be used. By the way, this table will have a lot of rows. It is essential that the API for accessing it is optimised for the kind of access that is done during large quiz attempts.
3. The reasons for this simple approach to naming is that when the data is loaded from the database, the responses become a PHP array on the state. (I hope you realise that this design page is still incomplete, which is why this is not yet explained.) Obviously arrays can only have keys and values, which is why the only way to avoid collisions is a naming convention. The other reason is, as I said, question_responses is critical for performance. Therefore, the fewer columns there, the better.
Also, student input, extra info from the question type, and extra information from the interaction model are the only things that should be storing things in this table, and it is easy to distinguish three things with a prefix.
--Tim Hunt 19:05, 3 October 2009 (UTC)

Also some ideas that may not be directly related to it:

1. I'm long dreaming about an ability to tag question categories (maybe quesitons too), and then filter them by that tags on various occasions (from listing and editing trought random questions to import/export). Tree hierarchy isn't sufficient. I have a category tree organizied by topic in the course, but then on the each topic I have one demo, several training and many control categories. And sometimes I need something to be done with all demo categories or something like that.

2. there are several questions (in core and not) - shortanswer, numerical, calculated - that shares a model of having several answers, with finding a matching one by some rules (multichoice partialy fits there in single choice mode). What do you think about abstracting it in abstract_multianswer qtype (oh, existing multianswer probably better to be known as multiquestion)? It isn't really good to place functions, that similar to all them, in root question type class.

3. Also I'm wondering, for questions with sub-states in Incomplete state, would it be easy to create a Replay on reviewing? This should be taken into consideration too IMHO.

--Oleg Sychev 12:48, 3 October 2009 (UTC)

1. The question bank is indeed a completely different issue.
2. I hope the new code will make it easier to make good use of inheritance between question types. Yes, more abstract classes is probably good. Both the one you suggest, and also a base class for t/f and multichoice.
3. Yes, with the new question engine, anything can be replayed. (Just replay the sequence of student input, which is all in the database.)
--Tim Hunt 19:05, 3 October 2009 (UTC)

Question: how should behave a question with many sub-states?

Now for a question about how should more complex qtype use this system. Let's imagine a complex question type with many sub-states in Incomplete state i.e. question that should pass several states with submitting data and complete redrawing (page reloading? or AJAX maybe) after each submit before it finally get graded. Let's say that student should complete a sequence of steps to achive the result).

The questions are:

  1. Can such question work in deffered feedback (and similar) model without AJAX, causing page reloads using it's own controls on each sub-state (except last step just before grading), or interaction model will prevent it?
  2. Such question should have several states records for incomplete state with identical (except ID) rows in question_states, holding different information in question_responses. Is this OK and desirable or there should be a field for substate in question_states?
  3. Would it be easy to aggregate information for several states from question_responses using you API or it would be better to duplicate necessary data on each state?
  4. Should such question be tied up to do grading only on the last step or they can use 'grade' field for Incomplete state (oh, the alternative would be adding '_partialgrade' to question_responses of course...but that's mean even more rows there)

--Oleg Sychev 17:21, 5 October 2009 (UTC)

1. Yes. The question type will need to provide its own interaction model to do this, but it will work.
2. There is no problem with almost identical rows in question_states (both id and timestamp would change) with different information in question_responses. I do not want an extra column for sub-state. If this is really important, the question type or interaction model can set $responses['_substate'] = 'whatever' (which will then get stored as a row in question_responses. Or, it can re-compute the substate from the responses each time, whichever is more convenient.
3. Yes, will be easy to aggregate the responses from all previous states. It would be unnecessarily inefficient to duplicate the data.
4. The question interaction model can choose at which stage to start assigning a grade. An example would be the current adaptive mode, where the student gets a grade after their first try, but subsequent tries may increase that, until the quiz attempt is closed.
I have chosen that the question engine will not support AJAX. It is more robust if it just has to process HTTP POST requests that reload the page. However, that does not prevent question types from using JavaScript on the client side to take student input like drag and drop and set hidden form fields that are then send to be graded by submitting a page of the quiz. (Providing accessibility is considered.)
--Tim Hunt 20:56, 5 October 2009 (UTC)

1. "The question type will need to provide its own interaction model to do this" I maybe not correctly understand you intentions about interaction models. If a question will provide it's own interaction model, does this mean that it will not work on a quiz with a standard model like deffered feedback? I thinked of a model as something selected in the module settings, that questions should get and respect... What will become on a quiz with mix of a core questions and questions with their own interaction models? --Oleg Sychev 10:27, 6 October 2009 (UTC)

You need to read this disucussion again. --Tim Hunt 19:52, 6 October 2009 (UTC)

2. "The question enginge will not support AJAX" ... i.e. if a question need some complex server-side processing (like answer parsing) and redrawing of itself after some info is subitted (but still remains uncomplete) it is forced to reload a page? Or a question type can decide to support AJAX on it's own, so it's reloading won't affect others? --Oleg Sychev 10:27, 6 October 2009 (UTC)

Ah, if a particular question type wanted to offer client-side validation backed up be ajax calls to validation logic on the server, that would be fine. What is not allowed would be ajax code that tries to submit part of a students answer. A key simplifying assumption that helps use ensure that the system is robust and reproducible is that a student's interaction with a question can be represented as a sequence of calls to question_usage_by_activity::process_response($qnumber, $response). That is, as a sequence of synchronous submissions. --Tim Hunt 19:52, 6 October 2009 (UTC)

1. Oh, sorry, now I see. I guess the tricky part would be to have a good set of selection of 'modes' that allows enought of flexibility and clarity to the quiz/lessson/whatever user and in the same time allows a question type authors enought flexibility and clarity too. Maybe starting to collect use cases may help there. There are some things that can be done too:

    1. The alternative may be to use some checkboxes, similar to review options now, to control the quiz behaviour during attempt too. That's allows for more flexibility.
    2. Or (for really complex cases) you may decide to let quiz editing interface specify an interaction model while adding a question to the quiz. They still get default values based on the quiz setting and question type choice, but user will be able to ajust them as he want, if desirable. Then you can answer any complaint if it at all possible.

2. I have nothing against using process_response() on every submitting data (even throught AJAX call). I basically try to work around another assumption: all current question types can be graded after one submission of student's answer. What if a question requires to have several (different) submissions before it gets graded? Right now you can't use such questions in non-adaptive quizzes without working around quiz engine, as student can only "submit all and finish" quiz. Should these things be left to the question types work around (which means they can bypass process_response() and must write quite a bit of code) or to create some sort of API to support it while ensuring using a process_response(). --Oleg Sychev 10:37, 7 October 2009 (UTC)

2. The API will support multi-stage questions, although I am still working out the details. That is also why there is the GaveUp state in the state diagram. If the student only completely stage one (or does not answer a question at all) before clicking submit all and finish, then the question goes into the gave up state, which is different from being graded with grade 0.--Tim Hunt 17:06, 7 October 2009 (UTC)

Subplugins code structure

I suggest to try and be consistent with the internal structure of every subplugin in Moodle.

  • All subplugins to be stored in a directory with the name of the subplugin type. So, your interaction subplugins (qim) could be stored in qim/ instead of the current interactions/
  • Whole the subplugin public API (the implementation of the subplugin interface) to be defined in lib.php. So I guess your current model.php could be renamed to lib.php
  • It may be found useful in a near future if we keep version info in version.php even if the subplugin does not have own DB tables.
  • Are the contributed (3rd party) interaction subplugins ready to be installed and/or uninstalled? Petr is working on a general install/uninstall support for all subplugins.

The goal here is to achieve a general required structure of every plugable Moodle component so the core may rely on some standard interface without additional hacks and exceptions. --David Mudrak 21:36, 12 October 2009 (UTC)

I disagree with most of this.
We have an array in get_plugin_types (in lib/moodlelib.php) that can freely convert names to paths, and few of the existing plugin types follow your proposed rule. Also, qim is short for (question interaction model) so you are proposing to store the code for these plugins in a files called question / (question interaction model) / deferredfeedback / lib.php. Whereas I am currently using a path like question / interaction / defferedfeedback / model.php.
I don't see any real benefit for calling the file lib.php rather than model.php. One advantage of model.php is that Eclipse has a shortcut (CTRL + SHIFT + R) to load a file from any folder just based on the filename, so it is better to have more distinctive names. Also, model.php only needs to be included if you are using the question engine. Arguably, it might be better for performance to save lib.php for more generic call-backs, if we ever find we need them. That way we only every need to load the code that is required in a particular place. model.php contains just the interaction model class definition, so the name says what it contains.
I don't currently see any need for a version.php, but naturally it would be easy to add one. However, at the moment, for all other plugin types, we only have a version.php if there are DB tables. I will follow the general policy on this.
I don't understand what you mean by "Are the contributed (3rd party) interaction subplugins ready to be installed and/or uninstalled?". They will install just like any other Moodle plugin, however, no-one has written any 3rd-party ones yet. I guess one of the ones I need to write for the OU will be 3rd-party.
Does that reasoning seem sound?--Tim Hunt 07:08, 13 October 2009 (UTC)
I do not like the argument that an Eclipse feature is a reason for a codebase structure. However I accept the argument that lib.php shall be reserved for some general call-backs, like post_install(), uninstall(), reset() etc which will be needed once we fully support subplugins uninstallation. In this cases, I would consistently use locallib.php
Sure, version.php can be easy to be added later once we support it. I can imagine some sort of compatibility checking, so for example a qim subplugin can require qengine version xxxx or higher.
The last question was if you plan that qim can be eventually added/removed as real plugins. In workshop, I had to sort out some issues, for example the admin decides to uninstall a grading strategy and such a grading strategy was used at some workshops. However, if qim does not contain own data, you have no such issues.
p.s. do not get me wrong and thanks for all these discussions. Proposing things, asking questions and reflecting objections is my way of learning Moodle development. --David Mudrak 08:24, 13 October 2009 (UTC)

Absolutely, discussion is important to ensure we get things right. Thank you for taking time to look at this. And you raise in important point. An attempt may use an interaction model, and then the admin may uninstall it. With the question types we have the special 'missing' question type to handle the corresponding case as gracefully as possible. I probably need to create qim_missing too.--Tim Hunt 09:37, 13 October 2009 (UTC)