Difference between revisions of "Talk:Question Engine 2:Design"

Jump to: navigation, search
m (Question: how should behave a question with many sub-states?)
Line 69: Line 69:
  
 
: --[[User:Tim Hunt|Tim Hunt]] 20:56, 5 October 2009 (UTC)
 
: --[[User:Tim Hunt|Tim Hunt]] 20:56, 5 October 2009 (UTC)
 +
 
1. "The question type will need to provide its own interaction model to do this" I maybe not correctly understand you intentions about interaction models. If a question will provide it's own interaction model, does this mean that it will not work on a quiz with a standard model like deffered feedback? I thinked of a model as something selected  in the module settings, that questions should get and respect... What will become on a quiz with mix of a core questions and questions with their own interaction models?
 
1. "The question type will need to provide its own interaction model to do this" I maybe not correctly understand you intentions about interaction models. If a question will provide it's own interaction model, does this mean that it will not work on a quiz with a standard model like deffered feedback? I thinked of a model as something selected  in the module settings, that questions should get and respect... What will become on a quiz with mix of a core questions and questions with their own interaction models?
 +
--[[User:Oleg Sychev|Oleg Sychev]] 10:27, 6 October 2009 (UTC)
 +
 +
: You need to read [[Question_Engine_2:Overview#Brief_digression:_who_controls_the_model.2C_the_quiz_or_the_question.3F|this disucussion]] again. --[[User:Tim Hunt|Tim Hunt]] 19:52, 6 October 2009 (UTC)
  
 
2. "The question enginge will not support AJAX" ... i.e. if a question need some complex server-side processing (like answer parsing) and redrawing of itself after some info is subitted (but still remains uncomplete) it is forced to reload a page? Or a question type can decide to support AJAX on it's own, so it's reloading won't affect others?
 
2. "The question enginge will not support AJAX" ... i.e. if a question need some complex server-side processing (like answer parsing) and redrawing of itself after some info is subitted (but still remains uncomplete) it is forced to reload a page? Or a question type can decide to support AJAX on it's own, so it's reloading won't affect others?
 
--[[User:Oleg Sychev|Oleg Sychev]] 10:27, 6 October 2009 (UTC)
 
--[[User:Oleg Sychev|Oleg Sychev]] 10:27, 6 October 2009 (UTC)
 +
 +
: Ah, if a particular question type wanted to offer client-side validation backed up be ajax calls to validation logic on the server, that would be fine. What is not allowed would be ajax code that tries to submit part of a students answer. A key simplifying assumption that helps use ensure that the system is robust and reproducible is that a student's interaction with a question can be represented as a sequence of calls to question_usage_by_activity::process_response($qnumber, $response). That is, as a sequence of synchronous submissions. --[[User:Tim Hunt|Tim Hunt]] 19:52, 6 October 2009 (UTC)

Revision as of 19:52, 6 October 2009

Some thoughts

Some ideas from reading this proposal:

1. question_states table store state as an Integer. However, some complex qtypes (like our planned training) may want to have custom sub-states (especially for Incomplete state). They can be stored in question_responses but that is somewhat inconsistent. What do you think about adding a field for substate in questions_states or making state field a float, so substates can be after comma while states before?

2. question_responses table structure is generic and can be used by many other modules (like assignment). I think that it will may be good to have a library of functions that work with such tables.

3. Also I wonder is naming convention (!, _) a good way to resolve conflicts in the names in question_responses table, or an additional field for the source of data (i.e. qtype, interaction model etc) would be better. The later is definitely more scalable IMHO.

--Oleg Sychev 12:48, 3 October 2009 (UTC)

Thank you Oleg. Good questions.
1. The purpose of the state field in the the question_states table is for communication. Communication between particular question types/interaction modules, and whatever is using the question engine. That is why I tried to have the smallest possible number of states. The less the outside world knows, the more freedom the question engine has.
So, I expect particular questions will do different things with the complete and incomplete states. For example a maths question that needed the student to type a complex syntax might use the Incomplete state for both the initial situation where the answer is blank, but also if the student types an answer that cannot be validated. However, all the quiz needs to know (for the navigation and the summary page in 2.0) is to tell the student that they have not finished working on that question.
Similarly, in the new Interactive mode, you might image in needs a 'Try again' state, but actually that is another Incomplete state.
So, actually, fewer states are more flexible.
2. That structure is indeed generic. In fact Moodle already has similar other structures, for example in the config and config_plugins tables. The they key point about this instance of this table is that it is linked to the question states, and the way it will be used. By the way, this table will have a lot of rows. It is essential that the API for accessing it is optimised for the kind of access that is done during large quiz attempts.
3. The reasons for this simple approach to naming is that when the data is loaded from the database, the responses become a PHP array on the state. (I hope you realise that this design page is still incomplete, which is why this is not yet explained.) Obviously arrays can only have keys and values, which is why the only way to avoid collisions is a naming convention. The other reason is, as I said, question_responses is critical for performance. Therefore, the fewer columns there, the better.
Also, student input, extra info from the question type, and extra information from the interaction model are the only things that should be storing things in this table, and it is easy to distinguish three things with a prefix.
--Tim Hunt 19:05, 3 October 2009 (UTC)

Also some ideas that may not be directly related to it:

1. I'm long dreaming about an ability to tag question categories (maybe quesitons too), and then filter them by that tags on various occasions (from listing and editing trought random questions to import/export). Tree hierarchy isn't sufficient. I have a category tree organizied by topic in the course, but then on the each topic I have one demo, several training and many control categories. And sometimes I need something to be done with all demo categories or something like that.

2. there are several questions (in core and not) - shortanswer, numerical, calculated - that shares a model of having several answers, with finding a matching one by some rules (multichoice partialy fits there in single choice mode). What do you think about abstracting it in abstract_multianswer qtype (oh, existing multianswer probably better to be known as multiquestion)? It isn't really good to place functions, that similar to all them, in root question type class.

3. Also I'm wondering, for questions with sub-states in Incomplete state, would it be easy to create a Replay on reviewing? This should be taken into consideration too IMHO.

--Oleg Sychev 12:48, 3 October 2009 (UTC)

1. The question bank is indeed a completely different issue.
2. I hope the new code will make it easier to make good use of inheritance between question types. Yes, more abstract classes is probably good. Both the one you suggest, and also a base class for t/f and multichoice.
3. Yes, with the new question engine, anything can be replayed. (Just replay the sequence of student input, which is all in the database.)
--Tim Hunt 19:05, 3 October 2009 (UTC)

Question: how should behave a question with many sub-states?

Now for a question about how should more complex qtype use this system. Let's imagine a complex question type with many sub-states in Incomplete state i.e. question that should pass several states with submitting data and complete redrawing (page reloading? or AJAX maybe) after each submit before it finally get graded. Let's say that student should complete a sequence of steps to achive the result).

The questions are:

  1. Can such question work in deffered feedback (and similar) model without AJAX, causing page reloads using it's own controls on each sub-state (except last step just before grading), or interaction model will prevent it?
  2. Such question should have several states records for incomplete state with identical (except ID) rows in question_states, holding different information in question_responses. Is this OK and desirable or there should be a field for substate in question_states?
  3. Would it be easy to aggregate information for several states from question_responses using you API or it would be better to duplicate necessary data on each state?
  4. Should such question be tied up to do grading only on the last step or they can use 'grade' field for Incomplete state (oh, the alternative would be adding '_partialgrade' to question_responses of course...but that's mean even more rows there)

--Oleg Sychev 17:21, 5 October 2009 (UTC)

1. Yes. The question type will need to provide its own interaction model to do this, but it will work.
2. There is no problem with almost identical rows in question_states (both id and timestamp would change) with different information in question_responses. I do not want an extra column for sub-state. If this is really important, the question type or interaction model can set $responses['_substate'] = 'whatever' (which will then get stored as a row in question_responses. Or, it can re-compute the substate from the responses each time, whichever is more convenient.
3. Yes, will be easy to aggregate the responses from all previous states. It would be unnecessarily inefficient to duplicate the data.
4. The question interaction model can choose at which stage to start assigning a grade. An example would be the current adaptive mode, where the student gets a grade after their first try, but subsequent tries may increase that, until the quiz attempt is closed.
I have chosen that the question engine will not support AJAX. It is more robust if it just has to process HTTP POST requests that reload the page. However, that does not prevent question types from using JavaScript on the client side to take student input like drag and drop and set hidden form fields that are then send to be graded by submitting a page of the quiz. (Providing accessibility is considered.)
--Tim Hunt 20:56, 5 October 2009 (UTC)

1. "The question type will need to provide its own interaction model to do this" I maybe not correctly understand you intentions about interaction models. If a question will provide it's own interaction model, does this mean that it will not work on a quiz with a standard model like deffered feedback? I thinked of a model as something selected in the module settings, that questions should get and respect... What will become on a quiz with mix of a core questions and questions with their own interaction models? --Oleg Sychev 10:27, 6 October 2009 (UTC)

You need to read this disucussion again. --Tim Hunt 19:52, 6 October 2009 (UTC)

2. "The question enginge will not support AJAX" ... i.e. if a question need some complex server-side processing (like answer parsing) and redrawing of itself after some info is subitted (but still remains uncomplete) it is forced to reload a page? Or a question type can decide to support AJAX on it's own, so it's reloading won't affect others? --Oleg Sychev 10:27, 6 October 2009 (UTC)

Ah, if a particular question type wanted to offer client-side validation backed up be ajax calls to validation logic on the server, that would be fine. What is not allowed would be ajax code that tries to submit part of a students answer. A key simplifying assumption that helps use ensure that the system is robust and reproducible is that a student's interaction with a question can be represented as a sequence of calls to question_usage_by_activity::process_response($qnumber, $response). That is, as a sequence of synchronous submissions. --Tim Hunt 19:52, 6 October 2009 (UTC)