Question Engine 2:Overview - Revision history

Tobias.reischmann: /* Normalise the database structure */

2015-06-03T10:25:51Z

Normalise the database structure

← Older revision		Revision as of 10:25, 3 June 2015
Line 9:		Line 9:
	At the moment, the some parts of the question_sessions and question_states tables and their relations are not normalised. This is a technical term for describing database designs. Basically, if the tables are normalised, then each 'fact' is stored in exactly one place. A normalised database is much less likely to get into an inconsistent state. Changing to a normalised structure should therefore increase robustness.		At the moment, the some parts of the question_sessions and question_states tables and their relations are not normalised. This is a technical term for describing database designs. Basically, if the tables are normalised, then each 'fact' is stored in exactly one place. A normalised database is much less likely to get into an inconsistent state. Changing to a normalised structure should therefore increase robustness.

	In addition, I wish to change the tables, so ~~the~~ the responses received from the student are stored in a much more 'raw' form. That will mean that the responses can be saved much earlier in the sequence of processing, which will again increase robustness. It will also allow the sequence of responses from the student to be replayed more easily, making it easier to continue after a server crash, to regrade, and to write automatic test scripts.		In addition, I wish to change the tables, so that the responses received from the student are stored in a much more 'raw' form. That will mean that the responses can be saved much earlier in the sequence of processing, which will again increase robustness. It will also allow the sequence of responses from the student to be replayed more easily, making it easier to continue after a server crash, to regrade, and to write automatic test scripts.

	The detailed design describes the new database structure.		The detailed design describes the new database structure.

	(+++Robustness, +Correctness)		(+++Robustness, +Correctness)


	==New concept: Question behaviours==		==New concept: Question behaviours==

Scyrma: /* Brief digression: who controls the model, the quiz or the question? */

2012-07-12T17:17:52Z

Brief digression: who controls the model, the quiz or the question?

← Older revision		Revision as of 17:17, 12 July 2012
Line 103:		Line 103:
	At the moment, there is a setting in the quiz that lets you change all the questions in the quiz from adaptive mode to non-adaptive mode at the flick of a switch. Or, at least, it allows you to re-use the same questions in both adaptive and non-adaptive mode. This suggest that the behaviour is a property of the quiz.		At the moment, there is a setting in the quiz that lets you change all the questions in the quiz from adaptive mode to non-adaptive mode at the flick of a switch. Or, at least, it allows you to re-use the same questions in both adaptive and non-adaptive mode. This suggest that the behaviour is a property of the quiz.

	On the other hand, suppose that you wanted to introduce the feature that, in adaptive mode, the student gets different amounts of feedback after each try at getting the question right. For example, the first time they get it wrong the only get a brief hint. If they are wrong the second time they get a more detailed comment, and so on. In order to do this, you need more data (the various hints) in order to define an adaptive question that is irrelevant to a non-adaptive question. Also, certain question types (e.g. Opaque, from contrib) have to, of necessity, ignore the adaptive, non-adaptive setting. And I suggested above that, manually graded question types like Essay should really be considered to have a separate behaviour. This suggests that the behaviour is a ~~properly~~ of the individual question. (Although usability considerations suggest that a single quiz should probably be constructed from questions with similar behaviour.)		On the other hand, suppose that you wanted to introduce the feature that, in adaptive mode, the student gets different amounts of feedback after each try at getting the question right. For example, the first time they get it wrong the only get a brief hint. If they are wrong the second time they get a more detailed comment, and so on. In order to do this, you need more data (the various hints) in order to define an adaptive question that is irrelevant to a non-adaptive question. Also, certain question types (e.g. Opaque, from contrib) have to, of necessity, ignore the adaptive, non-adaptive setting. And I suggested above that, manually graded question types like Essay should really be considered to have a separate behaviour. This suggests that the behaviour is a property of the individual question. (Although usability considerations suggest that a single quiz should probably be constructed from questions with similar behaviour.)

	I eventually concluded that both answers are right. That is, it is a good idea for the quiz to have a setting like adaptive/non-adaptive that sets out the teacher's intention of how each question in this quiz should behave. However, the exact choice of behaviour to use is up to the different question types. When a quiz attempts is started, each question type is asked something like "A quiz attempt is starting using 'adaptive mode'. Exactly which behaviour should be used for questions of this type in this attempt?" This gives questions that can only work in a certain way (for example essay questions) a chance to override the quiz setting.		I eventually concluded that both answers are right. That is, it is a good idea for the quiz to have a setting like adaptive/non-adaptive that sets out the teacher's intention of how each question in this quiz should behave. However, the exact choice of behaviour to use is up to the different question types. When a quiz attempts is started, each question type is asked something like "A quiz attempt is starting using 'adaptive mode'. Exactly which behaviour should be used for questions of this type in this attempt?" This gives questions that can only work in a certain way (for example essay questions) a chance to override the quiz setting.

Scyrma: /* Brief digression: who controls the model, the quiz or the question? */

2012-07-12T17:16:33Z

Brief digression: who controls the model, the quiz or the question?

← Older revision		Revision as of 17:16, 12 July 2012
Line 101:		Line 101:
	The question is: Is the behaviour a property of the question, or the quiz?		The question is: Is the behaviour a property of the question, or the quiz?

	At the moment, there is a setting in the quiz that lets you change all the questions in the quiz from adaptive mode to non-~~adptive~~ mode at the flick of a switch. Or, at least, it allows you to re-use the same questions in both adaptive and non-adaptive mode. This suggest that the behaviour is a property of the quiz.		At the moment, there is a setting in the quiz that lets you change all the questions in the quiz from adaptive mode to non-adaptive mode at the flick of a switch. Or, at least, it allows you to re-use the same questions in both adaptive and non-adaptive mode. This suggest that the behaviour is a property of the quiz.

	On the other hand, suppose that you wanted to introduce the feature that, in adaptive mode, the student gets different amounts of feedback after each try at getting the question right. For example, the first time they get it wrong the only get a brief hint. If they are wrong the second time they get a more detailed comment, and so on. In order to do this, you need more data (the various hints) in order to define an adaptive question that is irrelevant to a non-adaptive question. Also, certain question types (e.g. Opaque, from contrib) have to, of necessity, ignore the adaptive, non-adaptive setting. And I suggested above that, manually graded question types like Essay should really be considered to have a separate behaviour. This suggests that the behaviour is a properly of the individual question. (Although usability considerations suggest that a single quiz should probably be constructed from questions with similar behaviour.)		On the other hand, suppose that you wanted to introduce the feature that, in adaptive mode, the student gets different amounts of feedback after each try at getting the question right. For example, the first time they get it wrong the only get a brief hint. If they are wrong the second time they get a more detailed comment, and so on. In order to do this, you need more data (the various hints) in order to define an adaptive question that is irrelevant to a non-adaptive question. Also, certain question types (e.g. Opaque, from contrib) have to, of necessity, ignore the adaptive, non-adaptive setting. And I suggested above that, manually graded question types like Essay should really be considered to have a separate behaviour. This suggests that the behaviour is a properly of the individual question. (Although usability considerations suggest that a single quiz should probably be constructed from questions with similar behaviour.)

	I eventually concluded that both answers are right. That is, it is a good idea for the quiz to have a setting like adaptive/non-adaptive that sets out the teacher's intention of how each question in this quiz should behave. However, the exact choice of behaviour to use is up to the different question types. When a quiz attempts is started, each question type is asked something like "A quiz attempt is starting using 'adaptive mode'. Exactly which behaviour should be used for questions of this type in this attempt?" This gives questions that can only work in a certain way (for example essay questions) a chance to override the quiz setting.		I eventually concluded that both answers are right. That is, it is a good idea for the quiz to have a setting like adaptive/non-adaptive that sets out the teacher's intention of how each question in this quiz should behave. However, the exact choice of behaviour to use is up to the different question types. When a quiz attempts is started, each question type is asked something like "A quiz attempt is starting using 'adaptive mode'. Exactly which behaviour should be used for questions of this type in this attempt?" This gives questions that can only work in a certain way (for example essay questions) a chance to override the quiz setting.



	==Clarifying question states==		==Clarifying question states==

Scyrma: missing [ to set link up

2012-07-12T17:15:51Z

missing [ to set link up

← Older revision		Revision as of 17:15, 12 July 2012
Line 196:		Line 196:
	Since the question_set_attempt class keeps track of all the data that is needed when processing questions, it should be very easy to write automatic tests (unit tests) for all the other parts of the question engine. That should greatly help in eliminating bugs.		Since the question_set_attempt class keeps track of all the data that is needed when processing questions, it should be very easy to write automatic tests (unit tests) for all the other parts of the question engine. That should greatly help in eliminating bugs.

	I am intending to take a http://en.wikipedia.org/wiki/Test-driven_development test-driven] approach to implementing this proposal.		I am intending to take a [http://en.wikipedia.org/wiki/Test-driven_development test-driven] approach to implementing this proposal.

	(+++Correctness, ++Robustness)		(+++Correctness, ++Robustness)

Janaranjan: /* What are the parts of a question? */

2011-10-10T07:55:50Z

What are the parts of a question?

← Older revision		Revision as of 07:55, 10 October 2011
Line 124:		Line 124:
	==What are the parts of a question?==		==What are the parts of a question?==

	When we come to think of outputting a question on-screen, we can now see that ~~there~~ there are several things that affect what it will look like:		When we come to think of outputting a question on-screen, we can now see that there are several things that affect what it will look like:
	# The question type (multichoice, shortanswer, essay, ...)		# The question type (multichoice, shortanswer, essay, ...)
	# The behaviour (deferred feedback, interactive, CBM, ...)		# The behaviour (deferred feedback, interactive, CBM, ...)

TimHunt at 17:04, 26 April 2010

2010-04-26T17:04:56Z

Show changes

TimHunt at 10:06, 8 December 2009

2009-12-08T10:06:27Z

← Older revision		Revision as of 10:06, 8 December 2009
Line 1:		Line 1:
			{{Template:Question_engine_2}}
	This page outlines how I think the question engine should work in Moodle 2.0 or 2.1.		This page outlines how I think the question engine should work in Moodle 2.0 or 2.1.

Line 205:		Line 206:

	* Back to [[Question_Engine_2\|Question Engine 2]]		* Back to [[Question_Engine_2\|Question Engine 2]]

	~~{{CategoryDeveloper}}~~
	~~[[Category:Quiz]]~~

TimHunt: /* What are the parts of a question? */

2009-11-05T11:24:27Z

What are the parts of a question?

← Older revision		Revision as of 11:24, 5 November 2009
Line 142:		Line 142:
	The full list of options that control which of those bits are visible at any time are:		The full list of options that control which of those bits are visible at any time are:

			; correctness : (hidden / visible) whether the student gets told whether their answer was (in/partially)correct in the status summary under the question number, or instead are told something vague like 'Finished'.
	; marks : (hidden / max only / actual mark and max / actual mark and max with explanation ) whether the student can see the number of marks available, and how many marks they got, and how much detail of the marking to display.		; marks : (hidden / max only / actual mark and max / actual mark and max with explanation ) whether the student can see the number of marks available, and how many marks they got, and how much detail of the marking to display.
	; marks d.p. : (0 .. 7) how many decimal places marks are displayed to.		; marks d.p. : (0 .. 7) how many decimal places marks are displayed to.

TimHunt: /* What are the parts of a question? */

2009-11-05T11:11:00Z

What are the parts of a question?

← Older revision		Revision as of 11:11, 5 November 2009
Line 138:		Line 138:
	# The outcome from submitting that response. This has a yellow background.		# The outcome from submitting that response. This has a yellow background.

	[[Image:Parts of a ~~question~~.png]]		[[Image:Parts of a question2.png]]

	The full list of options that control which of those bits are visible at any time are:		The full list of options that control which of those bits are visible at any time are:

			; marks : (hidden / max only / actual mark and max / actual mark and max with explanation ) whether the student can see the number of marks available, and how many marks they got, and how much detail of the marking to display.
			; marks d.p. : (0 .. 7) how many decimal places marks are displayed to.
			; flags: (hidden / visible / visible and editable) this is the feature that lets students bookmark or flag a question in an attempt for later reference.
	; read-only: Whether the question just shows the response was already entered, or whether it gives controls to the user enter/change their response.		; read-only: Whether the question just shows the response was already entered, or whether it gives controls to the user enter/change their response.
	~~; flags: (hidden / visible / visible and editable) this is the feature that lets students bookmark or flag a question in an attempt for later reference.~~
	; specific feedback: (hidden / visible) feedback that relates to the particular response the student entered.		; specific feedback: (hidden / visible) feedback that relates to the particular response the student entered.
	; ~~correct response~~: (hidden / visible) whether the ~~automatically generated message or other indication of the correct answer~~ is visible.		; general feedback : (hidden / visible) whether the general feedback (same for all students) is visible.
	; ~~correctness~~: (hidden / visible) whether the automatically generated message ~~'Your~~ answer ~~was partially correct' or similar~~ is visible.		; correct response : (hidden / visible) whether the automatically generated message or other indication of the correct answer is visible.
	; manual comment: (hidden / visible / visible and editable) whether the comment manually added by the teacher is visible and editable by the current user.		; manual comment: (hidden / visible / visible and editable) whether the comment manually added by the teacher is visible and editable by the current user.
	; marks : (hidden / max available only / actual mark and max / actual mark and max with explanation ) whether the student can see the number of marks available, and how many marks they got, and how much detail of the marking to display.
	~~; marks d.p. : (0 .. 7) how many decimal places marks are displayed to.~~
	~~; general feedback : (hidden / visible) whether the general feedback (same for all students) is visible.~~
	; response history : (hidden / visible) whether the list of steps the student went through to answer the question is displayed.		; response history : (hidden / visible) whether the list of steps the student went through to answer the question is displayed.

TimHunt: /* What are the parts of a question? */

2009-10-27T13:47:35Z

What are the parts of a question?

← Older revision		Revision as of 13:47, 27 October 2009
Line 129:		Line 129:
	# The options from the particular activity (whether any marks are available for this question, are students allowed to see their grades, how many decimal places to use to display grades, does this user have permission to make a manual comment ...)		# The options from the particular activity (whether any marks are available for this question, are students allowed to see their grades, how many decimal places to use to display grades, does this user have permission to make a manual comment ...)

	Also, we want different question type and interaction models to have the freedom to display whatever they like to the students, but we also want all the questions in a quiz to look ~~similar,~~ and ~~operate in a similar way~~, so the experience is not ~~unnecessarily~~ confusing.		Also, we want different question type and interaction models to have the freedom to display whatever they like to the students; but we also want all the questions in a quiz to look and behave consistently, so that the experience is not confusing.

	In ~~this~~ sketch, I try to summarise ~~what~~ a typical question ~~might look like~~. ~~That~~ is ~~what bits might be there~~. The flexibility comes from the question type or interaction model ~~controlling what goes~~ inside those bits. ~~Using annotation~~, ~~I indicate which display options control which bits~~ of the question ~~are visible~~.		In the sketch below, I try to summarise the bits that may be present in a typical question. If all questions can have these bits in this order, then that is probably enough consistency. The flexibility comes from allowing the question type or interaction model to put whatever it likes inside those bits.

			We can divide what you see when looking at a question is into three main parts:
			# Information (meta-data): Which question number this is, a summary of what state it is currently in, what grade you have got for it (or how many marks are available, if any), whether you have flagged it. This has a grey background in the sketch below.
			# The question and the response to it. This has a blue background in the sketch.
			# The outcome from submitting that response. This has a yellow background.

	[[Image:Parts of a question.png]]		[[Image:Parts of a question.png]]

	The full list of ~~display option settings~~ are:		The full list of options that control which of those bits are visible at any time are:

	; read-only: Whether the question just shows the response was already entered, or whether it gives controls to the user enter/change their response.		; read-only: Whether the question just shows the response was already entered, or whether it gives controls to the user enter/change their response.
Line 148:		Line 153:
	; response history : (hidden / visible) whether the list of steps the student went through to answer the question is displayed.		; response history : (hidden / visible) whether the list of steps the student went through to answer the question is displayed.

	These options are initially set by whatever it is that is using the question~~. For~~ example ~~they may initially be set by~~ the quiz. Then these options are modified by the interaction model~~, so the interaction model insures~~ that no feedback is displayed until the student has submitted ~~their~~ answer, ~~and~~ that after they have submitted their final answer, the question only appears in read-only mode~~, and so on.~~		These options are initially set by whatever it is that is using the question (for example the quiz will initialise them from the quiz settings). Then these options are modified by the interaction model. For example it should ensure that no feedback is displayed until the student has actually submitted an answer, or that after they have submitted their final answer, the question only appears in read-only mode.

	~~The major subdivision of what you see when looking at a question is into three parts:~~
	# Information (meta-data): Which question number this is, a summary of what state it is currently in, what grade you have got for it (or how many marks are available), whether you have flagged it. This has a grey background in the sketch.
	~~# The question and a the response to it. This has a blue background in the sketch.~~
	~~# The outcome from submitting that response. This has a yellow background~~.

	==Reorganise the code==		==Reorganise the code==