MoodleDocs - User contributions [en]

User:Jeff Forssell

2024-01-25T13:46:03Z

Jeffjeff:

I was Born In Buffalo NY USA 1945. I have lived in Sweden since 1964.

I have a Swedish Fil. Lic. (about Masters degree) in physics and have been a Math Physics teacher for many years. I worked with adult distant education at the [[http://web.archive.org/web/20041009224241/http://www.cfl.se/?sid=60 Swedish Agency for Flexible Learning]], in Härnösand from 1996-2008. After that I've taken Moodle certification 100, 101 and 200 and am starting to learn PHP. Any interesting jobs, especially in English-speaking countries, especially involving Moodle, would be interesting to hear about!). I have given distance consulting to several Moodle sites and have registered a firm, Tupo Media.

I have also worked 2*2 years in Tanzania. (see: https://tupomedia.se/?s=tanzania .) I speak (Am.) English, Swedish and Swahili (and rusty German in an emergency and Duolingo Spanish in a pinch)

I have a site at [http://www.tupo.biz http://www.TupoMedia.se]

Main special interest are the quiz questions that are flexible, give teaching feedback, able to interpret mathematical answers including units, that collect feedback from users.

I have written a page about feedback "for all elements", not just quiz questions here: <nowiki>https://docs.moodle.org/dev/Feedback_link_for_all_elements</nowiki>

I would still like that to be created, and I still don't think it has been created. Developer Hunt said it should be "easy" when Moodle went over to the "new 2.0. If that's true I could even imagine that I could privately pay someone to do that, or help me get started at least. I have almost no PHP experience.

Feedback link for all elements

2018-12-21T13:05:03Z

Jeffjeff:

This is a work in progress when I find the time. Sometimes there is obvious repetition that I haven't cleaned up yet.

The web is more and more social in that the users/readers of content are more and more invited to influence the content. This is especially true for support pages that are for helping ie. (these links may need updating) [http://documents.google.com/support/spreadsheets/bin/answer.py?answer=87589 Google "was this helpful?"]; [http://office.microsoft.com/en-us/powerpoint/HP030713791033.aspx?pid=CH063500491033 Microsofts ditto].

Therefore it seems very important that this feature is lacking in Moodle, especially considering its social constructivistic underpinnings. It seems natural that this should be implemented, indeed almost a glaring miss.

There is a form for collecting feedback ([https://docs.moodle.org/en/Development:Feedback documentation] [http://moodle.org/mod/data/view.php?d=13&rid=95 download]) in Moodle which can be used for making course evaluations etc. The problem is that it is difficult to get people to spontaneously report suggestions, criticisms etc at all. If they have to find and go to another page, formulate a description of where they came from and what they want, felt etc, there are few that will do it, and the quality of the feedback will often lack crucial details so it may be useless.

Collecting feedback in a general form at the end of a course has many lacks. The students are poorly motivated (they won't be getting any use of changes that are made). Many annoyances, suggestions will have been forgotten. Those that are remembered will be difficult to describe accurately.

One way of nicely collecting feedback for an HTML page is to use a forum page as if it were an ordinary webpage. Then, after the content you put into the top, there is a forum at the bottom which can receive feedback and can have some soc. const. advantages over my proposal. But this wouldn't work for other kinds of pages/elements.

Having some kind of general course feedback early in a course is a good idea and can, of course, meet the first problem (that the pupil's feedback usually doesn't effect the on-going course).

=Which Elements?=
==Page==
The properties that would be good to include:
*No response
*General rating
*Lacking:
**Linkage
**Content
**Language
***grammar
***spelling
*Suggestions
* Automatically (javascript) generated info about the browser, operating system, screen and window size, plugin availability. Often the user wouldn't even know these things, and they could be critically helpful for the page editor.

Here is an example of [http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm a page feedback form]. The form has many of the things that I'd like to include. I would be like to have a small page with expandable fields connected to checked types. Only the first two checks are implemented in this. It does have a JavaScript that in the background includes auto-generated information in the form submission.

This microSoft page had some interesting questions:

Provide feedback on this information
Did this information solve your problem?
Yes
No
I don't know

Was this information relevant?
Yes
No

How much effort did you personally put forth to use this article?
Very low
Low
Moderate
High
Very high

What can we do to improve this information?

==Quiz questions (even lesson and feedback and questionnaire?)==
This is very important. If I had to chose between Page and Question, I'd chose question feedback.

Since evaluation questions are very important to the learners, they are very motivated to see that they are clear.

This can also be a way to make students an active part of the quiz creation/improvement process. Dennis Daniels has been trying to [http://video.google.com/videoplay?docid=5678154150724088700&q=dennis+daniels+moodle+quiz&total=35&start=0&num=10&so=0&type=search&plindex=7 push for this possibility]. He calls it "question challenge". A possible use scenario: The teacher quickly prepares a quiz with minimal answer alternatives and makes it available to the fastest students. These are encouraged to add more answer alternatives with motivations, and react to the existing formulation, grading and feedback. The teacher reviews their feedback and easily incorporates the useful suggestions into the test before the main body of students try the quiz.

There are many possible parts that can be specific to the feedback for questions.

Is the question itself clear? This might be able to be covered by the same fields that are of interest for general feedback for pages.

This CBM possibility has since been implemented in newer Moodles: Is my answer considered wrong though I feel it is right? It would be good to include a "certainty" rating. Certainty Based Marking, CBM, is a very valuable form of quiz feedback. It would be good to think about it at the same time as other quiz feedback. It is often a 3-step scale.: C1 C2 C3 where 1 is quite uncertain and 3 is very certain. Many teachers would appreciate the ability to connect the certainty to the grading of the question. But even while the grading implementation has not been successfully made yet, it would be valuable feedback for the teacher. "Hmm a lot of students have this right, but almost all of them are very uncertain about it." or "Hmm, this student was very certain about this incorrect answer.

Is the feedback I get for my answer: unclear, incorrect, possible to improve?

Does the user have other answers that (s)he was tempted to try? Usually people make similar mistakes and this could be a valuable resource for giving better feedback to common mistakes and for creating good "distractors" for multiple choice questions.

When a quiz editor receives feedback on a question it should be easy to use it to develop the question. Prepared fields may depend on the question type. "the answer I would have used", "an answer I might have used", "my motivation". The first 2 could be used to make new answer alternatives. The last as foundation for feedback for that answer. There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.

===Question feedback form fields===

There are prepared fields below to help you formulate your
feedback, but in case you feel like
"''I would rather just explain in my own words''": ...............
............................................................
............................................................

[_]question/[X]word in this question unclear
(check on word-> expand to: which word: ____________ )
a)possible interpretation 1 __________________________
b)possible interpretation(s) 2, ... _____________________________

[_]grade questionable, motivation: ______________________________
(this perspective maybe the teacher should be able to choose not to ask about)

answer feedback [_]wrong/[_]unclear (this would be for the answer that was active
when the feedback button was pushed, and automatically included)
What was wrong/unclear: ________________________________
Suggested improvement: _________________________________

Other possible answers:
[_]Best possible, [_]acceptable, [_]should get partial credit, [_]Not right but probable
(if last two are checked expand to: Suggested credit [____]% )
answer: ________________________________________________
motivation: ____________________________________________
(how one would get this answer.)

[_]More possible answers? (gives more fields like above)

Social bookmarking is important since what's good for the goose isn't always good for the gander! The grade goal of students varies and this can mean that a highly rated resource for one person could be useless for another. Therefore a rating should be able to be connected to information about the user. Anonymous (though visible for admin if breaking netiquette) ratings are better than none, though the value would be increased if complemented by some info like grade goal, special branch, language proficiency, disabilities, etc.

=Who would the feedback go to?=

There could be some parts of a Moodle distribution that are default set to collect feedback. The pages that are for administrators, teacher editors should be of interest to Moodle developers. Help pages should collect to the documenters. It should be possible to turn off these default settings if a site doesn't want to participate.

The other parts could choose at different levels, admin, course developer, teacher, to invite feedback for specific courses, pages, questions and which recipients would get it.

Another consideration is making feedback easier/possible for people using Moodle off line. There are many situations, especially in many places in developing countries, where Internet access is poor, slow or nonexistent. Moodle is still usable on LAN or even a single computer or on a USB memory stick that can be used on different computers, some on line some off line. It would be good if there were preparations made for saving up feedback for when coming on line or for exporting to a file that could be sent on a floppy, through a mobile phone etc to someone on line.

==Some Examples==

Developers have made a new version of some component in Moodle and are really not sure about if it will be easily understood and function well. They can than choose to set a VERY Inviting feedback link. "This page is new and we would really like suggestions about how to make it better"
The feedback choice will be default in Moodle packages and the developers will get the feedback from any site that doesn't turn it off (and has a user giving feedback). The form should contain all the fields, as many as possible automatically, that would be needed to interface with the Tracker: first for search, then for entering a new bug/feature.

A school administrator would like to collect any views that anyone using the site has about pages and turns on a more general feedback invitation: "If this page doesn't meet your needs, please tell us!" (S)he choses to have the feedback go to (her)him and to the course teacher in charge of the course containing the page. Some administrators have wished for a channel for getting info, questions, suggestions even from non-registered visitors (usually on the front page but perhaps even in other guest open courses). Feedback from non-registered visitors should probably include a reCaptcha antispam protection.

A teacher could want to have students rate pages to be able to give other students a Social bookmarking guide especially if there is a lot of varied material that everybody isn't expected to go through. Or compare the webpages with the textbook. [the book was better][Both were needed][the webpage better][Neither helped much]. This particular type of feedback would be good to be able to display for the students, perhaps tagged according to grade goals, previous experience, math love/hate, etc.

=Links=

[http://tracker.moodle.org/browse/MDL-18067 The Moodle tracker issue for this suggestion.]

[http://blog.pbwiki.com/2008/03/28/the-foolproof-5-step-way-to-answer-tough-questions/ A blog post about answering tough questions] includes "Check afterwards to see if the questioner feels satisfied."

Feedback link for all elements

2014-11-23T11:57:05Z

Jeffjeff: /* Quiz questions (even lesson and feedback and questionnaire?) */ CBM has been implemented in Moodle

This is a work in progress when I find the time. Sometimes there is obvious repetition that I haven't cleaned up yet.

The web is more and more social in that the users/readers of content are more and more invited to influence the content. This is especially true for support pages that are for helping ie. [http://documents.google.com/support/spreadsheets/bin/answer.py?answer=87589 Google "was this helpful?"]; [http://office.microsoft.com/en-us/powerpoint/HP030713791033.aspx?pid=CH063500491033 Microsofts ditto].

Therefore it seems very important that this feature is lacking in Moodle, especially considering its social constructivistic underpinnings. It seems natural that this should be implemented, indeed almost a glaring miss.

There is a form for collecting feedback ([https://docs.moodle.org/en/Development:Feedback documentation] [http://moodle.org/mod/data/view.php?d=13&rid=95 download]) in Moodle which can be used for making course evaluations etc. The problem is that it is difficult to get people to spontaneously report suggestions, criticisms etc at all. If they have to find and go to another page, formulate a description of where they came from and what they want, felt etc, there are few that will do it, and the quality of the feedback will often lack crucial details so it may be useless.

Collecting feedback in a general form at the end of a course has many lacks. The students are poorly motivated (they won't be getting any use of changes that are made). Many annoyances, suggestions will have been forgotten. Those that are remembered will be difficult to describe accurately.

One way of nicely collecting feedback for an HTML page is to use a forum page as if it were an ordinary webpage. Then, after the content you put into the top, there is a forum at the bottom which can receive feedback and can have some soc. const. advantages over my proposal. But this wouldn't work for other kinds of pages/elements.

Having some kind of general course feedback early in a course is a good idea and can, of course, meet the first problem (that the pupil's feedback usually doesn't effect the on-going course).

=Which Elements?=
==Page==
The properties that would be good to include:
*No response
*General rating
*Lacking:
**Linkage
**Content
**Language
***grammar
***spelling
*Suggestions
* Automatically (javascript) generated info about the browser, operating system, screen and window size, plugin availability. Often the user wouldn't even know these things, and they could be critically helpful for the page editor.

Here is an example of [http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm a page feedback form]. The form has many of the things that I'd like to include. I would be like to have a small page with expandable fields connected to checked types. Only the first two checks are implemented in this. It does have a JavaScript that in the background includes auto-generated information in the form submission.

This microSoft page had some interesting questions:

Provide feedback on this information
Did this information solve your problem?
Yes
No
I don't know

Was this information relevant?
Yes
No

How much effort did you personally put forth to use this article?
Very low
Low
Moderate
High
Very high

What can we do to improve this information?

==Quiz questions (even lesson and feedback and questionnaire?)==
This is very important. If I had to chose between Page and Question, I'd chose question feedback.

Since evaluation questions are very important to the learners, they are very motivated to see that they are clear.

This can also be a way to make students an active part of the quiz creation/improvement process. Dennis Daniels has been trying to [http://video.google.com/videoplay?docid=5678154150724088700&q=dennis+daniels+moodle+quiz&total=35&start=0&num=10&so=0&type=search&plindex=7 push for this possibility]. He calls it "question challenge". A possible use scenario: The teacher quickly prepares a quiz with minimal answer alternatives and makes it available to the fastest students. These are encouraged to add more answer alternatives with motivations, and react to the existing formulation, grading and feedback. The teacher reviews their feedback and easily incorporates the useful suggestions into the test before the main body of students try the quiz.

There are many possible parts that can be specific to the feedback for questions.

Is the question itself clear? This might be able to be covered by the same fields that are of interest for general feedback for pages.

This CBM possibility has since been implemented in newer Moodles: Is my answer considered wrong though I feel it is right? It would be good to include a "certainty" rating. Certainty Based Marking, CBM, is a very valuable form of quiz feedback. It would be good to think about it at the same time as other quiz feedback. It is often a 3-step scale.: C1 C2 C3 where 1 is quite uncertain and 3 is very certain. Many teachers would appreciate the ability to connect the certainty to the grading of the question. But even while the grading implementation has not been successfully made yet, it would be valuable feedback for the teacher. "Hmm a lot of students have this right, but almost all of them are very uncertain about it." or "Hmm, this student was very certain about this incorrect answer.

Is the feedback I get for my answer: unclear, incorrect, possible to improve?

Does the user have other answers that (s)he was tempted to try? Usually people make similar mistakes and this could be a valuable resource for giving better feedback to common mistakes and for creating good "distractors" for multiple choice questions.

When a quiz editor receives feedback on a question it should be easy to use it to develop the question. Prepared fields may depend on the question type. "the answer I would have used", "an answer I might have used", "my motivation". The first 2 could be used to make new answer alternatives. The last as foundation for feedback for that answer. There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.

===Question feedback form fields===

There are prepared fields below to help you formulate your
feedback, but in case you feel like
"''I would rather just explain in my own words''": ...............
............................................................
............................................................

[_]question/[X]word in this question unclear
(check on word-> expand to: which word: ____________ )
a)possible interpretation 1 __________________________
b)possible interpretation(s) 2, ... _____________________________

[_]grade questionable, motivation: ______________________________
(this perspective maybe the teacher should be able to choose not to ask about)

answer feedback [_]wrong/[_]unclear (this would be for the answer that was active
when the feedback button was pushed, and automatically included)
What was wrong/unclear: ________________________________
Suggested improvement: _________________________________

Other possible answers:
[_]Best possible, [_]acceptable, [_]should get partial credit, [_]Not right but probable
(if last two are checked expand to: Suggested credit [____]% )
answer: ________________________________________________
motivation: ____________________________________________
(how one would get this answer.)

[_]More possible answers? (gives more fields like above)

Social bookmarking is important since what's good for the goose isn't always good for the gander! The grade goal of students varies and this can mean that a highly rated resource for one person could be useless for another. Therefore a rating should be able to be connected to information about the user. Anonymous (though visible for admin if breaking netiquette) ratings are better than none, though the value would be increased if complemented by some info like grade goal, special branch, language proficiency, disabilities, etc.

=Who would the feedback go to?=

There could be some parts of a Moodle distribution that are default set to collect feedback. The pages that are for administrators, teacher editors should be of interest to Moodle developers. Help pages should collect to the documenters. It should be possible to turn off these default settings if a site doesn't want to participate.

The other parts could choose at different levels, admin, course developer, teacher, to invite feedback for specific courses, pages, questions and which recipients would get it.

Another consideration is making feedback easier/possible for people using Moodle off line. There are many situations, especially in many places in developing countries, where Internet access is poor, slow or nonexistent. Moodle is still usable on LAN or even a single computer or on a USB memory stick that can be used on different computers, some on line some off line. It would be good if there were preparations made for saving up feedback for when coming on line or for exporting to a file that could be sent on a floppy, through a mobile phone etc to someone on line.

==Some Examples==

Developers have made a new version of some component in Moodle and are really not sure about if it will be easily understood and function well. They can than choose to set a VERY Inviting feedback link. "This page is new and we would really like suggestions about how to make it better"
The feedback choice will be default in Moodle packages and the developers will get the feedback from any site that doesn't turn it off (and has a user giving feedback). The form should contain all the fields, as many as possible automatically, that would be needed to interface with the Tracker: first for search, then for entering a new bug/feature.

A school administrator would like to collect any views that anyone using the site has about pages and turns on a more general feedback invitation: "If this page doesn't meet your needs, please tell us!" (S)he choses to have the feedback go to (her)him and to the course teacher in charge of the course containing the page. Some administrators have wished for a channel for getting info, questions, suggestions even from non-registered visitors (usually on the front page but perhaps even in other guest open courses). Feedback from non-registered visitors should probably include a reCaptcha antispam protection.

A teacher could want to have students rate pages to be able to give other students a Social bookmarking guide especially if there is a lot of varied material that everybody isn't expected to go through. Or compare the webpages with the textbook. [the book was better][Both were needed][the webpage better][Neither helped much]. This particular type of feedback would be good to be able to display for the students, perhaps tagged according to grade goals, previous experience, math love/hate, etc.

=Links=

[http://tracker.moodle.org/browse/MDL-18067 The Moodle tracker issue for this suggestion.]

[http://blog.pbwiki.com/2008/03/28/the-foolproof-5-step-way-to-answer-tough-questions/ A blog post about answering tough questions] includes "Check afterwards to see if the questioner feels satisfied."

User:Jeff Forssell

2010-11-23T09:52:05Z

Jeffjeff:

User:Jeff Forssell

2010-11-22T12:07:24Z

Jeffjeff:

Talk:Setting up Netbeans

2010-08-15T21:04:39Z

Jeffjeff:

== CVS instructions ==

I copied the instructions from [[User:Gary_Anderson| Gary Anderson]]'s ''NetBeans for Moodle Development course'' (http://moodle2.seattleacademy.org/wiki/index.php/Get_the_Moodle_Software, http://moodle2.seattleacademy.org/course/view.php?id=5). There might be some redundancies which still have to be amended. --[[User:Frank Ralf|Frank Ralf]] 09:43, 20 May 2009 (UTC)

: I was in the Gary Andersson course and there was a prepared package there with both NetBeans and Moodle. But Right now I can't get to that course at all. I am setting up on another computer and would like to have the latest NetBeans 6.7, and set up Moodle preferably both 1.9.5+ and HEAD 2.0.

: When I see the instructions here I don't see anything about getting the xamp server setup.

: I also wonder about if one should maybe use a complete install package first than connect that to NetBeans and after that try doing CVS updating through NetBeans. (I maybe wrong, but I assume that doing a complete install via CVS would take much longer.) I also assume that I must than create an extra Moodle database in MySQL and of course a separate folder if I want to install 2.0 also. [[User:Jeff Forssell|Jeff Forssell]] 13:02, 9 July 2009 (UTC)

:: Hi Jeff, If you want to use CVS you should do a manual installation.
::* Setting up XAMPP is nothing more than downloading and unpacking it, really painless. The web server's directory will be \htdocs\.
::* Then you install NetBeans 6.7 and do separate CVS checkouts for HEAD (2.0) and 1.9.5 into separate directories (e.g. \htdocs\moodle19\ and \htdocs\moodle20\.
::* Then you can run Moodle 1.9.5 under "localhost/moodle19/" in your browser and Moodle "localhost/moodle20/".
:: Further details for manually setting up Moodle you'll find under [[Windows installation using XAMPP]]. hth --[[User:Frank Ralf|Frank Ralf]] 13:42, 9 July 2009 (UTC)

:::Thanx

:::I've done lots of xampp installs of the Moodle complete packages, but never just xampp alone. Should I interpret your answer as I have to install Moodle via CVS if I want to use CVS to keep it updated? And that it won't be ridiculously long time doing the original checkout? In the GA course we did a CVS update of the original install, but I don't know if he had done that package by doing a CVS checkout. (It just doesn't feel like that was what was going on!)
:::: The first checkout might take some time, but as far as I remember it's done in about 15 minutes. --[[User:Frank Ralf|Frank Ralf]] 16:02, 9 July 2009 (UTC)

::: Oh well I suppose it's time to learn to create MySQL databases for Moodle without complete packages or Fantastico! [[User:Jeff Forssell|Jeff Forssell]] 15:04, 9 July 2009 (UTC)
:::: XAMPP comes with PHPMyAdmin for MySQL administration. That's a very useful tool and worth exploring. --[[User:Frank Ralf|Frank Ralf]] 16:02, 9 July 2009 (UTC)

:::: Just another thing: Moodle 2.0 requires quite recent versions of PHP and MySQL. So for using HEAD I had to install a new version of XAMPP (I have two versions installed on my development machine which is no problem, you only should not run them at the same time). --[[User:Frank Ralf|Frank Ralf]] 16:14, 9 July 2009 (UTC)
::::: I have now CVSed moodle 1.9 with NetBeans - it took less than 10 mins. Created a new project. Then it started "scanning projects" - I didn't know what it is doing and it wasn't finished after more than 10 minutes. I see now that it has stopped. Anyhow now I\ve also done the same with HEAD / moodle20.

:::::I have some hesitation about trying to "run" them. Anything one should do before that other than creating the projects? [[User:Jeff Forssell|Jeff Forssell]] 08:55, 10 July 2009 (UTC)

::::: I see when trying to RUN that my moodle 2.0 is in http://localhost/moodle20/moodle/install.php while i thought it would be in http://localhost/moodle20/install.php. Maybe that's good because the NB proj files are in a folder there too.

:::::When I get to the "web address" page that is greyed out. I was hoping to try out setting the computer name so I could access from others on my intranet. How can I change that (from "localhost")?

:::::: Hi Jeff, I would suggest moving this discussion to the appropriate forum to give more people the chance to join. This here is quite a private place and should preferably only be used for issues directly connected to this Moodle Docs page. Thanks in advance! --[[User:Frank Ralf|Frank Ralf]] 10:29, 10 July 2009 (UTC)

::::::: Which is "the appropriate forum" for this subject? [[User:Jeff Forssell|Jeff Forssell]] 21:04, 15 August 2010 (UTC)
:::::: "localhost" is always the local PC itself. You can access it from other computers by using the PC's IP address (use "ipconfig" in a DOS box) or by modifying the "hosts" file on your PC.

Feedback link for all elements

2010-05-05T10:48:56Z

Jeffjeff: /* Page */ how much effort did YOU put forth to use page?

This is a work in progress when I find the time. Sometimes there is obvious repetition that I haven't cleaned up yet.

The web is more and more social in that the users/readers of content are more and more invited to influence the content. This is especially true for support pages that are for helping ie. [http://documents.google.com/support/spreadsheets/bin/answer.py?answer=87589 Google "was this helpful?"]; [http://office.microsoft.com/en-us/powerpoint/HP030713791033.aspx?pid=CH063500491033 Microsofts ditto].

Therefore it seems very important that this feature is lacking in Moodle, especially considering its social constructivistic underpinnings. It seems natural that this should be implemented, indeed almost a glaring miss.

There is a form for collecting feedback ([https://docs.moodle.org/en/Development:Feedback documentation] [http://moodle.org/mod/data/view.php?d=13&rid=95 download]) in Moodle which can be used for making course evaluations etc. The problem is that it is difficult to get people to spontaneously report suggestions, criticisms etc at all. If they have to find and go to another page, formulate a description of where they came from and what they want, felt etc, there are few that will do it, and the quality of the feedback will often lack crucial details so it may be useless.

Collecting feedback in a general form at the end of a course has many lacks. The students are poorly motivated (they won't be getting any use of changes that are made). Many annoyances, suggestions will have been forgotten. Those that are remembered will be difficult to describe accurately.

One way of nicely collecting feedback for an HTML page is to use a forum page as if it were an ordinary webpage. Then, after the content you put into the top, there is a forum at the bottom which can receive feedback and can have some soc. const. advantages over my proposal. But this wouldn't work for other kinds of pages/elements.

Having some kind of general course feedback early in a course is a good idea and can, of course, meet the first problem (that the pupil's feedback usually doesn't effect the on-going course).

=Which Elements?=
==Page==
The properties that would be good to include:
*No response
*General rating
*Lacking:
**Linkage
**Content
**Language
***grammar
***spelling
*Suggestions
* Automatically (javascript) generated info about the browser, operating system, screen and window size, plugin availability. Often the user wouldn't even know these things, and they could be critically helpful for the page editor.

Here is an example of [http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm a page feedback form]. The form has many of the things that I'd like to include. I would be like to have a small page with expandable fields connected to checked types. Only the first two checks are implemented in this. It does have a JavaScript that in the background includes auto-generated information in the form submission.

This microSoft page had some interesting questions:

Provide feedback on this information
Did this information solve your problem?
Yes
No
I don't know

Was this information relevant?
Yes
No

How much effort did you personally put forth to use this article?
Very low
Low
Moderate
High
Very high

What can we do to improve this information?

==Quiz questions (even lesson and feedback and questionnaire?)==
This is very important. If I had to chose between Page and Question, I'd chose question feedback.

Since evaluation questions are very important to the learners, they are very motivated to see that they are clear.

This can also be a way to make students an active part of the quiz creation/improvement process. Dennis Daniels has been trying to [http://video.google.com/videoplay?docid=5678154150724088700&q=dennis+daniels+moodle+quiz&total=35&start=0&num=10&so=0&type=search&plindex=7 push for this possibility]. He calls it "question challenge". A possible use scenario: The teacher quickly prepares a quiz with minimal answer alternatives and makes it available to the fastest students. These are encouraged to add more answer alternatives with motivations, and react to the existing formulation, grading and feedback. The teacher reviews their feedback and easily incorporates the useful suggestions into the test before the main body of students try the quiz.

There are many possible parts that can be specific to the feedback for questions.

Is the question itself clear? This might be able to be covered by the same fields that are of interest for general feedback for pages.

Is my answer considered wrong though I feel it is right? It would be good to include a "certainty" rating. Certainty Based Marking, CBM, is a very valuable form of quiz feedback. It would be good to think about it at the same time as other quiz feedback. It is often a 3-step scale.: C1 C2 C3 where 1 is quite uncertain and 3 is very certain. Many teachers would appreciate the ability to connect the certainty to the grading of the question. But even while the grading implementation has not been successfully made yet, it would be valuable feedback for the teacher. "Hmm a lot of students have this right, but almost all of them are very uncertain about it." or "Hmm, this student was very certain about this incorrect answer."

Is the feedback I get for my answer: unclear, incorrect, possible to improve?

Does the user have other answers that (s)he was tempted to try? Usually people make similar mistakes and this could be a valuable resource for giving better feedback to common mistakes and for creating good "distractors" for multiple choice questions.

When a quiz editor receives feedback on a question it should be easy to use it to develop the question. Prepared fields may depend on the question type. "the answer I would have used", "an answer I might have used", "my motivation". The first 2 could be used to make new answer alternatives. The last as foundation for feedback for that answer. There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.

===Question feedback form fields===

There are prepared fields below to help you formulate your
feedback, but in case you feel like
"''I would rather just explain in my own words''": ...............
............................................................
............................................................

[_]question/[X]word in this question unclear
(check on word-> expand to: which word: ____________ )
a)possible interpretation 1 __________________________
b)possible interpretation(s) 2, ... _____________________________

[_]grade questionable, motivation: ______________________________
(this perspective maybe the teacher should be able to choose not to ask about)

answer feedback [_]wrong/[_]unclear (this would be for the answer that was active
when the feedback button was pushed, and automatically included)
What was wrong/unclear: ________________________________
Suggested improvement: _________________________________

Other possible answers:
[_]Best possible, [_]acceptable, [_]should get partial credit, [_]Not right but probable
(if last two are checked expand to: Suggested credit [____]% )
answer: ________________________________________________
motivation: ____________________________________________
(how one would get this answer.)

[_]More possible answers? (gives more fields like above)

Social bookmarking is important since what's good for the goose isn't always good for the gander! The grade goal of students varies and this can mean that a highly rated resource for one person could be useless for another. Therefore a rating should be able to be connected to information about the user. Anonymous (though visible for admin if breaking netiquette) ratings are better than none, though the value would be increased if complemented by some info like grade goal, special branch, language proficiency, disabilities, etc.

=Who would the feedback go to?=

There could be some parts of a Moodle distribution that are default set to collect feedback. The pages that are for administrators, teacher editors should be of interest to Moodle developers. Help pages should collect to the documenters. It should be possible to turn off these default settings if a site doesn't want to participate.

The other parts could choose at different levels, admin, course developer, teacher, to invite feedback for specific courses, pages, questions and which recipients would get it.

Another consideration is making feedback easier/possible for people using Moodle off line. There are many situations, especially in many places in developing countries, where Internet access is poor, slow or nonexistent. Moodle is still usable on LAN or even a single computer or on a USB memory stick that can be used on different computers, some on line some off line. It would be good if there were preparations made for saving up feedback for when coming on line or for exporting to a file that could be sent on a floppy, through a mobile phone etc to someone on line.

==Some Examples==

Developers have made a new version of some component in Moodle and are really not sure about if it will be easily understood and function well. They can than choose to set a VERY Inviting feedback link. "This page is new and we would really like suggestions about how to make it better"
The feedback choice will be default in Moodle packages and the developers will get the feedback from any site that doesn't turn it off (and has a user giving feedback). The form should contain all the fields, as many as possible automatically, that would be needed to interface with the Tracker: first for search, then for entering a new bug/feature.

A school administrator would like to collect any views that anyone using the site has about pages and turns on a more general feedback invitation: "If this page doesn't meet your needs, please tell us!" (S)he choses to have the feedback go to (her)him and to the course teacher in charge of the course containing the page. Some administrators have wished for a channel for getting info, questions, suggestions even from non-registered visitors (usually on the front page but perhaps even in other guest open courses). Feedback from non-registered visitors should probably include a reCaptcha antispam protection.

A teacher could want to have students rate pages to be able to give other students a Social bookmarking guide especially if there is a lot of varied material that everybody isn't expected to go through. Or compare the webpages with the textbook. [the book was better][Both were needed][the webpage better][Neither helped much]. This particular type of feedback would be good to be able to display for the students, perhaps tagged according to grade goals, previous experience, math love/hate, etc.

=Links=

[http://tracker.moodle.org/browse/MDL-18067 The Moodle tracker issue for this suggestion.]

[http://blog.pbwiki.com/2008/03/28/the-foolproof-5-step-way-to-answer-tough-questions/ A blog post about answering tough questions] includes "Check afterwards to see if the questioner feels satisfied."

Setting up Netbeans

2009-10-16T07:55:20Z

Jeffjeff: /* Coding faster */

[http://www.netbeans.org/features/php/index.html NetBeans] has got a good PHP support. You find a host of information on the website (tutorials, developer blog, screen casts, etc.).

== Features ==
* CVS integration: see all changes, lines deletion, diff in real time, show annotations, diff history...
* Ctrl+Click: Go to declaration
* Export/Import Diff Patch
* Easy navigation
* List of functions
* Code completion
* Instant rename
* HTML, CSS, JavaScript support
* MySQL manager
* Quick Search
* Very few bugs

== Installation ==

* I recommend downloading the latest [http://www.netbeans.org/downloads/indexB.html 6.7.1 version with PHP support ]. It's only a 26 MB download.
* Install and run it.

== Set up for Moodle development ==

* Checkout your Moodle project with a '''CVS client''' - see [[CVS for Administrators]] or [[CVS for developers|CVS for Developers]].

* Open File > New Project > PHP > PHP Application > Next
: You're going to set the project now. ''Name'', ''Location'' and ''Folder'' are used by NetBeans and are not related to the source code. So you can choose whatever you like, except your source folder. ''Sources'' has to be your checked out Moodle branch/head folder. The rest is clear enough. Don't forget to choose UTF-8 for ''Default Encoding''.

: Example:

Project Name: Moodle 1.9 Stable
Project Location: C:\Users\jerome\Documents\NetBeansProjects
Project Folder: C:\Users\jerome\Documents\NetBeansProjects\Moodle 1.9 Stable
Project Sources: C:\Users\jerome\Projects\branch19_STABLE\moodle
Project URL: http://localhost/moodle19/
Index File: index.php
Create: unchecked
Default Encoding: UTF-8
Set as Main Project: unchecked

* Click on Finish.

* Start coding!

== CVS with NetBeans ==

NetBeans comes with '''integrated CVS support''' which might be the easiest way to check out Moodle.

* In NetBeans, select Window->Versioning->CVS->Checkout
* Select Team->CVS->Checkout
* Enter into CVS Root: '':pserver:anonymous@us.cvs.moodle.org:/cvsroot/moodle''
: (Non-US-residents might use one of the other [https://docs.moodle.org/en/CVS_for_Administrators#CVS_Servers Moodle CVS servers] nearer to them.)
* Click ''Next''
* Browse or enter into ''Module:'' moodle
* Browse or enter into ''Branch:'' MOODLE_19_STABLE
* Browse or enter into ''Local Folder:'' C:\xampp\htdocs
* Click ''Finish'' (and wait a few minutes for Moodle to be checked out)
* When you get the dialog box "Do you want to create an IDE project from the checked-out sources?", Click "Create Project..."
* Select PHP Application with Existing Sources, and click Next
* Browse or enter into ''Sources Folder'' C:\xampp\htdocs\moodle
* Enter into ''Project Name:'' moodle
* Keep the other defaults and click next
* ''Run As:'' should have selected ''Local Web Site (running on local web server)''
* Enter into Project URL http://localhost/moodle/
* Browse or enter into ''Index File:'' index.php
* Click ''Finish''

=== Few warnings ===
Some of these warnings could also apply to other IDE:
* If you want to delete a file from your computer but not from CVS, delete it from Windows Explorer/Nautilus/Finder. Otherwise your next commit could delete the file from CVS. In case you have already deleted a wrong file from Netbeans: with Windows Explorer/Nautilus/Finder, delete all the folder content of this deleted file (including the CVS folder) and update the folder.
* If you rename files and that other people are working on them, Netbeans could end up to mess up your CVS folder (even though that is quite rare). Then Netbeans CVS will refuse to update your code displaying a no explicit error as '''Update Failed'''. In this case, delete the content of the damaged folder with Windows Explorer/Nautilus/Finder. Then update this folder.
* If you create a patch or a new php file with Netbeans on Microsoft Windows, please check that it's a Unix format file. You may want to use another software to create Unix format patch/php file.

== Optimization ==

1. You may want to run Netbeans with the Sun JDK. Netbeans seems to work a bit better with the Sun JDK. Download it from [http://java.sun.com/javase/downloads/index.jsp]. You'll have to edit Netbeans config file. Open netbeans/etc/netbeans.conf. Then uncomment and edit:

netbeans_jdkhome="your_JDK_path"

2. To change the Netbeans look and feel run netbeans in the command line with this parameter:
"netbeans" --laf javax.swing.plaf.metal.MetalLookAndFeel

3. If Netbeans starts to slow down, give it more memory
"netbeans" -J-Xmx600m

== Coding faster ==
You may want to have a look at the [http://wiki.netbeans.org/KeymapProfileFor60 keyboard shortcuts]. 
You can also follow the [http://blogs.sun.com/netbeansphp/ Netbeans PHP blog].

== See also: ==
* Moodle forum: [http://moodle.org/mod/forum/discuss.php?d=112972 NetBeans 6.5 for moodle/PHP/debugging is a better experience v.s Eclipse]
* [http://www.netbeans.org/kb/trails/php.html NetBeans PHP Learning Trail]
* [http://wiki.netbeans.org/PHP NetBeans PHP Wiki]

[[Category:NetBeans]]
[[Category:Developer tools|NetBeans]]

Talk:Tracking Moodle CVS with git

2009-10-14T10:43:26Z

Jeffjeff: /* Anyone got any experience with NetBeans and GIT? */

I am currently doing a bit of an overhaul of this page to remove obsolete coigto stuff - dont really see a reason why anyone would use it or want docs for it.--[[User:Dan Poltawski|Dan Poltawski]] 17:04, 1 October 2008 (CDT)
:Great! Thanks! --[[User:Eloy Lafuente (stronk7)|Eloy Lafuente (stronk7)]] 17:05, 1 October 2008 (CDT)

== Mass merge ==

"...If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename..."

can anybody please elaborate a bit? Provide and example? TIA --[[User:David Mudrak|David Mudrak]] 15:30, 18 December 2008 (CST)

== Anyone got any experience with NetBeans and GIT? ==

I've just started getting into CVS and am using netBeans. I like NetBeans so far and using the usual CVS seems to work OK.

GIT seem to be "quite the rage" now. Now I'm wondering should I try to get into that while I'm still learning CVS? A google search reveals at least one plugin for GIT and NetBeans: NBgit. [[User:Jeff Forssell|Jeff Forssell]] 10:43, 14 October 2009 (UTC)

=== CVS Export ===

Is that command example even correct - "GIT_DIR=../moodle-19/.git git cvsexportcommit b38cb61" doesn't look like any correct syntax and is nothing like the example in the manual page.. --[[User:Howard Miller|Howard Miller]] 10:22, 14 October 2009 (UTC)

$ export GIT_DIR=~/project/.git
$ cd ~/project_cvs_checkout
$ git cvsexportcommit -v <commit-sha1>
$ cvs commit -F .msg <files>

Also, this paragraph:

''"If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this."''

...doesn't mean anything to me at all (what's Cogito and cg-update?). It needs contextualised in some way. This is probably a very useful feature while Moodle is CVS and could use some more explanation. --[[User:Howard Miller|Howard Miller]] 10:22, 14 October 2009 (UTC)

Open protocol for accessing question engines

2009-10-14T08:18:16Z

Jeffjeff: /* The parts of the system */ tale->take

==History==

Previously (2004 to early 2005) there was an attempt (by the Serving Maths Project, Gustav Delius and others) to create something called [http://mantis.york.ac.uk/moodle/course/view.php?id=14 Remote Question Protocol]. This was a web service interface designed to allow question from different question engines (for example [http://web.mat.bham.ac.uk/C.J.Sangwin/aim/ AIM] and [http://stack.bham.ac.uk/ STACK]) to be included in Moodle quizzes. The effort ran out of resources at a time when the code was not in a working state. Several groups have an interest in getting RQP working again, although no one is currently working on it.

During 2005, The Open University (mainly Sam Marshall) developed in-house an online assessment system called [http://www.open.ac.uk/openmarkexamples/ OpenMark]. Its architecture was of separate question engines and a test navigator, communicating via a web service link which we call Opaque (Open Protocol for Accessing QUestion Engines).

Then in May 2006, I (Tim Hunt) took over maintenance of the quiz module from Gustav. At which point I learned about RQP. I was immediately struck by how similar Opaque and RQP were conceptually, although they differed in details.

In autumn 2006, the OU wanted to be able to include OM question in Moodle quizzes. Our choices are to either
# change OM to use RQP; or
# change (add to) Moodle to support Opaque.
We have pretty-much decided to do 2., because it is less work for us, and becuase Opaque currenly works and RQP doesn't. Therefore, RQP implementing question engines will have to change anyway, and since Opaque is conceptually similar to RQP, changing to Opaque won't be much more work that changed to whatever the next iteration of RQP would have been.

OpenMark was released under an open source license (hosted at [http://openmark.dev.java.net/ dev.java.net]) in Spring 2007; and STACK 2.0 (http://stack.bham.ac.uk/) was released just before Christmas 2007, so there are now at least two freely available Opaque compatible systems.

==Rationale==

Why are people interested in this sort of protocol? The answer is separation of concerns. The things an online assessment system has to do are:
# Identify students, test, which students can do which tests at what times, and which questions make up each test.
# Record the scores and other information that results from students attempting test, and provide reporting fascilities for this information.
# Display questions to students and process the responses the students make to generate scores, feedback, etc.
# Allow teachers to create and configure the questions.

Moodle is very good at points 1 and 2, there is a lot of boring book-keeping do be done here, and it is already implemented in Moodle.

For the developers who are interested in Opaque/RQP, the really interesting part of an assessment system is 3. They want to be able to pose really intersting types of questions to students to really make them think, and then use sophisticated algorithms to analyse the student's response and give targeted feedback when they get things wrong.

Doing this may require specialist tools. For example, STACK and AIM both use Computer algebra systems to set and mark interesting mathematical questions. OpenMark lets question authors write answer marking and feedback generation algorithms in Java code, with a library of helpful classes and functions. It is simply not feasible to rewrite these question engines in PHP and turn them into Moodle question types. Hence the search for another way to get these questions into Moodle.

Of course, each of STACK, AIM, OM, etc. could programme their own solutions to the administration parts 1 and 2. However, this is a needless duplication of effort. It does not allow questions from different question engines to be combined in the same test. And people who are interested in writing advanced question engines are normally not very interested in writing student administration systems.

Finally, there is point 4, question authoring. Authoring advanced question types is a complicated task that will probably need a specialised interface. For example STACK provides its own editing interface. OM questions are authored in a Java IDE, then complied to a JAR file and uploaded to the server. This is all beyond the scope of Opaque. RQP started trying to address authoring at about the point that it was abandoned.

Our approach is that Opaque should be optimised point 3 above - presenting questions to students and processing their responses. This is the point where interoperability is required. Interoperability is hard, so let us keep the scope of the protocol as small as possible. Interoperability of student data and results is a separate problem. Authoring of advanced question types is specialised to the paticular question engine, and interoperability is not possible.

All this is in the context of an online assessment system. Therefore, we are interested in questions that get presented to students in their web browser as HTML, with accompanying CSS, Javascript, Images and other media files. Student responses get sent back as form sumissions, that is HTTP POST requests. Concievably the trendy XMLHttpRequest could be used instead, but it is not really necessary. Opaque only works with form submissions, which is simple and sufficient.

==Overview of Opaque==

Opaque is a web service protocol based on SOAP.

===The parts of the system===

[[Image:OPAQUE_block_diagram.png]]

Opaque is only used at the point when questions are being presented to students, and possibly later when a student's attempt is being reviewed by a teacher.

It allows the test management system (TMS) to delegate the rendering of questions, the scoring of responses and the generation of feedback to a remote question engine.

There may be a separate question bank where question definitions are stored, or question definitions may be stored within the question engine (e.g. Stack) or the test management system may also take the role of question bank (e.g. OM).

The TMS takes full responsibility for authenticating students and controlling which questions they see. It asks an appropriate question engine to render each question. The question just had to process the questions and responses it is told to process.

Although Opaque is designed to allow interoperability between arbitrary different '''types''' of question engines and TMSs, the expectation is that any online assessment system will only use particular instances of each component.

===Identifying questions===

When the TMS asks the question engine to render a particular qustion, it needs to identify that question. This is done using three pieces of information:

;questionbank base URL
:If the question engine is designed to fetch question definitions from a remote question bank, then this is base URL for fetching questions from. For example '''<nowiki>http://example.com/questionbank</nowiki>'''.
;question id
:This is a string conatining only the characters [_a-z0-9.], and also matching the syntactic rules for Java package names. That is, it is one or more 'words' separated by '.'s, each word matching the regexp '''[_a-z][_a-z0-9]*'''. This identifies the question withing the question bank. For example '''calculus.diff01a''' or '''_12345'''.
;question version
:This is a string of the fomat major.minor, where major and minor are both decimal integers with no leading zero. Version with the same major version number should all behave roughly the same way. minor version changes are things like bug fixes. Therefore, a test definition would probably only specify which major version of the question to use, and the most recent minor version would be picked by the TMS. The full version number is needed for resuming interrupted sessions and reviewing student responsees. See [[#uestion_sessions_and_the_reproducability_guarantee|reproducability guarantee]] below. For example '''1.1''', '''1.2'''.

If the question engine fetches question definitions from a remote question bank, then it will fetch a particular question by requesting the URL ''questionbankBaseURL''/''questionId''.''questionVersion'' (e.g. '''<nowiki>http://example.com/questionbank/calculus.diff01a.1.1</nowiki>'''). This returns a single block of data (e.g. OM questions are jar files conaining XML, media files and java code). The format of this data is up to the question engine. It is the responsibility of the TMS to only ask question engines to render questions that they can understand. In the question engine supports different question formats, then it may use the mime type of the response from the question bank to decide how to process this question.

If the question engine stores its own questions, then it can ignore the questionbank base URL, and use the question id and question version to uniquely identify the requested question within its datastore.

===Data passed when a question session is started===

When a new question session is stared, certain information is passed from the test navigator to the question engine:

;randomseed: Any randomisation done be the question engine should be seeded with this number. [[#Question_sessions_and_the_reproducability_guarantee|See below for details]].
;userid: For information only, for example to improve the logging done at the question engine.
;language: Information about the current user's preferred language.
;passkey: This may be used to improve security. The value passed is md5($secret . $userID), where $secret is a shared secret string (8 characters or fewer) specific to this question engine.

The question engine may ignore any of these pieces of data.

===What can be included in rendered questions===

When a question engine is asked to render a question, it returns a response which includes the following parts:

;String XHTML
:Well-formed XML which, when inserted into the body or div of an HTML page, will result in valid HTML.
;String CSS
:any CSS needed to style the content. The TMS will cache this, and include a link to it in the head of the HTML page.
;Resource[] resources
:an array of resources which the TMS must cache and make available.

A Resource has the following fields:

;byte[] content
;String encoding
;String filename
;String mimeType

The content is what is served when the resource is requested. It is served with the specified encoding and mimeType. The filename is used in the URL the resource will be made available at. If the question requires any javascript, this should be returned as a resource.

Both the XHTML and CSS may contain placeholders of the form %%''identifier''%%. These are substituted by the TMS. Recognised placeholders include:

;%%RESOURCE%%
:Path [relative or absolute] at which resources will become available. This should not include the terminating /. Example: If a resource has the name myfile.png, then <img src="%%RESOURCES%%/myfile.png"/> should work to include that image.
;%%IDPREFIX%%
:This should be prepended to any id or class attribute in the XHTML and references to them in the CSS or Javascript. This allows multiple questions to be diplayed on one page, without conflict. [Note, this is not currently implemented since OM only ever displays one question per page. There is a problem here with javascript, since placeholders are not substituted in resources.
;%%%%
:Replaced with %%.

For perfomance reasons, whenever the TMS asks the question engine to render a question, it may send a list of resources that it already has cached for that question. The question engine may choose not to resend any of these resources.

===How the student response is represented===

The TMS receives the student's response as a HTML form submission, that is, as a HTTP POST request.

Because there might be more than one question per page, the post response to the TMS may contain many post variables. The TMS separates out the ones with the appropriate %%IDPREFIX%%, and forwards just those key->value pairs to the question engine (with idprefix removed from the key name), when it asks it to process that response.

(Another problem with multiple questions per page is that at the moment for form tag is included in the HTML returned by the question engine, whereas multiple questios per page needs a single form for all questions, provided by the TMS.)

===What sort of results can be returned===

Any request to the question engine to process some results will return HTML and resources etc. to be displayed to the student. In addition, it may (if the student has finished answering the question) include some stuctured results data to be stored in the results database. The Results structure includes:

;String actionSummary
:A log of the actions the student took in getting to their final answer. Intended for reports seen by test administrators.
;String answerLine
:A one-line summary of the student's final answer. For general reporting.
;int attempts
:Number of attempts it took the student to get their final answer.
;String questionLine
:A one line summary of the question that was asked. For reporting. Most useful when a question has different random variants.
;Map<String, int> scores
:The scores for this question. Normally the score will be a single integer, and the corresponding key will by the empty string. However, in some situations (e.g. psychological tests) the question may want to report scores against different 'axes'.

To repeat, these are stored in a database by the TMS, and used for reporting and computing overall test scores without needing to contact the question engines. It it is necessary to reconstruct exactly what the student saw on a particular question, and exactly what they entered, then this can be done using the question engine. See the next section.

===Question sessions and the reproducability guarantee===

Opaque is a stateful protocol, but in a weak sense.

In the protocol, a student's attempt on a question will require a call to the start() method, then one or more calls to the process() method. These calls may be separated by several minutes, but it could be any length of time.

Normally, what will happen is that the the start() call will return a question session id, and then each call to process() will refer to this session. This works most of the time, but on it's own would not be robust. What if a studnet stopped working on a question one day and expected to come back the next day and resume where they left off? What if the question engine server crashed?

To solve these problems, the question engine must guarantee that given a sequence of calls to start() and process() with exactly the same arguments, it will always return the same responses. The TMS should store all the information needed to recreate the sequence of calls it made on behalf of the student (that is, the question identity, the random seed, and the values submitted by students in each POST request). This means that if a question session is interrupted or lost, it can just be recreated.

Question sessions are designed to improve performance since the question definition only has to be read and processed once per session. They should also make implementing question engines easier, since state does not have to be saved in persistent storage, and most languages designed for writing server-side applications have session handling mechanisms.

If the TMS knows that a studnet has suspended thier attempt on a question (for example by navigating to a different question) it should call the stop() method to inform the question engine that that question session is no longer needed. The question enging may discard any question session that has been inactive for a period of time.

===What happens in a typical attempt on a question===

This diagram indicates a typical sequence of calls that happen when a student attempts a question.

[[Image:OPAQUE_sequence_diagram.png]]

==The Opaque data structures==

These are documented in IDL (which I don't really understand, but I am familiar with it from other docs (e.g. http://www.w3.org/TR/2000/REC-DOM-Level-2-Core-20001113/) and it is nicely language independent.)

===StartReturn===

interface StartReturn {
readonly attribute string questionSession;
readonly attribute string XHTML;
readonly attribute string CSS;
readonly attribute string progressInfo;
readonly attribute Resource[] resources;
}

===Resource===

interface Resource {
readonly attribute BLOB content;
readonly attribute string encoding;
readonly attribute string filename;
readonly attribute string mimeType;
}

===ProcessReturn===

interface ProcessReturn {
readonly attribute string XHTML;
readonly attribute string CSS;
readonly attribute string progressInfo;
readonly attribute boolean questionEnd;
readonly attribute Resource[] resources;
readonly attribute Results questionLine;
}

===Results===

interface Results {
readonly attribute string questionLine;
readonly attribute string answerLine;
readonly attribute string actionSummary;
readonly attribute integer attempts;
readonly attribute Map<string, float> scores;
readonly attribute Map<string, string> customResults;
}

==The Opaque API methods==

At the moment, I am unlikely to find time to document these in details. Sorry.

===String getEngineInfo()===

This returns some XML data as a string (byte array). The format of the response is

<pre><nowiki>
<engineinfo>
<name>[question engine]</name> ''''
<usedmemory>['123 bytes' or '45 KB' or '67 MB']</usedmemory> ''''
<activesessions>[Number of question sessions active]</activesessions> ''''
</nowiki></engineinfo>
</pre>

This method is only ever used for system monitoring - that is to ask a question engine "are you still there?". It is not essential to the functioning of the system.

The tags inside the outer <engineinfo> tag may be in any order. Other tags may also be included.

===String getQuestionMetadata(String questionID, String questionVersion, String questionBaseURL)===

This returns a snipped of XML like this:

<pre><nowiki>
<questionmetadata>
<scoring>
<marks>3</marks>
</scoring>
<plainmode>yes</plainmode>
</questionmetadata>
</nowiki></pre>

The scoring section is basically giving the maximum score for the question, but because a question can report scores against different 'axes', this may be more complicated.

plain mode is a feature of the OpenMark system designed to enhance accessibility. Since OpenMark use a lot of JavaScript and CSS which may not be accessible to some people, then every question should provide an alternative rendering using just plain HTML and standard from controls. When a student chooses to take a quiz in plain mode, they see the plain mode version of each question. However, not all questions provide a plain-mode version, in which case they should return ,plainmode>no</plainmode>.

===StartReturn start(String questionID, String questionVersion, String questionBaseURL, Map<String, String> initialParams, String[])===

===ProcessReturn process(String questionSession, Map<String, String> response)===

===stop(String questionSession)===

==Opaque WSDL==

<pre><nowiki>
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://om.open.ac.uk/" xmlns:apachesoap="http://xml.apache.org/xml-soap" xmlns:impl="http://om.open.ac.uk/" xmlns:intf="http://om.open.ac.uk/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<wsdl:types>
<schema targetNamespace="http://om.open.ac.uk/" xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://schemas.xmlsoap.org/soap/encoding/"/>

<complexType name="ArrayOf_soapenc_string">
<complexContent>
<restriction base="soapenc:Array">
<attribute ref="soapenc:arrayType" wsdl:arrayType="soapenc:string[]"/>
</restriction>
</complexContent>
</complexType>

<complexType name="Resource">
<sequence>
<element name="content" nillable="true" type="soapenc:base64Binary"/>
<element name="encoding" nillable="true" type="soapenc:string"/>
<element name="filename" nillable="true" type="soapenc:string"/>
<element name="mimeType" nillable="true" type="soapenc:string"/>
</sequence>
</complexType>

<complexType name="ArrayOfResource">
<complexContent>
<restriction base="soapenc:Array">
<attribute ref="soapenc:arrayType" wsdl:arrayType="impl:Resource[]"/>
</restriction>
</complexContent>
</complexType>

<complexType name="StartReturn">
<sequence>
<element name="CSS" nillable="true" type="soapenc:string"/>
<element name="XHTML" nillable="true" type="soapenc:string"/>
<element name="progressInfo" nillable="true" type="soapenc:string"/>
<element name="questionSession" nillable="true" type="soapenc:string"/>
<element name="resources" nillable="true" type="impl:ArrayOfResource"/>
</sequence>
</complexType>

<complexType name="OmException">
<sequence/>
</complexType>

<complexType name="CustomResult">
<sequence>
<element name="name" nillable="true" type="soapenc:string"/>
<element name="value" nillable="true" type="soapenc:string"/>
</sequence>
</complexType>

<complexType name="ArrayOfCustomResult">
<complexContent>
<restriction base="soapenc:Array">
<attribute ref="soapenc:arrayType" wsdl:arrayType="impl:CustomResult[]"/>
</restriction>
</complexContent>
</complexType>

<complexType name="Score">
<sequence>
<element name="axis" nillable="true" type="soapenc:string"/>
<element name="marks" type="xsd:int"/>
</sequence>
</complexType>

<complexType name="ArrayOfScore">
<complexContent>
<restriction base="soapenc:Array">
<attribute ref="soapenc:arrayType" wsdl:arrayType="impl:Score[]"/>
</restriction>
</complexContent>
</complexType>

<complexType name="Results">
<sequence>
<element name="actionSummary" nillable="true" type="soapenc:string"/>
<element name="answerLine" nillable="true" type="soapenc:string"/>
<element name="attempts" type="xsd:int"/>
<element name="customResults" nillable="true" type="impl:ArrayOfCustomResult"/>
<element name="questionLine" nillable="true" type="soapenc:string"/>
<element name="scores" nillable="true" type="impl:ArrayOfScore"/>
</sequence>
</complexType>

<complexType name="ProcessReturn">
<sequence>
<element name="CSS" nillable="true" type="soapenc:string"/>
<element name="XHTML" nillable="true" type="soapenc:string"/>
<element name="progressInfo" nillable="true" type="soapenc:string"/>
<element name="questionEnd" type="xsd:boolean"/>
<element name="resources" nillable="true" type="impl:ArrayOfResource"/>
<element name="results" nillable="true" type="impl:Results"/>
</sequence>
</complexType>

</schema>

</wsdl:types>

<wsdl:message name="getEngineInfoResponse">
<wsdl:part name="getEngineInfoReturn" type="soapenc:string"/>
</wsdl:message>

<wsdl:message name="processRequest">
<wsdl:part name="questionSession" type="soapenc:string"/>
<wsdl:part name="names" type="impl:ArrayOf_soapenc_string"/>
<wsdl:part name="values" type="impl:ArrayOf_soapenc_string"/>
</wsdl:message>

<wsdl:message name="getEngineInfoRequest">
</wsdl:message>

<wsdl:message name="getQuestionMetadataResponse">
<wsdl:part name="getQuestionMetadataReturn" type="soapenc:string"/>
</wsdl:message>

<wsdl:message name="processResponse">
<wsdl:part name="processReturn" type="impl:ProcessReturn"/>
</wsdl:message>

<wsdl:message name="stopResponse">
</wsdl:message>

<wsdl:message name="OmException">
<wsdl:part name="fault" type="impl:OmException"/>
</wsdl:message>

<wsdl:message name="startResponse">
<wsdl:part name="startReturn" type="impl:StartReturn"/>
</wsdl:message>

<wsdl:message name="stopRequest">
<wsdl:part name="questionSession" type="soapenc:string"/>
</wsdl:message>

<wsdl:message name="getQuestionMetadataRequest">
<wsdl:part name="questionID" type="soapenc:string"/>
<wsdl:part name="questionVersion" type="soapenc:string"/>
<wsdl:part name="questionBaseURL" type="soapenc:string"/>
</wsdl:message>

<wsdl:message name="startRequest">
<wsdl:part name="questionID" type="soapenc:string"/>
<wsdl:part name="questionVersion" type="soapenc:string"/>
<wsdl:part name="questionBaseURL" type="soapenc:string"/>
<wsdl:part name="initialParamNames" type="impl:ArrayOf_soapenc_string"/>
<wsdl:part name="initialParamValues" type="impl:ArrayOf_soapenc_string"/>
<wsdl:part name="cachedResources" type="impl:ArrayOf_soapenc_string"/>
</wsdl:message>

<wsdl:portType name="OmService">

<wsdl:operation name="start" parameterOrder="questionID questionVersion questionBaseURL initialParamNames initialParamValues cachedResources">
<wsdl:input message="impl:startRequest" name="startRequest"/>
<wsdl:output message="impl:startResponse" name="startResponse"/>
<wsdl:fault message="impl:OmException" name="OmException"/>
</wsdl:operation>

<wsdl:operation name="stop" parameterOrder="questionSession">
<wsdl:input message="impl:stopRequest" name="stopRequest"/>
<wsdl:output message="impl:stopResponse" name="stopResponse"/>
<wsdl:fault message="impl:OmException" name="OmException"/>
</wsdl:operation>

<wsdl:operation name="process" parameterOrder="questionSession names values">
<wsdl:input message="impl:processRequest" name="processRequest"/>
<wsdl:output message="impl:processResponse" name="processResponse"/>
<wsdl:fault message="impl:OmException" name="OmException"/>
</wsdl:operation>

<wsdl:operation name="getEngineInfo">
<wsdl:input message="impl:getEngineInfoRequest" name="getEngineInfoRequest"/>
<wsdl:output message="impl:getEngineInfoResponse" name="getEngineInfoResponse"/>
</wsdl:operation>

<wsdl:operation name="getQuestionMetadata" parameterOrder="questionID questionVersion questionBaseURL">
<wsdl:input message="impl:getQuestionMetadataRequest" name="getQuestionMetadataRequest"/>
<wsdl:output message="impl:getQuestionMetadataResponse" name="getQuestionMetadataResponse"/>
<wsdl:fault message="impl:OmException" name="OmException"/>
</wsdl:operation>

</wsdl:portType>

<wsdl:binding name="OmSoapBinding" type="impl:OmService">
<wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>

<wsdl:operation name="start">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="startRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:input>
<wsdl:output name="startResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:output>

<wsdl:fault name="OmException">
<wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="OmException" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:fault>
</wsdl:operation>

<wsdl:operation name="stop">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="stopRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:input>

<wsdl:output name="stopResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:output>
<wsdl:fault name="OmException">
<wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="OmException" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:fault>
</wsdl:operation>

<wsdl:operation name="process">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="processRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:input>
<wsdl:output name="processResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:output>
<wsdl:fault name="OmException">
<wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="OmException" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:fault>
</wsdl:operation>

<wsdl:operation name="getEngineInfo">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="getEngineInfoRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:input>
<wsdl:output name="getEngineInfoResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:output>
</wsdl:operation>

<wsdl:operation name="getQuestionMetadata">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="getQuestionMetadataRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:input>
<wsdl:output name="getQuestionMetadataResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:output>
<wsdl:fault name="OmException">
<wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="OmException" namespace="http://om.open.ac.uk/" use="encoded"/>
</wsdl:fault>
</wsdl:operation>

</wsdl:binding>

<wsdl:service name="OmServiceService">
<wsdl:port binding="impl:OmSoapBinding" name="Om">
<wsdlsoap:address location="http://kestrel.open.ac.uk/om-qe/services/Om"/>
</wsdl:port>
</wsdl:service>

</wsdl:definitions>
</nowiki></pre>

-----

See also:
* [[Opaque question type]] for instructions on installing and using the Moodle Opaque question type.

[[Category:Open protocol ]]

Talk:Tracking Moodle CVS with git

2009-09-19T16:10:21Z

Jeffjeff: /* Anyone got any experience with NetBeans and GIT? */ new section

Numerical question units and intervals

2009-09-18T11:00:16Z

Jeffjeff: /* Allow fraction type 2 1/2 as 2.5 */ link to wikipedia about intervals

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours. (o) or (_) is a radio button.

==Allow evaluation of expressions==

Checking this would accept
'''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.

[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)

To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.

Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.

Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.

(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))

[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.

[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)

But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2

These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.

[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will
usually want to answer the first question as his/her first option
(tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up at NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys
for which targets. People that are really into this kind of thing can
use "Mouseless Browsing" extension for FireFox)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpreted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]

Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that that boundary not included
in the [http://en.wikipedia.org/wiki/Interval_%28mathematics%29 interval]. The .. notation for an interval is already used in the GIFT format.

Another useful possibility: that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==

===How the question has treated use of units===

The actual philosophy of units in the question at present (2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options==

differentiated unit feedback, which is all of:
* If there is no SPACE between the number and unit "There should always be a space between the number and the unit, except for degree sign ° and other planal angle units."
* translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
* feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
* feedback for "possibly right number" if unit is adjusted
* feedback for "Unit not recognized, either misspelled, wrong type, or just not expected"

(o) missed unit gives standard retry reduction of points
(_)no reduction for missed unit
(_)this reduction [___]

[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions] (#17 I don't understand. Otherwise a good collection of rules, at least about SI units. #16 is a rule that could be good to be able to choose to enforce.)

===Unit: [ ] (optional)===

Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"

Alternative Units:
Multiplier:[ ] Unit: [_______]

There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==

''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]

alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented with similar code to number 4 below.

2) Hints option: option to include a click-able link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) An easy reordering tool for alternative answers. It might be possible to have two models:
*Automatic based on grade for alternative. Because the question engine processes them in order it seems logical that alternative answer patterns that have higher points should be tested before others.
*BASIC numbering 10 20 30 40 so if I wanted 40 moved to between 20 and 30 I could change it to 25

5) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Numerical question units and intervals

2009-09-18T10:42:53Z

Jeffjeff: /* Allow fraction type 2 1/2 as 2.5 */ point out .. for intervals in GIFT

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours. (o) or (_) is a radio button.

==Allow evaluation of expressions==

Checking this would accept
'''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.

[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)

To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.

Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.

Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.

(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))

[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.

[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)

But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2

These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.

[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will
usually want to answer the first question as his/her first option
(tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up at NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys
for which targets. People that are really into this kind of thing can
use "Mouseless Browsing" extension for FireFox)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpreted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]

Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that that boundary not included
in the interval. The .. notation for an interval is already used in the GIFT format.

Another useful possibility: that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==

===How the question has treated use of units===

The actual philosophy of units in the question at present (2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options==

differentiated unit feedback, which is all of:
* If there is no SPACE between the number and unit "There should always be a space between the number and the unit, except for degree sign ° and other planal angle units."
* translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
* feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
* feedback for "possibly right number" if unit is adjusted
* feedback for "Unit not recognized, either misspelled, wrong type, or just not expected"

(o) missed unit gives standard retry reduction of points
(_)no reduction for missed unit
(_)this reduction [___]

[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions] (#17 I don't understand. Otherwise a good collection of rules, at least about SI units. #16 is a rule that could be good to be able to choose to enforce.)

===Unit: [ ] (optional)===

Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"

Alternative Units:
Multiplier:[ ] Unit: [_______]

There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==

''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]

alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented with similar code to number 4 below.

2) Hints option: option to include a click-able link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) An easy reordering tool for alternative answers. It might be possible to have two models:
*Automatic based on grade for alternative. Because the question engine processes them in order it seems logical that alternative answer patterns that have higher points should be tested before others.
*BASIC numbering 10 20 30 40 so if I wanted 40 moved to between 20 and 30 I could change it to 25

5) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Numerical question units and intervals

2009-09-18T10:37:37Z

Jeffjeff: /* options for all [maybe?] questiontypes (including numerical): */ easy reordering of alternative answers

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours. (o) or (_) is a radio button.

==Allow evaluation of expressions==

Checking this would accept
'''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.

[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)

To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.

Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.

Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.

(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))

[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.

[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)

But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2

These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.

[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will
usually want to answer the first question as his/her first option
(tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up at NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys
for which targets. People that are really into this kind of thing can
use "Mouseless Browsing" extension for FireFox)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpreted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]

Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that boundary not included
in the intervall or that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==

===How the question has treated use of units===

The actual philosophy of units in the question at present (2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options==

differentiated unit feedback, which is all of:
* If there is no SPACE between the number and unit "There should always be a space between the number and the unit, except for degree sign ° and other planal angle units."
* translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
* feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
* feedback for "possibly right number" if unit is adjusted
* feedback for "Unit not recognized, either misspelled, wrong type, or just not expected"

(o) missed unit gives standard retry reduction of points
(_)no reduction for missed unit
(_)this reduction [___]

[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions] (#17 I don't understand. Otherwise a good collection of rules, at least about SI units. #16 is a rule that could be good to be able to choose to enforce.)

===Unit: [ ] (optional)===

Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"

Alternative Units:
Multiplier:[ ] Unit: [_______]

There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==

''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]

alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented with similar code to number 4 below.

2) Hints option: option to include a click-able link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) An easy reordering tool for alternative answers. It might be possible to have two models:
*Automatic based on grade for alternative. Because the question engine processes them in order it seems logical that alternative answer patterns that have higher points should be tested before others.
*BASIC numbering 10 20 30 40 so if I wanted 40 moved to between 20 and 30 I could change it to 25

5) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Numerical question units and intervals

2009-09-11T07:27:34Z

Jeffjeff: /* Allow fraction type 2 1/2 as 2.5 */ shortened some lines, spelling correction, mouseless browsing

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours.

==Allow evaluation of expressions==

Checking this would accept '''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.
[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)
To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.
Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.
Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.
(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))
[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.
[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)
But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2
These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.
[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will
usually want to answer the first question as his/her first option
(tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up at NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys
for which targets. People that are really into this kind of thing can
use "Mouseless Browsing" extension for FireFox)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpreted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]
Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that boundary not included
in the intervall or that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==
===How the question has treated use of units===
The actual philosophy of units in the question at present (-2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options===
differentiated unit feedback, which is all of:
If there is no SPACE between the number and unit "There should always be a
space between the number and the unit, except for degree sign ° and
other planal angle units."
translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
feedback for "possibly right number" if unit is adjusted
feedback for "Unit not recognized, either misspelled, wrong type,
or just not expected"
(o) missed unit gives standard retry reduction of points (_)no reduction
for missed unit (_)this reduction [___]
[[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions]] (#17 I don't
understand. Otherwise a good collection of rules, at least about SI units.
#16 is a rule that could be good to be able to choose to enforce.)

===Unit: [ ] (optional)===
Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"
Alternative Units:
Multiplier:[ ] Unit: [_______]
There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==
''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]
alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented witn similar code to number 4 below.

2) Hints option: option to include a clickable link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Numerical question units and intervals

2009-09-09T16:00:05Z

Jeffjeff: /* Unit feedback options= */ small spelling misstakes-checked the link

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours.

==Allow evaluation of expressions==

Checking this would accept '''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.
[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)
To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.
Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.
Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.
(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))
[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.
[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)
But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2
These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.
[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will usually want to
answer the first question as his/her first option (tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up att NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys for which targets)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpretted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]
Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that boundary not included in intervall
or that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==
===How the question has treated use of units===
The actual philosophy of units in the question at present (-2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options===
differentiated unit feedback, which is all of:
If there is no SPACE between the number and unit "There should always be a
space between the number and the unit, except for degree sign ° and
other planal angle units."
translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
feedback for "possibly right number" if unit is adjusted
feedback for "Unit not recognized, either misspelled, wrong type,
or just not expected"
(o) missed unit gives standard retry reduction of points (_)no reduction
for missed unit (_)this reduction [___]
[[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions]] (#17 I don't
understand. Otherwise a good collection of rules, at least about SI units.
#16 is a rule that could be good to be able to choose to enforce.)

===Unit: [ ] (optional)===
Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"
Alternative Units:
Multiplier:[ ] Unit: [_______]
There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==
''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]
alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented witn similar code to number 4 below.

2) Hints option: option to include a clickable link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Numerical question units and intervals

2009-08-24T06:21:58Z

Jeffjeff: /* Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical): */ changed the link to an example feedback form (the old one no lon

This is my proposal for how the numerical question function and interface should be improved. [_] means checkboxes that teachers could use for specifying different question behaviours.

==Allow evaluation of expressions==

Checking this would accept '''5/2 0.5*5 5^2/10 5^(5-3)/10 25*10^-1 sqrt(25)/2'''

The main value of calculated or numerical question is that the student really does the
calculation and gives a real number (which means in decimal format).
I suggested that you use the short answer (or the multichoice) question type with the different
answer forms that you allow.
[[User:Pierre Pichet|Pierre Pichet]] 04:25, 6 September 2007 (CDT)
To me the real value of the numerical question type is it's ability to really see the
answers as numbers and be able to see what things are equivalent. I have for many years
used a PERL script which could treat 3/2 and 1+1/2 and 1.5 and (1,5 for us Swedes) as
the same thing, and especially to accept well defined intervals.
Often I am more interested in students knowing HOW to create the right expression moreso
than doing the last step, evaluating it. It is also interesting when looking at the
statistics for a question, to see how they "thought" which can aid the teacher when
adding new feedback for other answers.
Real numbers include rational numbers include 1/3 which cannot be correctly represented
as a decimal. (Which doesn't bother me as a physicist, but a pure matematician ought to
value being able to use expressions like 1/3.
(Even if a programmer may say it'll only work from 0.33333333333333333333333333333 to
0.333333333333333333333334 :-))
[[User:Jeff Forssell|Jeff Forssell]] 06:21, 6 September 2007 (CDT)

If your answers are necessarily expressed as rational numbers then the short answer type
is the way to do it. You could use as many answers as necessary to express the various
formats that you accept as correct.
[[User:Pierre Pichet|Pierre Pichet]] 07:21, 6 September 2007 (CDT)
But that would mean that the teacher looses the possibility to give feedback for numerical
intervals. Having questions that are as open as possible for different kinds of answers
while still being able to give meaningful feedback is one of the strengths of Moodle that
I like and would like to increase![[User:Jeff Forssell|Jeff Forssell]] 07:01, 4 October 2007 (CDT)

This is not the case if you use as many answers as you want to express the desired feedback.
Just defined the other answers in the range that you think reflect a given problem.
i.e. if a factor of 2 reflect a given error use this value as an answer and write the feedback text.
[[User:Pierre Pichet|Pierre Pichet]] 15:40, 9 October 2008 (CDT)

5/2 0.5*5 5^2/2 5^(5-3)/2 25*10^-1 sqrt(25)/2
These value are not the same 5^2/2 and 5^(5-3)/2 are equal to 12.5!
I think it is very difficult to define which expressions are to be accepted. The numerical answer should somehow be the result. Otherweise the answer to a question like sqrt(50-2*17)=? could be the question itself.
[[User: Stefan Büchler|Stefan Büchler]] 25. Jan. 08

I have for many years used a Perl script that worked very well with numerical questions. It had the ability to evaluate expressions also and compare the value with the intervals that gave various feedback. Usually I felt that it was a good possibility. A lot of times in physics the main problem is figuring out HOW to get the result and the the actual computation can be, easy though time consuming. (Also good if they weren't getting the right answer, I could see HOW they were figuring.) But sometimes I wanted to be able to turn it off, just because of the possibility of "the answer being the question". That's why I suggest that this possibility should not be the default, but something that a teacher can chose If s/he wants it. (thanks for the pointing out the unequal expressions, which I have {hopefully} corrected. [[User:Jeff Forssell|Jeff Forssell]] 08:26, 22 September 2008 (CDT)

==Allow fraction type 2 1/2 as 2.5==

Translate "½" as "+.5" but give warning "Avoid ½ character in numerical expressions! We have translated ½ as +.5 to evaluate your answer."

In the student question interface the answer box should:
1) have an easy way to access with keyboard
a) good "tab order": when opening a quiz question page the student will usually want to
answer the first question as his/her first option (tab order =10)
Assuming there are no other taborders set this would be first.
"BASIC" numbering allows later refinements without renumbering.
if available "send in question" (tab order =15)
answer next question (tab order= 20)
if available "send in question" (tab order =25)
(how this looks would depend on how many question blanks are on a page
and would end up att NEXT page link)
b) Acesskeys (This is mined territory since there are no real standards (or
rather several DIFFERENT standards) It would probably be best to let
people chose whether they want this activated and maybe which keys for which targets)
2) Have a (?) after the answer box linked to a popup explanation, which depends
on which options were checked by question author, saying for example:
This is for a numerical answer. The answer can contain a unit at the right
end after a space. As decimal character you may use . or , . Mixed fractions
can be written as 2 1/2 which is interpretted as 2+1/2 (2½) In this question
you may enter an expression which is evaluated to a number before comparing to
the correction key. [link to examples of allowable expressions]

Answer 1:[ 2.5 ] Accepted error ± [0.5] Grade: [100%]
Feedback:

Answer 2: [ '''100.4..100.5)''' ] Accepted error ± [_empty_] Grade: [90%]
Which would be interpreted '''110.4 <= right < 100.5'''
i.e. the round parentheses would mean that boundary not included in intervall
or that one could enter evaluation strings like
'''110.4<=ans<100.5||100.5<ans<=100.6'''
''(I know that this particular thing can be addressed by having
100.5 +/-0 first than 100.5 +/-0.1)''
Feedback:

Answer 3: ...
Feedback: (for answer 3)

Show 3 more alternative answer fields

==Units mandatory or not==
===How the question has treated use of units===
The actual philosophy of units in the question at present (-2008) is that the units are used to allow the student to answer the question in another unit than the one prescribed or implied by the question.
However this unit must have been specified when creating the question.

This is why you set a multiplier to convert from your implied or expressed unit with a default 1 multiplier.
For example, if you enter a unit of 'cm' here, and the accepted answer is 15, then the answers '15cm' and '15' are both accepted as correct. You can also specify a multiplier. So, if your main answer was 5500 with unit W, you can also add the unit kW with a multiplier of 0.001. This means that the answers '5500', '5500W' or '5.5kW' would all be marked correct. Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.
For the response where the multiplier is 1 the unit is not mandatory.i.e.'5500', '5500W'
However when the multiplier is not 1 then the units is mandatory. i.e. '5.5kW'

===How most science/math teachers expect students to answer regarding units===
Usually a numerical answer without a unit is not considered complete. (At least in Sweden and when I grew up in the states 1960s) If one has an automatic correction of numerical values only, one must sacrifice the openness of the question by stipulating units.

What Moodle's questions have often provided is the possibility of allowing quite open responses from the students and still it can give meaningful feedback and points. If we must say "your answer must be expressed in kilonewtons" we won't be able to discover the weaknesses that might have lead to the student thinking Joules. Knowing what unit is involved it an important part of the ability we try to teach the students and we must be able to evaluate it. That is not possible with the question's present way of functioning.

A student can answer "5.2" and gets full points without writing any unit whereas a student answering "52 millimeters" would get zero if no multiplier was stipulated OR the unit only expressed as "mm" or "millimeter". The student with the 5.2 answer might have even been thinking "m" rather than the the teacher's expected "cm". This would not be the case if it were a "paper" test/assignment.

==Unit feedback options===
differentiated unit feedback, which is all of:
If there is no SPACE between the number and unit "There should always be a
space between the number and the unit, except for degree sign ° and
other planal angle units."
translate dimensionless "units": % = *0.01; ppm=*1E-6 ppb. promille
feedback for lacking unit [_]Feedback for accepted UNIT but wrong number
feedback for "possibly right number" if unit is adjusted
feedback for "Unit not recognized, either misspelled, wrong type,
or just not expected"
(o) missed unit gives standard retry reduction of points (_)no reduction
for missed unit (_)this reduction [___]
[[http://physics.nist.gov/cuu/Units/checklist.html|SI Unit rules and style conventions]] (#17 I don't
understand. Otherwise a good colection of rules, at least about SI units.
#16 is a rule that could be good to be abe to choose to enforce.)

===Unit: [ ] (optional)===
Should accept arrays like '''kg, kilogram, kilograms , kilogramme, {"kilo gram", keelograhm} '''
where the ones enclosed in {} would be given feedback like
''We have interpretted '''keelograhm''' as a misspelling av '''kg''' Be aware that small spelling
mistakes like '''M''' in stead of '''m''' can lead to huge errors M=Mega=million
m=milli= one thousandth''"
Alternative Units:
Multiplier:[ ] Unit: [_______]
There should be the same possibility of unit arrays here.

Allow prefix units ($,£ maybe there are others )

==The Multiplier should have an explanation (?)==
''If the student answers with this unit, the student answer will be multiplied with this number
before being compared with the expected answer. Example:
Answer [_10__] Unit [cm]
alternative unit
multiplier [_2.54_] unit [", in, in., inch]
(That " could be a tricky unit to handle!)

==Then there are some things that I would be interested in having as options for all [maybe?] questiontypes (including numerical):==

1) "Challenge" That a student is given (perhaps with a linked "!" or "Hmm.." or button) a chance to give his/her preferred answer and his/her motivation (or why they didn't understand, like, agree with the grade or feedback they got for their answer or how the question was phrased.) This might be implemented witn similar code to number 4 below.

2) Hints option: option to include a clickable link that could display a/some hint(s), if the student didn't know how to attack a question. (It is possible, though hardly for html-illiterate, to use the OverLib.js which is included in Moodle to do this even now.

3) Certainty (in the past sometimes called "confidence") declaration option
[__answer___] certainty: Hi(o)|Med( )|Low( )
if possible with CBM/CBG grading scale including negative

4) All Internet pages (that the maintainer cares about) should have a [[Feedback link for all elements|feedback link]] something like this:

[[http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm Completely satisfied with this page?]]

Talk:Checkbox

2009-08-04T09:24:29Z

Jeffjeff: Checkbox label AFTER

== Checkbox label ==

why AFTER (which is bolded in the text):
"Create a checkbox with an id attribute. Add a <label for="id_of_checkbox"> element '''after''' the checkbox. As the text content of the label element, write the result of checking the box: "

I seem to recall having placed a label so it includes both the checkbox and the text belonging to it. I have sometimes used that to put in a tooltip with a title="when you put a check here, life will be wonderful ;-)" in the label tag and it would be good if that came up while hoovering over the checkbox itself. Isn't it also possible to make the checkbox "mouseclickable" even on the text which belongs to the checkbox? [[User:Jeff Forssell|Jeff Forssell]] 09:24, 4 August 2009 (UTC)

Talk:Setting up Netbeans

2009-07-10T09:58:39Z

Jeffjeff:

Talk:Setting up Netbeans

2009-07-10T08:55:52Z

Jeffjeff:

Setting up Netbeans

2009-07-10T07:48:02Z

Jeffjeff: /* CVS with NetBeans */

[http://www.netbeans.org/features/php/index.html NetBeans] has got a good PHP support. You find a host of information on the website (tutorials, developer blog, screen casts, etc.).

== Features ==
* CVS integration: see all changes, lines deletion, diff in real time, show annotations, diff history...
* Ctrl+Click: Go to declaration
* Export/Import Diff Patch
* Easy navigation
* List of functions
* Code completion
* Instant rename
* HTML, CSS, JavaScript support
* MySQL manager
* Quick Search
* Very few bugs

== Installation ==

* I recommend to download the latest [http://www.netbeans.org/downloads/index.html 6.5 version with PHP support ]. It's only a 26 MB download.
* Install and run it.

== Set up for Moodle development ==

* Checkout your Moodle project with a '''CVS client''' - see [[CVS for Administrators]] or [[CVS for developers|CVS for Developers]]. I personally prefer to use an external client in order to realise the CVS Delete operation on my own.

* Open File > New Project > PHP > PHP Application > Next
: You're going to set the project now. ''Name'', ''Location'' and ''Folder'' are used by NetBeans and are not related to the source code. So you can choose whatever you like, except your source folder. ''Sources'' has to be your checked out Moodle branch/head folder. The rest is clear enough. Don't forget to choose UTF-8 for ''Default Encoding''.

: Example:

Project Name: Moodle 1.9 Stable
Project Location: C:\Users\jerome\Documents\NetBeansProjects
Project Folder: C:\Users\jerome\Documents\NetBeansProjects\Moodle 1.9 Stable
Project Sources: C:\Users\jerome\Projects\branch19_STABLE\moodle
Project URL: http://localhost/moodle19/
Index File: index.php
Create: unchecked
Default Encoding: UTF-8
Set as Main Project: unchecked

* Click on Finish.

* Start coding!

== CVS with NetBeans ==

NetBeans comes with '''integrated CVS support''' which might be the easiest way to check out Moodle.

* In NetBeans, select Window->Versioning->CVS->Checkout
* Select Team->CVS->Checkout
* Enter into CVS Root: '':pserver:anonymous@us.cvs.moodle.org:/cvsroot/moodle''
: (Non-US-residents might use one of the other [https://docs.moodle.org/en/CVS_for_Administrators#CVS_Servers Moodle CVS servers] nearer to them.)
* Click ''Next''
* Browse or enter into ''Module:'' moodle
* Browse or enter into ''Branch:'' MOODLE_19_STABLE
* Browse or enter into ''Local Folder:'' C:\xampp\htdocs
* Click ''Finish'' (and wait a few minutes for Moodle to be checked out)
* When you get the dialog box "Do you want to create an IDE project from the checked-out sources?", Click "Create Project..."
* Select PHP Application with Existing Sources, and click Next
* Browse or enter into ''Sources Folder'' C:\xampp\htdocs\moodle
* Enter into ''Project Name:'' moodle
* Keep the other defaults and click next
* ''Run As:'' should have selected ''Local Web Site (running on local web server)''
* Enter into Project URL http://localhost/moodle/
* Browse or enter into ''Index File:'' index.php
* Click ''Finish''

== Optimization ==

1. You could want to run Netbeans with the Sun JDK. Netbeans seems to work a bit better with the Sun JDK. Download it from there: [http://java.sun.com/javase/downloads/index.jsp]. You'll have to edit Netbeans config file. Open netbeans/etc/netbeans.conf. Then uncomment and edit:

netbeans_jdkhome="your_JDK_path"

2. To change the Netbeans look and feel run netbeans in command line with this parameter:
"netbeans" --laf javax.swing.plaf.metal.MetalLookAndFeel

3. If Netbeans starts to slow down, give it more memory
"netbeans" -J-Xmx600m

== See also: ==
* Moodle forum: [http://moodle.org/mod/forum/discuss.php?d=112972 NetBeans 6.5 for moodle/PHP/debugging is a better experience v.s Eclipse]
* [http://www.netbeans.org/kb/trails/php.html NetBeans PHP Learning Trail]
* [http://wiki.netbeans.org/PHP NetBeans PHP Wiki]
* [http://blogs.sun.com/netbeansphp/ NetBeans for PHP Blog]

[[Category:NetBeans]]
[[Category:Developer tools|NetBeans]]

Talk:Setting up Netbeans

2009-07-09T15:05:06Z

Jeffjeff: /* CVS instructions */

Talk:Setting up Netbeans

2009-07-09T15:04:27Z

Jeffjeff: /* CVS instructions */

Talk:Setting up Netbeans

2009-07-09T13:02:12Z

Jeffjeff: /* CVS instructions */ including xamp and 2 Moodles?

Talk:Places to search for lang strings

2009-07-09T09:56:58Z

Jeffjeff:

Not sure where to put this, but here goes. Often a Moodle user will like to adjust a particular sting the he sees when using Moodle. It is often a problem to find WHICH PHP file is the one that it is associated with. I think I have even tried doing a search in a local moodle install to try to find a string and I don't think that worked as well as I had hoped. The best thing would be a search option built into Moodle. Could a browser plugin like firebug or webdeveloper be of any assistance?

Also the link to phpdocs at the bottom of the page doesn't work [[User:Jeff Forssell|Jeff Forssell]] 08:55, 8 July 2009 (UTC)

: You might like MDL-19467 - implemented in HEAD.--[[User:Tim Hunt|Tim Hunt]] 13:58, 8 July 2009 (UTC)

That sounds good. But it is only for 2.0? Is there any sandbox to try it in? Or just by making a local 2.0 beta install?
I have spent an hour now trying to find what HEAD really means. I have a vague contextual feeling from many Using Moodle posts, but I would like to understand it. Could you put in something here: https://docs.moodle.org/en/index.php?title=HEAD&action=edit&redlink=1 [[User:Jeff Forssell|Jeff Forssell]] 09:31, 9 July 2009 (UTC)

Talk:Places to search for lang strings

2009-07-09T09:31:15Z

Jeffjeff:

Not sure where to put this, but here goes. Often a Moodle user will like to adjust a particular sting the he sees when using Moodle. It is often a problem to find WHICH PHP file is the one that it is associated with. I think I have even tried doing a search in a local moodle install to try to find a string and I don't think that worked as well as I had hoped. The best thing would be a search option built into Moodle. Could a browser plugin like firebug or webdeveloper be of any assistance?

Also the link to phpdocs at the bottom of the page doesn't work [[User:Jeff Forssell|Jeff Forssell]] 08:55, 8 July 2009 (UTC)

: You might like MDL-19467 - implemented in HEAD.--[[User:Tim Hunt|Tim Hunt]] 13:58, 8 July 2009 (UTC)

That sounds good. But it is only for 2.0? Is there any sandbox to try it in? Or just by making a local 2.0 beta install?
I have spent an hour now trying to find what HEAD really means. I have a vagues contextual feeling from many Usin Moodle posts, but i would like to understand it. Could you puit in something here: https://docs.moodle.org/en/index.php?title=HEAD&action=edit&redlink=1 [[User:Jeff Forssell|Jeff Forssell]] 09:31, 9 July 2009 (UTC)

Developer documentation

2009-07-09T08:27:41Z

Jeffjeff: /* Resources */ wish for an explanation of what HEAD is

This [[:Category:Developer|Developer]] section of Moodle Docs is aimed at developers who contribute to the Moodle code, plugins, themes, and so on.
* If you manage a Moodle site, [[Administrator documentation]] may suit your needs better.
* If you teach using Moodle, try [[Teacher documentation]].

'''Note:''' New developer documentation pages should be added to the ''Development namespace'' by typing <code>Development:</code> before the new page name i.e. <code><nowiki>[[New page name]]</nowiki></code>. If you are a developer, you probably want to change your [[Special:Preferences|preferences]] to include the Development namespace in searches. 

A page may be added to the Developer category by adding the template <code><nowiki>{{CategoryDeveloper}}</nowiki></code> to the bottom of the page. - If required, you can use <code><nowiki>[[Category:Sort key]]</nowiki></code> to provide a sort key other than the default page name.

==How Moodle development works==

The [[Overview|overview of the Moodle development process]] explains how Moodle development occurs and how people become Moodle developers. Current plans are listed on the [[Roadmap]].

You can also enrol in one of the [http://dev.moodle.org Moodle Developer Courses].

==Guidelines==

The following guidelines are crucial reading for anyone wanting to contribute to the Moodle code base:
*[[Coding|Coding guidelines]] have to be followed by all Moodle developers
*[[Moodle design goals]] spells out the basic design goals behind Moodle
*[[Interface guidelines]] aim to provide a common feel to the Moodle user interface
*[[CVS (developer)|Moodle CVS for developers]] explains how to work with the Moodle code in CVS
*[[Tracker]] explains the Moodle Tracker for keeping track of bugs, issues, feature requests etc
*[[Working with the Community|Working with the Community]] explains how to engage with the dev community and discuss changes
*[[Unit tests|Unit tests]] explains how to run the unit tests, and how to write new test cases.
*[[Fast portable SQL]] shows SQL techniques that are fast, efficient, and known to work on all supported DBs.

==Documentation for core components==

This section is for documentation of specific components of the existing core Moodle code. Discussion of components that are under discussion or in development can be found in the [[Developer notes|developer notes]] or on the [[Roadmap|roadmap]].

The documents below give a general overview. For detailed function-by-function documentation, see the [http://phpdocs.moodle.org/ phpDocumentor] documentation that is automatically generated from the comments in the code.

And don't forget that the most up-to-date and detailed description of how the code works is the code itself, and you can [http://xref.moodle.org/nav.html?index.html browse the code online] using [[PHPXref|PHPXref]].

===Core components that affect everything===

*[[Database schema introduction|The database schema]]
*[[What happens when you require config.php|What happens when you require config.php]]
*lib/moodlelib.php
*[[lib/weblib.php|lib/weblib.php]] for outputting stuff
*[[JavaScript_functions|JavaScript function available on the client side]]
*[[XMLDB_Documentation|Database abstraction layer]] @ v[[1.7]]
*[[Roles|Roles and Capabilities system]] @ v[[1.7]] for controlling who can do what
*[[lib/formslib.php|Forms library]] @ v[[1.8]] for creating accessible and secure HTML forms that let users edit things
*[[Using_the_file_API|File API]] @ v[[2.0]] for managing files stored by Moodle

===Core libraries with a more specific uses===

*[[Authentication API]]
*[[Cookieless Sessions]]
*[[Email processing]]
*[[Environment checking|Environment checking]] before install, check the user's server to ensure Moodle will work there.
*[[Groups|Groups system]]
*[[Grades|Gradebook]]
*[[Moodle Network|Moodle Network]]
*[[Question engine]]
*[[Stats package]]
*[[UTF-8 migration|Migration to UTF-8]] @ v[[:Category:Moodle 1.6|1.6]]
*[http://developer.yahoo.com/yui YUI JavaScript library] - YUI was selected as the official AJAX library for Moodle.
*[[lib/graphlib|lib/graphlib]]
*[[Admin settings|Admin settings]]

===Modules included in the standard distribution===

*[[Lesson Specification|Lesson Specification]]
*[[Quiz developer docs|Quiz module]]
*[[SCORM schema|SCORM module 1.5 schema]]

==How you can contribute==

===Make a new plugin===

The M in Moodle stands for modular, and the easiest, most maintainable way to add new functionality to Moodle is by using one of the many plugin APIs. There are many types of plugin you can write:
*[[Modules|Activity modules]], see also [[NEWMODULE Documentation]] (work in progress)
*[[Admin reports|Admin reports]]
*[[Assignment types|Assignment types]]
*[[Authentication plugins|Authentication plugins]]
*[[Blocks|Blocks]]
*[[Course formats]]
*[[Course Report Plugins|Course reports]]
*[[Database fields|Database fields]]
*[[Database presets|Database presets]]
*[[Enrolment plugins|Enrolment plugins]]
*[[Filters|Filters]]
*[[Gradebook plugins|Gradebook plugins]] (1.9 onwards)
**[[Gradebook_Report_Tutorial|Gradebook report]]
**[[Gradebook export|Gradebook export]]
**[[Gradebook import|Gradebook import]]
*[[Writing_a_Portfolio_Plugin|Portfolio plugins]] (2.0 onwards)
*[[Question_type_plugin_how_to|Question types]]
*[[Question import/export formats|Question import/export formats]]
*[[How to write a quiz report plugin|Quiz reports]]
*[[Repository plugins|Repository plugins]] (2.0 onwards)
*[[Resource types|Resource types]]
*[[Search engine adapters|Search engine adapters]]

General information that applies to all types of plugins
*[[Places to search for lang strings|Where to put language strings for your plugin]]
*[[Installing and upgrading plugin database tables|Defining the database tables for your plugin]]

Please see the [[Guidelines for contributed code|Guidelines for contributed code]] for an overview of how to contribute to the Moodle code.

Sometimes it is not possible to write a proper plugin for what you want to do, in which case you may have to resort to using the [[Local_customisation|local customisations]] hook.

===Change core code===

Some types of change can only be made by editing the core Moodle code. Such changes are much harder to maintain than plugins. If you want your core change to be considered for inclusion in the official Moodle release, you need to create an issue in the [[Tracker|tracker]], and attach your change as a [[How_to_create_a_patch|patch]]. It is also a good idea to discuss your ideas in the forums first. See [[Overview#Major_Development]] for more details.

===Ways to contribute that do not involve PHP programming===

*[[Themes|Create Moodle themes]]
*[[Translation|Translate Moodle into other languages]]
*[[MoodleDocs:Guidelines for contributors|Help document Moodle]]
*[[Tests|Join the testing effort]], which involves [[Tracker|participating in the bug tracker]]

==Plans for the future==

Ideas for and details of planned future features of Moodle are initially discussed on the forums in the [http://moodle.org/course/view.php?id=5 Using Moodle] course at moodle.org. That developer discussions are intermixed with user discussions in the same forums may seem strange at first but is one of the reasons for the success of Moodle. It is important that both end-users and developers discuss the future features together.

Once ideas begin to crystallize on the forums they can be summarized in this wiki, either as part of the [[Roadmap|roadmap]] or in the form of [[Developer notes|developer notes]]. These pages then form the basis for further discussion in the forums.

*[[Roadmap]]
*[[Developer notes|Developer notes]]
*[[Student projects]]
*[[Developer meetings]]

== Resources ==

*[[Developer FAQ]] - frequently asked questions, especially useful for newcomers to Moodle
*[[Finding_your_way_into_the_Moodle_code|Finding your way into the Moodle code]] - also aimed at newcomers
*[http://tracker.moodle.org/ Moodle tracker] - bug reports, feature requests and other tracked issues
**[[Firefox tracker search]] - How to setup a firefox quicksearch to easily navigate to moodle bugs
**[[Firefox tracker search#Firefox Search Plugins|Firefox Search Plugins]] - Find tracked issues even more easily
*[[Unmerged files]] - changes on the stable branch in CVS that have not been merged to [[HEAD]]
*Browse the code online:
**[http://cvs.moodle.org/moodle/ the code with a complete change history from CVS]
**[http://xref.moodle.org/index.html the code, with links generated by PHPXref]
*[http://phpdocs.moodle.org/ Moodle PHP doc reference] - compiled from the comment attached to each class and function in the code
*[[Database Schema|Database Schema]] - for recent releases
*[http://moodle.org/course/view.php?id=5#4 Development news and discussion] section of Using Moodle course
**especially the [http://moodle.org/mod/forum/view.php?id=55 General developer forum]
**[[Filters used on the Moodle.org forums|cool tricks you can use in the moodle.org forums]]

== Tools ==
Some tools people use when working on Moodle code:

=== IDEs ===

* [[Setting_up_Netbeans|Setting up NetBeans for Moodle development]] - NetBeans for PHP is a great out-of-the-box editor.
* [[Setting_up_Eclipse|Setting up Eclipse for Moodle development]] - Eclipse is a great editor to use for php development, if you can work out how to set it up.
* [[vim|Setting up Vim for Moodle development]]

=== Browser add-ons ===
*[http://getfirebug.com Firebug], see [[Firebug]].
* [[Web developer extension]]
* [https://addons.mozilla.org/en-US/firefox/addon/394 ViewSourceWith] - The main goal is to view page source with external applications, but you can do a lot of other things as well.

=== Miscellaneous ===
*[[ctags|Ctags]] - Using a tags file to navigate code
*[[W3C_validation|W3C HTML validator]] - Moodle has built in support to make using it easier.
*[[Windows Installer|Windows Installer]] - Windows Installer documentation for developer.

See also: [http://dev.moodle.org/mod/forum/view.php?id=18 Useful Development Tools forum]in the [http://dev.moodle.org/course/view.php?id=2 Introduction to Moodle Programming course]

==See also==

*[http://security.moodle.org/ Moodle Security Centre]
*[http://moodle.com/partners/ Moodle Partners] - providers of custom Moodle development services

[[Category:Developer tools]]

[[ru:Development:Краткий обзор]]

Talk:Places to search for lang strings

2009-07-08T08:55:24Z

Jeffjeff:

Talk:Places to search for lang strings

2009-07-08T08:54:20Z

Jeffjeff: New page: Not sure where to put this, but here goes. Often a Moodle user will like to adjust a particular sting the he sees when using Moodle. It is often a problem to find WHICH PHP file is the one...

Talk:Coding style

2009-05-27T07:17:14Z

Jeffjeff: Trailing spaces to be left when they are in a CVS original??

== PHP5 constructor ==

Should we enforce the PHP5 constructor __construct() instead of $classname() [[User:Nicolas Connault|Nicolas Connault]] 19:55, 19 May 2009 (UTC)
:+1. I think we agreed about that some time ago (when igniting some 2.0 developments) --[[User:Eloy Lafuente (stronk7)|Eloy Lafuente (stronk7)]] 00:03, 20 May 2009 (UTC)

== Inline comments ==

I have been using since ages ago "///" with outer alignment (I think that it was caused by some agreement long time ago, not my decision), for example:

<code php>
function check_moodle_environment(..., ..., ..., ...) {

$status = true;

/// This are cached per request
static $result = true;
static $env_results;
static $cache_exists = false;

/// if we have results cached, use them
if ($cache_exists) {
$environment_results = $env_results;
/// No cache exists, calculate everything
} else {
/// Get the more recent version before the requested
if (!$version = get_latest_version_available($version, $env_select)) {
$status = false;
}
....
....
</code>
Is that allowed, or only the "//" with inner alignment as commented at [[Coding style#Inline comments|inline comments]] --[[User:Eloy Lafuente (stronk7)|Eloy Lafuente (stronk7)]] 00:03, 20 May 2009 (UTC)
==Trailing spaces==
"Lines should not contain trailing spaces. In order to facilitate this convention, most editors can be configured to strip trailing spaces, such as upon a save operation. However, if you are editing an existing Moodle file and are planning to submit your changes to CVS, please switch off that feature so that whitespace changes do not pollute CVS history (other developers will have trouble viewing what you've done if every other line has been edited for whitespace)."
Wouldn't the CVS history only get 'polluted' when someone fetched a file that DIDN'T follow this principle and returned it following the principle? Isn't that the price that has to be paid to clean things up? Otherwise this is pretty scary for someone who feels: "Great I can turn on my editor to autostrip those when saving. BUT I have to remember to turn that off if submitting it! (Or become a 'polluter'!)"
And what happens if I have already saved it locally while working on (and striped those trailing spaces) and later try to commit it. Maybe my ignorance about CVS is making me pose a "silly" question. [[User:Jeff Forssell|Jeff Forssell]] 07:17, 27 May 2009 (UTC)

Patching forms tutorial

2009-04-03T06:56:53Z

Jeffjeff: overflowing lines in pre block

by Gary Anderson

One of the easiest local modifications that an institution can make to customize their Moodle site is to make simple changes to the form elements that appear on pages. This permits one to set different default values when creating new items, allows one to hide unused input elements to simplify the interface, and lets one identify certain more complicated elements as "advanced" so that features are still availale while simplifying the interface for most uses.

This tutorial will demonstrate how to do such modifications.

'''Warning'''
Making custom changes to the PHP scripting code in Moodle should be done carefully.
You should always test your code before using it on your production site and should
backup the original scripts.
You should always comment your code so you know what has been changed.
Care should also be taken during upgrades in the code to make sure things are not
broken (like conflicts during a CVS update).

Having said that, these patches are among the simplest and the recovery from an error is easy: simply replace the file with original from the core distribution.

'''Moodle Forms Overview'''

Rather than placing HTML form elements directly on the page, most well-written Moodle code will use the Forms Library. This allows a standard way to create user input elements and also centralizes code making it easier to override certain behaviors creating a greater consistency.

An entire form is an "object" with methods and attributes. In many places in the Moodle code, the instance of the object is identified by the variable $mform, but you can use any variable.

Common methods of the form object look like:

* $mform->addelement(<elementtype>,<elementidentifier>,<displayname>,<options>);

This creates a new input element on a page.

While forms can be produced in any scripting page which outputs code, you will often find code for printing out forms in a file like mod_form.php or in index.php of the affected component. The examine the URL of the page being viewed as a starting point for finding the file to be changed.

Always comment your code so you can change or remove it later.. For single line modifications, considering putting in your new code, and then end it with //<yourusername> <date> <the old code>. This allows for retention of existing line numbers.

==Changing defaults==

The '''setDefault''' method is used to establish the initial value used in the form and allows you to to define the most common setting for use in your institution.

Example: Change the default number of points for a default assignment from 100 points to 10 points.

Solution:

In the file /mod/assignment/mod_form.php, find

$mform->setDefault('grade', 100);

Change the default to 10, and comment out the previous code with your initial and dates:

$mform->setDefault('grade', 10); //GVA 20080608 $mform->setDefault('grade', 100);

==Hiding fields==

Form elements are created for a page using the '''addElement''' method. Various types of element are text, select, date_time_selector, htmleditor, etc.

There is also the type "hidden" that defines the element on a page and allows you to set a default value, stores previous values from a submission etc. To simplify your forms for you users, simply change the type of the field during when addElement is called to hidden.

Example: Your site does not want to use meta courses and you want to remove it from the course creation/edit page. In the file /course/edit_form.php

Change:

$mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

to

$mform->addElement('hidden', 'metacourse', get_string('managemeta'), $meta); //GVA 20080610 $mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

==Setting fields to "Advanced"==

For elements that are infrequently used in your institution, you can still simplify the interface while making options still available to your 'Power Users' by marking elements as "advanced".

These elements only appear to the user when they click the "Set Advanced" button.

To identify the element as advanced, call the '''setAdvanced''' method after the element has been created.

Example:

Your typical user is local and hence it will be rare that they will use a different time zone. To clean up the interface, set the option to change the user time zone so that it is shown only when the user clicks "Set Advanced".

Solution:

In the file /user/editlib.php, find:

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);

Set the field to advanced using the setAdvanced method

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);
$mform->setAdvanced('timezone');

==Other Examples==
Here are some other example of making simple changes to forms
===Change the maximum allowed grade===
The maximum grade for a module is set in the standard form to 100. To change this to a different value (such as 200) in version 1.9, make this change in the file '''/lib/form/modgrade.php'''

Find:

for ($i=100; $i>=1; $i--) {

Change to:

for ($i=200; $i>=1; $i--) { // GVA 20080616 for ($i=100; $i>=1; $i--) {

===Set a standard start date and number of weeks for courses===

By default, Moodle starts courses on the day following course creation and gives them a duration of 10 weeks. As many institutions have standard terms, setting the default start date and duration will allow for more uniformity, fewer mistakes, and will ease course creation.

In version 1.9, edit the file '''/course/edit_form.php'''. Insert the following lines after the affected fields are defined (perhaps on lines just before $this->add_action_buttons();)

//GVA 20080618 Number of weeks for default course. Should go through final exam
$mform->setDefault('numsections', 13);
//GVA 20080618 Set default course start date. Always start on a Monday to have weeks work right
$mform->setDefault('startdate', strtotime("1 Sep 2008"));

[[Category:Tutorial]]

Talk:Patching forms tutorial

2009-04-03T06:31:34Z

Jeffjeff: New page: the example: Change the maximum allowed grade The maximum grade for a module is set in the standard form to 100. To change this to a different value (such as 200) in version 1.9, make ...

the example:
Change the maximum allowed grade
The maximum grade for a module is set in the standard form to 100.
To change this to a different value (such as 200) in version 1.9,
make this change in the file /lib/form/modgrade.php
Find:
for ($i=100; $i>=1; $i--) {

seems very strange, at least without seeing the code around it (which I haven't yet done).
I think it would be good to say something about how that works. The previous examples are easy to follow what they are doing.
--[[User:Jeff Forssell|Jeff Forssell]] 06:31, 3 April 2009 (UTC)

Patching forms tutorial

2009-04-03T06:24:19Z

Jeffjeff: /* Setting fields to "Advanced" */

by Gary Anderson

One of the easiest local modifications that an institution can make to customize their Moodle site is to make simple changes to the form elements that appear on pages. This permits one to set different default values when creating new items, allows one to hide unused input elements to simplify the interface, and lets one identify certain more complicated elements as "advanced" so that features are still availale while simplifying the interface for most uses.

This tutorial will demonstrate how to do such modifications.

'''Warning'''
Making custom changes to the PHP scripting code in Moodle should be done carefully.
You should always test your code before using it on your production site and should backup the original scripts.
You should always comment your code so you know what has been changed.
Care should also be taken during upgrades in the code to make sure things are not broken (like conflicts during a CVS update).

Having said that, these patches are among the simplest and the recovery from an error is easy: simply replace the file with original from the core distribution.

'''Moodle Forms Overview'''

Rather than placing HTML form elements directly on the page, most well-written Moodle code will use the Forms Library. This allows a standard way to create user input elements and also centralizes code making it easier to override certain behaviors creating a greater consistency.

An entire form is an "object" with methods and attributes. In many places in the Moodle code, the instance of the object is identified by the variable $mform, but you can use any variable.

Common methods of the form object look like:

* $mform->addelement(<elementtype>,<elementidentifier>,<displayname>,<options>);

This creates a new input element on a page.

While forms can be produced in any scripting page which outputs code, you will often find code for printing out forms in a file like mod_form.php or in index.php of the affected component. The examine the URL of the page being viewed as a starting point for finding the file to be changed.

Always comment your code so you can change or remove it later.. For single line modifications, considering putting in your new code, and then end it with //<yourusername> <date> <the old code>. This allows for retention of existing line numbers.

==Changing defaults==

The '''setDefault''' method is used to establish the initial value used in the form and allows you to to define the most common setting for use in your institution.

Example: Change the default number of points for a default assignment from 100 points to 10 points.

Solution:

In the file /mod/assignment/mod_form.php, find

$mform->setDefault('grade', 100);

Change the default to 10, and comment out the previous code with your initial and dates:

$mform->setDefault('grade', 10); //GVA 20080608 $mform->setDefault('grade', 100);

==Hiding fields==

Form elements are created for a page using the '''addElement''' method. Various types of element are text, select, date_time_selector, htmleditor, etc.

There is also the type "hidden" that defines the element on a page and allows you to set a default value, stores previous values from a submission etc. To simplify your forms for you users, simply change the type of the field during when addElement is called to hidden.

Example: Your site does not want to use meta courses and you want to remove it from the course creation/edit page. In the file /course/edit_form.php

Change:

$mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

to

$mform->addElement('hidden', 'metacourse', get_string('managemeta'), $meta); //GVA 20080610 $mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

==Setting fields to "Advanced"==

For elements that are infrequently used in your institution, you can still simplify the interface while making options still available to your 'Power Users' by marking elements as "advanced".

These elements only appear to the user when they click the "Set Advanced" button.

To identify the element as advanced, call the '''setAdvanced''' method after the element has been created.

Example:

Your typical user is local and hence it will be rare that they will use a different time zone. To clean up the interface, set the option to change the user time zone so that it is shown only when the user clicks "Set Advanced".

Solution:

In the file /user/editlib.php, find:

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);

Set the field to advanced using the setAdvanced method

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);
$mform->setAdvanced('timezone');

==Other Examples==
Here are some other example of making simple changes to forms
===Change the maximum allowed grade===
The maximum grade for a module is set in the standard form to 100. To change this to a different value (such as 200) in version 1.9, make this change in the file '''/lib/form/modgrade.php'''

Find:

for ($i=100; $i>=1; $i--) {

Change to:

for ($i=200; $i>=1; $i--) { // GVA 20080616 for ($i=100; $i>=1; $i--) {

===Set a standard start date and number of weeks for courses===

By default, Moodle starts courses on the day following course creation and gives them a duration of 10 weeks. As many institutions have standard terms, setting the default start date and duration will allow for more uniformity, fewer mistakes, and will ease course creation.

In version 1.9, edit the file '''/course/edit_form.php'''. Insert the following lines after the affected fields are defined (perhaps on lines just before $this->add_action_buttons();)

//GVA 20080618 Number of weeks for default course. Should go through final exam
$mform->setDefault('numsections', 13);
//GVA 20080618 Set default course start date. Always start on a Monday to have weeks work right
$mform->setDefault('startdate', strtotime("1 Sep 2008"));

[[Category:Tutorial]]

Patching forms tutorial

2009-04-03T06:20:09Z

Jeffjeff: and to end eisting existing

by Gary Anderson

One of the easiest local modifications that an institution can make to customize their Moodle site is to make simple changes to the form elements that appear on pages. This permits one to set different default values when creating new items, allows one to hide unused input elements to simplify the interface, and lets one identify certain more complicated elements as "advanced" so that features are still availale while simplifying the interface for most uses.

This tutorial will demonstrate how to do such modifications.

'''Warning'''
Making custom changes to the PHP scripting code in Moodle should be done carefully.
You should always test your code before using it on your production site and should backup the original scripts.
You should always comment your code so you know what has been changed.
Care should also be taken during upgrades in the code to make sure things are not broken (like conflicts during a CVS update).

Having said that, these patches are among the simplest and the recovery from an error is easy: simply replace the file with original from the core distribution.

'''Moodle Forms Overview'''

Rather than placing HTML form elements directly on the page, most well-written Moodle code will use the Forms Library. This allows a standard way to create user input elements and also centralizes code making it easier to override certain behaviors creating a greater consistency.

An entire form is an "object" with methods and attributes. In many places in the Moodle code, the instance of the object is identified by the variable $mform, but you can use any variable.

Common methods of the form object look like:

* $mform->addelement(<elementtype>,<elementidentifier>,<displayname>,<options>);

This creates a new input element on a page.

While forms can be produced in any scripting page which outputs code, you will often find code for printing out forms in a file like mod_form.php or in index.php of the affected component. The examine the URL of the page being viewed as a starting point for finding the file to be changed.

Always comment your code so you can change or remove it later.. For single line modifications, considering putting in your new code, and then end it with //<yourusername> <date> <the old code>. This allows for retention of existing line numbers.

==Changing defaults==

The '''setDefault''' method is used to establish the initial value used in the form and allows you to to define the most common setting for use in your institution.

Example: Change the default number of points for a default assignment from 100 points to 10 points.

Solution:

In the file /mod/assignment/mod_form.php, find

$mform->setDefault('grade', 100);

Change the default to 10, and comment out the previous code with your initial and dates:

$mform->setDefault('grade', 10); //GVA 20080608 $mform->setDefault('grade', 100);

==Hiding fields==

Form elements are created for a page using the '''addElement''' method. Various types of element are text, select, date_time_selector, htmleditor, etc.

There is also the type "hidden" that defines the element on a page and allows you to set a default value, stores previous values from a submission etc. To simplify your forms for you users, simply change the type of the field during when addElement is called to hidden.

Example: Your site does not want to use meta courses and you want to remove it from the course creation/edit page. In the file /course/edit_form.php

Change:

$mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

to

$mform->addElement('hidden', 'metacourse', get_string('managemeta'), $meta); //GVA 20080610 $mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

==Setting fields to "Advanced"==

For elements that are infrequently used in your institution, you can still simplify the interface while making options still available to your 'Power Users' by marking elements as "advanced".

These elements only appear to the user when the click the "Set Advanced" button.

To identify the element as advanced, call the '''setAdvanced''' method after the element has been created.

Example:

Your typical user is local and hence it will be rare that they will use a different time zone. To clean up the interface, set the option to change the user time zone so that it is shown only when the user clicks "Set Advanced".

Solution:

In the file /user/editlib.php, find:

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);

Set the field to advanced using the setAdvanced method

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);
$mform->setAdvanced('timezone');

==Other Examples==
Here are some other example of making simple changes to forms
===Change the maximum allowed grade===
The maximum grade for a module is set in the standard form to 100. To change this to a different value (such as 200) in version 1.9, make this change in the file '''/lib/form/modgrade.php'''

Find:

for ($i=100; $i>=1; $i--) {

Change to:

for ($i=200; $i>=1; $i--) { // GVA 20080616 for ($i=100; $i>=1; $i--) {

===Set a standard start date and number of weeks for courses===

By default, Moodle starts courses on the day following course creation and gives them a duration of 10 weeks. As many institutions have standard terms, setting the default start date and duration will allow for more uniformity, fewer mistakes, and will ease course creation.

In version 1.9, edit the file '''/course/edit_form.php'''. Insert the following lines after the affected fields are defined (perhaps on lines just before $this->add_action_buttons();)

//GVA 20080618 Number of weeks for default course. Should go through final exam
$mform->setDefault('numsections', 13);
//GVA 20080618 Set default course start date. Always start on a Monday to have weeks work right
$mform->setDefault('startdate', strtotime("1 Sep 2008"));

[[Category:Tutorial]]

Patching forms tutorial

2009-04-03T06:17:24Z

Jeffjeff: my to by

by Gary Anderson

One of the easiest local modifications that an institution can make to customize their Moodle site is to make simple changes to the form elements that appear on pages. This permits one to set different default values when creating new items, allows one to hide unused input elements to simplify the interface, and lets one identify certain more complicated elements as "advanced" so that features are still availale while simplifying the interface for most uses.

This tutorial will demonstrate how to do such modifications.

'''Warning'''
Making custom changes to the PHP scripting code in Moodle should be done carefully.
You should always test your code before using it on your production site and should backup the original scripts.
You should always comment your code so you know what has been changed.
Care should also be taken during upgrades in the code to make sure things are not broken (like conflicts during a CVS update).

Having said that, these patches are among the simplest and the recovery from an error is easy: simply replace the file with original from the core distribution.

'''Moodle Forms Overview'''

Rather than placing HTML form elements directly on the page, most well-written Moodle code will use the Forms Library. This allows a standard way to create user input elements and also centralizes code making it easier to override certain behaviors creating a greater consistency.

An entire form is an "object" with methods and attributes. In many places in the Moodle code, the instance of the object is identified by the variable $mform, but you can use any variable.

Common methods of the form object look like:

* $mform->addelement(<elementtype>,<elementidentifier>,<displayname>,<options>);

This creates a new input element on a page.

While forms can be produced in any scripting page which outputs code, you will often find code for printing out forms in a file like mod_form.php or in index.php of the affected component. The examine the URL of the page being viewed as a starting point for finding the file to be changed.

Always comment your code so you can change or remove it later.. For single line modifications, considering putting in your new code, and then and it with //<yourusername> <date> <the old code>. This allows for retention of eisting line numbers.

==Changing defaults==

The '''setDefault''' method is used to establish the initial value used in the form and allows you to to define the most common setting for use in your institution.

Example: Change the default number of points for a default assignment from 100 points to 10 points.

Solution:

In the file /mod/assignment/mod_form.php, find

$mform->setDefault('grade', 100);

Change the default to 10, and comment out the previous code with your initial and dates:

$mform->setDefault('grade', 10); //GVA 20080608 $mform->setDefault('grade', 100);

==Hiding fields==

Form elements are created for a page using the '''addElement''' method. Various types of element are text, select, date_time_selector, htmleditor, etc.

There is also the type "hidden" that defines the element on a page and allows you to set a default value, stores previous values from a submission etc. To simplify your forms for you users, simply change the type of the field during when addElement is called to hidden.

Example: Your site does not want to use meta courses and you want to remove it from the course creation/edit page. In the file /course/edit_form.php

Change:

$mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

to

$mform->addElement('hidden', 'metacourse', get_string('managemeta'), $meta); //GVA 20080610 $mform->addElement('select', 'metacourse', get_string('managemeta'), $meta);

==Setting fields to "Advanced"==

For elements that are infrequently used in your institution, you can still simplify the interface while making options still available to your 'Power Users' by marking elements as "advanced".

These elements only appear to the user when the click the "Set Advanced" button.

To identify the element as advanced, call the '''setAdvanced''' method after the element has been created.

Example:

Your typical user is local and hence it will be rare that they will use a different time zone. To clean up the interface, set the option to change the user time zone so that it is shown only when the user clicks "Set Advanced".

Solution:

In the file /user/editlib.php, find:

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);

Set the field to advanced using the setAdvanced method

$mform->addElement('select', 'timezone', get_string('timezone'), $choices);
$mform->setAdvanced('timezone');

==Other Examples==
Here are some other example of making simple changes to forms
===Change the maximum allowed grade===
The maximum grade for a module is set in the standard form to 100. To change this to a different value (such as 200) in version 1.9, make this change in the file '''/lib/form/modgrade.php'''

Find:

for ($i=100; $i>=1; $i--) {

Change to:

for ($i=200; $i>=1; $i--) { // GVA 20080616 for ($i=100; $i>=1; $i--) {

===Set a standard start date and number of weeks for courses===

By default, Moodle starts courses on the day following course creation and gives them a duration of 10 weeks. As many institutions have standard terms, setting the default start date and duration will allow for more uniformity, fewer mistakes, and will ease course creation.

In version 1.9, edit the file '''/course/edit_form.php'''. Insert the following lines after the affected fields are defined (perhaps on lines just before $this->add_action_buttons();)

//GVA 20080618 Number of weeks for default course. Should go through final exam
$mform->setDefault('numsections', 13);
//GVA 20080618 Set default course start date. Always start on a Monday to have weeks work right
$mform->setDefault('startdate', strtotime("1 Sep 2008"));

[[Category:Tutorial]]

Filter enable/disable by context

2009-03-13T07:44:29Z

Jeffjeff: /* New settings page in each context */

{{Moodle 2.0}}

We want to make configuration of filters more flexible, so, for example, you can do things like
* Disable the glossary auto-linking filter in a quiz.
* Enable the TeX filter only in the Maths course category.

Please discuss this proposal in [http://moodle.org/mod/forum/discuss.php?d=118580 this General Developer Forum thread].

==Overview==

Most filter settings will still be controlled at the system level. Administrators will still choose which filters enabled for the whole site, and, more importantly, which ones are not available at all. Also, any filter settings (for example which media types for the multimedia filter, and the paths for the TeX filter) will also still be set globally for the whole site by the administrator.

However, in addition to the overall on/off switch for each filter, we will store a separate active/inactive state for each filter in each context. (Don't worry, we don't actually store a row in the database for each context, mostly we use inheritance.)

This proposal relies on some of the proposed navigation changes for Moodle 2.0, to give us a place to add a new 'Text filtering' setting link for each context.

==Detailed design==

===Changes to the filter admin screen===

On Administration ► Plugins ► Filters ► Manage filters, the existing Disable/Enable column will be split into two.
* '''Enabled?''' will control whether this filter can be used at all on this site.
* '''Active?''' controls whether this filter is in use globally. (If the plugin in not enabled, you cannot activate it.)

(A filter may need to be enabled, but not active globally, for example in the "TeX filter only in the maths course" use case.)

We will also add a Delete column to this table, to let administrators delete all the related configuration from the database if they uninstall a plugin.

===New settings page in each context===

There will be a "Text filtering" settings page for each context (apart from block contexts), just like there are Assign roles and Override permissions pages.

The page will have a title something like "Text filtering in Course: Maths 101". It will contain a table with one row for each filter that the administrator has enabled. For each filter, there will be a drop-down with three choices: Inherit, Active and Inactive. For the inherit choice, it will display the value that would be inherited in brackets, for example "Inherit (Active)". (Would these be better as radio buttons?) There will be a Save changes button at the bottom.

[[Image:Filters_example_1.png]] [[Image:Filters_example_2.png]]

===New capability moodle/filter:manage===

This controls access to the Text filtering settings page.

By default Administrators, Course creators and Teachers will have this capability.

===New database table filter_active===

This table replaces $CFG->textfilters.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) AUTO-INCREMENT || Unique id
|-
| filter || VARCHAR(32) || e.g. 'filter/tex' or 'mod/glossary'
|-
| contextid || INT(10) || Foreign key references context.id
|-
| active || INT(4) || 1 = active, -1 = inactive. As a special case, if contextid == $systemcontext->id, then -9999 is used to mean that the administrator has disabled this filter.
|-
| sortorder || INT(10) || This is only used when contextid == $systemcontext->id. It stores the filter sort-order, numbering from 1. In other contexts, this column should contain 0.
|}

Whenever a context is deleted, we must delete the corresponding rows from this table.

We will not delete data from here when a filter is disabled globally, so that if the administrator disables then re-enables the filter, the setting in all the different contexts are not lost.

===Retrieving the active filters for a page===

One of the Navigation 2.0 changes is that each page will be linked to a specific context. (That is, $PAGE->context will always to set to something sensible.)

Text will be filtered according to the page it appears on, not the context it belongs to. This is normally fine. For example, we want question text to be filtered according to the quiz that the question appears in.

With that in mind, suppose $contextids is the list of ids of this context and all its parents. Then the list of active filters is:

<code sql>
SELECT f.filter
FROM filter_active f
JOIN context ctx ON f.contextid = cxt.id
WHERE ctx.id IN ($contextids)
GROUP BY filter
HAVING MAX(f.active * ctx.depth) > -MIN(f.active * ctx.depth)
ORDER BY MAX(f.sortorder)
</code>

====Why does this work?====

To understand what that query is doing, you need to look at the results of

<code sql>
SELECT f.filter, f.contextid, ctx.depth, f.active, ctx.depth * f.active AS active_x_depth, f.sortorder
FROM filter_active f
JOIN context ctx ON f.contextid = cxt.id
WHERE ctx.id IN ($contextids)
ORDER BY f.filter, ctx.depth
</code>

That shows the data that is being aggregated by the GROUP BY and HAVING clauses. I will explain with reference to a specific example:

Suppose we are in a Quiz in the Maths 101 course in the Maths category in the system. $contextids = 1,3,10,22 (order reversed relative to the previous sentence - system context first). Then suppose that the above query returns

{| class="nicetable"
! filter !! contextid !! depth !! active !! active_x_depth !! f.sortorder
|-
| filter/mediaplugin || 1 || 1 || 1 || 1 || 1
|-
| filter/multilang || 1 || 1 || -1 || -1 || 1
|-
| filter/tex || 1 || 1 || -1 || -1 || 3
|-
| filter/tex || 3 || 2 || 1 || 2 || 0
|-
| filter/tidy || 1 || 1 || -9999 || -9999 || 4
|-
| filter/tidy || 22 || 4 || 1 || 4 || 0
|-
| mod/glossary || 1 || 1 || 1 || 1 || 2
|-
| mod/glossary || 3 || 2 || -1 || -2 || 0
|-
| mod/glossary || 10 || 3 || 1 || 3 || 0
|-
| mod/glossary || 22 || 4 || -1 || -4 || 0
|}

; filter/mediaplugin
: This is enabled, and active globally, and there are no overrides.
:* MAX(f.active * ctx.depth) = 1
:* MIN(f.active * ctx.depth) = 1
: 1 > -1, so this filter will be returned by the get active filters query.

; filter/multilang
: This is enabled, but not active globally, and there are no overrides.
:* MAX(f.active * ctx.depth) = -1
:* MIN(f.active * ctx.depth) = -1
: -1 ≯ 1, so this filter will '''not''' be returned by the get active filters query.

; filter/tex
: This is enabled, but not active globally. However, it has been activated in the Maths category.
:* MAX(f.active * ctx.depth) = 2
:* MIN(f.active * ctx.depth) = -1
: 2 > 1, so this filter will be returned by the get active filters query.

; filter/tidy
: This is '''not''' enabled, but there is an old override in the database
:* MAX(f.active * ctx.depth) = 4
:* MIN(f.active * ctx.depth) = -9999
: 4 ≯ 9999, so this filter will '''not''' be returned by the get active filters query.

; filter/glossary
: A slightly silly example with the maximum number of overrides possible. The most specific override is the one that says that the filter is not active in the quiz.
:* MAX(f.active * ctx.depth) = 3
:* MIN(f.active * ctx.depth) = -4
: 3 ≯ 4, so this filter will '''not''' be returned by the get active filters query.

; filter/myfilter
: What! there is no filter/myfilter in the data above. What is going on? This is the situation when the administrator has just installed the myfilter plugin, but has not yet been to the manage filters administration page to update the filter settings. What happens?
: Since there is nothing in the database, this filter is '''not''' returned by the get active filters query. That is, a newly installed filter is not active until the administrator explicitly activates it.

'''Sort order''': Note that sortorder is zero except when contextid = 1. So MAX(f.sortorder) is just a clever trick to pull out the sortorder from the system context, ignoring all the zeros. There are other tricks that could be used instead, but this one is probably quite efficient.

===Changes to text caching===

The md5key used in the cache_text table will be changed to user the contextid, rather than the courseid, in $hashstr. See format_text in weblib.php.

===Backup and restore===

This will be similar in some ways to the roles backup and restore code, in that it needs to do work for every context in the backup file.

On backup (in lib/backuplib.php), everywhere it does write_role_overrides_xml and write_role_assignments_xml we need to add a call to a new function write_filter_actives_xml. (I think we need a write_context_data_xml function to avoid duplicating the calls to those three functions in lots of places.) That function can write XML like
<code xml>
<FILTERACTIVES>
<FILTERACTIVE>
<FILTER>filter/tex</FILTER>
<ACTIVE>+1</ACTIVE>
<FILTERACTIVE>
<FILTERACTIVE>
<FILTER>mod/glossary</FILTER>
<ACTIVE>-1</ACTIVE>
<FILTERACTIVE>
</FILTERACTIVES>
</code>

In the function restore_execute in backup/restorelib.php, probably just after the bit that calls restore_roles_settings, add a call to restore_filter_actives. This needs to read each FILTERACTIVE record from the backup file, check whether the corresponding filters in installed on this Moodle site, and if it is, write the record to the filter_active table using the recoded contextid.

==Possible problems==

===Terminology===

I have been using enabled/disabled for the on-off switch that the administrator had, and active/inactive for the setting that can be inherited or changed in each context. However, it is confusing. Does anyone have a better suggestion?

===Is filtering by page context really the right thing===

Filtering according to page context might cause strange results on pages that aggregate content from different contexts.

For example, we normally think that the user's profile page is in the user context. That would mean that the Recent activity report there will be filtered according to the filter settings for the user context. (These will almost certainly inherit unmodified from the System context.) That would be strange if the TeX filter was only enabled in the Maths course. However, it may work better (both here and elsewhere) if we set $PAGE->context to the course context for profile pages like this.

On the other hand, if the parent looking at their child's recent activity does not have access to the course, it is good that they do not see the effects of the glossary auto-linking filter, which would like them to glossary entries they do not have permission to see.

The real problem page, would, of course, be the My Moodle page. That could aggregate content from any number of different contexts.

Or, we may find that we have to add a $contextid parameter to the format_text function, and find some way to avoid the performance problems that causes. (It would be really bad to query the database with a query like the one above for each call to format_text. Or even for each context that contributes content to the My Moodle page.)

==See also==

* [[Navigation_2.0]]
* MDL-7336
* [http://moodle.org/mod/forum/discuss.php?d=118580 this General Developer Forum thread].

{{CategoryDeveloper}}
[[Category:Filter]]

Quiz UI redesign scenarios - Background

2009-03-12T10:19:05Z

Jeffjeff: /* Paul Pedagogue */

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where scenarios were commented. Now inactive, please edit this wiki directly or create a new thread].

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]
== Background/Personas ==

An exam can be used at the beginning of a course to evaluate the initial level of the students' understanding, during the course to provide checkpoints for students, or at the end of the course. In any case, quizzes/exams often have a dual role: they give the teacher information about the student's skills, but more '''importantly they are a pedagogical tool: while answering, a student generates knowledge and learns in the process'''.

'''If it seems to you that your idea of using exam is very different from any of the current personas, you may add a new one or [http://moodle.org/mod/forum/discuss.php?d=99351 tell us about this in the forum thread dedicated for persona/scenario discussion]'''

Key questions for this part of the exam making:

* What are the pedagogigal goals for the teacher?
* Does the exam use essay questions or automatically evaluated (or what)?
* What initiates the lifecycle of an exam?

----

=== Mack Marketing ===

* '''Mack's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
''Mack'' is a marketing teacher. He likes golf, but not on rainy days.
* He has quite a bit of experience using the internet, mainly for webmail

He also teaches online a bit, but mostly he has normal classes. In addition to regular, time-limited exams, he also uses take-home exams (with deadlines for the questions/case assignments) and group exams (a group works together on a a case and the work is evaluated).

====Exams: why and what====

While defining the goals of his course, he ends up with the conclusion that an exam with essay questions is needed at the end of the course for evaluating students' learning. He does not think exams are a very good way of evaluating learning, since they mostly evaluate what students happen to remember. And even though he considers grading essay answers an extra workload, he still sees this as the only way '''to effectively evaluate students' conceptual understanding''', especially on his basic courses where there are more students.

He uses quite similar exams on different courses, our focus is on a Introduction to Marketing exam, which is about:

* Evaluating conceptual learning; can apply what they have learned to real-life cases
* Essay questions
* Qualitative feedback; using templates as basis

Hypothesis: Since his exams are so specialized, Mack is afraid that if another teacher takes up his courses, that teacher will not understand all the nuances he has taken into account while designing his exams. That is why he writes notes about each exam whole and archives these notes that explain the choices he has made in the exam design.

====Feedback====
He plans '''to give feedback about students' answers: for him, this enables the exam to work as a pedagogical tool''' and he does not require the exam results to be absolutely comparable with the exam results of courses of previous semesters. He uses some templates for feedback, and based on that he then writes feedback for each student personally.

----

=== Ida Informatics ===

* '''Ida's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

Moodle community contributors to Ida: [http://moodle.org/user/view.php?id=81449&course=5 Gary Anderson]
----

''Ida'' is a teacher in her fifties, still likes teaching but is more-or-less secretly already anticipating her retirement. She teaches informatics. Her teaching responsibilities include lecturing a couple of basic courses with dozens or even some hundreds of students at a time.

====Exams: why and what====
A lot of her time is spent planning '''a course of basic information retrieval to 200 first year students each semester'''.
The exam is about:
* Evaluating learning of basic factual knowledge of her field
* Multiple choice questions
* for '''testing students' basic factual knowledge of her field''' automatically graded questions, such as multiple questions, suffice well enough (versus Mack's essay questions for testing conceptual understanding).
* Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.

Ida sees exams beneficial since they provide '''measurable grounds on which to compare and grade learning''' on a '''numerical (quantitative) scale''' and for quick feedback to students. She wants to have her exam as automated as possible. She does not think exams have much pedagogical purpose, and would actually rather not even grade her students, but since the educational system demands comparable grades, she does it this way.

(In an ideal world, she would instantly start using a tool that would analyze her course material automatically, generate essay questions and then even grade them using linguistic analysis or some other method.)

----

===Harvey Historian===
* '''Harvey's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey is a history professor in a university. He mostly concentrates on his research, but also does some teaching of basic courses he knows by heart, since he has been doing it for a couple of decades.

====Exams: why and what====
He uses exams to test the students' understanding of specific eras from different points of view, and students' ability to see the connections between different events in history.

Sometimes Harvey also takes a year off, and the exam bank is an excellent resource for a less experienced teacher to refer to for finding out what is essential to teach about a course.

====Feedback====
Harvey gives brief feedback to students about what would have made a better grade. Since he has seen most things in the past already, he does not use any feedbacks for template since he has all of it pretty much in the back of his head.

----
===Paul Pedagogue===
* '''Paul's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Paul_Pedagogue|Paul: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Paul_Pedagogue|Paul: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
Moodle community contributors to Paul: [https://docs.moodle.org/en/User:Jeff_Forssell Jeff Forssell]
----

Paul is almost 60 years old and is interested in all kinds of fields: science, gardening, languages, politics and music. He spends a lot of time on Internet and is delighted that it is now possible for everyone to contribute. He isn't a trained programmer, but is not afraid to dabble in code to fine tune things to the way he wants to work. Though he loves discussing directly with students, he feels that in a dynamic Internet distance course he can have very personal contact with the students. Good interactive questions can give him freedom from repetitious correcting while giving good guidance to students. He has many distance students, but still has some classroom students.

====Grading hinders learning====
Paul uses interactive questions, in "quizzes", primarily as a learning tool. He actually would prefer to have "official evaluation" (grading) done by some other instance, like the higher school or workplace that the student is aspiring to. This would free the teacher from the schizophrenic Helper/Judge dichotomy that prevents students and teachers from really working together in trust toward the common goal: lasting knowledge and methods for the student that will help him/er reach his/er goals. Grading in a class can also lead to diminished student interest in helping one's classmates, though this is one of the best ways to really learn something.

====Questions are seeds for dialogue====
Questions for learning have been the prime tool for many teachers from Socrates to Martin D! Questions usually don't have one RIGHT answer, so it is very important to allow for many alternative answers and this can make pattern matching (like "regular expressions") a high priority. Since the question is used for teaching it is important to make it provide as much dialog as possible. He will generally not like a simplified interface that doesn't have possibilities for rich feedback for varied answers. If possible he will try to give some kind of encouragement/understanding, for common mistakes as well a some '''clue to help''' the student in his '''processing of the problem''' (preferably without giving the "right answer"). His quizzes give students ways they can assess their own learning and, in a problem-based learning style, build up their understanding. If he has distance pupils, he will create many "'''interactive question exercise follow-ups'''" (using quizzes) as soon as he notices things that are common pitfalls. For an answer that isn't the "right one" his '''feedback will tend to be cautious''': "We didn't recognize your answer as being one that we expected. Could you describe how you reached this answer?"

===="Organic growth" of interactive questions====
Since he sees the questions as part of a developing dialog he values any characteristics that increase the dynamics. Since he's not usually interested in "objective grading", he doesn't like the case that a "quiz" can not have new questions added once someone has "done the test". On the other hand possibilities of increasing the feedback/input of the students are exciting. Students should always be able to [[https://docs.moodle.org/en/Feedback_link_for_all_elements#Quiz_questions_.28even_lesson_and_feedback_and_questionnaire.3F.29 comment on the questions]] and ideally be able to suggest other answer alternatives (with their motivations) that could be, ideally, automatically, on teacher review, added to the questions. As a help in emphasizing that his interactive questions are a "work in progress", the student interface should be able to turn off or play down points (example: '''''preliminary''' points 0.6 of 1.0'').

Even if his interest in grades is minimal, a CBG, "certainty based grading", system is an exciting way of getting certainty feedback about the things that students are unsure of (in spite of giving what they think/hope is the right answer) as well as pointing out the seriousness of being very sure of something wrong. This kind of information is important for developing the teaching (including new interactive questions).

----

=== Fred ===
* '''Fred's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

'''We are still missing quite a few details about Fred. If you are or know a Fred, please tell us about him.'''

Fred is an teacher of an advanced computer sciences course. Several other courses are prerequisites for his course.

====Exams: why and what====
Fred's exams are:
* At the beginning of a course,
* for evaluating the starting level of the students on his course;
** formative assessment of understanding of the meaning of basic concepts
** Multiple choice
* given periodically to assess the class's ability to grasp new concepts
** multiple choice
** free response (code response)

'''Fred '''uses an exam to '''evaluate the starting level for his students'''. Some students are given a grade for the whole course just for the exam if they do well enough.

----

===Susan Support===
* '''Susan's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Susan works in the IT support of her school.

She does introductory lectures about Moodle and other learning environments. She also tutors teachers hand-by-hand, and the drill is roughly as follows:
* Explain basic principles of Quiz
* Go through the help material
* Teach the question bank separately at first, before looking at the Quiz, it is easier to digest this way (Since Moodle 1.6)
* Make an sample exam

Teachers also send her their exams so that she would make the exam for them.

----

===Grace Grader===
* '''Grace's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Is a trainee and only does [[Quiz UI redesign scenarios - Grading et Feedback|grading]] according to instructions.

Quiz UI redesign scenarios - Entering questions into an exam

2009-03-12T09:48:05Z

Jeffjeff: /* Paul Pedagogue */ first draft

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where you can comment on the scenarios]

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]

==Organizing questions and adding them into an exam==

Unless exams are very simple, teachers often have a natural set of categories to organize questions in, from course curriculum or from the actual material of the course. In tech-speak, this can be regarded as metadata of the actual questions.
----
=== Mack Marketing ===
* [[Quiz_UI_redesign_scenarios_-_Background#Mack_Marketing|Mack's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* '''Mack: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
* carefully crafted, can't reuse the questions (but may be able to use for reference)

Mack primarily creates just exams. For him, exams are one-time wholes and each question is a part of the whole, tightly bound to that one exam.

However, Mack can use his questions as templates, or at least sources for inspiration when creating new exams. Thus he keeps a copy all his exams, sorted by date. If he had time, he sometimes wonders, he would also create an thematical index of all the questions, so that whenever he needed a question from a certain theme, he could just look at the index and see which exams contained a question he needs.

Sometimes Mack finds himself in a real hurry when a student wants a renewal (another try or another, separate date for taking the exam for the first time) for an exam, since some, even the more rarely taken exams he has don't have question banks. In this case, he might be willing to compromise how unique wholes his exams are, so that he can have a question bank full of spare questions, ready for use for surprising renewal exams.
----

=== Ida Informatics ===
* [[Quiz_UI_redesign_scenarios_-_Background#Ida_Informatics|Ida's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* '''Ida: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
Based on the themes that come from course curriculum, Ida has organized her questions in categories. Under most themes she has further divided the questions into different difficulty levels and for most a given exam, she usually has about 60% of easier and 40% of more difficult questions. Since she has a lot (i.e. dozens ... thousands) of questions for each course, she would find it useful to categorize questions based on different criterias – that is, to be capable of tagging questions, as well as searching for particular questions based on their content, tags or other properties.

To distribute questions according to a formula, Ida would like to be able to create a test of 100 questions dispersed something like the following: 10% of questions from category 1, 15% of questions from category 2, 25% of questions from category 3, etc. The questions would randomly appear on the exam. BUT on the other hand, there are times when Ida wants to select the exact questions for the exam and even place them in exact spots on the exam. OR select the exact questions but have Moodle randomly distribute them on the exam. (Do you think she'll ever be satisfied???)

=== Harvey Historian ===
* [[Quiz_UI_redesign_scenarios_-_Background#Harvey_Historian|Harvey's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* '''Harvey: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
Each time the exam is taken, Harvey picks one or two questions for each exam book from his comprehensive selection of questions.
----
===Grace Grader===
* [[Quiz_UI_redesign_scenarios_-_Background#Grace_Grader|Grace's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* '''Grace: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
----
===Susan Support===
* [[Quiz_UI_redesign_scenarios_-_Background#Susan_Support|Susan's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* '''Susan: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
--------

=== Fred ===
* [[Quiz_UI_redesign_scenarios_-_Background#Fred|Fred's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* '''Fred: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
----
===Paul Pedagogue===
* [[Quiz_UI_redesign_scenarios_-_Background#Paul_Pedagogue|Paul's persona]]
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Paul_Pedagogue|Paul: Preparing questions]]
* '''Paul: Organizing questions and adding them into an exam'''
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

====National Exam====
As I've pointed out on other pages, Paul uses National Central Exams on paper for grading. He doesn't change them unless there is some unusual imbalance in material coverage that would be unfair to his students.

====Organizing Interactive ("quiz") Questions====
He and several other colleagues like to share resources like quiz questions. They try to organize them in categories that will help. But being able to tag them would be much more flexible. There are many questions that can be useful in many different subjects and levels. A good search function and a meta marking system would also be valuable.

====Random questions====
Since Paul's big interest is not absolute grading he is not especially interested in random questions (which are usually used to making cheating more difficult). He might have some training quizzes toward the end of sections/courses that could randomly choose some questions out of a larger bank.
----

Quiz UI redesign scenarios - Preparing questions

2009-03-12T09:32:28Z

Jeffjeff: /* Paul Pedagogue */ first draft

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]

== Preparing questions ==
Users of various question types: your feedback is especially needed at this point. Which goals, pedagogical or other kinds, make you choose different question types for your exams?

=== Mack Marketing===
* [[Quiz_UI_redesign_scenarios_-_Background#Mack_Marketing|Mack's persona]]
* '''Mack: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

* Questions come from a common question bank

====Prepreparation: sketching questions====
Sketching: While designing the course itself and its topics, at least for the first time, Mack often does '''early "sketching" of his marketing exam questions''' for at least some of the topics. When a new course instance starts to which new themes have been added, he may return to this sketching phase.

====Material for questions====
The exam is '''structured according to the curriculum''', making sure that every essential subject field has been covered.

'''To determine a reasonable difficulty level for the exam''' (and to get feedback about his classroom teaching), he also uses assignments during the course. The exam content also varies based on which themes Mack has had time for/covered during a given course instance.

When designing the exam and its questions, Mack then takes
* Literary on course themes;
** scientific articles, textbooks etc.
** Web pages (does multitasking while having the exam making application, such as Word, open in another window)
** Current, topical articles from newspapers
* Question sketches (see above)
* The course curriculum which
**lists course literature
**gives a broad structure of the course.
* Question metadata (see below)

He will use some of these also as '''material students need to read before or during the exam''' and refer to in their exam answers.

=====Questions from students=====
Mack has also asked students to create relevant exam questions during the course, and some of the questions he has gotten have proven surprisingly good. He then has taken the best ones and imported them to his question bank (which is a Word document).

=====Question metadata=====
To make decisions about what to actually use in his exam, Mack uses any information about the past usefulness of individual questions.

The school Mack teaches in has a common question bank and thus some of the '''questions he uses may have been written by someone else''' than him so any background information to understand the motives behind the question and how it has worked is needed.

With old/existing exams, questions, exercises, Mack uses the following kinds of metadata (information ''about'' questions)
* Feedback about issues '''from students''' (for example, in understanding a question)
**As some exams occur only once per semester or sometimes even more rarely, he tends to forget to take into account student feedback, which he reseives currently usually by email or in the classroom. Sometimes students also give feedback about questions in their exam answers.
* '''issues''' discovered while reading students' exam answers (for example, the student has misunderstood the question and answered to a different question or from a wrong point of view)
* '''statistical data''' about how students have done in previous exams replying a certain question
**In addition to written feedback, Mack has also graded students' answers to his own questions from previous semesters numerically, so he can use '''statistical tools to get an idea about which questions have worked before'''.

While creating a question, Mack notes down two more pieces of metadata:
* '''a model/template answer''' to use as grading criteria when he gets the exam answers from students.
* '''feedback templates''', which he can easily customize in order to give qualitative feedback to students about their exam answers

This information about the questions (metadata) would need to be stored with the questions themselves so that they would be easy to find where they are needed.

====Writing questions====

When Mack has all the material there, he starts to write questions for each of the course themes/topics based on them. He does this category by category, trying to balance each category appropriately. Mack's questions are often based on real-world cases, which the student needs to apply his knowledge to.

Based on how the course has gone that time, he looks at what has seemed relevant that time around - the courses themselves can be very dynamic. He then makes the selection of some of the questions based on that.

''While creating a question, Mack notes down another piece of metadata: '''a model/template answer to use as grading criteria when he gets the exam answers from students'''.''

=== Ida Informatics ===

* [[Quiz_UI_redesign_scenarios_-_Background#Ida_Informatics|Ida's persona]]
* '''Ida: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

====Material====
To create her extensive question banks, Ida uses
* existing questions as templates,
** she has written notes (metadata) about the usefulness of those questions alongside the questions
* the material she has covered during the lectures, which is a fairly '''standardized selection of themes''' since the exam results need to be comparable across semesters.

Ida needs to, however, adjust the selections of questions in different themes, as well as the number of questions from different categories, based on also the statistics she can see about how well students have done with different questions.

=====Question metadata=====
Ida also edits her question material outside Moodle - exporting and importing questions as needed – since the web interface does not provide functionality such as question bank wide text search and replace. It is thus crucial for her that the import/export mechanism works without error, or at least reports to her when some part of her data could not be preserved. It would be important to preserve question metadata even when it is exported and imported.

Even though her questions are selected randomly from categories to exams, she sometimes wants to primarily use questions that have not been seen by students before, to gather data about how well those questions work. For this, she needs to know when a question has been used.

She gets feedback about the exam from students, but since she does not use any question types where students could freely type their answers, he has not gotten feedback about exam questions with the exams. Also feedback about how a student has understood a question in the first place is useful, since if a student has answered incorrectly but the answer makes sense in relation to how the student has understood the question, it might still be just to give him/her a good grade for that question.

====Writing questions====
Ida considers exam and question design very intensive and solitary work, where she wants to lock herself out from the world around her for one or more days, possibly not even answering the phone.

As a consequence, does this fairly seldom, once per year or possibly not even that often, and wants to get a fairly finalized set of questions as a result.

She takes her early ideas about the course themes and possible multiple choice answer options she may have come to think of while designing the course content. She then tries to create a sufficient amount of multiple choices for each theme, one theme at a time, for the different difficulty levels. The most important considerations are to make sure the questions are '''not obvious or ambiguous''' and that they '''are valid, actually test the understanding they are designed to test'''. She also needs to check for any typos or wrong/imprecise terms, since they can affect how well students understand questions.

For Ida, the central resource is the question bank, which she tries to complete at once and then refine later. However, since the work demands full attention, she finds it hard to find time, in the middle of everyday work, to orient herself to this refining and editing the questions based on feedback. Here, the tools she uses could perhaps support her memory so that there would be less of a barrier to get the refining done.

For Ida, an essential goal is to be capable to perceive all of the questions of a given theme at once, in order to evaluate whether the theme has been covered to a sufficient degree. If she decides it hasn't, she can then add new questions from a new angle to her collection.

====Conclusions====
Teachers could have an option to have a “give feedback about this question” button in the exam with each/some of the questions which would give a separate writing area for the student to write in. This would allow student-entered metadata to be stored with the questions and presented to the teacher in context.

----

===Harvey Historian===
* [[Quiz_UI_redesign_scenarios_-_Background#Harvey_Historian|Harvey's persona]]
* '''Harvey: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey has a history exam that every student needs to take at some point of their studies. The exam covers five books, which are all in the status of a classic in Harvey's subject field. There are only so many questions that can be made from each of these books, and as Harvey has taught the course for two decades, he has managed to gather pretty much all variations of them.

Harvey has also given some of his courses in other languages than his native one, while doing teacher exchange. For those exams, he has translated and adapted his questions to fit the local cultural context. However, the information about the questions (metadata) is shared between different exams, and also the grading is made comparable between different-language exams. The students choose when they come to the exam, which language they want to take the language in.

----

=== Fred ===
* [[Quiz_UI_redesign_scenarios_-_Background#Fred|Fred's persona]]
* '''Fred: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

As Fred's course is an advanced one with other courses as prerequisites, he has a test at the beginning of a course to see what is the general level of a given year's students. In this test, he uses some of the more difficult questions of the exams of the prerequisite courses.

He also uses the prerequisite courses' exams for reference, since those exams demonstrate quite well which subjects have been taught on those courses - he is painfully aware that the official curriculum often does not correspond tightly with the actual course content.
----

===Grace Grader===
* [[Quiz_UI_redesign_scenarios_-_Background#Grace_Grader|Grace's persona]]
* '''Grace: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

----
===Susan Support===
* [[Quiz_UI_redesign_scenarios_-_Background#Susan_Support|Susan's persona]]
* '''Susan: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

----
===Paul Pedagogue===
* [[Quiz_UI_redesign_scenarios_-_Background#Paul_Pedagogue|Paul's persona]]
* '''Paul: Preparing questions'''
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Paul_Pedagogue|Paul: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

====National Exam grading cop-out====
If Paul were to use his own high goals, too many students would get low grades compared to if they went to other teachers, but he doesn't want to just give arbitrary "nice grades". Since Paul finds grading to be "not his thing", he chooses to use the National Central Exams as the basis for his grading. They have a well-defined grading scheme and the questions are often quite varied and interesting (to him, though students tend to prefer the usual "cook book problems"). The exams are given on paper, in a controlled room at the end of the course and he grades them according to the official scheme and uses that grade for the whole course unless some other interaction with the student has given him reason to adjust that grade. He saves notes on various student answers so that he can give tutorial feedback when these tests are not longer secret and can be used for practice. He always offers to students to have an oral session after the exam if the student wishes it. (Some students can get exam black-outs, be better at talking than writing etc.) He hopes that there will be acceptably secure on line testing alternatives in the future, since he finds correcting work uninspiring.

====Practice past National Exams====
Since both he and the students want them to perform well on the National exams, he starts by making practice questions based on the previous exams. He tries to provide tutorial process feedback for as many answers as possible. Multiple choice questions are easy. More open questions can be more challenging to make into automatic correction questions. Sometimes they have to be broken up into a suite of questions checking whether the student can do the various steps in the process. Though he presents the whole (previous) exams as a package to accustom the students to how the exams work, he would like to be able to tag the questions to be easily found and used in different parts of the course.

====Preparing for laboratory====
His distance students have to do a large number of laboratory exercises during a single day at the school. To make sure that they can get as much as possible out of the economically limited time, there are some things they must do before coming to a lab day. They must read descriptions about safety routines, how to make and record measurements. There are also some laboratory videos and Java and FLASH simulations they are to do some exercises on. To make sure they have understood the reading and the simulations they have to do a quiz. It will even include screenshots from the videos/simulations in the follow-up questions. There can be some theoretical readings and exercises that should best be mastered before the labs also. The questions following up these things would typically have feedback with links back to the materials/simulations to be mastered. Students would be encouraged to contact the teacher if the material isn't helping. (The teacher has a channel through the quiz statistics to see where the weaknesses are otherwise.)

====General Follow-ups====
Especially for distance students it is important to give many checkpoints to see if the steps toward the goal are being met. Paul will try to have a couple questions on each Internet page that the student can use to check that. They will usually just have a popup answer. It would be really nice to be able to '''pull in a couple of Moodle quiz questions into any page'''. Then you could have a rich feedback opportunity and be able to collect student answers so you can improve the material.

----

Quiz UI redesign scenarios - Preparing questions

2009-03-12T07:25:11Z

Jeffjeff: /* Material for questions */

Quiz UI redesign scenarios - Entering questions into an exam

2009-03-12T07:11:53Z

Jeffjeff: /* Ida Informatics */

User:Jeff Forssell

2009-03-12T07:03:37Z

Jeffjeff:

Quiz UI redesign scenarios - Background

2009-03-12T06:52:48Z

Jeffjeff: /* Paul Pedagogue */ link to jeff

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where scenarios were commented. Now inactive, please edit this wiki directly or create a new thread].

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]
== Background/Personas ==

An exam can be used at the beginning of a course to evaluate the initial level of the students' understanding, during the course to provide checkpoints for students, or at the end of the course. In any case, quizzes/exams often have a dual role: they give the teacher information about the student's skills, but more '''importantly they are a pedagogical tool: while answering, a student generates knowledge and learns in the process'''.

'''If it seems to you that your idea of using exam is very different from any of the current personas, you may add a new one or [http://moodle.org/mod/forum/discuss.php?d=99351 tell us about this in the forum thread dedicated for persona/scenario discussion]'''

Key questions for this part of the exam making:

* What are the pedagogigal goals for the teacher?
* Does the exam use essay questions or automatically evaluated (or what)?
* What initiates the lifecycle of an exam?

----

=== Mack Marketing ===

* '''Mack's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
''Mack'' is a marketing teacher. He likes golf, but not on rainy days.
* He has quite a bit of experience using the internet, mainly for webmail

He also teaches online a bit, but mostly he has normal classes. In addition to regular, time-limited exams, he also uses take-home exams (with deadlines for the questions/case assignments) and group exams (a group works together on a a case and the work is evaluated).

====Exams: why and what====

While defining the goals of his course, he ends up with the conclusion that an exam with essay questions is needed at the end of the course for evaluating students' learning. He does not think exams are a very good way of evaluating learning, since they mostly evaluate what students happen to remember. And even though he considers grading essay answers an extra workload, he still sees this as the only way '''to effectively evaluate students' conceptual understanding''', especially on his basic courses where there are more students.

He uses quite similar exams on different courses, our focus is on a Introduction to Marketing exam, which is about:

* Evaluating conceptual learning; can apply what they have learned to real-life cases
* Essay questions
* Qualitative feedback; using templates as basis

Hypothesis: Since his exams are so specialized, Mack is afraid that if another teacher takes up his courses, that teacher will not understand all the nuances he has taken into account while designing his exams. That is why he writes notes about each exam whole and archives these notes that explain the choices he has made in the exam design.

====Feedback====
He plans '''to give feedback about students' answers: for him, this enables the exam to work as a pedagogical tool''' and he does not require the exam results to be absolutely comparable with the exam results of courses of previous semesters. He uses some templates for feedback, and based on that he then writes feedback for each student personally.

----

=== Ida Informatics ===

* '''Ida's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

Moodle community contributors to Ida: [http://moodle.org/user/view.php?id=81449&course=5 Gary Anderson]
----

''Ida'' is a teacher in her fifties, still likes teaching but is more-or-less secretly already anticipating her retirement. She teaches informatics. Her teaching responsibilities include lecturing a couple of basic courses with dozens or even some hundreds of students at a time.

====Exams: why and what====
A lot of her time is spent planning '''a course of basic information retrieval to 200 first year students each semester'''.
The exam is about:
* Evaluating learning of basic factual knowledge of her field
* Multiple choice questions
* for '''testing students' basic factual knowledge of her field''' automatically graded questions, such as multiple questions, suffice well enough (versus Mack's essay questions for testing conceptual understanding).
* Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.

Ida sees exams beneficial since they provide '''measurable grounds on which to compare and grade learning''' on a '''numerical (quantitative) scale''' and for quick feedback to students. She wants to have her exam as automated as possible. She does not think exams have much pedagogical purpose, and would actually rather not even grade her students, but since the educational system demands comparable grades, she does it this way.

(In an ideal world, she would instantly start using a tool that would analyze her course material automatically, generate essay questions and then even grade them using linguistic analysis or some other method.)

----

===Harvey Historian===
* '''Harvey's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey is a history professor in a university. He mostly concentrates on his research, but also does some teaching of basic courses he knows by heart, since he has been doing it for a couple of decades.

====Exams: why and what====
He uses exams to test the students' understanding of specific eras from different points of view, and students' ability to see the connections between different events in history.

Sometimes Harvey also takes a year off, and the exam bank is an excellent resource for a less experienced teacher to refer to for finding out what is essential to teach about a course.

====Feedback====
Harvey gives brief feedback to students about what would have made a better grade. Since he has seen most things in the past already, he does not use any feedbacks for template since he has all of it pretty much in the back of his head.

----
===Paul Pedagogue===
* '''Paul's persona'''

* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
Moodle community contributors to Paul: [https://docs.moodle.org/en/User:Jeff_Forssell Jeff Forssell]
----

Paul is almost 60 years old and is interested in all kinds of fields: science, gardening, languages, politics and music. He spends a lot of time on Internet and is delighted that it is now possible for everyone to contribute. He isn't a trained programmer, but is not afraid to dabble in code to fine tune things to the way he wants to work. Though he loves discussing directly with students, he feels that in a dynamic Internet distance course he can have very personal contact with the students. Good interactive questions can give him freedom from repetitious correcting while giving good guidance to students. He has many distance students, but still has some classroom students.

====Grading hinders learning====
Paul uses interactive questions, in "quizzes", primarily as a learning tool. He actually would prefer to have "official evaluation" (grading) done by some other instance, like the higher school or workplace that the student is aspiring to. This would free the teacher from the schizophrenic Helper/Judge dichotomy that prevents students and teachers from really working together in trust toward the common goal: lasting knowledge and methods for the student that will help him/er reach his/er goals. Grading in a class can also lead to diminished student interest in helping one's classmates, though this is one of the best ways to really learn something.

====Questions are seeds for dialogue====
Questions for learning have been the prime tool for many teachers from Socrates to Martin D! Questions usually don't have one RIGHT answer, so it is very important to allow for many alternative answers and this can make pattern matching (like "regular expressions") a high priority. Since the question is used for teaching it is important to make it provide as much dialog as possible. He will generally not like a simplified interface that doesn't have possibilities for rich feedback for varied answers. If possible he will try to give some kind of encouragement/understanding, for common mistakes as well a some '''clue to help''' the student in his '''processing of the problem''' (preferably without giving the "right answer"). His quizzes give students ways they can assess their own learning and, in a problem-based learning style, build up their understanding. If he has distance pupils, he will create many "'''interactive question exercise follow-ups'''" (using quizzes) as soon as he notices things that are common pitfalls. For an answer that isn't the "right one" his '''feedback will tend to be cautious''': "We didn't recognize your answer as being one that we expected. Could you describe how you reached this answer?"

===="Organic growth" of interactive questions====
Since he sees the questions as part of a developing dialog he values any characteristics that increase the dynamics. Since he's not usually interested in "objective grading", he doesn't like the case that a "quiz" can not have new questions added once someone has "done the test". On the other hand possibilities of increasing the feedback/input of the students are exciting. Students should always be able to [[https://docs.moodle.org/en/Feedback_link_for_all_elements#Quiz_questions_.28even_lesson_and_feedback_and_questionnaire.3F.29 comment on the questions]] and ideally be able to suggest other answer alternatives (with their motivations) that could be, ideally, automatically, on teacher review, added to the questions. As a help in emphasizing that his interactive questions are a "work in progress", the student interface should be able to turn off or play down points (example: '''''preliminary''' points 0.6 of 1.0'').

Even if his interest in grades is minimal, a CBG, "certainty based grading", system is an exciting way of getting certainty feedback about the things that students are unsure of (in spite of giving what they think/hope is the right answer) as well as pointing out the seriousness of being very sure of something wrong. This kind of information is important for developing the teaching (including new interactive questions).

----

=== Fred ===
* '''Fred's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

'''We are still missing quite a few details about Fred. If you are or know a Fred, please tell us about him.'''

Fred is an teacher of an advanced computer sciences course. Several other courses are prerequisites for his course.

====Exams: why and what====
Fred's exams are:
* At the beginning of a course,
* for evaluating the starting level of the students on his course;
** formative assessment of understanding of the meaning of basic concepts
** Multiple choice
* given periodically to assess the class's ability to grasp new concepts
** multiple choice
** free response (code response)

'''Fred '''uses an exam to '''evaluate the starting level for his students'''. Some students are given a grade for the whole course just for the exam if they do well enough.

----

===Susan Support===
* '''Susan's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Susan works in the IT support of her school.

She does introductory lectures about Moodle and other learning environments. She also tutors teachers hand-by-hand, and the drill is roughly as follows:
* Explain basic principles of Quiz
* Go through the help material
* Teach the question bank separately at first, before looking at the Quiz, it is easier to digest this way (Since Moodle 1.6)
* Make an sample exam

Teachers also send her their exams so that she would make the exam for them.

----

===Grace Grader===
* '''Grace's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Is a trainee and only does [[Quiz UI redesign scenarios - Grading et Feedback|grading]] according to instructions.

Quiz UI redesign scenarios - Background

2009-03-12T06:50:10Z

Jeffjeff: /* Paul Pedagogue */ persona draft

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where scenarios were commented. Now inactive, please edit this wiki directly or create a new thread].

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]
== Background/Personas ==

An exam can be used at the beginning of a course to evaluate the initial level of the students' understanding, during the course to provide checkpoints for students, or at the end of the course. In any case, quizzes/exams often have a dual role: they give the teacher information about the student's skills, but more '''importantly they are a pedagogical tool: while answering, a student generates knowledge and learns in the process'''.

'''If it seems to you that your idea of using exam is very different from any of the current personas, you may add a new one or [http://moodle.org/mod/forum/discuss.php?d=99351 tell us about this in the forum thread dedicated for persona/scenario discussion]'''

Key questions for this part of the exam making:

* What are the pedagogigal goals for the teacher?
* Does the exam use essay questions or automatically evaluated (or what)?
* What initiates the lifecycle of an exam?

----

=== Mack Marketing ===

* '''Mack's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
''Mack'' is a marketing teacher. He likes golf, but not on rainy days.
* He has quite a bit of experience using the internet, mainly for webmail

He also teaches online a bit, but mostly he has normal classes. In addition to regular, time-limited exams, he also uses take-home exams (with deadlines for the questions/case assignments) and group exams (a group works together on a a case and the work is evaluated).

====Exams: why and what====

While defining the goals of his course, he ends up with the conclusion that an exam with essay questions is needed at the end of the course for evaluating students' learning. He does not think exams are a very good way of evaluating learning, since they mostly evaluate what students happen to remember. And even though he considers grading essay answers an extra workload, he still sees this as the only way '''to effectively evaluate students' conceptual understanding''', especially on his basic courses where there are more students.

He uses quite similar exams on different courses, our focus is on a Introduction to Marketing exam, which is about:

* Evaluating conceptual learning; can apply what they have learned to real-life cases
* Essay questions
* Qualitative feedback; using templates as basis

Hypothesis: Since his exams are so specialized, Mack is afraid that if another teacher takes up his courses, that teacher will not understand all the nuances he has taken into account while designing his exams. That is why he writes notes about each exam whole and archives these notes that explain the choices he has made in the exam design.

====Feedback====
He plans '''to give feedback about students' answers: for him, this enables the exam to work as a pedagogical tool''' and he does not require the exam results to be absolutely comparable with the exam results of courses of previous semesters. He uses some templates for feedback, and based on that he then writes feedback for each student personally.

----

=== Ida Informatics ===

* '''Ida's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

Moodle community contributors to Ida: [http://moodle.org/user/view.php?id=81449&course=5 Gary Anderson]
----

''Ida'' is a teacher in her fifties, still likes teaching but is more-or-less secretly already anticipating her retirement. She teaches informatics. Her teaching responsibilities include lecturing a couple of basic courses with dozens or even some hundreds of students at a time.

====Exams: why and what====
A lot of her time is spent planning '''a course of basic information retrieval to 200 first year students each semester'''.
The exam is about:
* Evaluating learning of basic factual knowledge of her field
* Multiple choice questions
* for '''testing students' basic factual knowledge of her field''' automatically graded questions, such as multiple questions, suffice well enough (versus Mack's essay questions for testing conceptual understanding).
* Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.

Ida sees exams beneficial since they provide '''measurable grounds on which to compare and grade learning''' on a '''numerical (quantitative) scale''' and for quick feedback to students. She wants to have her exam as automated as possible. She does not think exams have much pedagogical purpose, and would actually rather not even grade her students, but since the educational system demands comparable grades, she does it this way.

(In an ideal world, she would instantly start using a tool that would analyze her course material automatically, generate essay questions and then even grade them using linguistic analysis or some other method.)

----

===Harvey Historian===
* '''Harvey's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey is a history professor in a university. He mostly concentrates on his research, but also does some teaching of basic courses he knows by heart, since he has been doing it for a couple of decades.

====Exams: why and what====
He uses exams to test the students' understanding of specific eras from different points of view, and students' ability to see the connections between different events in history.

Sometimes Harvey also takes a year off, and the exam bank is an excellent resource for a less experienced teacher to refer to for finding out what is essential to teach about a course.

====Feedback====
Harvey gives brief feedback to students about what would have made a better grade. Since he has seen most things in the past already, he does not use any feedbacks for template since he has all of it pretty much in the back of his head.

----
===Paul Pedagogue===
* '''Paul's persona'''

* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
* Moodle community contributor for Paul Pedagogue Jeff Forssell
----

Paul is almost 60 years old and is interested in all kinds of fields: science, gardening, languages, politics and music. He spends a lot of time on Internet and is delighted that it is now possible for everyone to contribute. He isn't a trained programmer, but is not afraid to dabble in code to fine tune things to the way he wants to work. Though he loves discussing directly with students, he feels that in a dynamic Internet distance course he can have very personal contact with the students. Good interactive questions can give him freedom from repetitious correcting while giving good guidance to students. He has many distance students, but still has some classroom students.

====Grading hinders learning====
Paul uses interactive questions, in "quizzes", primarily as a learning tool. He actually would prefer to have "official evaluation" (grading) done by some other instance, like the higher school or workplace that the student is aspiring to. This would free the teacher from the schizophrenic Helper/Judge dichotomy that prevents students and teachers from really working together in trust toward the common goal: lasting knowledge and methods for the student that will help him/er reach his/er goals. Grading in a class can also lead to diminished student interest in helping one's classmates, though this is one of the best ways to really learn something.

====Questions are seeds for dialogue====
Questions for learning have been the prime tool for many teachers from Socrates to Martin D! Questions usually don't have one RIGHT answer, so it is very important to allow for many alternative answers and this can make pattern matching (like "regular expressions") a high priority. Since the question is used for teaching it is important to make it provide as much dialog as possible. He will generally not like a simplified interface that doesn't have possibilities for rich feedback for varied answers. If possible he will try to give some kind of encouragement/understanding, for common mistakes as well a some '''clue to help''' the student in his '''processing of the problem''' (preferably without giving the "right answer"). His quizzes give students ways they can assess their own learning and, in a problem-based learning style, build up their understanding. If he has distance pupils, he will create many "'''interactive question exercise follow-ups'''" (using quizzes) as soon as he notices things that are common pitfalls. For an answer that isn't the "right one" his '''feedback will tend to be cautious''': "We didn't recognize your answer as being one that we expected. Could you describe how you reached this answer?"

===="Organic growth" of interactive questions====
Since he sees the questions as part of a developing dialog he values any characteristics that increase the dynamics. Since he's not usually interested in "objective grading", he doesn't like the case that a "quiz" can not have new questions added once someone has "done the test". On the other hand possibilities of increasing the feedback/input of the students are exciting. Students should always be able to [[https://docs.moodle.org/en/Feedback_link_for_all_elements#Quiz_questions_.28even_lesson_and_feedback_and_questionnaire.3F.29 comment on the questions]] and ideally be able to suggest other answer alternatives (with their motivations) that could be, ideally, automatically, on teacher review, added to the questions. As a help in emphasizing that his interactive questions are a "work in progress", the student interface should be able to turn off or play down points (example: '''''preliminary''' points 0.6 of 1.0'').

Even if his interest in grades is minimal, a CBG, "certainty based grading", system is an exciting way of getting certainty feedback about the things that students are unsure of (in spite of giving what they think/hope is the right answer) as well as pointing out the seriousness of being very sure of something wrong. This kind of information is important for developing the teaching (including new interactive questions).

----

=== Fred ===
* '''Fred's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

'''We are still missing quite a few details about Fred. If you are or know a Fred, please tell us about him.'''

Fred is an teacher of an advanced computer sciences course. Several other courses are prerequisites for his course.

====Exams: why and what====
Fred's exams are:
* At the beginning of a course,
* for evaluating the starting level of the students on his course;
** formative assessment of understanding of the meaning of basic concepts
** Multiple choice
* given periodically to assess the class's ability to grasp new concepts
** multiple choice
** free response (code response)

'''Fred '''uses an exam to '''evaluate the starting level for his students'''. Some students are given a grade for the whole course just for the exam if they do well enough.

----

===Susan Support===
* '''Susan's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Susan works in the IT support of her school.

She does introductory lectures about Moodle and other learning environments. She also tutors teachers hand-by-hand, and the drill is roughly as follows:
* Explain basic principles of Quiz
* Go through the help material
* Teach the question bank separately at first, before looking at the Quiz, it is easier to digest this way (Since Moodle 1.6)
* Make an sample exam

Teachers also send her their exams so that she would make the exam for them.

----

===Grace Grader===
* '''Grace's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Is a trainee and only does [[Quiz UI redesign scenarios - Grading et Feedback|grading]] according to instructions.

Quiz UI redesign scenarios - Background

2009-03-12T05:47:10Z

Jeffjeff: /* Paul Pedagogue */ headings and other changes

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where scenarios were commented. Now inactive, please edit this wiki directly or create a new thread].

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]
== Background/Personas ==

An exam can be used at the beginning of a course to evaluate the initial level of the students' understanding, during the course to provide checkpoints for students, or at the end of the course. In any case, quizzes/exams often have a dual role: they give the teacher information about the student's skills, but more '''importantly they are a pedagogical tool: while answering, a student generates knowledge and learns in the process'''.

'''If it seems to you that your idea of using exam is very different from any of the current personas, you may add a new one or [http://moodle.org/mod/forum/discuss.php?d=99351 tell us about this in the forum thread dedicated for persona/scenario discussion]'''

Key questions for this part of the exam making:

* What are the pedagogigal goals for the teacher?
* Does the exam use essay questions or automatically evaluated (or what)?
* What initiates the lifecycle of an exam?

----

=== Mack Marketing ===

* '''Mack's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
''Mack'' is a marketing teacher. He likes golf, but not on rainy days.
* He has quite a bit of experience using the internet, mainly for webmail

He also teaches online a bit, but mostly he has normal classes. In addition to regular, time-limited exams, he also uses take-home exams (with deadlines for the questions/case assignments) and group exams (a group works together on a a case and the work is evaluated).

====Exams: why and what====

While defining the goals of his course, he ends up with the conclusion that an exam with essay questions is needed at the end of the course for evaluating students' learning. He does not think exams are a very good way of evaluating learning, since they mostly evaluate what students happen to remember. And even though he considers grading essay answers an extra workload, he still sees this as the only way '''to effectively evaluate students' conceptual understanding''', especially on his basic courses where there are more students.

He uses quite similar exams on different courses, our focus is on a Introduction to Marketing exam, which is about:

* Evaluating conceptual learning; can apply what they have learned to real-life cases
* Essay questions
* Qualitative feedback; using templates as basis

Hypothesis: Since his exams are so specialized, Mack is afraid that if another teacher takes up his courses, that teacher will not understand all the nuances he has taken into account while designing his exams. That is why he writes notes about each exam whole and archives these notes that explain the choices he has made in the exam design.

====Feedback====
He plans '''to give feedback about students' answers: for him, this enables the exam to work as a pedagogical tool''' and he does not require the exam results to be absolutely comparable with the exam results of courses of previous semesters. He uses some templates for feedback, and based on that he then writes feedback for each student personally.

----

=== Ida Informatics ===

* '''Ida's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

Moodle community contributors to Ida: [http://moodle.org/user/view.php?id=81449&course=5 Gary Anderson]
----

''Ida'' is a teacher in her fifties, still likes teaching but is more-or-less secretly already anticipating her retirement. She teaches informatics. Her teaching responsibilities include lecturing a couple of basic courses with dozens or even some hundreds of students at a time.

====Exams: why and what====
A lot of her time is spent planning '''a course of basic information retrieval to 200 first year students each semester'''.
The exam is about:
* Evaluating learning of basic factual knowledge of her field
* Multiple choice questions
* for '''testing students' basic factual knowledge of her field''' automatically graded questions, such as multiple questions, suffice well enough (versus Mack's essay questions for testing conceptual understanding).
* Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.

Ida sees exams beneficial since they provide '''measurable grounds on which to compare and grade learning''' on a '''numerical (quantitative) scale''' and for quick feedback to students. She wants to have her exam as automated as possible. She does not think exams have much pedagogical purpose, and would actually rather not even grade her students, but since the educational system demands comparable grades, she does it this way.

(In an ideal world, she would instantly start using a tool that would analyze her course material automatically, generate essay questions and then even grade them using linguistic analysis or some other method.)

----

===Harvey Historian===
* '''Harvey's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey is a history professor in a university. He mostly concentrates on his research, but also does some teaching of basic courses he knows by heart, since he has been doing it for a couple of decades.

====Exams: why and what====
He uses exams to test the students' understanding of specific eras from different points of view, and students' ability to see the connections between different events in history.

Sometimes Harvey also takes a year off, and the exam bank is an excellent resource for a less experienced teacher to refer to for finding out what is essential to teach about a course.

====Feedback====
Harvey gives brief feedback to students about what would have made a better grade. Since he has seen most things in the past already, he does not use any feedbacks for template since he has all of it pretty much in the back of his head.

----
===Paul Pedagogue===
* '''Paul's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Paul_Pedagogue|Paul: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Paul_Pedagogue|Paul: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Paul's profile was proposed by [http://moodle.org/mod/forum/discuss.php?d=99351#p439124 Joseph Rézeau in the forums].

====Grading hinders learning====
Paul uses interactive questions, in "quizzes", primarily as a learning tool. He actually would prefer to have "official evaluation" (grading) done by some other instance, like the higher school or workplace that the student is aspiring to. This would free the teacher from the schizophrenic Helper/Judge dichotomy that prevents students and teachers from really working together in trust toward the common goal: lasting knowledge and methods for the student that will help him/er reach his/er goals. Grading in a class can also lead to diminished student interest in helping one's classmates, though this is one of the best ways to really learn something.

====Questions are seeds for dialogue====
Questions for learning have been the prime tool for many teachers from Socrates to Martin D! Questions usually don't have one RIGHT answer, so it is very important to allow for many alternative answers and this can make pattern matching (like "regular expressions") a high priority. Since the question is used for teaching it is important to make it provide as much dialog as possible. He will generally not like a simplified interface that doesn't have possibilities for rich feedback for varied answers. If possible he will try to give some kind of encouragement/understanding, for common mistakes as well a some '''clue to help''' the student in his '''processing of the problem''' (preferably without giving the "right answer"). His quizzes give students ways they can assess their own learning and, in a problem-based learning style, build up their understanding. If he has distance pupils, he will create many "'''interactive question exercise follow-ups'''" (using quizzes) as soon as he notices things that are common pitfalls. For an answer that isn't the "right one" his '''feedback will tend to be cautious''': "We didn't recognize your answer as being one that we expected. Could you describe how you reached this answer?"

===="Organic growth" of interactive questions====
Since he sees the questions as part of a developing dialog he values any characteristics that increase the dynamics. Since he's not usually interested in "objective grading", he doesn't like the case that a "quiz" can not have new questions added once someone has "done the test". On the other hand possibilities of increasing the feedback/input of the students are exciting. Students should always be able to [[https://docs.moodle.org/en/Feedback_link_for_all_elements#Quiz_questions_.28even_lesson_and_feedback_and_questionnaire.3F.29 comment on the questions]] and ideally be able to suggest other answer alternatives (with their motivations) that could be, ideally, automatically, on teacher review, added to the questions. As a help in emphasizing that his interactive questions are a "work in progress", the student interface should be able to turn off or play down points (example: '''''preliminary''' points 0.6 of 1.0'').

Even if his interest in grades is minimal, a CBG, "certainty based grading", system is an exciting way of getting certainty feedback about the things that students are unsure of (in spite of giving what they think/hope is the right answer) as well as pointing out the seriousness of being very sure of something wrong. This kind of information is important for developing the teaching (including new interactive questions).

----

=== Fred ===
* '''Fred's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

'''We are still missing quite a few details about Fred. If you are or know a Fred, please tell us about him.'''

Fred is an teacher of an advanced computer sciences course. Several other courses are prerequisites for his course.

====Exams: why and what====
Fred's exams are:
* At the beginning of a course,
* for evaluating the starting level of the students on his course;
** formative assessment of understanding of the meaning of basic concepts
** Multiple choice
* given periodically to assess the class's ability to grasp new concepts
** multiple choice
** free response (code response)

'''Fred '''uses an exam to '''evaluate the starting level for his students'''. Some students are given a grade for the whole course just for the exam if they do well enough.

----

===Susan Support===
* '''Susan's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Susan works in the IT support of her school.

She does introductory lectures about Moodle and other learning environments. She also tutors teachers hand-by-hand, and the drill is roughly as follows:
* Explain basic principles of Quiz
* Go through the help material
* Teach the question bank separately at first, before looking at the Quiz, it is easier to digest this way (Since Moodle 1.6)
* Make an sample exam

Teachers also send her their exams so that she would make the exam for them.

----

===Grace Grader===
* '''Grace's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Is a trainee and only does [[Quiz UI redesign scenarios - Grading et Feedback|grading]] according to instructions.

Quiz UI redesign scenarios - Background

2009-03-11T13:52:24Z

Jeffjeff: /* Paul Pedagogue */ feedback, dialog, student process support, grading not his favorite

Back to [[Quiz_UI_redesign_scenarios_-_Scenarios_index|Scenarios index]]

[http://moodle.org/mod/forum/discuss.php?d=99351 Discussion thread, where scenarios were commented. Now inactive, please edit this wiki directly or create a new thread].

[http://www.youtube.com/watch?v=hyfLOSadJW4 Introductory video]
== Background/Personas ==

An exam can be used at the beginning of a course to evaluate the initial level of the students' understanding, during the course to provide checkpoints for students, or at the end of the course. In any case, quizzes/exams often have a dual role: they give the teacher information about the student's skills, but more '''importantly they are a pedagogical tool: while answering, a student generates knowledge and learns in the process'''.

'''If it seems to you that your idea of using exam is very different from any of the current personas, you may add a new one or [http://moodle.org/mod/forum/discuss.php?d=99351 tell us about this in the forum thread dedicated for persona/scenario discussion]'''

Key questions for this part of the exam making:

* What are the pedagogigal goals for the teacher?
* Does the exam use essay questions or automatically evaluated (or what)?
* What initiates the lifecycle of an exam?

----

=== Mack Marketing ===

* '''Mack's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Mack_Marketing|Mack: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Mack_Marketing|Mack: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----
''Mack'' is a marketing teacher. He likes golf, but not on rainy days.
* He has quite a bit of experience using the internet, mainly for webmail

He also teaches online a bit, but mostly he has normal classes. In addition to regular, time-limited exams, he also uses take-home exams (with deadlines for the questions/case assignments) and group exams (a group works together on a a case and the work is evaluated).

====Exams: why and what====

While defining the goals of his course, he ends up with the conclusion that an exam with essay questions is needed at the end of the course for evaluating students' learning. He does not think exams are a very good way of evaluating learning, since they mostly evaluate what students happen to remember. And even though he considers grading essay answers an extra workload, he still sees this as the only way '''to effectively evaluate students' conceptual understanding''', especially on his basic courses where there are more students.

He uses quite similar exams on different courses, our focus is on a Introduction to Marketing exam, which is about:

* Evaluating conceptual learning; can apply what they have learned to real-life cases
* Essay questions
* Qualitative feedback; using templates as basis

Hypothesis: Since his exams are so specialized, Mack is afraid that if another teacher takes up his courses, that teacher will not understand all the nuances he has taken into account while designing his exams. That is why he writes notes about each exam whole and archives these notes that explain the choices he has made in the exam design.

====Feedback====
He plans '''to give feedback about students' answers: for him, this enables the exam to work as a pedagogical tool''' and he does not require the exam results to be absolutely comparable with the exam results of courses of previous semesters. He uses some templates for feedback, and based on that he then writes feedback for each student personally.

----

=== Ida Informatics ===

* '''Ida's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Ida_Informatics|Ida: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Ida_Informatics|Ida: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]

Moodle community contributors to Ida: [http://moodle.org/user/view.php?id=81449&course=5 Gary Anderson]
----

''Ida'' is a teacher in her fifties, still likes teaching but is more-or-less secretly already anticipating her retirement. She teaches informatics. Her teaching responsibilities include lecturing a couple of basic courses with dozens or even some hundreds of students at a time.

====Exams: why and what====
A lot of her time is spent planning '''a course of basic information retrieval to 200 first year students each semester'''.
The exam is about:
* Evaluating learning of basic factual knowledge of her field
* Multiple choice questions
* for '''testing students' basic factual knowledge of her field''' automatically graded questions, such as multiple questions, suffice well enough (versus Mack's essay questions for testing conceptual understanding).
* Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.

Ida sees exams beneficial since they provide '''measurable grounds on which to compare and grade learning''' on a '''numerical (quantitative) scale''' and for quick feedback to students. She wants to have her exam as automated as possible. She does not think exams have much pedagogical purpose, and would actually rather not even grade her students, but since the educational system demands comparable grades, she does it this way.

(In an ideal world, she would instantly start using a tool that would analyze her course material automatically, generate essay questions and then even grade them using linguistic analysis or some other method.)

----

===Harvey Historian===
* '''Harvey's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Harvey_Historian|Harvey: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Harvey_Historian|Harvey: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Harvey is a history professor in a university. He mostly concentrates on his research, but also does some teaching of basic courses he knows by heart, since he has been doing it for a couple of decades.

====Exams: why and what====
He uses exams to test the students' understanding of specific eras from different points of view, and students' ability to see the connections between different events in history.

Sometimes Harvey also takes a year off, and the exam bank is an excellent resource for a less experienced teacher to refer to for finding out what is essential to teach about a course.

====Feedback====
Harvey gives brief feedback to students about what would have made a better grade. Since he has seen most things in the past already, he does not use any feedbacks for template since he has all of it pretty much in the back of his head.

----
===Paul Pedagogue===
* '''Paul's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Paul_Pedagogue|Paul: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Paul_Pedagogue|Paul: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Paul's profile was proposed by [http://moodle.org/mod/forum/discuss.php?d=99351#p439124 Joseph Rézeau in the forums].

Paul uses interactive questions, in "quizzes", primarily as a learning tool. He actually would prefer to have "official evaluation" (grading) done by some other instance, like the higher school or workplace that the student is aspiring to. This would free the teacher from the schizophrenic Helper/Judge dichotomy that prevents students and teachers from really working together in trust toward the common goal: lasting knowledge and methods for the student that will help him/er reach his/er goals.

Questions for learning have been the prime tool for many teachers from Socrates to Martin D! Questions usually don't have one RIGHT answer, so it is very important to allow for many alternative answers and this can make pattern matching (like "regular expressions") a high priority. Since the question is used for teaching it is important to make it provide as much dialog as possible. He will generally not like a simplified interface that doesn't have possibilities for rich feedback for varied answers. If possible he will try to give some kind of encouragement/understanding, for common mistakes as well a some clue to help the student in his processing of the problem (preferably without giving the "right answer"). His quizzes give students ways they can assess their own learning and, in a problem-based learning style, build up their understanding. If he has distance pupils, he will create many "interactive question exercise follow-ups" (using quizzes) as soon as he notices things that are common pitfalls. For an answer that isn't the "right one" his feedback will tend to be cautious: "We didn't recognize your answer as being one that we expected. Could you describe how you reached this answer?"

Since he sees the questions as part of a developing dialog he values any characteristics that increase the dynamics. Since he's not usually interested in "objective grading", he doesn't like the case that a "quiz" can not have new questions added once someone has "done the test". On the other hand possibilities of increasing the feedback/input of the students are exciting. Students should always be able to [[https://docs.moodle.org/en/Feedback_link_for_all_elements#Quiz_questions_.28even_lesson_and_feedback_and_questionnaire.3F.29 comment on the questions]] and ideally be able to suggest other answer alternatives (with their motivations) that could be, ideally, automatically, on teacher review, added to the questions. Even if his interest in grades is minimal, a CBG, "certainty based grading", system is an exciting way of getting feedback about the things that students are unsure of (in spite of giving what they think/hope is the right answer) as well as pointing out the seriousness of being very sure of something wrong.

----

=== Fred ===
* '''Fred's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Fred|Fred: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Fred|Fred: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

'''We are still missing quite a few details about Fred. If you are or know a Fred, please tell us about him.'''

Fred is an teacher of an advanced computer sciences course. Several other courses are prerequisites for his course.

====Exams: why and what====
Fred's exams are:
* At the beginning of a course,
* for evaluating the starting level of the students on his course;
** formative assessment of understanding of the meaning of basic concepts
** Multiple choice
* given periodically to assess the class's ability to grasp new concepts
** multiple choice
** free response (code response)

'''Fred '''uses an exam to '''evaluate the starting level for his students'''. Some students are given a grade for the whole course just for the exam if they do well enough.

----

===Susan Support===
* '''Susan's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Susan_Support|Susan: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Susan_Support|Susan: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Susan works in the IT support of her school.

She does introductory lectures about Moodle and other learning environments. She also tutors teachers hand-by-hand, and the drill is roughly as follows:
* Explain basic principles of Quiz
* Go through the help material
* Teach the question bank separately at first, before looking at the Quiz, it is easier to digest this way (Since Moodle 1.6)
* Make an sample exam

Teachers also send her their exams so that she would make the exam for them.

----

===Grace Grader===
* '''Grace's persona'''
* [[Quiz_UI_redesign_scenarios_-_Preparing_questions#Grace_Grader|Grace: Preparing questions]]
* [[Quiz_UI_redesign_scenarios_-_Entering_questions_into_an_exam#Grace_Grader|Grace: Organizing questions and adding them into an exam]]
* [http://moodle.org/mod/forum/discuss.php?d=99351 Forum discussion to comment on scenarios]
----

Is a trainee and only does [[Quiz UI redesign scenarios - Grading et Feedback|grading]] according to instructions.

Flagging questions during a quiz attempt

2009-02-13T08:22:17Z

Jeffjeff: missing word

{{Moodle 2.0}}

We want students to be able to mark certain questions as they are doing them for whatever purpose, for example:
* because they want to go back and check their answer before submitting the quiz.
* because the results will only be released later, and they particularly want to remember to check whether they got this question right.
* because they want to ask their friend or teacher something relating to that question later.
This should be a simple toggleable marker, so a question is either flagged or not. The flag is controlled by the person attempting the quiz. Other people (for example a teacher reviewing the quiz) can see but not change the state.

Teachers should be able to configure whether flags are used on a particular quiz.

This is MDL-16263. Please [http://moodle.org/mod/forum/discuss.php?d=104487 discuss this proposal in the Quiz forum].

==Code organisation==

Flagging of questions actually be implemented as a feature of the question engine, because it is potentially useful shared functionality. However any user of the question engine will have control over whether the flagging interface appears using the $cmoptions parameter to the print_question function.

In addition, some other parts of the quiz interface will be enhanced to show the flagged state of questions.

We will just store the current state of the flag for each attempt at a question (per question session), so there will not be any history kept.

==Interface within questions==

The flags will be displayed to the left of the questions, under the question number and marks, if the $cmoptions->flags is set in the call to print_question. There are three modes: QUESTION_FLAGSHIDDEN, QUESTION_FLAGSSHOWN and QUESTION_FLAGSEDITABLE. QUESTION_FLAGSSHOWN means display the current state of the flag, but don't let it be changed. QUESTION_FLAGSEDITABLE means show the flag with UI to change it.

If JavaScript is on, the flag state will be toggled using an Ajax interface when it is clicked on. If JavaScript is off, there will be a check-box to show the state, which will be saved when that page of the quiz is submitted. Even with Ajax on, there will be a checkbox, and the flag image will be in its <label>, but the checkbox will be hidden.

==Configuration options within the quiz==

There will be a new capability moodle/question:flag, which can be overridden in the quiz context to control whether a particular quiz uses flagging.

==Interface within the quiz==

If flags are on, the quiz will use QUESTION_FLAGSEDITABLE in attempt.php, and on the review.php and reviewquestion.php page when a user is reviewing their own attempt. It will use QUESTION_FLAGSSHOWN on the two review pages when you are reviewing someone else's attempt.

In addition, if flags are in use
* they will be shown on summary.php.
* they may also be shown in the navigation block. (I will try it, is see if it looks good or if it is just confusing.)

==Database changes==

There will be a new column
flagged INT(2) NOT NULL DEFAULT 0
in the question_sessions table.

This will be accessed and updated through a few functions in questionlib.php.

==Recording state changes==

If JavaScript is on, there will be a new script question/toggleflag.php to receive the Ajax state update requests.

Also, in the standard response processing (question_process_responses) there will be additional code to get the state of the flag checkbox and store it. '''TODO'''

However, for the review page, with JavaScript off, we need a 'Save flags' button, which will need a script to go to. '''TODO'''

==Security of the Ajax interface==

One difficulty with the Ajax interface is that the flags are managed by the question bank, but the question bank does not associate the question session with a particular user. (This is to give maximum flexibility to the quiz and other modules using the question engine.) So, when the question bank receives a request saying update the flag for session 12345 from a particular user, how does it know whether to allow it?

We use two techniques to ensure that a call to question/toggleflag.php is valid.

First, question_session has an id column as a primary key, but the pair (attemptid, questionid) also uniquely identifies the row. toggleflag.php requires all three ids to be sent, and checks that they are consistent. That makes it harder to forge requests. However it does not really check that the user sending the request should be allowed to change the flag on this question session.

Second, toggleflag.php requires a checksum parameter to be sent. This must be
md5($attemptid . "_" . $user->secret . "_" . $questionid . "_" . $sessionid);
This can be computed when the flag is being output in the calling page, and in toggleflag.php we can check it against the same quantity for $USER, after require_login(). ($user->secret is a random string stored for each user that is not accessible anywhere.)

(This problem does not arise with the normal response processing because the quiz only asks the question bank to process responses after checking the user's identity.)

==See also==

* [[Developer_notes#Quiz_.26_question_types|Other quiz & question bank developer notes]]
* MDL-16263.
* [http://moodle.org/mod/forum/discuss.php?d=104487 The forum thread to discuss this proposal]
* [http://moodle.org/mod/forum/discuss.php?d=97196 Previous forum thread requesting this feature]

[[Category:Quiz]]

File API

2009-02-02T05:34:23Z

Jeffjeff: /* Moving files around */

{{Moodle 2.0}}
The file API is for managing all the files stored by Moodle. If you are interested in how the file API works internally, see [[File_API]]. The page is just about what you need to know to use the file API. Related is the [[Repository API]], which lets users get files into Moodle.

==Overview of how it works==

===File areas===

===Serving files to users===

==Getting files from the user==

===Getting a file from a Moodle form===

===Getting a file with content from the HTML editor===

==Giving a file to the user==

==Moving files around==

For example, if you have just built a file at the path

$from_zip_file =
$CFG->dataroot."/temp/backup/".$preferences->backup_unique_code."/".$preferences->backup_name;

And you want to move it into the course_backup file area, do

$context = get_context_instance(CONTEXT_COURSE, $preferences->backup_course);
$fs = get_file_storage();
$file_record = array('contextid'=>$context->id, 'filearea'=>'course_backup',
'itemid'=>0, 'filepath'=>'/', 'filename'=>$preferences->backup_name,
'timecreated'=>time(), 'timemodified'=>time());
$fs->create_file_from_pathname($file_record, $from_zip_file);

==See also==

* [[File_API]] how the File API works internally.
* [[Roadmap|Moodle 2.0 roadmap]]

[[Category:Files]]

Changing the Moodle Question Engine

2009-01-29T18:27:53Z

Jeffjeff: /* Code flow */ missing word

__TOC__

The Question Engine is the part of the question code in Moodle that deals with what happens when a question is attempted by a Student, or later reviewed by the Teacher or the student themselves or included in reports. Roughly speaking this is half the question code, the other part being the Question Bank, which allows people to create, edit, organise, import, export and share questions. The Question Engine is the part that has to perform when you have a large class of students taking a quiz simultaneously.

[[Image:Sydney_harbour.jpg|frame|An irrelevant but pretty picture of Sydney, taken from the hotel room where most of this was written.]]This document is a by-product of three years working on the Moodle quiz and question code. During that time I have fixed hundreds of bugs and feature requests. Some of the bugs with the Question Engine have requried careful fixing in order to avoid regressions, and there are other issues that are hard to do anything about at the moment.

As well as bug fixes, I have also overseen major developments in the quiz/question system. For example Jamie's changes to the Question Bank, Ollie's new quiz editing interface, and Mahmoud's new quiz navigation that I added to Core Moodle while refactoring the quiz attempt code.

Over the three years, I have also followed the Moodle quiz forum and seen what it is that people struggle with, complain about, want to change (and also what they like!). For example, having coached some people when writing new question types I realise that creating a new questions type is more difficult than it needs to be.

The changes proposed here fix parts of the question engine code that I have been dissatisfied with since about the first time I looked at it. There are several reasons why I have not addressed them before now.

# To fix them properly, we must change how the data is stored in some of the key question tables. In some sites, these tables contain millions of rows, and the processing required on update is non-trivial and is different for different question types. This is not something to undertake lightly.
# It took me about two and a half years to work out this solution. Some parts of this proposal where clear from the start, but the exact notion of question interaction models, and how they would relate to question types, only crystallised recently in my mind. Because of 1. I did not want to change anything until I was sure what I was proposing was right, not just in general, but in all the details.
# The mundane reason that there have been plenty of other things to work on, and I have not had time to address this until now.
# These are not critical problems. There have been no major changes to the question engine since the 1.6 release. Clearly there are not fundamental flaws with the current system. It is just not as good as it could be.

As this is a major change, I will spend some time justifying it, before describing the change itself. My estimate is that doing what I propose here is a couple of month's solid work.

==Goals of an online assessment system==

Before going further, I think it is worth being clear about what properties we require of a question engine. In my view, we want:

1. Correctness
2. Robustness
3. Richness
4. Efficiency

Let me explain in more detail, and summarise how I thing Moodle currently performs.

===1. Correctness===

This says that, assuming nothing untoward happens, given certain inputs (question definitions, student responses, ...); and an understanding (specification) of what is supposed to happen; then the quiz module will always produce the right outputs (feedback, grades, ...). This is most important, because without this, you can't rely on anything.

I think Moodle does quite well in this regard, mainly due to many committed users who will report bugs as they are encountered, and smaller group of developers who then fix those bugs.

===2. Robustness===

The previous section started, "assuming nothing untoward happens." Robustness is about what happens if something does go wrong. What if the server crashes in the middle of a test? What if the connection to the database is lost in the middle of processing one student's submission? What if the PHP process runs out of memory in the middle of a re-grade? What if an impatient user clicks submit repeatedly or closes their browser window? And so on.

In some of these cases, it is inevitable that some data will be lost. But robustness says that no matter what happens, the student's quiz attempt must be left in a consistent state (presumably either just before, or just after the last submission) and no more than the last submission's worth of data is lost.

I think that this is currently a weakness for Moodle. If the server goes down at the end of a timed test, bad things happen. See the list of bugs below.

===3. Richness===

This is about supporting a rich range of educational experiences. One of Martin's 'five laws' is that we learn when creating something for another person to see. An online assessment system can sometimes take the place of the other person. The question is, how effective a substitute can it be in how many situations?

Moodle does quite well here, with adaptive mode, plug-in-able question types and so on. However, we could do better. At the moment, writing a new question type is harder than it need be. Also, the interaction model like adaptive or non-adaptive mode is hard coded into the question engine which makes it harder to support new interactions like certainty based marking.

===4. Efficiency===

Given all the above, how many simultaneous users can be supported with a particular amount of hardware?

Moodle is neither flagrantly inefficient, nor as efficient as it could be. We currently lack easy-to-perform performance measurement. We should establish some, so we can estimate current hardware requirements, and observe the effect of other developments on efficiency.

==Why things need to change==

I have just summarised how well Moodle meets my four goals. I will now presents a different picture, by listing bugs and feature requests in the Moodle tracker that I think can best be resolved by the changes I propose.

===Correctness issues===

{| class="nicetable"
! Key !! Votes !! Watchers !! Summary
|-
| MDL-3030 || 13 || 7 || attempts should be auto-closed
|-
| MDL-3936 || 2 || 1 || Essay Question used in score before grading
|-
| MDL-9303 || 2 || 5 || General Feedback not shown for essay questions on review screen
|-
| MDL-9327 || 4 || 3 || Text in Essay type questions is not saved in second or more attempts when the teacher has not graded previous attempts
|-
| MDL-13289 || 0 || 0 || Essay question with adaptive mode not show feedback
|-
| MDL-17681 || 0 || 0 || Manually assigned grades cannot easily by re-scaled when the marks for a question is changed
|}

===Robustness issues===

{| class="nicetable"
! Key !! Votes !! Watchers !! Summary
|-
| MDL-11852 || 0 || 1 || Quiz does not score properly
|-
| MDL-12344 || 0 || 0 || Student takes quiz, upon completion of quiz receives the following
|-
| MDL-12665 || 0 || 0 || error: Must Specify course id, shorname, or idnumber quiz attempts sometimes show zero despite correct answers
|-
| MDL-13073 || 0 || 0 || Regrading (really) messes up grades
|-
| MDL-13246 || 0 || 1 || Quiz does not save score. Shows 0/20
|-
| MDL-14873 || 0 || 2 || quiz answers submit freezes (occationally)
|-
| MDL-15305 || 1 || 0 || Quiz doesn't work after submit all and finish button is selected - get a "No quiz with id=0" or "No attempt with id=0" error message
|-
| MDL-16451 || 0 || 1 || Quiz scores 0 with secure window; Grades do not transfer
|-
| MDL-16490 || 0 || 1 || timed quiz generates bad 2nd attempt
|-
| MDL-16663 || 0 || 1 || errors and missing work
|}

(Many of these are intermittent and only affect some people. Sometimes bugs like this turn out to be the result of a corrupt table in MySQL. Still, we really should not be getting problems like this with a system that is sometimes relied on as much as the quiz is.)

===Richness issues===

{| class="nicetable"
! Key !! Votes !! Watchers !! Summary
|-
| MDL-1647 || 4 || 6 || Allowing negative marks for questions
|-
|| MDL-4860 || 0 || 0 || Questiontype giving hints
|-
| MDL-12821 || 0 || 1 || Essay quiz question redundant submit button
|-
| MDL-17894 || 0 || 0 || Support certainty/Confidence based marking
|-
| MDL-17895 || 0 || 0 || Allow question types to validate responses before grading
|-
| MDL-17896 || 0 || 0 || Question types like Opaque should not require nasty hacks in question_process_responses
|}

===Issues I don't propose to fix with this change===

... but which would become easier to fix as a result:

{| class="nicetable"
! Key !! Votes !! Watchers !! Summary
|-
| MDL-3452 || 0 || 0 || Use saved answers when marking late submissions
|-
| MDL-4309 || 0 || 3 || Ability for late quizzes (with penalty)
|-
| MDL-15596 || 0 || 0 || Allow a question to be used more than once in an attempt
|}

==How this part of the quiz currently works==

Before saying what I would like to change, I think it is best to summarise how the question engine currently works.

===Database tables===

There are three database tables that store data about students' attempts at questions.

====question_attempts====

This table is basically used to generate unique ids for the other tables relating to question attempts, and allow one to link the data in those tables to the other parts of Moodle that use the question bank, for example from a quiz attempt.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id used to link attempt question data to other things, e.g. quiz_attempts.uniqueid
|-
| modulename || NOT NULL VARCHAR(20) || e.g. 'quiz' the thing linked to.
|}

====question_sessions====

There is one row in this table for each question in an attempt.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id. Not used much.
|-
| attemptid || INT(10) NOT NULL REFERENCES question_attempts.id || Which attempt this data belongs to.
|-
| questionid || INT(10) NOT NULL REFERENCES question.id || Which question this is the attempt data for.
|-
| newest || INT(10) NOT NULL REFERENCES question_states.id || The latest state of this question in this attempt.
|-
| newgraded || INT(10) NOT NULL REFERENCES question_states.id || The latest graded state of this question in this attempt.
|-
| sumpenalty || NUMBER(12,7) NOT NULL || Used for adaptive mode scoring.
|-
| manualcomment || TEXT || A teacher's manual comment.
|}

(attemptid, questionid) is an alternate key, but that makes MDL-15596 impossible to satisfy.

====question_states====

There are several rows here for each question session, recording the different states that the question went through as the student attempted it. For example open, save, submit, manual_grade.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id. Not used much.
|-
| attempt || INT(10) NOT NULL REFERENCES question_attempts.id || Which attempt this data belongs to.
|-
| question || INT(10) NOT NULL REFERENCES question.id || Which question this is the attempt data for.
|-
| seq_number || INT(10) NOT NULL || Numbers the states within this question attempt.
|-
| answer || TEXT NOT NULL || A representation of the student's response.
|-
| timestamp || INT(10) NOT NULL || Timestamp of the event that lead to this state.
|-
| event || INT(4) NOT NULL || The type of state this is. One of the QUESTION_EVENTXXX constants from the top of questionlib.php.
|-
| grade || NUMBER(12,7) NOT NULL || The grade that had been earned by this point, after penalties had been taken into account.
|-
| raw_grade || NUMBER(12,7) NOT NULL || The grade that had been earned by this point, before penalties had been taken into account.
|-
| penalty || NUMBER(12,7) NOT NULL || Used by the adaptive mode scoring.
|}

(attempt, question, seq_number) is an alternate key (see the remark in the previous sub-section).

The student's response is stored in the answer column. It is up to each question type to convert the data received from the student into a single string in whatever format seems best, even though the response come from a HTTP post, and so is just an array of key => value pairs, which could easily be stored directly in the database without each question type having to write some nasty string concatenation/parsing code.

===Code flow===

A part of Moodle, for example the Quiz module, wishes to use questions. First, mod/quiz/attempt.php will render a page of questions by calling
# load_questions,
# load_question_states, and
# print_question one or more times.
All three of these functions delegte part of the work to the appropriate question type.

This outputs each question in its current state. Any form controls that are output have a name="" attribute that is a combination of a prefix determined by the question engine (e.q. q123_), and a main part determined by the question type.

When this page is submitted, the response goes to whichever script the question using code designate. In the case of the quiz it is mod/quiz/processattemptattempt.php. This calls
# load_questions,
# load_question_states,
# question_extract_responses,
# question_process_responses, and
# save_question_state.
# In the case of the quiz, it then saves the quiz_attempts row, which has been updated by some of the above methods.
Again, steps 1., 2., 4. and 5. delegate part of the work to the question types.

Processing manual grading by the teacher is similar. The sequece of calls is:
# load_questions,
# load_question_states,
# question_process_comment, and
# save_question_state.
# In the case of the quiz, it then saves the quiz_attempts row, which has been updated by some of the above methods.

One thing to note here is that a lot of processing is done before any data can be stored in the database.

Another point to note is that the question type classes are singleton classes. That is, only one instance of each question type is created. All the methods used by the question engine are passed a $question and possibly a $state object to tell the quetion type which question instance they should be processing. This is good design from several points of view. It works well with batch database queries (efficiency) and the question types to not have state, which might become inconsistent (robustness). However, there is a slight price here, since very rich question types cannot cache state in memory, so they may have to repeat processing (richness <-> efficiency trade off) however, they can store state in the database since they can put whatever they like in the question_states.answer field.

==Overview of the changes==

===Clarifying question states===

Althought there is a database table called question_states, the column there that stores the type of state is called event, and the values stored there are active verbs like open, save, grade. The question_process_responses function is written in terms of these actions too. One nasty part is that during the processing, the event type can be changed several times, before it is finally stored in the database as a state.

I would like to clearly separate the concept of the current state that a question is in, and the various actions that lead to a change of state.

(+Correctness, +Richness).

===Question interaction models===

At the moment, the various sequences of states that a question can move through is controlled by a combination of the compare_responses and grade_responses methods of the question types, and hard-coded logic in the question_process_responses function.

I would like to separate out the control of how a question moves through different states, into what I will call question interaction models. Currently there are two models supported by Moodle, and the choice is controlled by the adaptive mode setting in the quiz.

There is the deferred feedback model, which is what you get when adaptive mode is off. The student enters an answer to each question which is saved. Then, when they submit the quiz attempt, all the questions are graded. Then all the marks and feedback are available for review.

Alternatively, there is the adaptive model, where the student can submit one question at a time, get some feedback, and if they were wrong, immediately change their answer, with a reduced grade if they get it right the second time.

Currently, manually graded question use the same adaptive, or non-adaptive, models, with some slight changes. However, this does not work very well, as the number of essay question correctness bugs listed above shows. Really, manual grading should be treated as a separate interaction model.

So, what exactly does the interaction model control? Given the current state of the question, it limits the range of actions that are possible. For example, in adaptive mode, there is a submit button next to each question. In non-adaptive mode, the only option is to enter an answer and have it saved, until the submit all and finish button is pressed. That is, it rougly replaces question_extract_responses, question_process_responses, question_process_comment and save_question_state in the current code.

(+++Richness, ++Correctness, +Robustness)

===Simplified API for question types===

Although I said above "The student enters an answer to each question which is saved. Then when they submit the quiz attempt, all the questions are graded." In fact, that was a lie. Whenever a response is saved, the grade_responses method of the question type is called, even in the state is only being saved. This is confusing, to say the least, and very bad for performance in the case of a question type like JUnit (in contrib) which takes code submitted by the student, and compiles it in order to grade it.

So some of the API will only change to the extent that certian funcitons will in future only be called when one would expect them to be. I think this can be done in a backwards-compatible way.

Another change will be that, at the moment, question types have to implement tricky load/save_question_state methods that, basically, have to unserialise/serialise some of the state data in a custom way, so it can be stored in the answer column. This is silly design, and leads to extra, unnecessary database queries. By changing the database structure so that the students' responses are stored directly in a new table (after all, an HTTP POST request is really just an array of string key-value pairs, and there is a natural way to store that in a database) it should be posiblity to eliminate the need for these methods.

(+Richness, +Correctness, +Efficiency)

===Changed database structure===

At the moment, the some parts of the question_sessions and question_states tables and their relations are not normalised. This is a technical term for describing database designs. Basically, if the tables are normalised, then each 'fact' is stored in exactly one place. This makes it much less likely that the database can get into an inconsistent state. Changing to a normalised structure should therefore increase robustness.

In addition, I wish to change the tables, so the the responses recieved from the student are stored in a much more 'raw' form. That will mean that the responses can be saved much earlier in the sequence of processing, which will again increase robustness. It will also allow the sequence of responses from the student to be replayed more easily, making it easier to continue after a server crash, to regrade, and to write automatic test scripts.

(+++Robustness, +Correctness)

==Brief digression: who controls the model, the quiz or the question?==

Before proceeding to decribe in detail the design for how I will implement these changes, I want to digress and discuss one of the design decisions that it took a long time for me to resolve.

The question is, is the interaction model a property of the question, or the quiz?

At the moment, there is a setting in the quiz that lets you change all the questions in the quiz from adaptive mode to non-adptive mode at the flick of a switch. Or, at least, it allows you to re-use the same questions in both adaptive and non-adaptive mode. This suggest that the interaction model is a property of the quiz.

On the other hand, suppose that you wanted to introduce the feature that, in adaptive mode, the student gets different amounts of feedback after each try at getting the question right. For example, the first time they get it wrong the only get a brief hint. If they are wrong the second time they get a more detailed comment, and so on. In order to do this, you need more data (the various hints) in order to define an adaptive question that is irrelevant to a non-adaptive question. Also, certain question types (e.g. Opaque, from contrib) have to, of necessity, ignore the adaptive, non-adaptive setting. And I suggested above that, manually graded question types like Essay should really be considered to have a separate interaction model. This suggests that the interaction model is a properly of the individual question. (Although usability considerations suggest that a single quiz should probably be constructed from questions with similar interaction models.)

I evenutally concluded that both answers are right, and more importantly, worked out how a question engine could allow that to happen.

-----

{{Work in progress}}

''TODO finish the rest of this document.''

==Detailed design==

===New database structure===

====question_attempt_batches====

This is just a rename of question_attempts.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id used to link attempt question data to other things, e.g. quiz_attempts.uniqueid and question_attempts.batchid
|-
| modulename || NOT NULL VARCHAR(255) || e.g. 'quiz' the thing linked to.
|}

====question_attempts====

This replaces question_sessions. Question sessions is not a great name because session has other connotations in the context of web applications. I think it is right to use the question_attempt name here, because this tables has one row for each attempt at each question.

There is now no requirement for (attemptid, questionid) to be unique.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id. Linked to from question_states.attemptid.
|-
| batchid || INT(10) NOT NULL REFERENCES question_batch.id || Which attempt this data belongs to.
|-
| interactionmodel || VARCHAR(32) NOT NULL || The question interaction model that is managing this question attempt.
|-
| questionid || INT(10) NOT NULL REFERENCES question.id || Which question this is the attempt data for.
|-
| maxgrade || NUMBER(12,7) NOT NULL || The grade this question is marked out of in this attempt.
|-
| responsesummary || TEXT || This is a textual summary of the student's response (basically what you would expect to in the Quiz responses report).
|}

* Need to store maxgrade becuase it could come from anywhere, (e.g. quiz_question_instances, question.defaultgrade, ...). We need it available at various times (e.g. when displaying a question) so it is better to store it explicitly here.

====question_states====

Same purpose as the old question_states table, but simplified.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id. Linked to from question_states.stateid.
|-
| attemptid || INT(10) NOT NULL REFERENCES question_attempts.id || Which question attempt this data belongs to.
|-
| timestamp || INT(10) NOT NULL || Timestamp of the event that lead to this state.
|-
| state || INT(4) NOT NULL || The type of state this is. One of the QUESTION_STATEXXX constants from the top of questionlib.php.
|-
| grade || NUMBER(12,7) || The grade the student has earned for this question, on a scale of 0..1. Needs to be multiplied by question_attempts.maxgrade to get the true grade.
|}

* We store grade unscaled (as a value between 0.0 and 1.0) because that makes regrading easier. (You might think that you can adjust scaled grades later, and that is almost true, but if maxgrade used to be 0, then you can't change it to anything else.)

====question_responses====

This stores the data submitted by the student (a list of name => value pairs) that lead to the state stateid. This replaces the old question_states.answer.

There will be a convention that ordinary names like 'myvariable' should be used for submitted data belonging to the question type; names prefixed with a !, like '!myaction' should be used for data belonging to the question interaction model; and names prefixed with a _ can be used for internal things, for example, the random question might store '_realquestionid' attached to the 'open' state, or a question type that does a lot of expensive processing might store a '_cachedresult' value, so the expensive calculation does not need to be repeated when reviewing the attempt.

Note that, the old question_states.answer field used to save a lot of repetitive information from one state to the next, for example the other questionid for random questions, and the choices order for multiple-choice questions with shuffle-answers on. In future, this sort of repetitive information will not be saved. Instead, during question processing, the question types will be given access to the full state history.

{| class="nicetable"
! Column !! Type !! Comment
|-
| id || INT(10) NOT NULL AUTO INCREMENT || Unique id. Not used much.
|-
| stateid || INT(10) NOT NULL REFERENCES question_states.id || Which state the submission of this data lead to.
|-
| name || VARCHAR(20) NOT NULL || The name of the parameter received from the student.
|-
| value || TEXT || The value of the parameter.
|}

===Upgrading the database===

===New list of states that a question may be in===

===Interaction models to implement initially===

''I intend to draw state diagrams for each of these in due course.''

Note that the main blocker to supporting negative scoring in the quiz is that it is not compatible with adaptive mode. So separating the models will allow MDL-1647 to finally be resolved.

====Deferred feedback====

This is how the quiz currently work when adaptive mode is off. The student enters a response to each question, then does Submit all and finish at the end of their attempt. Only then do they get feedback and/or grades, depending on the review settings.

====Legacy adaptive mode====

This re-implements how adaptive mode currently works, so old attempts can still be reviewed. In future, adaptive mode will be replaced with ...

====Interactive====

This will replace adaptive mode. This is the model that has been used successfully for several years in the OU's OpenMark system. The OU has also modified Moodle to work like this, but because of the way the quiz core currently works, I was not happy just merging the OU changes into Moodle core. In a sense, this whole document grew out of my thoughts about how to implement the OU changes in Moodle core properly.

====Manually graded====

This is a new mode, specifically tailored to manually graded questions. Should make it possible to fix a lot of the outstanding manual grading correctness bugs above.

====Each attempt builds on last====

This is for the benefit of the quiz feature of the same name. It is just like deferred feedback model, but subsequent attempts are seeded with the student's last response from the previous attempt. I think it leads to slightly cleaner code to do this as a separate interaction model. For example, it does not make any sense to use each attempt builds on last in combination with adaptive mode, although Moodle currently lets you do that.

====Immediate feedback====

This would be like cross between deferred feedback an interactive. Each question would have a submit button beside it, like in interactive mode, so the student can submit it immediately and get the feedback while they still remember their thought processes while answering the question. However, unlike the interactive model, they would not then be allowed to change their answer.

====Certainty based marking with deferred feedback====

This takes any question that can use the deferred feedback model, and adds three radio buttons to the UI (Higher, medium, lower) for the student to use to indicate how certain they are that their answer is correct. Their choice is used to alter the grade the receive for the question.

====Certainty based marking with immediate feedback====

This is like immediate feedback mode with the certainty based marking feature.

===Changes to the question type API===

Can this be backwards compatible?

===The interaction model API===

==Proposed robustness and performance testing system==

A major change to the question engine should really only be contemplated in combination with the introduction of a test harness that makes it easy to run correctness, performance and reliablitity tests.

One adavntage of the way data will be stored in the new system is that everything originally submitted by the user will be stored in the database in a format very close to the one in which it was originally received by the web server. Therefore, it should be easy to write a script that replays saved quiz attempts. This is the basis of a test harness. I will create such a test script as part of this work.

==Time estimates==

* Completing the above design: a few days.
* ...

===Time scales===

It seems highly unlikely that this could be done before Moodle 2.0 branches. Now that I have worked out the details of this proposal, I would really like to do it as soon as possible, so that would be for Moodle 2.1. However, that is dependant on my employer (either moodle.com or The Open University). This change is sufficiently big that I would not want to undertake it on my own time.

==Summary==

This change will cost about X months development. In exchange, the question engine will be more reliable, and will support a richer ranged of teaching and learning activities, through extending the range of interaction models supported, and maing it easier to write new question types.

==See also==

* The place to discuss this is the [http://moodle.org/mod/forum/view.php?f=121 quiz forum].
*There is also a [http://moodle.org/mod/forum/discuss.php?d=114527#p502692 Thread in Lesson forum].
* See the lists of [[#Why_things_need_to_change|related tracker issues]] above.

[[Category:Quiz]]

Feedback link for all elements

2009-01-28T19:06:31Z

Jeffjeff: /* Links */

This is a work in progress when I find the time. Sometimes there is obvious repetition that I haven't cleaned up yet.

The web is more and more social in that the users/readers of content are more and more invited to influence the content. This is especially true for support pages that are for helping ie. [http://documents.google.com/support/spreadsheets/bin/answer.py?answer=87589 Google "was this helpful?"]; [http://office.microsoft.com/en-us/powerpoint/HP030713791033.aspx?pid=CH063500491033 Microsofts ditto].

Therefore it seems very important that this feature is lacking in Moodle, especially considering its social constructivistic underpinnings. It seems natural that this should be implemented, indeed almost a glaring miss.

There is a form for collecting feedback ([https://docs.moodle.org/en/Development:Feedback documentation] [http://moodle.org/mod/data/view.php?d=13&rid=95 download]) in Moodle which can be used for making course evaluations etc. The problem is that it is difficult to get people to spontaneously report suggestions, criticisms etc at all. If they have to find and go to another page, formulate a description of where they came from and what they want, felt etc, there are few that will do it, and the quality of the feedback will often lack crucial details so it may be useless.

Collecting feedback in a general form at the end of a course has many lacks. The students are poorly motivated (they won't be getting any use of changes that are made). Many annoyances, suggestions will have been forgotten. Those that are remembered will be difficult to describe accurately.

One way of nicely collecting feedback for an HTML page is to use a forum page as if it were an ordinary webpage. Then, after the content you put into the top, there is a forum at the bottom which can receive feedback and can have some soc. const. advantages over my proposal. But this wouldn't work for other kinds of pages/elements.

Having some kind of general course feedback early in a course is a good idea and can, of course, meet the first problem (that the pupil's feedback usually doesn't effect the on-going course).

=Which Elements?=
==Page==
The properties that would be good to include:
*No response
*General rating
*Lacking:
**Linkage
**Content
**Language
***grammar
***spelling
*Suggestions
* Automatically (javascript) generated info about the browser, operating system, screen and window size, plugin availability. Often the user wouldn't even know these things, and they could be critically helpful for the page editor.

Here is an example of [http://tupo.biz/kurser/gemensam/kommunikation/PageFeedbackFormExpand.htm a page feedback form]. The form has many of the things that I'd like to include. I would be like to have a small page with expandable fields connected to checked types. Only the first two checks are implemented in this. It does have a JavaScript that in the background includes auto-generated information in the form submission.

==Quiz questions (even lesson and feedback and questionnaire?)==
This is very important. If I had to chose between Page and Question, I'd chose question feedback.

Since evaluation questions are very important to the learners, they are very motivated to see that they are clear.

This can also be a way to make students an active part of the quiz creation/improvement process. Dennis Daniels has been trying to [http://video.google.com/videoplay?docid=5678154150724088700&q=dennis+daniels+moodle+quiz&total=35&start=0&num=10&so=0&type=search&plindex=7 push for this possibility]. He calls it "question challenge". A possible use scenario: The teacher quickly prepares a quiz with minimal answer alternatives and makes it available to the fastest students. These are encouraged to add more answer alternatives with motivations, and react to the existing formulation, grading and feedback. The teacher reviews their feedback and easily incorporates the useful suggestions into the test before the main body of students try the quiz.

There are many possible parts that can be specific to the feedback for questions.

Is the question itself clear? This might be able to be covered by the same fields that are of interest for general feedback for pages.

Is my answer considered wrong though I feel it is right? It would be good to include a "certainty" rating. Certainty Based Marking, CBM, is a very valuable form of quiz feedback. It would be good to think about it at the same time as other quiz feedback. It is often a 3-step scale.: C1 C2 C3 where 1 is quite uncertain and 3 is very certain. Many teachers would appreciate the ability to connect the certainty to the grading of the question. But even while the grading implementation has not been successfully made yet, it would be valuable feedback for the teacher. "Hmm a lot of students have this right, but almost all of them are very uncertain about it." or "Hmm, this student was very certain about this incorrect answer."

Is the feedback I get for my answer: unclear, incorrect, possible to improve?

Does the user have other answers that (s)he was tempted to try? Usually people make similar mistakes and this could be a valuable resource for giving better feedback to common mistakes and for creating good "distractors" for multiple choice questions.

When a quiz editor receives feedback on a question it should be easy to use it to develop the question. Prepared fields may depend on the question type. "the answer I would have used", "an answer I might have used", "my motivation". The first 2 could be used to make new answer alternatives. The last as foundation for feedback for that answer. There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.There should be a field for explaining how the learner got the answer. This could be the basis for the feedback field for that answer, if the teacher should chose to include that answer.

===Question feedback form fields===

There are prepared fields below to help you formulate your
feedback, but in case you feel like
"''I would rather just explain in my own words''": ...............
............................................................
............................................................

[_]question/[X]word in this question unclear
(check on word-> expand to: which word: ____________ )
a)possible interpretation 1 __________________________
b)possible interpretation(s) 2, ... _____________________________

[_]grade questionable, motivation: ______________________________
(this perspective maybe the teacher should be able to choose not to ask about)

answer feedback [_]wrong/[_]unclear (this would be for the answer that was active
when the feedback button was pushed, and automatically included)
What was wrong/unclear: ________________________________
Suggested improvement: _________________________________

Other possible answers:
[_]Best possible, [_]acceptable, [_]should get partial credit, [_]Not right but probable
(if last two are checked expand to: Suggested credit [____]% )
answer: ________________________________________________
motivation: ____________________________________________
(how one would get this answer.)

[_]More possible answers? (gives more fields like above)

Social bookmarking is important since what's good for the goose isn't always good for the gander! The grade goal of students varies and this can mean that a highly rated resource for one person could be useless for another. Therefore a rating should be able to be connected to information about the user. Anonymous (though visible for admin if breaking netiquette) ratings are better than none, though the value would be increased if complemented by some info like grade goal, special branch, language proficiency, disabilities, etc.

=Who would the feedback go to?=

There could be some parts of a Moodle distribution that are default set to collect feedback. The pages that are for administrators, teacher editors should be of interest to Moodle developers. Help pages should collect to the documenters. It should be possible to turn off these default settings if a site doesn't want to participate.

The other parts could choose at different levels, admin, course developer, teacher, to invite feedback for specific courses, pages, questions and which recipients would get it.

Another consideration is making feedback easier/possible for people using Moodle off line. There are many situations, especially in many places in developing countries, where Internet access is poor, slow or nonexistent. Moodle is still usable on LAN or even a single computer or on a USB memory stick that can be used on different computers, some on line some off line. It would be good if there were preparations made for saving up feedback for when coming on line or for exporting to a file that could be sent on a floppy, through a mobile phone etc to someone on line.

==Some Examples==

Developers have made a new version of some component in Moodle and are really not sure about if it will be easily understood and function well. They can than choose to set a VERY Inviting feedback link. "This page is new and we would really like suggestions about how to make it better"
The feedback choice will be default in Moodle packages and the developers will get the feedback from any site that doesn't turn it off (and has a user giving feedback). The form should contain all the fields, as many as possible automatically, that would be needed to interface with the Tracker: first for search, then for entering a new bug/feature.

A school administrator would like to collect any views that anyone using the site has about pages and turns on a more general feedback invitation: "If this page doesn't meet your needs, please tell us!" (S)he choses to have the feedback go to (her)him and to the course teacher in charge of the course containing the page. Some administrators have wished for a channel for getting info, questions, suggestions even from non-registered visitors (usually on the front page but perhaps even in other guest open courses). Feedback from non-registered visitors should probably include a reCaptcha antispam protection.

A teacher could want to have students rate pages to be able to give other students a Social bookmarking guide especially if there is a lot of varied material that everybody isn't expected to go through. Or compare the webpages with the textbook. [the book was better][Both were needed][the webpage better][Neither helped much]. This particular type of feedback would be good to be able to display for the students, perhaps tagged according to grade goals, previous experience, math love/hate, etc.

=Links=

[http://tracker.moodle.org/browse/MDL-18067 The Moodle tracker issue for this suggestion.]

[http://blog.pbwiki.com/2008/03/28/the-foolproof-5-step-way-to-answer-tough-questions/ A blog post about answering tough questions] includes "Check afterwards to see if the questioner feels satisfied."