MoodleDocs - User contributions [en]

Course contents block

2012-01-06T22:17:09Z

Mudrd8mz:

'''Course contents block''' produces a table of contents for the course - ie a list of all visible topics/weeks in your course. Clicking at one of these links will display that particular section (topic or week).

If the section has its name defined, the block uses it as the title for that section.

If the section name is not defined but there is the section summary (description) available, the block automatically extracts a suitable title from that summary. If you start summary with a heading (H1, H2, H3, etc), it will use such heading text. If your summary starts with a bold text, it will be used as a section title. If the summary consists of several paragraphs, the first one will be used. Technically spoken, the plain text content of the first non-empty HTML DOM node from the section summary is used as the summary title.

If the summary is empty, a customizable text "Unit X" (where X is the number) is displayed.

You can combine this feature with the multi-language filter to generate course contents in the user's language.

The block was written and is currently maintained by [[User:David Mudrak|David Mudrak]]

[[Image:course-contents-block-screenshot.png|thumb|The topic title is automatically extracted from the section summary|400px|left]]
 

==Installation==

There is a public source code repository for the block at [https://github.com/mudrd8mz/moodle-block_course_contents github.com]. You can either clone that repository or just download the latest package there. Follow the instructions provided by Github or see [[Git for Administrators]] for details. This may be what you want:

# cd /var/www/moodlesite/htdocs/blocks
# git clone git://github.com/mudrd8mz/moodle-block_course_contents.git course_contents
# cd course_contents
# git checkout -b local_21_STABLE origin/MOODLE_21_STABLE

If your Moodle dirroot is git checkout too, you may want to add the block directory into the list of ignored files:

# cd /var/www/moodlesite/htdocs
# echo /blocks/course_contents/ >> .git/info/exclude

To download the block in ZIP or TAR.GZ packages, follow [https://github.com/mudrd8mz/moodle-block_course_contents/tags this page]

==Examples==

{| class="nicetable"
! <center>Start of the section summary HTML</center>
! <center>Automatic course contents line</center>

|-
| <nowiki>Welcome! In this course, you will ...</nowiki>
| Welcome!

|-
| <nowiki><h1>Introduction</h1>In this course ...</nowiki>
| Introduction

|-
| <nowiki><h1>Lesson 1: Introduction</h1></nowiki>
| Lesson 1

|}

==See also==

* [http://moodle.org/plugins/pluginversions.php?plugin=block_course_contents Plugins database record]

[[Category:Block]]
[[Category:Contributed code]]

2011-11-10T15:57:05Z

Mudrd8mz:

File:grading-manage-initial.png

2011-11-10T15:00:39Z

Mudrd8mz:

File:grading-method-selection-modform.png

2011-11-10T14:44:41Z

Mudrd8mz: Screenshot of the modform grade section with the advanced grading method selector (single gradable area case)

Screenshot of the modform grade section with the advanced grading method selector (single gradable area case)

File:comicstrip-rubrics.png

2011-11-10T10:26:01Z

Mudrd8mz: A short comic strip about rubrics in Moodle 2.2

A short comic strip about rubrics in Moodle 2.2

Course contents block

2011-10-28T23:24:30Z

Mudrd8mz: Updating the page for 2.x versions

'''Course contents block''' produces a table of contents for the course - ie a list of all visible topics/weeks in your course. Clicking at one of these links will display that particular section (topic or week).

If the section has its name defined, the block uses it as the title for that section.

If the section name is not defined but there is the section summary (description) available, the block automatically extracts a suitable title from that summary. If you start summary with a heading (H1, H2, H3, etc), it will use such heading text. If your summary starts with a bold text, it will be used as a section title. If the summary consists of several paragraphs, the first one will be used. Technically spoken, the plain text content of the first non-empty HTML DOM node from the section summary is used as the summary title.

If the summary is empty, a customizable text "Unit X" (where X is the number) is displayed.

You can combine this feature with the multi-language filter to generate course contents in the user's language.

The block was written and is currently maintained by [[User:David Mudrak|David Mudrak]]

[[Image:course-contents-block-screenshot.png|thumb|The topic title is automatically extracted from the section summary|400px|left]]
 

==Installation==

There is a public source code repository for the block at [https://github.com/mudrd8mz/moodle-block_course_contents github.com]. You can either clone that repository or just download the latest package there. Follow the instructions provided by Github or see [[Git for Administrators]] for details. This may be what you want:

# cd /var/www/moodlesite/htdocs/blocks
# git clone git://github.com/mudrd8mz/moodle-block_course_contents.git course_contents
# cd course_contents
# git checkout -b local_21_STABLE origin/MOODLE_21_STABLE

If your Moodle dirroot is git checkout too, you may want to add the block directory into the list of ignored files:

# cd /var/www/moodlesite/htdocs
# echo /blocks/course_contents/ >> .git/info/exclude

To download the block in ZIP or TAR.GZ packages, follow [https://github.com/mudrd8mz/moodle-block_course_contents/tags this page]

==Examples==

{| class="nicetable"
! <center>Start of the section summary HTML</center>
! <center>Automatic course contents line</center>

|-
| <nowiki>Welcome! In this course, you will ...</nowiki>
| Welcome!

|-
| <nowiki><h1>Introduction</h1>In this course ...</nowiki>
| Introduction

|-
| <nowiki><h1>Lesson 1: Introduction</h1></nowiki>
| Lesson 1

|}

==See also==

* [http://moodle.org/mod/data/view.php?d=13&rid=2156 Modules and plugins database record]

[[Category:Block]]
[[Category:Contributed code]]

Custom menu items

2011-09-20T09:31:08Z

Mudrd8mz: Documented multilanguage support as promised in MDL-27073

{{Navigation}}
'''Please refer to [[TOC_with_notes#Navigation|these notes]] before editing this page.'''

You can configure a custom menu to be shown by themes. Each line consists of some menu text, a link URL (optional) and a tooltip title (optional), separated by pipe characters. You can specify a structure using hyphens. For example:

<pre><nowiki>
Moodle community|http://moodle.org
-Moodle free support|http://moodle.org/support
-Moodle development|http://moodle.org/development
--Moodle Tracker|http://tracker.moodle.org
--Moodle Docs|https://docs.moodle.org
-Moodle News|http://moodle.org/news
Moodle company
-Moodle commercial hosting|http://moodle.com/hosting
-Moodle commercial support|http://moodle.com/support
</nowiki></pre>

=== Multilanguage support ===
{{Moodle 2.1}}

Since Moodle 2.1, you can add a language code (or a comma separated list of codes) as the 4th item of the line. The line will be then printed if and only if the user has currently selected the listed language. For example:

<pre><nowiki>
English only|http://moodle.com|English only item|en
German only|http://moodle.de|Deutsch|de,de_du,de_kids
</nowiki></pre>

Upload users

2011-07-19T06:36:37Z

Mudrd8mz: The emailstop field not supported any more - see MDL-27804

{{Accounts}}
'''Please refer to [[TOC_with_notes#Accounts|these notes]] before editing this page.'''
Location: ''Administration > Users > Accounts > Upload users''

[[Image:Upload users preview.png|thumb|Upload users preview in Moodle 1.9]]
Firstly, note that it is usually not necessary to import users in bulk - to keep maintenance work down you should first explore forms of authentication that do not require manual maintenance, such as [[External database authentication|connecting to existing external databases]] or letting the [[Internal enrolment|users create their own accounts]]. See [[Manage authentication]] for more information.

If you are sure you want to import multiple user accounts from a text file, then you need to format your text file as follows:

==Upload users file format==

* Each line contains fields '''separated''' by commas (or other delimiters) without quotes (") and no trailing delimiter
* The first line is special, and contains fieldnames defining the format for the rest of the file.

*'''Required fields''': In any order
:<code>username, firstname, lastname, email</code>
:Validity checks are performed for:
#<code>username</code> can only contain alphabetical '''lowercase''' letters , numbers, hypen '-', underscore '_', period '.', or at-sign '@'
#<code>email</code> is in the form: ''name@example.com'' .

*'''Password field''': "password" field is optional if "Create password if needed" setting is chosen (default).
**If included, values should meet the requirements for the site's [[Site_policies#Password_policy|Password policy]]. To force password change for a particular user, set the password field to <code>changeme</code>.
**If omitted, a password will be generated for each user (during the next Cron job) and welcome e-mails sent out (not working in v2.0.2?).

*'''Optional fields''': To provide values other than the default include one or more of these
:<code>institution, department, city, country, lang, auth, ajax, timezone, idnumber, icq, phone1, phone2, address, url, description, mailformat, maildisplay, htmleditor, autosubscribe</code>

*'''Custom profile field names''': (Optional). xxxxx is the real custom user profile field name (i.e. the unique shortname)
:<code>profile_field_xxxxx</code>
: Create the custom fields BEFORE importing. Use the standard header. The "shortname" for your custom field is xxxxx (NB: as at v2.0.2, the shortname must be all lowercase, otherwise won't be recognised). The first record must include "profile_field_xxxxx".
:'''Example''': To create a custom field "genre", you must write a shortname "genre" in the new field, and write "profile_field_genre" in the header of the .csv file.

*'''Special fields''': Used for changing of usernames or deleting of users
:<code>oldusername</code>, <code>deleted</code>

*'''Enrolment fields''': (Optional):
:<code>course1, type1, role1, group1, enrolperiod1, course2, type2, role2, group2, enrolperiod2</code> etc.
**<code>course</code> is the "shortname" if present the user will be enrolled in those courses.
** <code>type</code> refers to the role to be used for associated course enrolment. Value 1 is default course role, 2 is legacy Teacher role and 3 is legacy Non-editing Teacher.
** You can use role field instead to specify roles directly - use either role short name or id (numeric names of roles are not supported).
** Users may be also assigned to groups in course (group1 in course1, group2 in course2, etc.).
*** A group is identified by name or id (numeric group names are not supported).
** From Moodle 2.0, you can set the enrolment duration, in days, for each course (<code>enrolperiod1</code> for <code>course1</code>, <code>enrolperiod2</code> for <code>course2</code>, etc.).

Commas within a field must be encoded as &#44 - the script will decode these back to commas.

For Boolean fields, use <code>0</code> for false and <code>1</code> for true.

To prevent users from receiving a large number of emails from courses or forced subscription forums use the '''maildigest'''. The options for this field are 0 = No digest, 1 = Complete digest and 2 = Digest with just subjects.

Here is an example of a valid upload file:

<code>username, password, firstname, lastname, email, course1, group1 
jonest, verysecret, Tom, Jones, jonest@someplace.edu, math102, Section 1 
reznort, somesecret, Trent, Reznor, reznort@someplace.edu,math102, Section 3</code>

==Updating existing accounts==
By default Moodle creates new user accounts, and skips lines where the <code>username</code> matches an existing account. Set "Upload Type" to '''Add new and update existing accounts''', and existing user account will be updated.

Include fieldname <code>oldusername</code> to updating existing accounts and change usernames. In the preview options, Set "Allow renames" to '''Yes''.
'''Warning''': errors updating existing accounts can affect your users badly. Be careful when using the options to update.

==After preview==
After the preprocessing, depending on the contents of the upload, the following may be available before final user creation
*Settings
** New user password
** Existing user details
** Existing user password
** Allow renames
** Allow deletes
** Prevent email addess duplicates
** Select for bulk operations

* Default values
** Authentication method: Manual account |no login | EMail-based self-registration
** Email display: Allow only other course members to see my email address | hide .. from everyone | allow anyone ...
** Email format Pretty HTML | plain text
** Email digest type: none|complete subjects
** Forum auto-subscribe: no |yes when I post
** When editing text: use HTML| standard web forms
** AJAX and Javascript: yes|no
** city/town
** country
** timezone
** Preferred language
** Description
** Web page
** ID number
** Institution
** Department
** Phone
** Mobile Phone
** Address

==After results ==
Users which were not added, will NOT be auto-enrolled in courses

----

Open another moodle browser and
* Site administration
** Courses
*** Add/edit courses; select the category, and the course
* Course administration
** Users
*** Enrolled Users
**** Enrol users and select the users flagged from other window

==Templates==

The default values are processed as templates in which the following codes are allowed:

* %l - will be replaced by the lastname
* %f - will be replaced by the firstname
* %u - will be replaced by the username
* %% - will be replaced by the %

Between the percent sign (%) and any code letter (l, f or u) the following modifiers are allowed:

* (-) minus sign - the information specified by the code letter will be converted to lowercase
* (+) plus sign - the information specified by the code letter will be converted to UPPERCASE
* (~) tilde sign - the information specified by the code letter will be converted to Title Case
* a decimal number - the information specified by the code letter will be truncated to that many characters

For example, if the firstname is John and the lastname is Doe, the following values will be obtained with the specified templates:

* %l%f = DoeJohn
* %l%1f = DoeJ
* %-l%+f = doeJOHN
* %-f_%-l = john_doe
* http://www.example.com/~%u/ = http://www.example.com/~jdoe/ (if the username is jdoe or %-1f%-l)

Template processing is done only on default values, and not on the values retrieved from the CSV file.

In order to create correct Moodle usernames, the username is always converted to lowercase. Moreover, if the "Allow extended characters in usernames" option in the Site policies page is off, characters different to letters, digits, dash (-) and dot (.) are removed. For example if the firstname is John Jr. and the lastname is Doe, the username %-f_%-l will produce john jr._doe when Allow extended characters in usernames is on, and johnjr.doe when off.

When the "New username duplicate handling" setting is set to Append counter, an auto-increment counter will be append to duplicate usernames produced by the template. For example, if the CSV file contains the users named John Doe, Jane Doe and Jenny Doe without explicit usernames, the default username is %-1f%-l and New username duplicate handling is set to Append counter, then the usernames produced will be jdoe, jdoe2 and jdoe3.

==Deleting accounts==

If the <code>deleted</code> field is present, users with value 1 for it will be deleted. In this case, all the fields may be omitted, except for <code>username</code>. After uploading the file, be sure to change the "Upload type" to "Update existing users only" and the "Allow deletes" option to "Yes".

Deleting and uploading accounts could be done with a single CSV file. For example, the following file will add the user Tom Jones and delete the user reznort:

username, firstname, lastname, deleted
jonest, Tom, Jones, 0
reznort, , , 1

==Encoding==

In Moodle 1.8 the file must be UTF-8. In Moodle 1.9 onwards, the encoding may be selected from a large list, including ISO-8859-1.

==Hints==

===Spreadsheet===

If you use a spreadsheet program such as Excel to create your .csv file, check the resulting output in a text editor before you upload it. It is possible to get trailing commas on each line from an empty field if you have added and deleted columns of information prior to saving the final file. Also check the character encoding. A csv file is a simple text file (ASCII or Unicode) that can be used to upload user accounts.

Excel translates passwords that begin with - (minus) or + (plus) as zero. Even when saving as .csv and saying "Yes" to "Keep this format, and leave out any incompatible features." Check for this before uploading, as a zero halts the upload process.

If you use a formula in Excel to create fields (for example, the concatenate function to create a user name), then remember to copy the cells with the formula and use special paste with values checked to make them into an acceptable data for a csv file.

===Country===
The country should be written as a two letter code, in capitals. For example, use BE for Belgium or NL for the Netherlands. Using "be" or "nl" as a country code will result in a database error.
:''Tip:'' If you are having trouble working out the two-letter code for a country, you can consult this Moodle source code file /moodle/lang/en_utf8/countries.php [http://cvs.moodle.org/moodle/lang/en_utf8/countries.php?view=markup&pathrev=MOODLE_19_STABLE or click here for a 1.9 STABLE list].
ISO Website: [http://www.iso.org/iso/country_codes/iso_3166_code_lists/english_country_names_and_code_elements.htm]

== See also ==
Moodle Docs:
*[[Flat file]]

Using Moodle forum discussions:
*[http://moodle.org/mod/forum/discuss.php?d=36851 Can I auto enroll from Excel?]
*[http://moodle.org/mod/forum/discuss.php?d=58215 Making Email Optional]
*[http://moodle.org/mod/forum/discuss.php?d=97903 Uploading users to custom roles]
*[http://moodle.org/mod/forum/discuss.php?d=181259 User upload option: standardise usernames]
*[http://moodle.org/mod/forum/discuss.php?d=144569 Matriculacion con flat file csv] - discussion in Spanish

[[Category:Authentication]]
[[Category:Enrolment]]
[[Category:Groups]]

[[fr:Importer des utilisateurs]]
[[ja:ユーザのアップロード]]
[[zh:上传用户]]
[[ru:Загрузка пользователей]]

Upgrading to Moodle 2.1

2011-07-01T11:06:10Z

Mudrd8mz: /* Now upgrade */

{{Work in progress}}

{{Moodle 2.1}}When upgrading to Moodle 2.1, you must have Moodle 2.0 or later. if you are using an earlier version of Moodle (for example 1.9.x) then you need to upgrade to Moodle 2.0.x first.

'''We advise that you test the upgrade first on a COPY of your production site, to make sure it works as you expect.'''

==System requirements==

* Moodle must be '''1.9''' or later
* PHP must be '''5.3.2''' or later
** Required PHP extensions: iconv, curl, ctype, zip, simplexml, spl, pcre, dom, xml, json
** Required PHP memory_limit at least 40MB (64MB or more recommended if you have a choice)
* Databases should be one of the following:
** MySQL 5.0.25 or later (InnoDB storage engine highly recommended)
** PostgreSQL 8.3 or later
** Oracle 10.2 or later
** MS SQL 2005 or later
* Any standards-supporting browser from the past few years, for example:
** Firefox 3 or later
** Safari 3 or later
** Google Chrome 4 or later
** Opera 9 or later
** MS Internet Explorer 7 or later (Even [http://googleenterprise.blogspot.com/2010/01/modern-browsers-for-modern-applications.html Google does not support IE6 any more])

==Moodle 2.1 planning==

The biggest change in Moodle 2.1 that affects the upgrade is the new question engine. If you have lots of quiz attempts, you need to plan this upgrade carefully. If you don't have lots of quiz attempts, you don't have to worry.

===Planning the question engine upgrade===

The upgrade from Moodle 2.0 to 2.1 needs to do a major transformation of all the quiz attempt data. This is something you need to plan for if you have a lot of quiz attempts.

To gauge whether this will be a problem for you, count the number of rows in the <tt>question_sessions</tt> table:

<code sql>
SELECT COUNT(1) FROM mdl_question_sessions;
</code>

* If this number is less than something like 10,000 then you have nothing to worry about.
* If this number is greater than something like 1,000,000 then you have to plan.

====A helpful plugin====

There is a useful plugin you can install to help you plan the upgrade which you can install from https://github.com/timhunt/moodle-local_qeupgradehelper. If you have a significant number of you are strongly advised to install it.

Once installed, you can access it using the '''Question engine upgrade helper''' link in the Site administration menu. This plugin has various features.

The '''List quizzes and attempts''' option will count the number of quiz attempts and question sessions (like the query above) but giving more detail. Again, this is a way to get an idea of the scope of the upgrade for you.

====What to do if you have a lot of attempts====

Here are some options to consider:

; Think about whether you really need to keep them all
: If you have not thought about your archiving strategy for old quiz attempts before, then this would be a really good time to do so. Do you really need to keep all the old data around? If not, deleting it before you upgrade to 2.1 would be a good idea.
: If you just need access to the old attempts for archival purposes, perhaps you could achieve that by keeping a copy of your old site from before the upgrade, and then only keeping and upgrading the current quiz attempts on your live system.
; Upgrade some of the attempts later
: The technical details for how to do this are in the script <tt>partialupgrade-example.php</tt> which you will need to edit.
: After the upgrade, you can complete the upgrade in one of two ways:
:# You can manually use the '''Question engine upgrade helper''' to upgrade the attempt data for any quiz that has not already been done. Use the '''List quizzes still to upgrade''' option.
:# You can set up cron to automatically complete the upgrade using the '''Configure cron''' option.

====How to check that the upgrade was successful====

If you want to verify that the upgrade worked successfully, here are some things to check:

* Obviously, look out for any error messages output, or written to the logs, during the upgrade.
* The upgrade also writes a file into <tt>moodledata/upgradelogs/qe_''{timestamp}''.html</tt>. This lists any places where there were problems with the data in your database, and the upgrade code had to make an assumption about how to proceed. This log file should be self-explanatory, if a bit technical, and it most cases each problem report will include a link to the quiz review page, so you can easily see the result of the upgrade.
* Finally, if you are testing the upgrade on a copy of your site, you can pick some example quizzes, and open the quiz reports or review pages from before, and after, the upgrade in separate browser windows, and compare to make sure the data is the same. Of course, there will be some differences (we hope the new system is better!) but this does let you check the data.

====What to do if something goes wrong during the upgrade====

I am sure you started by testing the upgrade on a copy of your live site, so if anything goes wrong, you can report the bug, wait for it to be fixed, and then try again.

If you wish to report a bug in the upgrade you may want to look at the instructions on [[Development:Question_Engine_2:Testing]] about how to make the bug report as useful as possible.

After the upgrade, you can use the Question engine upgrade helper plugin to re-do the upgrade of the attempts at any particular quiz. This should let you fix problems, even if you only discover them later. You can do this by using the '''List already upgrade quizzes that can be reset''' and '''List quizzes still to upgrade''' options.

==Before upgrading please... ==

'''NOTE''': The upgrade process will irreversibly modify the contents of your database '''and''' your moodledata file storage area. If something goes wrong you '''cannot''' go back. It is vital that you take good backups of both moodledata and the database in case you have problems with the upgrade. If you are not sure how see [[Site backup]] or ask in the moodle.org forums (explaining what your operating system is).

* It's a good idea to read the [[Latest release notes]] and the [[Moodle 2.1 release notes]]
* Always check your site to make sure it meets all system requirements for 2.1 in ''Administration > Server > [[Environment]]''
* '''Do a full database backup!'''
* '''Do a full moodledata backup'''
* '''Check your backups carefully'''
* Remember to purge PHP cache if using any PHP accelerator

==A word about optional plugins and themes==

If you have added optional or customised plugins (whether your own custom developments or from the modules and plugins database) or are using a non-core theme then these will probably work in Moodle 2.1+ if they worked in Moodle 2.0 with the following exceptions:
* Question types.
* Activity modules that use the question bank. (There a hardly any of these.)

Again, you are advice to test the upgrade on a copy of your Moodle site.

==Checking database schema - old sites==

If your 2.0 Moodle site has been upgraded through many prior versions it is possible that there will be some problems with the database schema (compared to a fresh 2.0 installation). This may cause the upgrade to fail. If your site started life prior to Moodle 2.0 it is a very good idea to check and correct the database schema before upgrading. See [[Verify Database Schema]]. You should also run the database integrity checks in the XMLDB editor, see the 'See also' for a link to extra scripts to check for other discrepancies.

==Now upgrade==

Once you have satisfied the requirements for Moodle 2.1, follow the instructions on the [[Upgrading|upgrading]] page.

On Linux servers, Moodle 2.1 supports running the [[CLI|upgrade from the command line]], rather than through a web browser. This is likely to be more reliable, particularly for large sites.

==After upgrade==

The config.php file from your 2.0 installation should work fine but if you take a look at config-dist.php that came with Moodle 2.0 there are more/different options available (e.g. database drivers and settings). It's a good idea to map your old config.php settings to a new one based on the 2.1 config-dist.php.

== Known and Discovered Issues ==

* If you get an error about 'handling of PHP float numbers', please see [[Installation_FAQ#Moodle_claims_PHP_float_handling_is_not_compatible|this FAQ entry]].
* You now need some additional PHP modules installed (e.g. intl and the zip extension). If you do not have these, installation them depends entirely on how your PHP was first installed and your operating system.

==See also==

* [[Moodle 2.1 release notes]]

[[Category:Installation]]

[[fr:Mise à jour à Moodle 2.1]]

Upgrading to Moodle 2.1

2011-07-01T10:57:47Z

Mudrd8mz: /* System requirements */

{{Work in progress}}

{{Moodle 2.1}}When upgrading to Moodle 2.1, you must have Moodle 2.0 or later. if you are using an earlier version of Moodle (for example 1.9.x) then you need to upgrade to Moodle 2.0.x first.

'''We advise that you test the upgrade first on a COPY of your production site, to make sure it works as you expect.'''

==System requirements==

* Moodle must be '''1.9''' or later
* PHP must be '''5.3.2''' or later
** Required PHP extensions: iconv, curl, ctype, zip, simplexml, spl, pcre, dom, xml, json
** Required PHP memory_limit at least 40MB (64MB or more recommended if you have a choice)
* Databases should be one of the following:
** MySQL 5.0.25 or later (InnoDB storage engine highly recommended)
** PostgreSQL 8.3 or later
** Oracle 10.2 or later
** MS SQL 2005 or later
* Any standards-supporting browser from the past few years, for example:
** Firefox 3 or later
** Safari 3 or later
** Google Chrome 4 or later
** Opera 9 or later
** MS Internet Explorer 7 or later (Even [http://googleenterprise.blogspot.com/2010/01/modern-browsers-for-modern-applications.html Google does not support IE6 any more])

==Moodle 2.1 planning==

The biggest change in Moodle 2.1 that affects the upgrade is the new question engine. If you have lots of quiz attempts, you need to plan this upgrade carefully. If you don't have lots of quiz attempts, you don't have to worry.

===Planning the question engine upgrade===

The upgrade from Moodle 2.0 to 2.1 needs to do a major transformation of all the quiz attempt data. This is something you need to plan for if you have a lot of quiz attempts.

To gauge whether this will be a problem for you, count the number of rows in the <tt>question_sessions</tt> table:

<code sql>
SELECT COUNT(1) FROM mdl_question_sessions;
</code>

* If this number is less than something like 10,000 then you have nothing to worry about.
* If this number is greater than something like 1,000,000 then you have to plan.

====A helpful plugin====

There is a useful plugin you can install to help you plan the upgrade which you can install from https://github.com/timhunt/moodle-local_qeupgradehelper. If you have a significant number of you are strongly advised to install it.

Once installed, you can access it using the '''Question engine upgrade helper''' link in the Site administration menu. This plugin has various features.

The '''List quizzes and attempts''' option will count the number of quiz attempts and question sessions (like the query above) but giving more detail. Again, this is a way to get an idea of the scope of the upgrade for you.

====What to do if you have a lot of attempts====

Here are some options to consider:

; Think about whether you really need to keep them all
: If you have not thought about your archiving strategy for old quiz attempts before, then this would be a really good time to do so. Do you really need to keep all the old data around? If not, deleting it before you upgrade to 2.1 would be a good idea.
: If you just need access to the old attempts for archival purposes, perhaps you could achieve that by keeping a copy of your old site from before the upgrade, and then only keeping and upgrading the current quiz attempts on your live system.
; Upgrade some of the attempts later
: The technical details for how to do this are in the script <tt>partialupgrade-example.php</tt> which you will need to edit.
: After the upgrade, you can complete the upgrade in one of two ways:
:# You can manually use the '''Question engine upgrade helper''' to upgrade the attempt data for any quiz that has not already been done. Use the '''List quizzes still to upgrade''' option.
:# You can set up cron to automatically complete the upgrade using the '''Configure cron''' option.

====How to check that the upgrade was successful====

If you want to verify that the upgrade worked successfully, here are some things to check:

* Obviously, look out for any error messages output, or written to the logs, during the upgrade.
* The upgrade also writes a file into <tt>moodledata/upgradelogs/qe_''{timestamp}''.html</tt>. This lists any places where there were problems with the data in your database, and the upgrade code had to make an assumption about how to proceed. This log file should be self-explanatory, if a bit technical, and it most cases each problem report will include a link to the quiz review page, so you can easily see the result of the upgrade.
* Finally, if you are testing the upgrade on a copy of your site, you can pick some example quizzes, and open the quiz reports or review pages from before, and after, the upgrade in separate browser windows, and compare to make sure the data is the same. Of course, there will be some differences (we hope the new system is better!) but this does let you check the data.

====What to do if something goes wrong during the upgrade====

I am sure you started by testing the upgrade on a copy of your live site, so if anything goes wrong, you can report the bug, wait for it to be fixed, and then try again.

If you wish to report a bug in the upgrade you may want to look at the instructions on [[Development:Question_Engine_2:Testing]] about how to make the bug report as useful as possible.

After the upgrade, you can use the Question engine upgrade helper plugin to re-do the upgrade of the attempts at any particular quiz. This should let you fix problems, even if you only discover them later. You can do this by using the '''List already upgrade quizzes that can be reset''' and '''List quizzes still to upgrade''' options.

==Before upgrading please... ==

'''NOTE''': The upgrade process will irreversibly modify the contents of your database '''and''' your moodledata file storage area. If something goes wrong you '''cannot''' go back. It is vital that you take good backups of both moodledata and the database in case you have problems with the upgrade. If you are not sure how see [[Site backup]] or ask in the moodle.org forums (explaining what your operating system is).

* It's a good idea to read the [[Latest release notes]] and the [[Moodle 2.1 release notes]]
* Always check your site to make sure it meets all system requirements for 2.1 in ''Administration > Server > [[Environment]]''
* '''Do a full database backup!'''
* '''Do a full moodledata backup'''
* '''Check your backups carefully'''
* Remember to purge PHP cache if using any PHP accelerator

==A word about optional plugins and themes==

If you have added optional or customised plugins (whether your own custom developments or from the modules and plugins database) or are using a non-core theme then these will probably work in Moodle 2.1+ if they worked in Moodle 2.0 with the following exceptions:
* Question types.
* Activity modules that use the question bank. (There a hardly any of these.)

Again, you are advice to test the upgrade on a copy of your Moodle site.

==Checking database schema - old sites==

If your 2.0 Moodle site has been upgraded through many prior versions it is possible that there will be some problems with the database schema (compared to a fresh 2.0 installation). This may cause the upgrade to fail. If your site started life prior to Moodle 2.0 it is a very good idea to check and correct the database schema before upgrading. See [[Verify Database Schema]]. You should also run the database integrity checks in the XMLDB editor, see the 'See also' for a link to extra scripts to check for other discrepancies.

==Now upgrade==

Once you have satisfied the requirements for Moodle 2.1, follow the instructions on the [[Upgrading|upgrading]] page.

Moodle 2.1 supports running the [[CLI|upgrade from the command line]], rather than through a web browser. This is likely to be more reliable, particularly for large sites.

==After upgrade==

The config.php file from your 2.0 installation should work fine but if you take a look at config-dist.php that came with Moodle 2.0 there are more/different options available (e.g. database drivers and settings). It's a good idea to map your old config.php settings to a new one based on the 2.1 config-dist.php.

== Known and Discovered Issues ==

* If you get an error about 'handling of PHP float numbers', please see [[Installation_FAQ#Moodle_claims_PHP_float_handling_is_not_compatible|this FAQ entry]].
* You now need some additional PHP modules installed (e.g. intl and the zip extension). If you do not have these, installation them depends entirely on how your PHP was first installed and your operating system.

==See also==

* [[Moodle 2.1 release notes]]

[[Category:Installation]]

[[fr:Mise à jour à Moodle 2.1]]

User:David Mudrak

2011-06-15T09:13:08Z

Mudrd8mz: Redirected page to dev:User:David Mudrak

#REDIRECT[[dev:User:David_Mudrak]]

Moodle 2.1 release notes/New question engine

2011-06-09T14:38:29Z

Mudrd8mz: Initial copy from the 2.1 Release notes page with links to dev pages fixed

This is a major rewrite of the Moodle question system to make it much more robust and to support lots of new possible functionality. It was implemented by developers at the [http://www.open.ac.uk/ Open University], primarily Tim Hunt. There is a site on OpenLearn that is running the new question engine (back-ported into Moodle 1.9) at http://labspace.open.ac.uk/course/view.php?id=3484. We recommend you start with 'A Moudle Quiz' (not a misspelling: M+OU+DLE). You'll need to register on the site and join the module to interact.

Details at [[dev:Question_Engine_2]] and in MDL-20636.

The main user-visible change is that the old Quiz setting '''Adaptive mode: yes/no''' has been replaced by a new option '''How questions behave'''. This provides a range of possible behaviours:
; Deferred feedback : what used to be called Adaptive mode: no, but with an important addition. With deferred feedback behaviour, it is finally the case that '''wrong answers to questions can give negative score''' - one of the oldest outstanding feature requests MDL-1647.
; Adaptive mode & Adaptive mode (no penalties): what used to be called Adaptive mode: yes
; Manual grading : used for essay questions (irrespective of what the quiz is set to) but you can now choose to have every question in the quiz manually graded, if you wish.
; Interactive mode : This is a new behaviour created by the OU. It is similar to the old adaptive mode, but with a few changes:
:* After submitting one answer, and reading the feedback, the student has to click a 'Try again' button before they can try a new response.
:* Once the student has got the question right, they can no longer change their response.
:* Once the student has got the question wrong too many times, they are just graded wrong (or partially correct) and get shown the feedback and can no longer change their answer.
:* There can be different feedback after each try the student makes.
; Immediate feedback : this is a bit like Adaptive/Interactive mode, in that the student can submit their response immediately during the quiz attempt, and get it graded. However, they can only submit one response, they cannot change it later.
; Deferred feedback or Immediate feedback with Certainty-based marking : with Certainty-based marking, the student does not only answer the question, but they also indicate how sure they are they got the question right. The grading is adjusted by the choice of certainty, so that students have to reflect honestly on their own level of knowledge in order to get the best mark.

There are corresponding changes in places that use questions, like the quiz and question preview. In particular the '''quiz manual grading report has been greatly improved'''.

The question types themselves have not changed very much in this release. The main change is '''students can attach files to essay question responses''' if the teacher allows it. Also, the teacher can choose whether the HTML editor is used.

The new system has extensive automated tests, which should ensure that bugs are much rarer in future.

== Warning! ==

* This change requires a major database upgrade. Currently the best information about preparing for this is on [[dev:Question_Engine_2:Testing#Testing_the_upgrade]].
* The one existing feature is its not possible to support in the new system is the Random short-answer matching question type. Therefore, we propose to move this question type out of the main Moodle release, and make it a optional plugin (which will not be compatible with Moodle 2.1 unless someone works out how to fix it). If this change will cause you problems, please explain in [http://moodle.org/mod/forum/discuss.php?d=176097 this Quiz forum thread].

== Note for question-type developers ==

Unfortunately, the extent of this change means that question types that are compatible with Moodle 2.0 will not work with Moodle 2.1 until they have been updated. Brief instructions for updating your question type are at [[dev:Developing_a_Question_Type#Converting_a_Moodle_2.0_question_type]]. If you need more help than this, please ask in the [http://moodle.org/mod/forum/view.php?id=737 Quiz forum]. I (Tim) will respond, and update the documentation. As a rough guide, I was able to upgrade most of the standard and OU-specific question types in about two days each. Of course, I know exactly how the new system works.

Development:Backup 2.0 - Converters review 2011-04

2011-05-04T20:12:54Z

Mudrd8mz: /* Related to integration with restore_controller & general behavior */ linking R0402

[[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]] > Review 2011-04
{{Template:Development:Backup 2.0}}{{Moodle_2.1}}

This page summarizes the results of the review performed (on April 2011) over the code available at https://github.com/mrmark/moodle related to the implementation of a general solution for the creation and support of moodle restore converters and its first implementation as one Moodle 1.9 => 2.x converter.

=== References ===

* MDL-22414 - the main tracker issue for this task.
* [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x article]] in Moodle Docs.
* [http://tracker.moodle.org/secure/attachment/23241/Moodle1.9to2.0Restore.pdf PDF with initial implementation plan].
* [https://github.com/mrmark/moodle/compare/master...converters_wip Development branch changes]

=== Nomenclature ===

* The review results are shown below as one list of numbered points, briefly explained, with link to corresponding MDL-xxxx (subtask of MDL-22414), where any discussion / agreement / coding should happen.
* All the MDL-xxxx issue created will begin with R04xx to note they have been ignited as part of this review process and the point they correspond to.
* Each point below includes one short "status" message in order to know the immediate actions that must be performed on it.
* New points can be added to the review as long as discussion / agreement happens, if necessary.

=== Global summary ===

* Overall, all code seems to be in correct places and be correct code.
* The main infrastructure parts (base/plan/moodle1 converters and plan/task/step architectures) are ok, though their APIs would need some improvements detailed below.
* There are a lot of stuff missing / todo. Files, users, enrolments, plugins, tricky activities, comments, grades...
* We need to push-push-push (4 weeks to freeze!).

=== The list ===

==== Related to integration with restore_controller & general behavior ====

* R0401: status: to code (MDL-27376) Right now, the to_moodle2_format() performs instantiation of all available converters + simple loop to get the conversions to perform. Instead, it should be possible to use exclusively static methods, returning origin format, target format and cost of the conversion, and then finding the best path or so. Note this is trivial right now but can be a useful general approach if new converters arrive (core and 3rd part).

* R0402: status: to code (MDL-27377) The can_convert() function can be static too, passing tempdir as param. Also, the same function (or another one, mandatory) should check if all the PHP/server requirements are satisfied or no (some converters can need extra stuff available in order to be able to perform conversions).

* R0403: status: to discuss. After each conversion, the original source directory is deleted and replaced by the converted one. This makes things complex to trace or perform repeated conversions and also, doesn't observe the $CFG->keeptempdirectoriesonbackup completely. We should be able to keep temp stuff undeleted if desired and, perhaps too, be able to instruct the restore controller about tempdir changes.

* R0404: status: to code. Exceptions: It needs to be decided if we are going to use own converter_exception/s or restore_exception/s and use them constantly along all the code base.

* R0405: status: to code. Logging: Converters stuff must inherit / use restore_controller logging facilities.

==== Related to the converters/moodle1 infrastructure ====

* R0406: status: to discuss. All the parser / processor / path_elelement stuff is part of the abstract plan_converter. That stuff should be moved to moodle1_converter as far as it is highly dependent / tidied with the conversion particularities.

* R0407: status: to discuss (not important). About converters, should class names and file names match, i.e. converter.class.php => moodle1_converter.class.php

* R0408: status: to discuss. On restore, we have both after_restore() methods available in steps (executed when the step execution ends) and in tasks (executed when the whole plan execution ends). Do we need this duality in plan_converter, or is it enough with current one. Right now it seems that we are using a lot these methods to do the work, and process_xxx() should be used instead (together with next point).

* R0409: status: to code. Since some weeks ago the notify_path_start/end() methods are available in the xml processor, should we start using them to call some stuff in converter, able to dispatch to start_xxxx() and end_xxxx() methods in the converter steps. It would help a lot with both starting/closing XML tags, creating and closing xml files, or any other pre/post stuff needing to be executed at certain stages.

* R0410: status: to discuss. The deprecated/new/renamed elements / mutate_datum stuff. Perhaps it should be moved from steps to path_elements instead, so each path element autocontains the basic transformations to be performed on data and they are applied automatically. It seems a better place for that definitions. Or, alternatively, we'll need deprecated/new/renamed/mutate_xxxx() methods, one for each path.

* R0411: status: to code. After looking current uses of the xml writer, it seems clear that we need some extra methods, able to print one tag completely, push/pop last open tag and other commodities in order to make coding easier.

* R0412: status: to code. The "trick" in the parser about dynamically modify /MOD paths seems ok (I cannot imagine another way to implement it). Similar magic will be needed for blocks and surely, the start/end implementation above will need a similar way to handle/dispatch processing.

* R0413: status: to code. Some raw accesses to the backup_ids_temp DB stuff must be modified to use helper functions. Also, the use of the table in get_context() can be really slow (text comparison). Look for some alternative.

* R0414: status: to code. The convert_file() method uses the backup_ids_temp to get sequential fileid values. It should be enough (completely safe), to use one static (in memory) generator instead. No concurrence problems at all IMO.

* R0415: status: to discuss. Support for plugins is a must as far as at least qtypes require it. Note that question types are going to suffer one major revamp on Moodle 2.1, so their final architecture is unknown right now (different from both 1.9 and 2.0).

* R0416: status: to code (tiny detail). In current moodle1_conversion, original_site_identifier should be generated only if there is one original identifier. Right now it saves one "?" that can lead to wrong results detecting same sites.

==== Related to coding in general ====

* <strike>R0417</strike>: status: done. Copyright messages are required on each new file. It must be the standard one with GPL3 msg and MR/Mark/whatever copyright.

* R0418: status: to code. We should have as many tests as possible, specially testing all the infrastructure stuff, in order to guarantee it continues working as expected after any refactor / modification. Right now it's practically inexistent.

* R0419: status: to do periodically. We need to keep the main repo for all the project "converters_wip" in sync (merged) with current moodle.git master branch in order to ensure stuff will land properly (conflicts-free) and converters will be able to use any new feature / improvement available.

==== Related to pending & tricky areas ====

* R0420: status: to code. There are a lot of areas which coding hasn't been ignited yet (surely to pending work in the list above, don't worry). Here there is a brief ordered list, annotating the ones that will be really tricky:
** Users and enrollments: All the users with their complete information must be saved to new users.xml file and each process_xxx() method must, continuously, be annotating needed uses in order to generate proper inforef.xml files. Enrolments will be all moved to manual ones (this needs confirmation).
** Workshop activity: Completely revamped in 2.0. It has proper subplugins. Upgrade code needs to be deeply analyzed and XML differences too to know how to implement it. (contact: David).
** Wiki activity: Also brand new in 2.0. Upgrade needs analysis and XML differences too. (contact: Dongsheng).
** Resource activity: Split into file/folder/page/url activities. The structure is simple (no user data) but the split makes it really tricky. Also very file-intensive. Upgrade code needs analysis and XML differences (contact: Petr).
** Files: 1.9 files can had site/course files (apart from the ones "owned" by the module @ moddata). They must be annotated and moved to new files.xml & storage. There is some code already but is not used in the general converter helper stuff (should be moodle1 converter specific instead?).
** Gradebook: There are important differences between how 1.9 and 2.0 backup information is handled. Scales / outcomes / grade items need annotation in inforef.xml files and also there are new xml files designed to store specifically grades (categories, items, grades...) information. Need analysis. (contact: Andrew).

====After review, organization====

This section will be used to annotate any agreement / conclusion / organization for the next weeks in case the Tracker isn't enough to do so. Feel free to use it, beloved colleagues, always using the R04xx nomenclature for referencing exact stuff, plz.

=== See also ===

* Go back to [[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]].

Development:Backup 2.0 - Converters review 2011-04

2011-05-04T20:05:43Z

Mudrd8mz: /* Related to integration with restore_controller & general behavior */ linking R0401

[[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]] > Review 2011-04
{{Template:Development:Backup 2.0}}{{Moodle_2.1}}

This page summarizes the results of the review performed (on April 2011) over the code available at https://github.com/mrmark/moodle related to the implementation of a general solution for the creation and support of moodle restore converters and its first implementation as one Moodle 1.9 => 2.x converter.

=== References ===

* MDL-22414 - the main tracker issue for this task.
* [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x article]] in Moodle Docs.
* [http://tracker.moodle.org/secure/attachment/23241/Moodle1.9to2.0Restore.pdf PDF with initial implementation plan].
* [https://github.com/mrmark/moodle/compare/master...converters_wip Development branch changes]

=== Nomenclature ===

* The review results are shown below as one list of numbered points, briefly explained, with link to corresponding MDL-xxxx (subtask of MDL-22414), where any discussion / agreement / coding should happen.
* All the MDL-xxxx issue created will begin with R04xx to note they have been ignited as part of this review process and the point they correspond to.
* Each point below includes one short "status" message in order to know the immediate actions that must be performed on it.
* New points can be added to the review as long as discussion / agreement happens, if necessary.

=== Global summary ===

* Overall, all code seems to be in correct places and be correct code.
* The main infrastructure parts (base/plan/moodle1 converters and plan/task/step architectures) are ok, though their APIs would need some improvements detailed below.
* There are a lot of stuff missing / todo. Files, users, enrolments, plugins, tricky activities, comments, grades...
* We need to push-push-push (4 weeks to freeze!).

=== The list ===

==== Related to integration with restore_controller & general behavior ====

* R0401: status: to code (MDL-27376) Right now, the to_moodle2_format() performs instantiation of all available converters + simple loop to get the conversions to perform. Instead, it should be possible to use exclusively static methods, returning origin format, target format and cost of the conversion, and then finding the best path or so. Note this is trivial right now but can be a useful general approach if new converters arrive (core and 3rd part).

* R0402: status: to code. The can_convert() function can be static too, passing tempdir as param. Also, the same function (or another one, mandatory) should check if all the PHP/server requirements are satisfied or no (some converters can need extra stuff available in order to be able to perform conversions).

* R0403: status: to discuss. After each conversion, the original source directory is deleted and replaced by the converted one. This makes things complex to trace or perform repeated conversions and also, doesn't observe the $CFG->keeptempdirectoriesonbackup completely. We should be able to keep temp stuff undeleted if desired and, perhaps too, be able to instruct the restore controller about tempdir changes.

* R0404: status: to code. Exceptions: It needs to be decided if we are going to use own converter_exception/s or restore_exception/s and use them constantly along all the code base.

* R0405: status: to code. Logging: Converters stuff must inherit / use restore_controller logging facilities.

==== Related to the converters/moodle1 infrastructure ====

* R0406: status: to discuss. All the parser / processor / path_elelement stuff is part of the abstract plan_converter. That stuff should be moved to moodle1_converter as far as it is highly dependent / tidied with the conversion particularities.

* R0407: status: to discuss (not important). About converters, should class names and file names match, i.e. converter.class.php => moodle1_converter.class.php

* R0408: status: to discuss. On restore, we have both after_restore() methods available in steps (executed when the step execution ends) and in tasks (executed when the whole plan execution ends). Do we need this duality in plan_converter, or is it enough with current one. Right now it seems that we are using a lot these methods to do the work, and process_xxx() should be used instead (together with next point).

* R0409: status: to code. Since some weeks ago the notify_path_start/end() methods are available in the xml processor, should we start using them to call some stuff in converter, able to dispatch to start_xxxx() and end_xxxx() methods in the converter steps. It would help a lot with both starting/closing XML tags, creating and closing xml files, or any other pre/post stuff needing to be executed at certain stages.

* R0410: status: to discuss. The deprecated/new/renamed elements / mutate_datum stuff. Perhaps it should be moved from steps to path_elements instead, so each path element autocontains the basic transformations to be performed on data and they are applied automatically. It seems a better place for that definitions. Or, alternatively, we'll need deprecated/new/renamed/mutate_xxxx() methods, one for each path.

* R0411: status: to code. After looking current uses of the xml writer, it seems clear that we need some extra methods, able to print one tag completely, push/pop last open tag and other commodities in order to make coding easier.

* R0412: status: to code. The "trick" in the parser about dynamically modify /MOD paths seems ok (I cannot imagine another way to implement it). Similar magic will be needed for blocks and surely, the start/end implementation above will need a similar way to handle/dispatch processing.

* R0413: status: to code. Some raw accesses to the backup_ids_temp DB stuff must be modified to use helper functions. Also, the use of the table in get_context() can be really slow (text comparison). Look for some alternative.

* R0414: status: to code. The convert_file() method uses the backup_ids_temp to get sequential fileid values. It should be enough (completely safe), to use one static (in memory) generator instead. No concurrence problems at all IMO.

* R0415: status: to discuss. Support for plugins is a must as far as at least qtypes require it. Note that question types are going to suffer one major revamp on Moodle 2.1, so their final architecture is unknown right now (different from both 1.9 and 2.0).

* R0416: status: to code (tiny detail). In current moodle1_conversion, original_site_identifier should be generated only if there is one original identifier. Right now it saves one "?" that can lead to wrong results detecting same sites.

==== Related to coding in general ====

* <strike>R0417</strike>: status: done. Copyright messages are required on each new file. It must be the standard one with GPL3 msg and MR/Mark/whatever copyright.

* R0418: status: to code. We should have as many tests as possible, specially testing all the infrastructure stuff, in order to guarantee it continues working as expected after any refactor / modification. Right now it's practically inexistent.

* R0419: status: to do periodically. We need to keep the main repo for all the project "converters_wip" in sync (merged) with current moodle.git master branch in order to ensure stuff will land properly (conflicts-free) and converters will be able to use any new feature / improvement available.

==== Related to pending & tricky areas ====

* R0420: status: to code. There are a lot of areas which coding hasn't been ignited yet (surely to pending work in the list above, don't worry). Here there is a brief ordered list, annotating the ones that will be really tricky:
** Users and enrollments: All the users with their complete information must be saved to new users.xml file and each process_xxx() method must, continuously, be annotating needed uses in order to generate proper inforef.xml files. Enrolments will be all moved to manual ones (this needs confirmation).
** Workshop activity: Completely revamped in 2.0. It has proper subplugins. Upgrade code needs to be deeply analyzed and XML differences too to know how to implement it. (contact: David).
** Wiki activity: Also brand new in 2.0. Upgrade needs analysis and XML differences too. (contact: Dongsheng).
** Resource activity: Split into file/folder/page/url activities. The structure is simple (no user data) but the split makes it really tricky. Also very file-intensive. Upgrade code needs analysis and XML differences (contact: Petr).
** Files: 1.9 files can had site/course files (apart from the ones "owned" by the module @ moddata). They must be annotated and moved to new files.xml & storage. There is some code already but is not used in the general converter helper stuff (should be moodle1 converter specific instead?).
** Gradebook: There are important differences between how 1.9 and 2.0 backup information is handled. Scales / outcomes / grade items need annotation in inforef.xml files and also there are new xml files designed to store specifically grades (categories, items, grades...) information. Need analysis. (contact: Andrew).

====After review, organization====

This section will be used to annotate any agreement / conclusion / organization for the next weeks in case the Tracker isn't enough to do so. Feel free to use it, beloved colleagues, always using the R04xx nomenclature for referencing exact stuff, plz.

=== See also ===

* Go back to [[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]].

Development:Backup 2.0 - Converters review 2011-04

2011-05-04T14:24:50Z

Mudrd8mz: /* Related to coding in general */ R0417 done on David's branch

[[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]] > Review 2011-04
{{Template:Development:Backup 2.0}}{{Moodle_2.1}}

This page summarizes the results of the review performed (on April 2011) over the code available at https://github.com/mrmark/moodle related to the implementation of a general solution for the creation and support of moodle restore converters and its first implementation as one Moodle 1.9 => 2.x converter.

=== References ===

* MDL-22414 - the main tracker issue for this task.
* [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x article]] in Moodle Docs.
* [http://tracker.moodle.org/secure/attachment/23241/Moodle1.9to2.0Restore.pdf PDF with initial implementation plan].
* [https://github.com/mrmark/moodle/compare/master...converters_wip Development branch changes]

=== Nomenclature ===

* The review results are shown below as one list of numbered points, briefly explained, with link to corresponding MDL-xxxx (subtask of MDL-22414), where any discussion / agreement / coding should happen.
* All the MDL-xxxx issue created will begin with R04xx to note they have been ignited as part of this review process and the point they correspond to.
* Each point below includes one short "status" message in order to know the immediate actions that must be performed on it.
* New points can be added to the review as long as discussion / agreement happens, if necessary.

=== Global summary ===

* Overall, all code seems to be in correct places and be correct code.
* The main infrastructure parts (base/plan/moodle1 converters and plan/task/step architectures) are ok, though their APIs would need some improvements detailed below.
* There are a lot of stuff missing / todo. Files, users, enrolments, plugins, tricky activities, comments, grades...
* We need to push-push-push (4 weeks to freeze!).

=== The list ===

==== Related to integration with restore_controller & general behavior ====

* R0401: status: to code. Right now, the to_moodle2_format() performs instantiation of all available converters + simple loop to get the conversions to perform. Instead, it should be possible to use exclusively static methods, returning origin format, target format and cost of the conversion, and then finding the best path or so. Note this is trivial right now but can be a useful general approach if new converters arrive (core and 3rd part).

* R0402: status: to code. The can_convert() function can be static too, passing tempdir as param. Also, the same function (or another one, mandatory) should check if all the PHP/server requirements are satisfied or no (some converters can need extra stuff available in order to be able to perform conversions).

* R0403: status: to discuss. After each conversion, the original source directory is deleted and replaced by the converted one. This makes things complex to trace or perform repeated conversions and also, doesn't observe the $CFG->keeptempdirectoriesonbackup completely. We should be able to keep temp stuff undeleted if desired and, perhaps too, be able to instruct the restore controller about tempdir changes.

* R0404: status: to code. Exceptions: It needs to be decided if we are going to use own converter_exception/s or restore_exception/s and use them constantly along all the code base.

* R0405: status: to code. Logging: Converters stuff must inherit / use restore_controller logging facilities.

==== Related to the converters/moodle1 infrastructure ====

* R0406: status: to discuss. All the parser / processor / path_elelement stuff is part of the abstract plan_converter. That stuff should be moved to moodle1_converter as far as it is highly dependent / tidied with the conversion particularities.

* R0407: status: to discuss (not important). About converters, should class names and file names match, i.e. converter.class.php => moodle1_converter.class.php

* R0408: status: to discuss. On restore, we have both after_restore() methods available in steps (executed when the step execution ends) and in tasks (executed when the whole plan execution ends). Do we need this duality in plan_converter, or is it enough with current one. Right now it seems that we are using a lot these methods to do the work, and process_xxx() should be used instead (together with next point).

* R0409: status: to code. Since some weeks ago the notify_path_start/end() methods are available in the xml processor, should we start using them to call some stuff in converter, able to dispatch to start_xxxx() and end_xxxx() methods in the converter steps. It would help a lot with both starting/closing XML tags, creating and closing xml files, or any other pre/post stuff needing to be executed at certain stages.

* R0410: status: to discuss. The deprecated/new/renamed elements / mutate_datum stuff. Perhaps it should be moved from steps to path_elements instead, so each path element autocontains the basic transformations to be performed on data and they are applied automatically. It seems a better place for that definitions. Or, alternatively, we'll need deprecated/new/renamed/mutate_xxxx() methods, one for each path.

* R0411: status: to code. After looking current uses of the xml writer, it seems clear that we need some extra methods, able to print one tag completely, push/pop last open tag and other commodities in order to make coding easier.

* R0412: status: to code. The "trick" in the parser about dynamically modify /MOD paths seems ok (I cannot imagine another way to implement it). Similar magic will be needed for blocks and surely, the start/end implementation above will need a similar way to handle/dispatch processing.

* R0413: status: to code. Some raw accesses to the backup_ids_temp DB stuff must be modified to use helper functions. Also, the use of the table in get_context() can be really slow (text comparison). Look for some alternative.

* R0414: status: to code. The convert_file() method uses the backup_ids_temp to get sequential fileid values. It should be enough (completely safe), to use one static (in memory) generator instead. No concurrence problems at all IMO.

* R0415: status: to discuss. Support for plugins is a must as far as at least qtypes require it. Note that question types are going to suffer one major revamp on Moodle 2.1, so their final architecture is unknown right now (different from both 1.9 and 2.0).

* R0416: status: to code (tiny detail). In current moodle1_conversion, original_site_identifier should be generated only if there is one original identifier. Right now it saves one "?" that can lead to wrong results detecting same sites.

==== Related to coding in general ====

* <strike>R0417</strike>: status: done. Copyright messages are required on each new file. It must be the standard one with GPL3 msg and MR/Mark/whatever copyright.

* R0418: status: to code. We should have as many tests as possible, specially testing all the infrastructure stuff, in order to guarantee it continues working as expected after any refactor / modification. Right now it's practically inexistent.

* R0419: status: to do periodically. We need to keep the main repo for all the project "converters_wip" in sync (merged) with current moodle.git master branch in order to ensure stuff will land properly (conflicts-free) and converters will be able to use any new feature / improvement available.

==== Related to pending & tricky areas ====

* R0420: status: to code. There are a lot of areas which coding hasn't been ignited yet (surely to pending work in the list above, don't worry). Here there is a brief ordered list, annotating the ones that will be really tricky:
** Users and enrollments: All the users with their complete information must be saved to new users.xml file and each process_xxx() method must, continuously, be annotating needed uses in order to generate proper inforef.xml files. Enrolments will be all moved to manual ones (this needs confirmation).
** Workshop activity: Completely revamped in 2.0. It has proper subplugins. Upgrade code needs to be deeply analyzed and XML differences too to know how to implement it. (contact: David).
** Wiki activity: Also brand new in 2.0. Upgrade needs analysis and XML differences too. (contact: Dongsheng).
** Resource activity: Split into file/folder/page/url activities. The structure is simple (no user data) but the split makes it really tricky. Also very file-intensive. Upgrade code needs analysis and XML differences (contact: Petr).
** Files: 1.9 files can had site/course files (apart from the ones "owned" by the module @ moddata). They must be annotated and moved to new files.xml & storage. There is some code already but is not used in the general converter helper stuff (should be moodle1 converter specific instead?).
** Gradebook: There are important differences between how 1.9 and 2.0 backup information is handled. Scales / outcomes / grade items need annotation in inforef.xml files and also there are new xml files designed to store specifically grades (categories, items, grades...) information. Need analysis. (contact: Andrew).

====After review, organization====

This section will be used to annotate any agreement / conclusion / organization for the next weeks in case the Tracker isn't enough to do so. Feel free to use it, beloved colleagues, always using the R04xx nomenclature for referencing exact stuff, plz.

=== See also ===

* Go back to [[Development:Backup 2.0|Backup 2.0]] > [[Development:Backup 2.0 - Provide upwards compatibility of Moodle 1.9.x backups|Backup 1.9 => 2.x main page]].

Language customization

2011-05-03T00:17:08Z

Mudrd8mz: The language customization feature has been there since ages, removing the since-2.0 part.

'''Note:''' This page describes how to change words or phrases in Moodle 2.0 onwards. For details of the process in versions prior to 2.0, see [[Language editing]].

{{Moodle 2.0}}Location: ''Site administration > Language > Language customization''

Words or phrases (in any language) used on the site may be easily changed using the language customization feature. For example, you may want to change the word "Course" to "Unit". The process consists of 4 steps:

# Check-out the strings
# Filter the strings you wish to customize
# Customize the strings
# Save and check-in the strings

== Quick instructions for the impatient ==

* Go to the ''Site administration > Language> Language customization'' page.
* Pick the language to customize from the pull down list.
* Click the "Check out strings to translator" button. And get ready for the change.
* Use the new "filter strings" interface to select the file to edit. Notice the files are grouped. For example, you will find the lesson strings under "mod" and moodle.php under core.
* After selecting the file, it is possible to use "string must contain" and other filters. For example, look at only string text that has "add" in the lesson.php set.
* Once you are done, you can save changes and pick another filter or PHP file to edit.
* When you want to save your edits, select "Save and check in string into files". This will store the changes you have made so all users will see them.

:''Tip:'' Do not see any string changes? Did you remember to use the "Save and check in string into files" button? Did you refresh your browser so it is not looking at a cached page? Did you edit the language file that is actually being used in your site, course or by the user?

== Background ==

Moodle is translated into many languages - see [http://download.moodle.org/langpack/2.0/] for their list and the translation completion status. The translations are distributed in so called language packages (or just lang packs) that are maintained by kind volunteers, community contributors and Moodle partners. Please read the page [[Translation]] first to understand how the whole localization machinery works.

Moodle site administrators can customize any language pack to fit their individual needs (for example to use the term "Unit" instead of "Course"). You are discouraged from direct editing the files coming as a part official language pack. Such changes would be silently overwritten during the next upgrade. Instead, you should create a local language pack that holds all your changes from the official pack.

Local language packs have the same structure as the official ones. They are saved in your Moodle data directory in moodledata/lang/xx_local/ folder where 'xx' is the code of the language. You have to have the official language pack installed before you can customize it. Local language pack should contain just strings you have customized - you should not copy whole official language packs.

When displaying a string, Moodle first looks if a local customization of it exists in moodledata/lang/xx_local/component_file.php. If so, it is used. If not, the string from the official language pack is used (eventually, if the string has not been translated yet, the original English version is displayed). Please note that the strings are cached for better performance so you have to purge Moodle caches after you modify a file in your xx_local pack (caches are purged automatically if you use the tool described below).

== Using Language customization tool ==

Moodle comes with a tool that allows you to edit your local language pack via web interface. This tool is available for the site administrators via block ''Settings > Site administration > Language > Language customization''. Please refer to the following workflow diagram.

[[image:customlang-process.png|800px|thumb|left|Workflow of the language customization (click to enlarge)]]
 

=== Check out strings into translator ===

At the Language customization first page, select a language to customize and press the button 'Check out string into translator'. During the checkout, Moodle loads the language strings from PHP files into its database. The language customization tool works with this database so the files in your xx_local pack are not touched unless your proceed to the final step of this workflow. If the xx_local language pack does not exist yet, Moodle automatically creates an empty one for you.

There is currently a problem with "execution time". If you get the "Fatal error: Maximum execution time of 180 seconds exceeded" message, you will have to press the button 'Check out string into translator' several times to get the operation completed. See [http://moodle.org/mod/forum/discuss.php?d=163375 forum discussion].

=== Filter the strings you want to customize ===

[[Image:Language_string_M2_filter.png|thumb|right|Moodle 2.0 language filter]]
After the checkout, use the filter form provided to find strings you want to customize for your site. The following filter options are available:

* ''Show strings of these components'' - if you know which component defines the string you look for (it is a second parameter of the get_string() function), choose it here. Eventually, select all options to search across all components.
* ''Customized only'' - check this field to display only those strings that are already present in your xx_local pack.
* ''Help only''' - check this field to display only help tooltips, that is the texts used when clicking the yellow question mark icon. Started in Moodle 2.0, help string identifiers must end with _help suffix.
* ''Modified only'' - displays only the strings that are modified in the current session. Note the difference between 'customized string' and 'modified string' here. We use the term 'customized' for strings that are saved on disk in your xx_local pack directory. While the term 'modified' represents the changes you have done since the last checkout. Customized strings (already saved in a file) are highlighted with green. Modified strings (not saved in a file yet) are highlighted with blue. You may want to use this option to overview your current work before you check it in.
* ''Only strings containing'' - insert a phrase that must appear in the string, either the English or the translated. For example, if you put a word 'student' here, you will get only those strings that mention this word. This can be effectively used for mass search and replace to change the terminology you use in your Moodle site.
* ''String identifier'' - if you know the string identifier (it is the first parameter of the get_string() function), type it here. For example, the names of activity modules are defined in strings 'modulename'.

Use any combination of filter settings to get the required set of strings and press the button 'Show strings'. We believe that the filter is powerful part of the whole tool and with clever settings it will allow you to find the required strings effectively.

=== Input your own translation ===

The strings that pass all the conditions defined in the filter are displayed in a table. To replace the standard translation, put your own into the field in the last column 'Local customization'. Do not forget to save the text you input via 'Save and continue editing' button.

If you want to delete your current customization, just delete the content of the input field and save the translator form. The modifications that are going to remove a customized string are highlighted in red.

=== Saving your work into files ===

Repeat the last two steps as needed to find all the strings you want to change. When you are ready to save your work into files in xx_local language pack, press 'Save and check in strings into files' button.

=== Writing the modifications into files ===

During the checkin, the contents of the translator database are dumped into files in moodledata/lang/xx_local/ directory. Note this operation removes this directory first and then re-creates it with the actual data. Therefore it is reasonable to not to touch the files directly after you have checked out them into the translator.

==See also==

* [[Language editing]] - for versions of Moodle prior to 2.0

[[Category:Language]]

Language editing

2011-05-03T00:15:25Z

Mudrd8mz: The 2.0 specific part moved to the Language customization page

'''Note:''' Language editing has been improved in Moodle 2.0 onwards. See [[Language customization]] for details.

Location: ''Administration > Language > Language editing''

== Edit words or phrases ==
[[Image:Editing-language-moodle-19.gif|thumb|Edit words or phrases in Moodle 1.9.5]]The language editing interface enables you to easily change any word or phrase used on the site. For example, you may want to change the word "Course" to "Area".

To edit a word or phrase:
#Access ''Administration > Language > Language editing''.
#Click the "Edit words or phrases" link in the middle of the page.
#Choose a file to edit. You may need to search through a few files before finding the file containing the word you wish to change. The file ''moodle.php'' contains all common site-wide phrases.
#Change the word or phrase.
#Click the "Save changes" button. The changed phrase will be highlighted in a different color.

Note: In versions of Moodle prior to 1.9, it is necessary to click the "Switch lang directory" button on the edit words or phrases page. A local language folder, ''parentlanguage_local'', will then be automatically created in ''moodledata/lang''. Files of edited strings will then be saved in this folder. This is necessary to prevent changes that you make being overwritten by a newer language pack when updating. In Moodle 1.9 onwards, the option to switch is no longer provided as edited strings are automatically saved in the local language folder.

If you wish to make further changes later, be sure to check that files of edited strings will again be saved to the folder ''parentlanguage_local'', switching folder if necessary.

==Changes in 1.9==
[[Image:screenshot-admin-lang-19.png|thumb|Language pack maintaining in Moodle 1.9]]
{{Moodle 1.9}}* From Moodle 1.9 onwards, only users with the capability [[Capabilities/moodle/site:langeditmaster|moodle/site:langeditmaster]] may modify the master language packages (i.e. those being saved in ''moodledata/lang/''). By default, the admin role has this capability set to "''prevent''". It is expected that only language maintainers will manually allow this for themselves. Language pack maintainers have an additional "Language pack maintaining" tab.
* From Moodle 1.9 onwards, only users with the capability [[Capabilities/moodle/site:langeditlocal|moodle/site:langeditlocal]] may customize the site translation (i.e. files being saved in ''moodledata/lang_local/''). Admins are allowed to do this by default.
* Added ability to edit language files in non-standard locations, i.e. string files for various types of plugin (e.g. blocks, database presets, 3rd party modules etc.)

== See also ==

* [[Language FAQ]]
* [[Translation]]
* [[Local language]]
* [[Development:Places to search for lang strings]]
* [[Language customization]] for language editing in Moodle 2.0
* [http://www.moodletutorials.org/view_video.php?viewkey=4a10e0db5e4b97fc2af3 Tutorial Showing How to Change a Word or Phrase in Moodle 1.9]
Using Moodle forum discussions:
* [http://moodle.org/mod/forum/discuss.php?d=49150 Local language]
* [http://moodle.org/mod/forum/discuss.php?d=78225 Editing help files]

[[Category:Language]]

[[es:admin/lang]]
[[fr:Langue]]
[[ja:言語]]
[[pt:Edição da língua]]
[[zh:语言]]
[[sk:Jazyk]]

Language customization

2011-05-03T00:12:27Z

Mudrd8mz: Merged the 2.0 instructions from the Language editing page

'''Note:''' This page describes how to change words or phrases in Moodle 2.0 onwards. For details of the process in versions prior to 2.0, see [[Language editing]].

{{Moodle 2.0}}Location: ''Site administration > Language > Language customization''

In Moodle 2.0 onwards, words or phrases (in any language) used on the site may be easily changed using the language customization feature. For example, you may want to change the word "Course" to "Unit". The process consists of 4 steps:

# Check-out the strings
# Filter the strings you wish to customize
# Customize the strings
# Save and check-in the strings

== Quick instructions for the impatient ==

* Go to the ''Site administration > Language> Language customization'' page.
* Pick the language to customize from the pull down list.
* Click the "Check out strings to translator" button. And get ready for the change.
* Use the new "filter strings" interface to select the file to edit. Notice the files are grouped. For example, you will find the lesson strings under "mod" and moodle.php under core.
* After selecting the file, it is possible to use "string must contain" and other filters. For example, look at only string text that has "add" in the lesson.php set.
* Once you are done, you can save changes and pick another filter or PHP file to edit.
* When you want to save your edits, select "Save and check in string into files". This will store the changes you have made so all users will see them.

:''Tip:'' Do not see any string changes? Did you remember to use the "Save and check in string into files" button? Did you refresh your browser so it is not looking at a cached page? Did you edit the language file that is actually being used in your site, course or by the user?

== Background ==

Moodle is translated into many languages - see [http://download.moodle.org/langpack/2.0/] for their list and the translation completion status. The translations are distributed in so called language packages (or just lang packs) that are maintained by kind volunteers, community contributors and Moodle partners. Please read the page [[Translation]] first to understand how the whole localization machinery works.

Moodle site administrators can customize any language pack to fit their individual needs (for example to use the term "Unit" instead of "Course"). You are discouraged from direct editing the files coming as a part official language pack. Such changes would be silently overwritten during the next upgrade. Instead, you should create a local language pack that holds all your changes from the official pack.

Local language packs have the same structure as the official ones. They are saved in your Moodle data directory in moodledata/lang/xx_local/ folder where 'xx' is the code of the language. You have to have the official language pack installed before you can customize it. Local language pack should contain just strings you have customized - you should not copy whole official language packs.

When displaying a string, Moodle first looks if a local customization of it exists in moodledata/lang/xx_local/component_file.php. If so, it is used. If not, the string from the official language pack is used (eventually, if the string has not been translated yet, the original English version is displayed). Please note that the strings are cached for better performance so you have to purge Moodle caches after you modify a file in your xx_local pack (caches are purged automatically if you use the tool described below).

== Using Language customization tool ==

Moodle comes with a tool that allows you to edit your local language pack via web interface. This tool is available for the site administrators via block ''Settings > Site administration > Language > Language customization''. Please refer to the following workflow diagram.

[[image:customlang-process.png|800px|thumb|left|Workflow of the language customization (click to enlarge)]]
 

=== Check out strings into translator ===

At the Language customization first page, select a language to customize and press the button 'Check out string into translator'. During the checkout, Moodle loads the language strings from PHP files into its database. The language customization tool works with this database so the files in your xx_local pack are not touched unless your proceed to the final step of this workflow. If the xx_local language pack does not exist yet, Moodle automatically creates an empty one for you.

There is currently a problem with "execution time". If you get the "Fatal error: Maximum execution time of 180 seconds exceeded" message, you will have to press the button 'Check out string into translator' several times to get the operation completed. See [http://moodle.org/mod/forum/discuss.php?d=163375 forum discussion].

=== Filter the strings you want to customize ===

[[Image:Language_string_M2_filter.png|thumb|right|Moodle 2.0 language filter]]
After the checkout, use the filter form provided to find strings you want to customize for your site. The following filter options are available:

* ''Show strings of these components'' - if you know which component defines the string you look for (it is a second parameter of the get_string() function), choose it here. Eventually, select all options to search across all components.
* ''Customized only'' - check this field to display only those strings that are already present in your xx_local pack.
* ''Help only''' - check this field to display only help tooltips, that is the texts used when clicking the yellow question mark icon. Started in Moodle 2.0, help string identifiers must end with _help suffix.
* ''Modified only'' - displays only the strings that are modified in the current session. Note the difference between 'customized string' and 'modified string' here. We use the term 'customized' for strings that are saved on disk in your xx_local pack directory. While the term 'modified' represents the changes you have done since the last checkout. Customized strings (already saved in a file) are highlighted with green. Modified strings (not saved in a file yet) are highlighted with blue. You may want to use this option to overview your current work before you check it in.
* ''Only strings containing'' - insert a phrase that must appear in the string, either the English or the translated. For example, if you put a word 'student' here, you will get only those strings that mention this word. This can be effectively used for mass search and replace to change the terminology you use in your Moodle site.
* ''String identifier'' - if you know the string identifier (it is the first parameter of the get_string() function), type it here. For example, the names of activity modules are defined in strings 'modulename'.

Use any combination of filter settings to get the required set of strings and press the button 'Show strings'. We believe that the filter is powerful part of the whole tool and with clever settings it will allow you to find the required strings effectively.

=== Input your own translation ===

The strings that pass all the conditions defined in the filter are displayed in a table. To replace the standard translation, put your own into the field in the last column 'Local customization'. Do not forget to save the text you input via 'Save and continue editing' button.

If you want to delete your current customization, just delete the content of the input field and save the translator form. The modifications that are going to remove a customized string are highlighted in red.

=== Saving your work into files ===

Repeat the last two steps as needed to find all the strings you want to change. When you are ready to save your work into files in xx_local language pack, press 'Save and check in strings into files' button.

=== Writing the modifications into files ===

During the checkin, the contents of the translator database are dumped into files in moodledata/lang/xx_local/ directory. Note this operation removes this directory first and then re-creates it with the actual data. Therefore it is reasonable to not to touch the files directly after you have checked out them into the translator.

==See also==

* [[Language editing]] - for versions of Moodle prior to 2.0

[[Category:Language]]

Development:Gradebook 2.x architecture

2011-04-29T23:24:33Z

Mudrd8mz: Added forum thread link into the infobox

{{Infobox Project
|name = Gradebook 2.x architecture
|state = Planning
|tracker = MDL-25423
|discussion = [http://moodle.org/mod/forum/discuss.php?d=174346 link]
|assignee = unassigned
}}

This page summarizes some initial ideas and suggestions that may help to resolve [[Development:Gradebook improvements#Stage 3|Stage 3]] of the Gradebook improvements project.

== Current issues ==

These are some of the most important issues with the current Gradebook design that this spec tries to deal with. See http://www.mindmeister.com/89537697/gradebook-2-x-issues-braindump for the full record of the braindumping session.

; The grades appear in the gradebook immediately (MDL-25439) : The grade values are stateless. We need a way how to store grades in the gradebook but exclude them from aggregations yet. Currently, the ''hiddenuntil'' feature tries to solve this but it is pretty limited and hacky (MDL-25440).
; Students and teachers may see different values : Because of how grades hiding is implemented currently, it is practically impossible to pre-populate whole grades tree and/or export reliable data (because they are dependent on the current role permissions).
; Unable to alter grades processing : There is a demand for alternate methods of grades processing (eg grades over 100% or negative grades) due to local conventions in various countries and school systems.
; Grading logic and UI implemented in activity modules : Grading tools like single point grade, scale-based grading or a new rubric-based grading should be handled by the Gradebook core with a rich set of interface toward activity modules (and eventually other plugin types).
; Manual and auto grades interaction : What should happen in this scenario has never been defined: 1. Student makes first attempt at the quiz, gets grade of 50%, this is pushed to the gradebook. 2. Teacher manually edits the grade to be 60%. 3. Student makes second quiz attempt and scores 75%. What should the gradebook show now? -- Tim Hunt
:: a similar situation happens in AMOS when the original English text is modified after it has been translated. If we record the timestamps, we can highlight such overridden grade as "outdated" (with a possibility to mark it as up-to-date hence un-highlight it). Still the manually edited value 60% should be the valid one IMHO --[[User:David Mudrak|David Mudrak]] 15:45, 14 April 2011 (WST)

== Suggested solutions and improvements ==

The goal is to make the gradebook so that '''simple things are easy and complex things are possible'''. That is for courses that just need to maintain a list of graded activities and calculate the final grade, the setup for teacher should be straightforward using default values. Advanced usage with a complex tree of grade items and calculations happening at several layers should be possible, though the setup may require more experience and insight into the internals.

According to this document, the gradebook to be still represented as a tree of grade items. There is no need to change this. Grade items may be associated with an activity module (multiple grade items per mod are supported) or a gradebook category. Grades are aggregated from the leaves of the tree to the root grade item that represents the total course grade.

[[image:gradebook-tree.png]]

In the example above, there are two grade items associated with modules in the course. These two are aggregated into a single item associated with a grades category. This category grade item is aggregated together with yet another grade item (that itself is populated via grader report) and form the course total grade.

=== Grades in core space or plugin space ===

Generally the grade values are to be stored by the gradebook in its own core tables. This will centralize and unify the storage and processing of the grades. For most modules like Assignment, Forum, Wiki, Database etc. (and eventually other plugin types like blocks if we ever decide to support grades there, too) this is enough as they do not need to keep the grades in their own tables.

This does not mean that the modules would be forced to give up advanced grades processing features. For example Quiz or Workshop modules must store a lot of additional data to calculate the final grades. So the Quiz will still hold the calculated grades for all user's attempts. But it will push the valid one (best attempt, average, maximum or whatever is set) to the gradebook. Similarly the Workshop will hold the information how the assessment forms were filled by peers. But the final grades for each student will be pushed into the gradebook tables via some nice API.

=== Grade approval process and grade values states ===

This is a suggestion to extend the grade_grades table so that every grade item can actually hold three independent values:

* ''Raw'' grade value - This is the value that comes from the source - from the activity module, from the grader report or as a result of the aggregation at lower levels. It behaves as an input buffer that just keeps the most recent incoming value.
* ''Overridden'' grade value - This is the value of the grade defined in the gradebook.
* ''Final'' grade value - This is a public value of the grade, it is used for aggregations at higher levels, it is being exported, displayed to users etc.

[[image:gradebook-buffer.png]]

The key point of this new scheme is how and when the ''final'' value is populated. The container (column) holding the final value is populated during the grade ''approval''. Whenever the gradebook tree is being recalculated, the following procedure is taken:

* If the value of overridden grade is not null, then use it as the final value (and break as we are finished)
* If there are no approval conditions defined, take the raw grade value and use it as the final value
* If there are some approval conditions defined, evaluate the list of conditions. If the evaluation passes, use the raw grade as the final grade value
* Otherwise, set the final grade to null and eventually indicate if there are some raw grade values available but on hold.

The approval conditions mechanism should support at least the following types of conditions:

* Date-based conditions - This would replace the current hidden-until feature
* Event-based conditions - Things like "take the grades into account only after all submissions are assessed"
* Manual trigger - special type of an event. The teacher can easily order the gradebook to "take the grades into account NOW" by clicking an icon or so

At the moment, we are unable to distinguish between the incoming ''raw'' grade and the outgoing ''final'' grade value. Therefore it is not easy to put a grade on hold. For teachers in schools, it is pretty natural to have some grades already known but not having them counted into the total grade yet.

Note that by default there are no approval conditions. So the raw grades becomes automatically final grades (backward compatible behaviour). So simple scenarios are kept simple while complex scenarios possible.

To be decided: do the conditions have to pass to take the overridden value into account, too? That is - if there are some approval conditions defined, is the overridden value dependent on them or not?

=== Grade values and their interpretation and display ===

Currently, grading configuration (like max grade, grade type etc) is stored by modules themselves, often independently on the associated grade item settings. This is a proposal to change it so:

* The modules themselves do not know anything about the actual grade setting. That is, they do not know max grade, min grade, the grade display type etc.
* All the grade items settings is stored and controlled exclusively by the the gradebook code
* There is a callback mechanism introduced so that the gradebook can push the required grades setting into the mod_edit form and the module settings node in the Settings block (compare with how enrol plugins push their settings into the course edit form and how they have their own setting in the course settings node)
* When pushing a grade into the gradebook (so that is saved as the ''raw'' grade value - see above), the activity module uses normalized decimal value from 0.00000 to 1.00000.
* The activity can ask the gradebook via its public API to format a given float value according the current grade item settings. "Hey gradebook, this student reached 0.81076. What shall I print to them?" and the gradebook would return "4/5" or "pretty good" or "81%" or whatever is defined there.

So the final, overridden and raw grade values are stored as normalized 0.00000 - 1.00000 decimals in grade_grades. Their actual interpretation depends on the grade item settings.

Question: should the final value depend on the actual grade item type? Example: let us have a grade item the type of which is three-points scale 1, 2, 3 and the raw grade value is 0.97654. During the approval, should the value be transformed to 1.00000 or should it stay original?

=== Introducing gradebook engines ===

The grade values stored in the tree and the aggregation and approval logic will be separated. Right now, the grade item itself contain the computation logic. A new class, so called gradebook engine, will be responsible for all grade values handling in the future. This should be pluggable and OOP-ish. This way, we should be able to implement alternative calculation engines, for example one that does not strip normalized values to <0; 1> interval but supports grades over 100% or negative grades.

=== Rubric support ===

At the moment, rubric assessment tool is supported by the workshop module only. There is a demand to use rubric as a general assessment tool across all activities. Rubric gradeitems will be probably stored separately and the gradebook/rubric engine will process them, resulting in a single 0.000000 - 1.00000 value to be pushed into the representing gradebook gradeitem. Alternatively, rubric items can be stored as ordinary gradebook gradeitems but they will be hidden from the normal gradebook report (only the result of the rubric aggregation would be displayed).

Rubric must not depend on standard scales as most rubric items will use their own scale that does not have much sense outside the context of the given rubric.

=== Grades over 100% ===

As this document suggests to separate the grades storage from the grade processing logic, there can be a separate gradebook engine designed as needed to process grades over 100%. Apparently the storage engine must support grade values > 1.00000 (and actually even grade values < 0.00000).

=== Problem with different values seen by the teacher and by students ===

The student views only show the final grades. And only final grades are exported to external systems. Teacher is able to view raw grades, too. It would be nice to have a way how the teacher could preview the final grades as they would look like if a set of raw grades were approved (dry-run approval).

== Example ==

Let us look at an example of how this mechanism could be used in a real course. Suppose there is a course called "Defence Against the Dark Arts" (D.A.D.A). The gradebook in that course is organized into three top categories (Autumn term, Spring term and Summer term), all using "Sum of grades" aggregation. The course total is aggregated as a sum, too.

This is how "Categories and items > Simple view" gradebook page looks like. Note the semaphore (traffic light) icons. The teacher can click the semaphore icon to open a page where approval conditions for the given grade item are configured. He can add more condition or remove/modify the current ones.

[[image:gradebook-buffer-ex0.png]]

The autumn term has just started and the students' current task is to attempt the preliminary quiz. They already got grades for their introduction essays. Right now, some students already have got a grade for the preliminary quiz while others are still waiting for the attempt.

The teacher has set up the grade items in the gradebook in the following way:

* There was a condition on the "Introduction essay" that the teacher had to manually approve the grades before they became available to students. He already did so and therefore the green semaphore icon is displayed at the column heading of the grader report and the final grades are displayed normally.
* There is a condition on the "Preliminary quiz" that the grades will be hidden from students until the quiz closes. Because the quiz has not closed yet, there is the red semaphore icon displayed and the grades are in parentheses (round brackets) and slightly dimmed. These are raw grades available to the teacher, students can't see them yet. Once the quiz module sends a signal via gradebook API, the raw grades will be approved and will become final grades.
* There is a date condition on the "Course total" item so that the total grade is not available to student until the end of the school year. Again, the teacher can see the raw values but students can't.
* No other grade items have approval conditions defined so no semaphore icon is displayed for them. Their raw grades becomes final grade automatically and immediately - see for example "Category total" for the Autumn term category.

[[image:gradebook-buffer-ex1.png]]

Clicking the semaphore icon in the heading of the grader report executes the dry-run switch (approval or hiding the grade item) so the teacher can see how the grades would change. Clicking the green semaphore simulates hiding the grades (so the raw grades would be displayed only), clicking the red semaphore simulates approving the grades. In both cases, a big red warning at the top of the screen "This is just a preview" or so is displayed.

Let us look closely at Hermione's grades, for instance. She got 20/20 for the introduction essay and got 10/10 for the preliminary quiz. However, she does not know the result of the quiz yet as that grade has not been approved yet. Therefore it is not aggregated into the category total. Her course total grade so far is 20/100 but again, this is not displayed to her yet.

Development talk:Gradebook 2.x architecture

2011-04-29T23:23:32Z

Mudrd8mz: Forum link

== PLEASE DO NOT USE THIS TALK PAGE ==

'''The issue is being discussed in the Gradebook forum. Please use the thread at http://moodle.org/mod/forum/discuss.php?d=174346 to comment on this spec'''

----

== Initial feedback from Tim ==

May I commend to you the wise words:
: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.” — Fred Brooks in “The Mythical Man-Month” 1975

The point is, we need to work out exactly what data we will be storing, and more importantly, what the meaning of what we store is. If we get that right, then the rest should be obvious. Of course, it is not at all clear what data structure, solves all the problems. Getting that right will take a lot of thought.--[[User:Tim Hunt|Tim Hunt]] 22:46, 30 March 2011 (UTC)

== More feedback from Tim ==

Moodle developers' chat about this: http://moodle.org/mod/cvsadmin/view.php?conversationid=7174#c266669

From there, I am a bit worried about how the following scenarios are handled:

Scenario 1:
* We have a course with a formative assessment that is open throughout the length of the course.
* Students can attempt the quiz at any time, and they can make multiple attempts.
* The grades for this formative quiz should appear in the gradebook the moment the student has finished an attempt, and should be visible to both students and teachers. (Although these grades do not affect the total course grade.)

Scenario 2:
* Summative test with a deadline.
* Students should only see their grades in the gradebook after the deadline as passed, and then the teacher has approved the grades.
* Before the deadline, teachers must be able to see the grades for their students (think Separate groups) in the gradebook, to monitor progress.

--[[User:Tim Hunt|Tim Hunt]] 06:39, 8 April 2011 (CDT)

Development:Gradebook 2.x architecture

2011-04-28T01:42:35Z

Mudrd8mz:

{{Infobox Project
|name = Gradebook 2.x architecture
|state = Planning
|tracker = MDL-25423
|discussion = N/A
|assignee = unassigned
}}

This page summarizes some initial ideas and suggestions that may help to resolve [[Development:Gradebook improvements#Stage 3|Stage 3]] of the Gradebook improvements project.

== Current issues ==

These are some of the most important issues with the current Gradebook design that this spec tries to deal with. See http://www.mindmeister.com/89537697/gradebook-2-x-issues-braindump for the full record of the braindumping session.

; The grades appear in the gradebook immediately (MDL-25439) : The grade values are stateless. We need a way how to store grades in the gradebook but exclude them from aggregations yet. Currently, the ''hiddenuntil'' feature tries to solve this but it is pretty limited and hacky (MDL-25440).
; Students and teachers may see different values : Because of how grades hiding is implemented currently, it is practically impossible to pre-populate whole grades tree and/or export reliable data (because they are dependent on the current role permissions).
; Unable to alter grades processing : There is a demand for alternate methods of grades processing (eg grades over 100% or negative grades) due to local conventions in various countries and school systems.
; Grading logic and UI implemented in activity modules : Grading tools like single point grade, scale-based grading or a new rubric-based grading should be handled by the Gradebook core with a rich set of interface toward activity modules (and eventually other plugin types).
; Manual and auto grades interaction : What should happen in this scenario has never been defined: 1. Student makes first attempt at the quiz, gets grade of 50%, this is pushed to the gradebook. 2. Teacher manually edits the grade to be 60%. 3. Student makes second quiz attempt and scores 75%. What should the gradebook show now? -- Tim Hunt
:: a similar situation happens in AMOS when the original English text is modified after it has been translated. If we record the timestamps, we can highlight such overridden grade as "outdated" (with a possibility to mark it as up-to-date hence un-highlight it). Still the manually edited value 60% should be the valid one IMHO --[[User:David Mudrak|David Mudrak]] 15:45, 14 April 2011 (WST)

== Suggested solutions and improvements ==

The goal is to make the gradebook so that '''simple things are easy and complex things are possible'''. That is for courses that just need to maintain a list of graded activities and calculate the final grade, the setup for teacher should be straightforward using default values. Advanced usage with a complex tree of grade items and calculations happening at several layers should be possible, though the setup may require more experience and insight into the internals.

According to this document, the gradebook to be still represented as a tree of grade items. There is no need to change this. Grade items may be associated with an activity module (multiple grade items per mod are supported) or a gradebook category. Grades are aggregated from the leaves of the tree to the root grade item that represents the total course grade.

[[image:gradebook-tree.png]]

In the example above, there are two grade items associated with modules in the course. These two are aggregated into a single item associated with a grades category. This category grade item is aggregated together with yet another grade item (that itself is populated via grader report) and form the course total grade.

=== Grades in core space or plugin space ===

Generally the grade values are to be stored by the gradebook in its own core tables. This will centralize and unify the storage and processing of the grades. For most modules like Assignment, Forum, Wiki, Database etc. (and eventually other plugin types like blocks if we ever decide to support grades there, too) this is enough as they do not need to keep the grades in their own tables.

This does not mean that the modules would be forced to give up advanced grades processing features. For example Quiz or Workshop modules must store a lot of additional data to calculate the final grades. So the Quiz will still hold the calculated grades for all user's attempts. But it will push the valid one (best attempt, average, maximum or whatever is set) to the gradebook. Similarly the Workshop will hold the information how the assessment forms were filled by peers. But the final grades for each student will be pushed into the gradebook tables via some nice API.

=== Grade approval process and grade values states ===

This is a suggestion to extend the grade_grades table so that every grade item can actually hold three independent values:

* ''Raw'' grade value - This is the value that comes from the source - from the activity module, from the grader report or as a result of the aggregation at lower levels. It behaves as an input buffer that just keeps the most recent incoming value.
* ''Overridden'' grade value - This is the value of the grade defined in the gradebook.
* ''Final'' grade value - This is a public value of the grade, it is used for aggregations at higher levels, it is being exported, displayed to users etc.

[[image:gradebook-buffer.png]]

The key point of this new scheme is how and when the ''final'' value is populated. The container (column) holding the final value is populated during the grade ''approval''. Whenever the gradebook tree is being recalculated, the following procedure is taken:

* If the value of overridden grade is not null, then use it as the final value (and break as we are finished)
* If there are no approval conditions defined, take the raw grade value and use it as the final value
* If there are some approval conditions defined, evaluate the list of conditions. If the evaluation passes, use the raw grade as the final grade value
* Otherwise, set the final grade to null and eventually indicate if there are some raw grade values available but on hold.

The approval conditions mechanism should support at least the following types of conditions:

* Date-based conditions - This would replace the current hidden-until feature
* Event-based conditions - Things like "take the grades into account only after all submissions are assessed"
* Manual trigger - special type of an event. The teacher can easily order the gradebook to "take the grades into account NOW" by clicking an icon or so

At the moment, we are unable to distinguish between the incoming ''raw'' grade and the outgoing ''final'' grade value. Therefore it is not easy to put a grade on hold. For teachers in schools, it is pretty natural to have some grades already known but not having them counted into the total grade yet.

Note that by default there are no approval conditions. So the raw grades becomes automatically final grades (backward compatible behaviour). So simple scenarios are kept simple while complex scenarios possible.

To be decided: do the conditions have to pass to take the overridden value into account, too? That is - if there are some approval conditions defined, is the overridden value dependent on them or not?

=== Grade values and their interpretation and display ===

Currently, grading configuration (like max grade, grade type etc) is stored by modules themselves, often independently on the associated grade item settings. This is a proposal to change it so:

* The modules themselves do not know anything about the actual grade setting. That is, they do not know max grade, min grade, the grade display type etc.
* All the grade items settings is stored and controlled exclusively by the the gradebook code
* There is a callback mechanism introduced so that the gradebook can push the required grades setting into the mod_edit form and the module settings node in the Settings block (compare with how enrol plugins push their settings into the course edit form and how they have their own setting in the course settings node)
* When pushing a grade into the gradebook (so that is saved as the ''raw'' grade value - see above), the activity module uses normalized decimal value from 0.00000 to 1.00000.
* The activity can ask the gradebook via its public API to format a given float value according the current grade item settings. "Hey gradebook, this student reached 0.81076. What shall I print to them?" and the gradebook would return "4/5" or "pretty good" or "81%" or whatever is defined there.

So the final, overridden and raw grade values are stored as normalized 0.00000 - 1.00000 decimals in grade_grades. Their actual interpretation depends on the grade item settings.

Question: should the final value depend on the actual grade item type? Example: let us have a grade item the type of which is three-points scale 1, 2, 3 and the raw grade value is 0.97654. During the approval, should the value be transformed to 1.00000 or should it stay original?

=== Introducing gradebook engines ===

The grade values stored in the tree and the aggregation and approval logic will be separated. Right now, the grade item itself contain the computation logic. A new class, so called gradebook engine, will be responsible for all grade values handling in the future. This should be pluggable and OOP-ish. This way, we should be able to implement alternative calculation engines, for example one that does not strip normalized values to <0; 1> interval but supports grades over 100% or negative grades.

=== Rubric support ===

At the moment, rubric assessment tool is supported by the workshop module only. There is a demand to use rubric as a general assessment tool across all activities. Rubric gradeitems will be probably stored separately and the gradebook/rubric engine will process them, resulting in a single 0.000000 - 1.00000 value to be pushed into the representing gradebook gradeitem. Alternatively, rubric items can be stored as ordinary gradebook gradeitems but they will be hidden from the normal gradebook report (only the result of the rubric aggregation would be displayed).

Rubric must not depend on standard scales as most rubric items will use their own scale that does not have much sense outside the context of the given rubric.

=== Grades over 100% ===

As this document suggests to separate the grades storage from the grade processing logic, there can be a separate gradebook engine designed as needed to process grades over 100%. Apparently the storage engine must support grade values > 1.00000 (and actually even grade values < 0.00000).

=== Problem with different values seen by the teacher and by students ===

The student views only show the final grades. And only final grades are exported to external systems. Teacher is able to view raw grades, too. It would be nice to have a way how the teacher could preview the final grades as they would look like if a set of raw grades were approved (dry-run approval).

== Example ==

Let us look at an example of how this mechanism could be used in a real course. Suppose there is a course called "Defence Against the Dark Arts" (D.A.D.A). The gradebook in that course is organized into three top categories (Autumn term, Spring term and Summer term), all using "Sum of grades" aggregation. The course total is aggregated as a sum, too.

This is how "Categories and items > Simple view" gradebook page looks like. Note the semaphore (traffic light) icons. The teacher can click the semaphore icon to open a page where approval conditions for the given grade item are configured. He can add more condition or remove/modify the current ones.

[[image:gradebook-buffer-ex0.png]]

The autumn term has just started and the students' current task is to attempt the preliminary quiz. They already got grades for their introduction essays. Right now, some students already have got a grade for the preliminary quiz while others are still waiting for the attempt.

The teacher has set up the grade items in the gradebook in the following way:

* There was a condition on the "Introduction essay" that the teacher had to manually approve the grades before they became available to students. He already did so and therefore the green semaphore icon is displayed at the column heading of the grader report and the final grades are displayed normally.
* There is a condition on the "Preliminary quiz" that the grades will be hidden from students until the quiz closes. Because the quiz has not closed yet, there is the red semaphore icon displayed and the grades are in parentheses (round brackets) and slightly dimmed. These are raw grades available to the teacher, students can't see them yet. Once the quiz module sends a signal via gradebook API, the raw grades will be approved and will become final grades.
* There is a date condition on the "Course total" item so that the total grade is not available to student until the end of the school year. Again, the teacher can see the raw values but students can't.
* No other grade items have approval conditions defined so no semaphore icon is displayed for them. Their raw grades becomes final grade automatically and immediately - see for example "Category total" for the Autumn term category.

[[image:gradebook-buffer-ex1.png]]

Clicking the semaphore icon in the heading of the grader report executes the dry-run switch (approval or hiding the grade item) so the teacher can see how the grades would change. Clicking the green semaphore simulates hiding the grades (so the raw grades would be displayed only), clicking the red semaphore simulates approving the grades. In both cases, a big red warning at the top of the screen "This is just a preview" or so is displayed.

Let us look closely at Hermione's grades, for instance. She got 20/20 for the introduction essay and got 10/10 for the preliminary quiz. However, she does not know the result of the quiz yet as that grade has not been approved yet. Therefore it is not aggregated into the category total. Her course total grade so far is 20/100 but again, this is not displayed to her yet.

File:gradebook-buffer-ex0.png

2011-04-28T01:35:11Z

Mudrd8mz: uploaded a new version of "File:gradebook-buffer-ex0.png"

Example structure of a demo gradebook

Development:Gradebook 2.x architecture

2011-04-28T01:29:11Z

Mudrd8mz: Example and first mockups

{{Infobox Project
|name = Gradebook 2.x architecture
|state = Planning
|tracker = MDL-25423
|discussion = N/A
|assignee = unassigned
}}

This page summarizes some initial ideas and suggestions that may help to resolve [[Development:Gradebook improvements#Stage 3|Stage 3]] of the Gradebook improvements project.

== Current issues ==

These are some of the most important issues with the current Gradebook design that this spec tries to deal with. See http://www.mindmeister.com/89537697/gradebook-2-x-issues-braindump for the full record of the braindumping session.

; The grades appear in the gradebook immediately (MDL-25439) : The grade values are stateless. We need a way how to store grades in the gradebook but exclude them from aggregations yet. Currently, the ''hiddenuntil'' feature tries to solve this but it is pretty limited and hacky (MDL-25440).
; Students and teachers may see different values : Because of how grades hiding is implemented currently, it is practically impossible to pre-populate whole grades tree and/or export reliable data (because they are dependent on the current role permissions).
; Unable to alter grades processing : There is a demand for alternate methods of grades processing (eg grades over 100% or negative grades) due to local conventions in various countries and school systems.
; Grading logic and UI implemented in activity modules : Grading tools like single point grade, scale-based grading or a new rubric-based grading should be handled by the Gradebook core with a rich set of interface toward activity modules (and eventually other plugin types).
; Manual and auto grades interaction : What should happen in this scenario has never been defined: 1. Student makes first attempt at the quiz, gets grade of 50%, this is pushed to the gradebook. 2. Teacher manually edits the grade to be 60%. 3. Student makes second quiz attempt and scores 75%. What should the gradebook show now? -- Tim Hunt
:: a similar situation happens in AMOS when the original English text is modified after it has been translated. If we record the timestamps, we can highlight such overridden grade as "outdated" (with a possibility to mark it as up-to-date hence un-highlight it). Still the manually edited value 60% should be the valid one IMHO --[[User:David Mudrak|David Mudrak]] 15:45, 14 April 2011 (WST)

== Suggested solutions and improvements ==

The goal is to make the gradebook so that '''simple things are easy and complex things are possible'''. That is for courses that just need to maintain a list of graded activities and calculate the final grade, the setup for teacher should be straightforward using default values. Advanced usage with a complex tree of grade items and calculations happening at several layers should be possible, though the setup may require more experience and insight into the internals.

According to this document, the gradebook to be still represented as a tree of grade items. There is no need to change this. Grade items may be associated with an activity module (multiple grade items per mod are supported) or a gradebook category. Grades are aggregated from the leaves of the tree to the root grade item that represents the total course grade.

[[image:gradebook-tree.png]]

In the example above, there are two grade items associated with modules in the course. These two are aggregated into a single item associated with a grades category. This category grade item is aggregated together with yet another grade item (that itself is populated via grader report) and form the course total grade.

=== Grades in core space or plugin space ===

Generally the grade values are to be stored by the gradebook in its own core tables. This will centralize and unify the storage and processing of the grades. For most modules like Assignment, Forum, Wiki, Database etc. (and eventually other plugin types like blocks if we ever decide to support grades there, too) this is enough as they do not need to keep the grades in their own tables.

This does not mean that the modules would be forced to give up advanced grades processing features. For example Quiz or Workshop modules must store a lot of additional data to calculate the final grades. So the Quiz will still hold the calculated grades for all user's attempts. But it will push the valid one (best attempt, average, maximum or whatever is set) to the gradebook. Similarly the Workshop will hold the information how the assessment forms were filled by peers. But the final grades for each student will be pushed into the gradebook tables via some nice API.

=== Grade approval process and grade values states ===

This is a suggestion to extend the grade_grades table so that every grade item can actually hold three independent values:

* ''Raw'' grade value - This is the value that comes from the source - from the activity module, from the grader report or as a result of the aggregation at lower levels. It behaves as an input buffer that just keeps the most recent incoming value.
* ''Overridden'' grade value - This is the value of the grade defined in the gradebook.
* ''Final'' grade value - This is a public value of the grade, it is used for aggregations at higher levels, it is being exported, displayed to users etc.

[[image:gradebook-buffer.png]]

The key point of this new scheme is how and when the ''final'' value is populated. The container (column) holding the final value is populated during the grade ''approval''. Whenever the gradebook tree is being recalculated, the following procedure is taken:

* If the value of overridden grade is not null, then use it as the final value (and break as we are finished)
* If there are no approval conditions defined, take the raw grade value and use it as the final value
* If there are some approval conditions defined, evaluate the list of conditions. If the evaluation passes, use the raw grade as the final grade value
* Otherwise, set the final grade to null and eventually indicate if there are some raw grade values available but on hold.

The approval conditions mechanism should support at least the following types of conditions:

* Date-based conditions - This would replace the current hidden-until feature
* Event-based conditions - Things like "take the grades into account only after all submissions are assessed"
* Manual trigger - special type of an event. The teacher can easily order the gradebook to "take the grades into account NOW" by clicking an icon or so

At the moment, we are unable to distinguish between the incoming ''raw'' grade and the outgoing ''final'' grade value. Therefore it is not easy to put a grade on hold. For teachers in schools, it is pretty natural to have some grades already known but not having them counted into the total grade yet.

Note that by default there are no approval conditions. So the raw grades becomes automatically final grades (backward compatible behaviour). So simple scenarios are kept simple while complex scenarios possible.

To be decided: do the conditions have to pass to take the overridden value into account, too? That is - if there are some approval conditions defined, is the overridden value dependent on them or not?

=== Grade values and their interpretation and display ===

Currently, grading configuration (like max grade, grade type etc) is stored by modules themselves, often independently on the associated grade item settings. This is a proposal to change it so:

* The modules themselves do not know anything about the actual grade setting. That is, they do not know max grade, min grade, the grade display type etc.
* All the grade items settings is stored and controlled exclusively by the the gradebook code
* There is a callback mechanism introduced so that the gradebook can push the required grades setting into the mod_edit form and the module settings node in the Settings block (compare with how enrol plugins push their settings into the course edit form and how they have their own setting in the course settings node)
* When pushing a grade into the gradebook (so that is saved as the ''raw'' grade value - see above), the activity module uses normalized decimal value from 0.00000 to 1.00000.
* The activity can ask the gradebook via its public API to format a given float value according the current grade item settings. "Hey gradebook, this student reached 0.81076. What shall I print to them?" and the gradebook would return "4/5" or "pretty good" or "81%" or whatever is defined there.

So the final, overridden and raw grade values are stored as normalized 0.00000 - 1.00000 decimals in grade_grades. Their actual interpretation depends on the grade item settings.

Question: should the final value depend on the actual grade item type? Example: let us have a grade item the type of which is three-points scale 1, 2, 3 and the raw grade value is 0.97654. During the approval, should the value be transformed to 1.00000 or should it stay original?

=== Introducing gradebook engines ===

The grade values stored in the tree and the aggregation and approval logic will be separated. Right now, the grade item itself contain the computation logic. A new class, so called gradebook engine, will be responsible for all grade values handling in the future. This should be pluggable and OOP-ish. This way, we should be able to implement alternative calculation engines, for example one that does not strip normalized values to <0; 1> interval but supports grades over 100% or negative grades.

=== Rubric support ===

At the moment, rubric assessment tool is supported by the workshop module only. There is a demand to use rubric as a general assessment tool across all activities. Rubric gradeitems will be probably stored separately and the gradebook/rubric engine will process them, resulting in a single 0.000000 - 1.00000 value to be pushed into the representing gradebook gradeitem. Alternatively, rubric items can be stored as ordinary gradebook gradeitems but they will be hidden from the normal gradebook report (only the result of the rubric aggregation would be displayed).

Rubric must not depend on standard scales as most rubric items will use their own scale that does not have much sense outside the context of the given rubric.

=== Grades over 100% ===

As this document suggests to separate the grades storage from the grade processing logic, there can be a separate gradebook engine designed as needed to process grades over 100%. Apparently the storage engine must support grade values > 1.00000 (and actually even grade values < 0.00000).

=== Problem with different values seen by the teacher and by students ===

The student views only show the final grades. And only final grades are exported to external systems. Teacher is able to view raw grades, too. It would be nice to have a way how the teacher could preview the final grades as they would look like if a set of raw grades were approved (dry-run approval).

== Example ==

Let us look at an example of how this mechanism could be used in a real course. Suppose there is a course called "Defence Against the Dark Arts" (D.A.D.A). The gradebook in that course is organized into three top categories (Autumn term, Spring term and Summer term), all using "Sum of grades" aggregation. The course total is aggregated as a sum, too.

[[image:gradebook-buffer-ex0.png]]

The autumn term has just started and the students' current task is to attempt the preliminary quiz. They already got grades for their introduction essays. Right now, some students already have got a grade for the preliminary quiz while others are still waiting for the attempt.

The teacher has set up the grade items in the gradebook in the following way:

* There was a condition on the "Introduction essay" that the teacher had to manually approve the grades before they became available to students. He already did so and therefore the green semaphore icon is displayed at the column heading of the grader report and the final grades are displayed normally.
* There is a condition on the "Preliminary quiz" that the grades will be hidden from students until the quiz closes. Because the quiz has not closed yet, there is the red semaphore icon displayed and the grades are in parentheses (round brackets) and slightly dimmed. These are raw grades available to the teacher, students can't see them yet. Once the quiz module sends a signal via gradebook API, the raw grades will be approved and will become final grades.
* There is a date condition on the "Course total" item so that the total grade is not available to student until the end of the school year. Again, the teacher can see the raw values but students can't.
* No other grade items have approval conditions defined so no semaphore icon is displayed for them. Their raw grades becomes final grade automatically and immediately - see for example "Category total" for the Autumn term category.

[[image:gradebook-buffer-ex1.png]]

The teacher can click the semaphore icon to open a page where approval conditions are configured. He can add more condition or remove/modify the current ones. Alternatively, clicking the semaphore can execute the dry-run approval so the teacher would see how the grades look like if the grade items is approved (with a big red warning at the top of the screen "This is just a preview" or so).

Let us look closely at Hermione's grades. She got 20/20 for the introduction essay and got 10/10 for the preliminary quiz. However, she does not know the result of the quiz yet as that grade has not been approved yet. Therefore it is not aggregated into the category total. Her course total grade so far is 20/100 but again, this is not displayed to her yet.

File:gradebook-buffer-ex1.png

2011-04-28T00:20:45Z

Mudrd8mz: UI mockup of the new gradebook features

UI mockup of the new gradebook features

File:gradebook-buffer-ex0.png

2011-04-28T00:19:39Z

Mudrd8mz: Example structure of a demo gradebook

Example structure of a demo gradebook

File:gradebook-buffer.png

2011-04-27T23:10:04Z

Mudrd8mz: uploaded a new version of "File:gradebook-buffer.png": Terminology change

Development:Gradebook 2.x architecture

2011-04-27T23:04:52Z

Mudrd8mz: Terminology change and added some notes from the meeting

User:David Mudrak

2011-04-27T11:50:11Z

Mudrd8mz: /* How to motivate me */

[[Image:mudrd8mz.jpg|right]] Hi! Welcome to my profile page. I am one of [http://moodle.com/hq Moodle HQ] developers, but my background includes ICT teaching and research into the theory of education. I am the author of several Moodle extensions and customizations. I translate Moodle into Czech and run Czech Moodle portal [http://moodle.cz moodle.cz].

==How to motivate me==

Would you like to support yet another code monkey? [[User:David Mudrak/Postcard|Send me a postcard!]]

==Standard color scheme I use in my code==

The first value is web safe but I like the second one more

{|
| style="background: #EEEE22;" | #EEEE22 #f3f2aa
| style="background: #AAEEAA;" | #AAEEAA #e7f1c3
| style="background: #AACCEE;" | #AACCEE #d2ebff
| style="background: #EEAAAA;" | #EEAAAA #ffd3d9
|}

==My contributions==

* [[Workshop module]]
* [[Translation|Moodle language translation tool]]
* [[Course contents block]]
* [[Subcourse module]]
* [[Stamp Collection module]]
* [[Course ordering and invoicing]]

==My images==

[[Image:question-categories.png|thumb|left]]
[[Image:question-contexts.png|thumb|left]]
[[Image:workshop_grades_calculation.png|thumb|left]]
[[Image:uml_output_renderers.png|thumb|left]]
[[Image:versions.png|thumb|left]]
[[Image:branches.png|thumb|left]]

 

==My drafts==

* [[Using GIT to backup moodledata]]
* [[Obsolete:Tutorial on using git in Moodle development]]
* [[Development:Commit cheat sheet]]

==My bookmarks==

* [[Development:Places to search for lang strings]]

==Miscellaneous==

* [[User:David Mudrak/vimrc|~/.vimrc]] - ViM configuration I use for Moodle development

User:David Mudrak/Postcard

2011-04-27T11:49:40Z

Mudrd8mz:

To motivate David for further work on Moodle, please send a postcard of your place to the following address. It will make yet another code monkey happy :-)

David Mudrak
Jecna 488/23
Liberec
460 15
Czech Republic

User:David Mudrak/Postcard

2011-04-27T11:44:03Z

Mudrd8mz: Created page with "To motivate David for further work on Moodle, please send a postcard of your place to the following address. It will make yet another coding monkey happy :-) David Mudrak ..."

To motivate David for further work on Moodle, please send a postcard of your place to the following address. It will make yet another coding monkey happy :-)

David Mudrak
Jecna 488/23
Liberec
460 15
Czech Republic

User:David Mudrak

2011-04-27T11:39:31Z

Mudrd8mz:

[[Image:mudrd8mz.jpg|right]] Hi! Welcome to my profile page. I am one of [http://moodle.com/hq Moodle HQ] developers, but my background includes ICT teaching and research into the theory of education. I am the author of several Moodle extensions and customizations. I translate Moodle into Czech and run Czech Moodle portal [http://moodle.cz moodle.cz].

==How to motivate me==

Would you like to support yet another coding monkey? [[User:David Mudrak/Postcard|Send me a postcard!]]

==Standard color scheme I use in my code==

The first value is web safe but I like the second one more

{|
| style="background: #EEEE22;" | #EEEE22 #f3f2aa
| style="background: #AAEEAA;" | #AAEEAA #e7f1c3
| style="background: #AACCEE;" | #AACCEE #d2ebff
| style="background: #EEAAAA;" | #EEAAAA #ffd3d9
|}

==My contributions==

* [[Workshop module]]
* [[Translation|Moodle language translation tool]]
* [[Course contents block]]
* [[Subcourse module]]
* [[Stamp Collection module]]
* [[Course ordering and invoicing]]

==My images==

[[Image:question-categories.png|thumb|left]]
[[Image:question-contexts.png|thumb|left]]
[[Image:workshop_grades_calculation.png|thumb|left]]
[[Image:uml_output_renderers.png|thumb|left]]
[[Image:versions.png|thumb|left]]
[[Image:branches.png|thumb|left]]

 

==My drafts==

* [[Using GIT to backup moodledata]]
* [[Obsolete:Tutorial on using git in Moodle development]]
* [[Development:Commit cheat sheet]]

==My bookmarks==

* [[Development:Places to search for lang strings]]

==Miscellaneous==

* [[User:David Mudrak/vimrc|~/.vimrc]] - ViM configuration I use for Moodle development

Development:Gradebook 2.x architecture

2011-04-14T07:46:18Z

Mudrd8mz: /* Current issues */ fixed a typo in the comment alignment

2011-04-05T14:16:51Z

Mudrd8mz: Do not use real MDL numbers in examples to avoid autolinking

This document is for helping you get started on Moodle development with Git. For further details of Git, see [[:Category:Git]].

== General workflow ==

[[image:git-pushpull-model.png|right|thumb|400px|Moodle development workflow with Git]]
In short, the Moodle development with Git looks like this:

* You as the contributor commit changes into your personal repository at your computer
* You push the changes into your public repository and ask for their inclusion via so called [[Development:PULL request|PULL request]]
* Moodle integrators pull the changes from your public repository and if they like them, they put them into Moodle integration repository
* The integrated change is tested and finally pushed into Moodle production repository
* You update your local repository with all changes from the production repository and the next cycle may start again

This workflow runs in roughly weekly cycles. The integration happens on Monday and the testing on Tuesday. On Wednesday, the production repository moodle.git is usually updated with changes from the last development week.

Most Moodle developers have their public repositories hosted at [http://github.com/ Github]. Alternatively you may want to try [http://gitorious.org Gitorious] or the legendary [http://repo.or.cz repo.or.cz]. In the examples in this guide we assume you'll set up your public repository at Github.

== Installing Git on your computer ==

Install Git on your computer. Most Linux distributions have Git available as a package to install. If you are on Mac, [http://code.google.com/p/git-osx-installer/ git-osx-installer] installs it in few click.

Immediately after the installation, set your name and contact e-mail. The name and e-mail will become part of your commits and they can't be changed later once your commits are accepted into the Moodle code. Therefore we ask contributors to use their real names written in capital letters, eg "John Smith" and not "john smith" or even "john5677".

git config --global user.name "Your Name"
git config --global user.email yourmail@domain.tld

Unless you are the repository maintainer, it is wise to set your Git to not push changes in file permissions:

git config --global core.filemode false

== Setting-up the public repository ==

1. Go to [http://github.com/ Github] and create an account.

2. Go to the [http://github.com/moodle/moodle official Moodle Github repository] and click on the Fork button. You now have your own Github Moodle repository.

3. Now you need to set up your SSH public key, so you can push to your Github Moodle repository from your local Moodle repository. On Mac you can go on this [http://help.github.com/mac-key-setup/ Github help page]. If you are on another system, go to your Github administration page, to the section SSH Public Keys, and you should see a link to a help page. Done? Good! That was the most difficult part!

== Setting-up the local repository at your computer ==

Create a local clone repository of your Github repository. In a terminal:

git clone git@github.com:YOUR_GITHUB_USERNAME/moodle.git YOUR_LOCAL_MOODLE_FOLDER

This command does several jobs for you. It creates a new folder, initializes an empty Git repository in it, sets your Github repository as the remote repository called 'origin' and makes a local checkout of the branch 'master' from it. The important point to remember now is that your Github repository is aliased as 'origin' for your local clone.

== Keeping your public repository up-to-date ==

[[image:git-sync-github.png|right|thumb|400px|Fetching changes from upstream and pushing them to github]]
Your fork at Github is not updated automatically. To keep it synced with the upstream Moodle repository, you have to fetch the recent changes from the official moodle.git and push them to your public repository. To avoid problems with this it is strongly recommended that you never modify the standard Moodle branches. ''Remember: never commit directly into master and MOODLE_xx_STABLE branches.'' In other words, always create topic branches to work on. In Gitspeak, the master branch and MOODLE_xx_STABLE branches should be always fast-forwardable.

To keep your public repository up-to-date, we will register remote repository git://git.moodle.org/moodle.git under 'upstream' alias. Then we create a script to be run regularly that fetches changes from the upstream repository and pushes them to your public repository. Note that this procedure will not affect your local working directory.

To register the upstream remote:

git remote add upstream git://git.moodle.org/moodle.git

The following commands can be used to keep the standard Moodle branches at your Github repository synced with the upstream repository. You may wish to store them in a script so that you can run it every week after the upstream repository is updated.

git fetch upstream
for BRANCH in MOODLE_19_STABLE MOODLE_20_STABLE master; do
git push github refs/remotes/upstream/$BRANCH:$BRANCH
done

=== How it works ===

The git-fetch command does not modify your current working dir (your checkout). It just downloads all recent changes from a remote repository and stores them into so called remote-tracking branches. The git-push command takes these remote-tracking branches from upstream and pushes them to Github under the same name. Understanding this fully requires a bit knowledge of Git internals - see gitrevisions(7) man page.

Note there is no need to switch the local branch during this. You can even execute this via cron at your machine. Just note that the upstream repository updates typically just once a week.

== Preparing a patch ==

As said earlier at this page, you never work on standard Moodle branches directly. Every time you are going to edit something, switch to a local branch. Fork the local branch off the standard branch you it should be merged to. So if you are working on a patch for 1.9 or 2.0, fork the branch off MOODLE_19_STABLE or MOODLE_20_STABLE, respectively. Patches for the next [[Moodle versions|major version]] should be based on the master branch.

git checkout -b MDL-xxxxx-nasty-bug origin/master

Note that if you forget to specify the starting point, the branch is based on the currently checked-out branch. It may not be what you want. It is recommended to always specify the starting point.

To check the current branch, run

git branch

The current branch is highlighted.

Now go and fix the issue with your favorite IDE. Check the status of the files, view the change to be committed and finally commit the change:

vim filename.php
git status
git diff
git commit -a

Note that this is safe as the commit is recorded just locally, nothing is sent to any server yet (as it would in CVS). To see history of the commits, use

git log

Once your local branch contains the change (note that it may consists of several patches) and you are happy with it, publish the branch at your public repository:

git push

Because we did not specify explicit remote repository, the 'origin' is used. Because we did not specify the branch to push, the Git will use the current branch and push it to the remote repository under the same name (by default, this is a subject of your configuration. See push.default config variable).

Now as your branch is published, you can ask Moodle core developers to review it and eventually integrate it into the standard Moodle repository.

=== Checking if a branch has already been merged ===

TODO

=== Deleting a branch in remote repository ===

git push origin :branchname

== Peer-reviewing someone else's code ==

To review a branch that someone else pushed into their public repository, you do not need to register a new remote (unless you work with such repository frequently, of course). Let us imagine your friend Alice pushed a work-in-progress branch called 'wip-feature' into her Github repository and asked you to review it. You need to know the read-only address of the repository and the name of the branch.

git fetch git://github.com/alice/moodle.git wip-feature

This will download all required data and will keep the pointer to the tip of the wip-feature branch in a local symbolic reference FETCH_HEAD. To see what's there on that branch, use

git log -p FETCH_HEAD

To create a new local branch called 'alice-wip-feature' containing the work by Alice, use

git checkout -b alice-wip-feature FETCH_HEAD

To merge Alice's work into your current branch:

git merge FETCH_HEAD

To see what would be merged into the current branch without actually modifying anything:

git diff ...FETCH_HEAD

== Rebasing a branch ==

Rebasing is a process when you cut off the branch from its current start point and transplant it to another point. Let us assume the following history exists:

A---B---C topic
/
D---E---F---G master

From this point, the result of the command:

git rebase master topic

would be:

A'--B'--C' topic
/
D---E---F---G master

and would end with 'topic' being your current branch.

You may be asked to rebase your branch submitted for the integration if the submitted branch was based on an outdated commit. The typical case is if you create a new branch as a fork off the upstream master branch on Tuesday. Then on Wednesday, the upstream master branch grows as all changes from the last integration cycle are merged to it. To make diff easy on Github for next weekly pull request review, you want to rebase your branch against the updated master.

git rebase master MDL-xxxxx-topic-branch

Note that rebasing effectively rewrites the history of the branch. '''Do not rebase the branch if there is a chance that somebody has already forked it and based their own branch on it.''' For this reason, many Git tutorials discourage from rebasing any branch that has been published. However in Moodle, all branches submitted for integration are potential subject of rebase (even though we try to not to do it often) and you should not base your own branches on them.

=== Conflicts during rebase ===

During the rebase procedure, conflicts may appear. git-status commands reports the conflicted files. Explore them carefully and fix them in your editor (like you would do with CVS). Then add the files with 'git add' command and continue.

vim conflicted.php
git add conflicted.php
git rebase --continue

== Cherry-picking changes from one branch to another ==

TODO

== See also ==

* [http://www.kernel.org/pub/software/scm/git/docs/everyday.html Everyday GIT With 20 Commands Or So]
* [http://gitref.org/ Git Reference]
* [http://progit.org/book/ Pro Git book]

[[Category:Git]]

Development:Commit cheat sheet

2011-04-05T10:30:23Z

Mudrd8mz: Initial version of the page